It would be nice if compare plug-ins can have individual views like the analysis plug-ins

Allow users to submit the same firmware with another name. Needed to support different devices sharing the same firmware. The current implementation feels like a bug as new submissions delete old ones.

Endpoint might be connected to /firmware.
Update can be given in URL as a URL-encoded list.

It should to scan a single firmware with yara.

It should be possible to apply tags to firmware Images.

Create some sort of compare buffer, to which one can append a firmware or a file object by clicking some button in the analysis view.
Maybe some sort of pop-up can be triggered that asks if compare shall be started once the second, third etc. item is selected.

hash > name > vendor based free-text search field in header bar á la github (search repository)

Two new features for the quicksearch would be appreciated:

• search for file name: e.g. type dropbear and find all dropbear binaries including /usr/bin/dropbear or /sbin/dropbearmulti.
• if only one result, show result directly. Could also be used in all other search requests.

Show the version numbers of components (frontend, backend, database) on the system health page

I'm trying to do some simple analysis on the paths of files in a firmware bin. I expected that the file_object.file_path string would be the path of the file as it appears in the firmware (i.e. squashfs-root/web/apache2/conf), but instead it's a giant string of hex digits plus the absolute path on my machine.

Is there a way to get what I'm looking for? If I expand the file tree after everything is analyzed, all the files and folders have the correct names, but the analysis doesn't seem to have access to that.

Each feature (Canary, NX, PIE, RELRO) should be considered individually.

e.g.
#Canary disabled + #Canary enabled = 100%

1. Step: Database should be dropdown instead of page.
2. Step: Reduce amount of tabs by merging some of them into new dropdown menues.

As of now the config file lists default plugins that are preselect on the upload page.

This could be enhanced to support custom presets. The upload page could then contain a dropdown menu to switch between presets.

Presets could be defined in config just as the default plugins are now.

Preset ideas:

• Full (duh)
• Fast (none selected)
• UNIX Firmware (init systems, passwords, software components ..)
• Component (binwalk, printable strings, ip finder)
• ..

In admin mode there are now 8 elements + logo + quicksearch button on the navbar.
Some of the elements should be reordered to allow for better overview.

e.g. profile and admin can be moved to one dropdown.

should be similar to browse database regarding html, flask code

https://pypi.python.org/pypi/geoip2

use local database (download during install)!

As of now the list of presets - set in the config - is determined in the upload view by the line
analysis_plugin_dict[analysis_plugin_dict | first][2] | sort
this
a) introduces a bug that is hit whenever analysis_plugin_dict | first returns unpacker
b) is overly complicated since the notion of first on a dictionary is not trivial what makes this hard to understand

Now a more straight forward solution would be to generate this list in the route and adding it as a variable to the render_template method.

It would be great if exists the posibility to create a report (HTML, for instance) given a firmware with all the information extracted.

name of function has been changed, but refactoring did not change it in the endpoint implementation.

How do I get the FACT Webserver to run on the IP address and not the Localhost address 127.0.0.1?

We need a function to track relations between firmwares. The most common relation might be "previous" version.

create_variety_data should be executed less often => It should be separated from the update_statistic script and get its own crontab entry. Execution once a week should be sufficient.

If the plaintext of a password resembles a password hash (i.e. all 13 letters long passwords resemble DES hashes), it is stored in plaintext in the database and the password check does not work (meaning the user can't log in) due to a (intended?) bug in flask-security

possible solution: validate the password beforehand with the same hash identifying function used by passlib and disallow such passwords

Parameter is used only in test code and can be omitted by fitting the binary service and a couple of other files.
The omission e.g. allows renaming of the storage directory.

search string is no longer correct due to new data structure

Add pages for user management (a GUI version of the user management command line tool) and allow users to change their password

Installation script support for Ubuntu 18.04 Bionic Beaver

I followed the directions outlined here, yet my plugin does not show up as a checkbox on the upload page.

Are there additional steps I have to perform to add a plugin?

If authentication is enabled, a comment should provide the logged in user as author automatically.

In some cases the calculation of the pie statistic fails raising the following error:

[2018-03-28 13:02:04][update_statistic][INFO]: Try to start Mongo Server...
[2018-03-28 13:02:04][MongoMgr][INFO]: start local mongo database
Traceback (most recent call last):
  File "src/update_statistic.py", line 48, in <module>
    sys.exit(main())
  File "src/update_statistic.py", line 38, in main
    updater.update_all_stats()
  File "/home/weidenba/git/FACT_core_github/src/statistic/update.py", line 45, in update_all_stats
    self.db.update_statistic('exploit_mitigations', self._get_exploit_mitigations_stats())
  File "/home/weidenba/git/FACT_core_github/src/statistic/update.py", line 102, in _get_exploit_mitigations_stats
    self.get_stats_pie(result, stats)
  File "/home/weidenba/git/FACT_core_github/src/statistic/update.py", line 175, in get_stats_pie
    total_amount_of_files = self.calculate_total_files_for_pie(pie_off, pie_on, pie_partial, pie_invalid)
  File "/home/weidenba/git/FACT_core_github/src/statistic/update.py", line 194, in calculate_total_files_for_pie
    total_amount_of_files = pie_on[0][1] + pie_off[0][1] + pie_partial[0][1] + pie_invalid[0][1]
IndexError: list index out of range

It would be very nice to use the binary search function via the API.
The return would then be the matching UIDs of the firmware images.

Two major bottlenecks:

• get_object method is called at least "number-of-compared-firmwares" x "number-of-objects-in-db" times, creating a miss basically each time as the firmware collection is queried first.
• file_coverage doesn't use the analysis_filter in get_object creating massive overhead

The download included files functions just provides one layer of files. It would be nice to get the whole file tree of unpacked files of all layers.

In some cases you might have different parts of a firmware, that you want to add. E.g. Kernel, bootloader, filesystem. Therefore Firmware meta data should provide a "module" entry.

apt is a wrapper for apt-get and does not provide a stable interface.
Citation:
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

For this reason the scripts should simply use apt-get instead.
The following lines still make use of apt:

bootstrap_common.sh:7:sudo apt install -y apt-transport-https
bootstrap_common.sh:8:sudo apt update
bootstrap_common.sh:9:sudo apt upgrade -y
bootstrap_common.sh:10:sudo apt autoremove -y
bootstrap_common.sh:11:sudo apt clean
bootstrap_db.sh:17:sudo -E apt update
bootstrap_db.sh:18:sudo -E apt install -y mongodb-org
bootstrap_db.sh:19:sudo -E apt upgrade -y

Happens during some integration and acceptance tests.

The intercom backend binding for "delete file" exists but doesn't get started. Therefore, the delete file jobs that are sent over the intercom are not executed.

There might be old delete jobs in the database. Moreover, these old jobs might collide with new analyses: Files that should previously have been deleted were again inserted into the database and the old delete jobs might delete them erroneously. Therefore, a safety check should be implemented to secure that the files that are to be deleted aren't used by existing firmware entries.

The System tab already provides a list of plug-ins. This list should be exteded by the plugin-versions

In short:
The file coverage compare plugin has the file type analysis plugin as dependency. When objects are stored in the database before file type is finished, the compare plugin can run into an error.

What can be done:
It has to be evaluated, what the reaction to such a missing dependency is.
If this is an unwanted szenario - because analysis should always finish before compare - we can error-log this case.
We could also catch the ensuing exception and ignore the affected objects.
The most complex way would be to add something like analysis dependencies to compare plugins. But this would likely involve besides some changes to the scheduling a re-write of nearly all plugins.

Comparing a file to a firmware causes a broken view

The password list, previously found here is not accessible anymore. Thus it has to either be added statically, or an alternative wordlist has to be added.

Good practice for REST APIs include rich responses.
This includes providing links to all content related to the request.

Examples include previous and next links when using browsing parameters as well as links to newly created resources (e.g. on firmware upload request).

The generation of yara signature files is based on their plugin directories name, while the base plugin looks for them based on the plugin name.
Once directory name and plugin name deviate, the signature file is not found.