Kali Linux 2021.1 is out: https://www.kali.org/downloads/

We need to test emba and all the deps on the new release

We have started to include license details in our version identifier configuration here: https://github.com/e-m-b-a/emba/blob/master/config/bin_version_strings.cfg

This is in a very early stage and need your help. If you know the license of some of the software components please include it directly in the configuration and bring up a pull request or comment to this issue with the license and the source for verification.

Current state

If you are running into cve-search issues please update your EMBA installation:

sudo service mongod stop
sudo /etc/init.d/redis-server stop
sudo /etc/init.d/docker restart         
sudo ifconfig emba_runs down                                                                                                                                                                                        
git clone https://github.com/e-m-b-a/emba.git new_emba
cd new_emba
sudo ./installer.sh -d

After testing you should be able to remove your old EMBA installation and the installed mongodb.

The following information is outdated:

Local network issues

With the latest updates (see #183) we have isolated the EMBA container from a networking perspective. This means EMBA is not able to reach the internet anymore. With this also some adjustments in the EMBA docker container and in the local mongodb configuration were needed.

The easiest way to get an updated EMBA installation are the following steps:

cd emba-directory
git pull
sudo docker pull embeddedanalyzer/emba
sudo ./installer.sh -d

If you are running into issues that the EMBA container is not able to access your mongodb installation you can check the following:

• The command ifconfig should show a new network interface with the following ip address:

└─$ ifconfig emba_runs
emba_runs: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.36.0.1  netmask 255.255.0.0  broadcast 172.36.255.255
        ether 02:42:cb:c9:37:80  txqueuelen 0  (Ethernet)
        RX packets 15643  bytes 964808 (942.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 43695  bytes 1435782652 (1.3 GiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

If this is not the case you can try to restart docker at all with the following command
└─$ sudo /etc/init.d/docker restart
Another way is to delete the network interface and re-initialize it:

└─$ sudo docker network rm emba_runs
└─$ sudo docker-compose up --no-start

Additionally, you can try the following command

└─$ sudo systemctl restart NetworkManager docker

• Your mongod.conf should have the following bindIp configuration:

└─$ grep bindIp /etc/mongod.conf
  bindIp: 172.36.0.1

Mongod is not starting up

EMBA is complaining that CVE search works not as expected. Checking if mongod is listening, does not return an open port:

└─$ sudo netstat -anpt | grep 27017 

Also a restart of mongod does not help!

Please check if your harddisk was running out of space (e.g. with df -h). If it is full, clean it up and retry it.

Mongod is not starting up - part 2

Check your mongodb log /var/log/mongodb/mongod.log for errors. One of the seen errors is the following:

{"t":{"$date":"2023-03-24T15:40:07.026+08:00"},"s":"E", "c":"STORAGE", "id":20557, "ctx":"initandlisten","msg":"DBException in initAndListen, terminating","attr":{"error":"IllegalOperation: Attempted to create a lock file on a read-only directory: /var/lib/mongodb"}}

Check this solution and adjust the permissions.

No results from CVE-search

Probably you are running into the issue that your network environment is working as expected and mongod is also listening on port 27017. If you are trying to query your CVE database you get no results:

Sometimes it happens that the CVE database is corrupt or not fully populated. So, please do a full reinitialization and update the following way:

└─$ source ./external/emba_venv/bin/activate
└─$ ./external/cve-search/sbin/db_updater.py -f

As from November 2023 cve-search switched to the new NIST API. From there on you can also use the following command:

└─$ cvexplore database initialize

Errors during database update

During a manual CVE database update you are running into the following errors:

It looks like this or this issue.

"Did not receive last-modified header in the response" Error during database update

During a manual CVE database update you are running into the following errors:

CVEDownloads - ERROR - Did not receive last-modified header in the response; setting to default (01-01-1970) and force update! Headers received: {'content-length': '93', 'cache-control': 'no-cache', 'content-type': 'text/html', 'connection': 'close'}

Error is gone by adjusting max_workers to 1 in https://github.com/cve-search/cve-search/blob/1f0b50aa46814e2a683a9b7b01da8bcc0403154e/lib/DownloadHandler.py#L121 like this

thread_map(self.download_site, sites, desc="Downloading files", max_workers=1)

Afterwards ./external/cve-search/sbin/db_updater.py -f should work fine (takes longer though).

Seems like there is a rate limit with https://nvd.nist.gov/
Source: cve-search/cve-search#890 (comment)

thx for documenting this issue to @brainsht

Python dependency warnings

If you get the following warnings you need to update the requests package.

/usr/local/lib/python3.10/dist-packages/requests/__init__.py:102: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (5.0.0)/charset_normalizer (2.0.12) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({})/charset_normalizer ({}) doesn't match a supported "

Update the requests package the following way should solve this issue:

└─$ source ./external/emba_venv/bin/activate
└─$ sudo pip install -U requests                               

Testing CVE-search manually

• Finally you can test it manually via starting the EMBA container and test the cve-search connection the following way:

└─$ sudo docker-compose run emba     
WARNING: The FIRMWARE variable is not set. Defaulting to a blank string.
WARNING: The LOG variable is not set. Defaulting to a blank string.
WARNING: The EMBA variable is not set. Defaulting to a blank string.
Creating emba_forked_emba_run ... done

┌──(root@d2a5960b73b2)-[/emba]
└─# source /external/emba_venv/bin/activate
┌──(root@d2a5960b73b2)-[/emba]
└─# /external/cve-search/bin/search.py -p busybox

This should show some CVE details regarding busybox.

Describe the bug
Installation failed

To Reproduce
Steps to reproduce the behavior:
clone repo
run: sudo ./installer.sh -d

Expected behavior
Expected no error

Screenshots
ERROR: The Compose file './docker-compose.yml' is invalid because:
networks.emba_runs value Additional properties are not allowed ('name' was unexpected)

Error detected - status code 1
Command: docker-compose up --no-start
Location: ./installer/I05_emba_docker_image_dl.sh, line 49
Stack Trace:
[1] I05_emba_docker_image_dl(): ./installer/I05_emba_docker_image_dl.sh, line 49 -> I05_emba_docker_image_dl
[2] main(): ./installer.sh, line 152 -> main -d

Desktop (please complete the following information):

• OS: Lubuntu 20.04

Additional context
Add any other context about the problem here.

I'm running in a fresh Ubuntu VM and cloned EMBA from git then ran the installer and yes, it seems to have failed to install the net-tools.

I could (untested!) imagine that the way you handle parameters with a dash (-) in it, that it somehow breaks because the dash is interpreted as a command line parameter switch. But that's just wild speculation.

Originally posted by @floyd-fuh in #118 (comment)

@elig0n found a problem in multiple areas: #29

A quick check shows 56 areas we should take a look:

└─$ grep for.*\;\ do modules/* -R | grep -v "{" | wc -l
56

I will close the original pull request and we document it via this issue.

Looks as the output of the cwe-checker has changed a bit.

Currently the installer only shows the dpkg deps and some others. It is not consistent and does not show our pip deps, our deps from cve-search and binwalk. We should show them all in the future

the error is these

i install the emba successfully before, but this i can't install successful

my enviroment is ubuntu18.04

After running docker-compose I got the following error:

---> 831e1fbfdc4e
Step 4/6 : ADD . /app
---> a575a222b90e
Step 5/6 : RUN yes | ./installer.sh
---> Running in e2ec779affe5
/bin/sh: 1: ./installer.sh: Permission denied
ERROR: Service 'emba' failed to build: The command '/bin/sh -c yes | ./installer.sh' returned a non-zero code: 126

The solution is to chmod +x installer.sh and docker-compose will succeed.

Thank you.

Describe the bug

Hello, while running the install (in embabox) a fresh install currently fails.

    default: [+] IP60_fact_extractor
    default: =================================================================
    default: fact-extractor
    default: Description: Wraps FACT unpack plugins into standalone utility. Should be able to extract most of the common container formats. (EMBA fork)
    default: Download-Size: 34 MB
    default: fact_extractor will be downloaded.
    default: 
    default: FACT-extractor will be downloaded and installed!
    default: Reading package lists...
    default: Building dependency tree...
    default: 
    default: Reading state information...
    default: The following additional packages will be installed:
    default:   libcurl4 libldap-2.5-0
    default: The following NEW packages will be installed:
    default:   libldap-2.5-0
    default: The following packages will be upgraded:
    default:   curl libcurl4
    default: 2 upgraded, 1 newly installed, 0 to remove and 948 not upgraded.
    default: Need to get 869 kB of archives.
    default: After this operation, 581 kB of additional disk space will be used.
    default: Do you want to continue? [Y/n]
    default: Abort.
    default: ------------------------------------------------------------------------------------------------
    default: Error detected - status code 1
    default: Command:  echo -e "Command:  $ORANGE$BASH_COMMAND$NC"
    default: Location:  ./installer/IP60_fact_extractor.sh, line 40
    default: Stack Trace:
    default:     [1] IP60_fact_extractor(): ./installer/IP60_fact_extractor.sh, line 40 -> IP60_fact_extractor
    default:     [2] main(): ./installer.sh, line 168 -> main -F
    default: 
    default: Important: Consider filling out a bug report at https://github.com/e-m-b-a/emba/issues
    default: ------------------------------------------------------------------------------------------------

It looks like the curl install needs at least a -y flag to install without interaction?
https://github.com/e-m-b-a/emba/blob/master/installer/IP60_fact_extractor.sh#L40

With the pull request from @Anemosx we now have full Docker support in emba.

As discussed here #66 (comment) this also includes new security issues. Nevertheless we see a massive improvement compared to running emba on your host.

Let's use this Issue to discuss further steps.

See here https://www.kali.org/blog/kali-linux-2021-4-release/

Testcases:

• Full installation working
  • emba run with profile default-scan
  • emba run with profile full-scan
  • emba run in dev mode with options -D -E -t -W
  • emba run in dev mode with options -D, -c, -E, -t, -W
• Docker build
• Default/docker installation working
  • emba run with profile default-scan

See #70

Let's install fact-extractor locally and in our emba docker image. With this we do not need the docker image of fact-extractor anymore and fix our security issues.

The output of objdump is misaligned:

I think we talked about it before but because I ran EMBA again and used these tools afterwards so I thought I let you know.

Is your feature request related to a problem? Please describe.
I think the password hash search and other analysis steps after unpacking in EMBA could be improved.

Describe the solution you'd like
I guess using https://github.com/returntocorp/semgrep is the best choice at the moment because of the huge amount of rules in https://github.com/returntocorp/semgrep-rules

Describe alternatives you've considered
If semgrep doesn't work for some reason there is also my standalone https://github.com/floyd-fuh/crass/blob/master/grep-it.sh script that might be easier to implement. As an fyi, I've been maintaining it for 7 years now and even though there is semgrep I'm not planning to stop. Btw. if you would like to keep your current greps we could integrate them as rules in grep-it.sh if you think something is missing.

Additional context
I know searching for CVEs is probably a quick win in many cases, but when you really want to do proper security research to pwn a device with custom/new approaches/techniques/ideas it is good to have a tool that only points to "fishy" areas (e.g. anything related to crypto as in https://github.com/floyd-fuh/crass/blob/f2da104b073f530fbadeda7578c39a377ebd296b/grep-it.sh#L318 ). In that regards I'm not sure (I really don't know as I've just started to use it) if semgrep allows an approach that is broad enough, whereas in CRASS I'm trying. I would still rather go with semgrep. Or both. If you use semgrep with --config=auto just make sure to tell your users that you are using a third-party (metrics are sent to their servers).

Currently it is not working without a firmware file:

Dep check should also work without any other parameters.

Is your feature request related to a problem? Please describe.

"Modern" android OTA updates contain a file called payload.bin that start with a magic "CrAU"
While emba does find some part of the file it would be nice to have full support for this (common) file format

Describe the solution you'd like
Starting from a zip obtained from

https://developers.google.com/android/ota
oriole-ota-sd1a.210817.015.a4-19a77b62.zip
https://dl.google.com/dl/android/aosp/oriole-ota-sd1a.210817.015.a4-19a77b62.zip

git clone https://github.com/vm03/payload_dumper.git
cd payload_dumper
pip install -r requirements.txt

unzip ../oriole-ota-sd1a.210817.015.a4-19a77b62.zip
python payload_dumper.py payload.bin

Processing system partition..........................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing system_ext partition..........................................................................................................................Done
Processing product partition.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Done
Processing vbmeta_system partition.Done
Processing boot partition................................Done
Processing vendor_boot partition................................Done
Processing dtbo partition........Done
Processing vbmeta partition.Done
Processing vbmeta_vendor partition.Done
Processing vendor partition......................................................................................................................................................................................................................................Done
Processing vendor_dlkm partition...........................Done
Processing bl1 partition.Done
Processing pbl partition.Done
Processing bl2 partition.Done
Processing abl partition.Done
Processing bl31 partition.Done
Processing tzsw partition...Done
Processing gsa partition.Done
Processing ldfw partition..Done
Processing modem partition...................................Done

The result is found in output
file output/*.img

output/bl1.img:           data
output/bl2.img:           data
output/bl31.img:          data
output/boot.img:          Android bootimg, kernel (0x150d94), ramdisk (0x630)
output/dtbo.img:          data
output/gsa.img:           data
output/ldfw.img:          data
output/modem.img:         POSIX tar archive (GNU)
output/pbl.img:           Dyalog APL version 165.7
output/product.img:       Linux rev 1.0 ext2 filesystem data, UUID=8e66e769-b7a9-574e-b7d2-513c40dbb996, volume name "product" (extents) (large files) (huge files)
output/system_ext.img:    Linux rev 1.0 ext2 filesystem data, UUID=fc31cba1-4585-5da0-9700-ecedd28b80ec, volume name "system_ext" (extents) (large files) (huge files)
output/system.img:        Linux rev 1.0 ext2 filesystem data, UUID=02e80408-f118-591d-90f7-5b2411e5859b (extents) (large files) (huge files)
output/tzsw.img:          data
output/vbmeta.img:        data
output/vbmeta_system.img: data
output/vbmeta_vendor.img: data
output/vendor_boot.img:   data
output/vendor_dlkm.img:   Linux rev 1.0 ext2 filesystem data, UUID=d550f889-ddd7-5920-bf31-ffd8c5dee97e, volume name "vendor_dlkm" (extents) (large files) (huge files)
output/vendor.img:        Linux rev 1.0 ext2 filesystem data, UUID=d8891240-d867-5fac-80a5-6e9859e0263d, volume name "vendor" (extents) (large files) (huge files)

It would be nice to integrate this .

Describe alternatives you've considered

Manual work

Additional context
https://www.thecustomdroid.com/how-to-extract-android-payload-bin-file/

New Kali Linux is available. We need to test everything on it

https://www.kali.org/get-kali/#kali-platforms

Based on #32 and #28 we have docker support.

Currently our setup does not support cve-search - https://cve-search.github.io/cve-search/

Describe the bug
I am using kali 2021.4 and I have to set up proxy to get internet access. I set the following parameters on my linux:
HTTP_PROXY=A.B.C.D:1234
HTTPS_PROXY=A.B.C.D:1234
NO_PROXY=localhost,127.0.,127.36.

echo "http_proxy=http://${HTTP_PROXY}/" >>/etc/environment &&
echo "https_proxy=http://${HTTPS_PROXY}/" >>/etc/environment &&
echo "no_proxy=${NO_PROXY}" >>/etc/environment

I set the proxy for APT:
touch /etc/apt/apt.conf.d/proxy.conf &&
echo "Acquire::http::Proxy "http://${HTTP_PROXY}";" >>/etc/apt/apt.conf.d/proxy.conf &&
echo "Acquire::https::Proxy "http://${HTTPS_PROXY}";" >>/etc/apt/apt.conf.d/proxy.conf

export:
export http_proxy=${HTTP_PROXY}
export https_proxy=${HTTPS_PROXY}
export no_proxy=${NO_PROXY}

proxy for wget as root:
cp /etc/wgetrc /root/.wgetrc
echo -e "use_proxy = on\nhttp_proxy = http://${HTTP_PROXY}\nhttps_proxy = http://${HTTPS_PROXY}\nftp_proxy = http://${HTTP_PROXY}" >>/root/.wgetrc

proxy for wget as kali:
cp /etc/wgetrc /home/kali/.wgetrc
echo -e "use_proxy = on\nhttp_proxy = http://${HTTP_PROXY}\nhttps_proxy = http://${HTTPS_PROXY}\nftp_proxy = http://${HTTP_PROXY}" >>/home/kali/.wgetrc

and I set proxy for Git:
git config --global http.proxy http://${HTTP_PROXY}
git config --global https.proxy http://${HTTPS_PROXY}

apt upgrade and reboot, and I git lone the emba, then cd ~emba, then:
sudo ./isntaller.sh -d

All the things are fine but after mongoDB installation the setup failes. This is on the screen:

Adding user mongodb' to group mongodb' ...
Adding user mongodb to group mongodb
Done.
Setting up mongodb-org-shell (4.4.12) ...
Setting up mongodb-database-tools (100.5.2) ...
Setting up mongodb-org-mongos (4.4.12) ...
Setting up mongodb-org-database-tools-extra (4.4.12) ...
Setting up mongodb-org-tools (4.4.12) ...
Setting up mongodb-org (4.4.12) ...
Processing triggers for man-db (2.9.4-2) ...
Processing triggers for kali-menu (2021.4.2) ...
Scanning processes...
Scanning linux images...

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.
Created symlink /etc/systemd/system/multi-user.target.wants/mongod.service → /lib/systemd/system/mongod.service.

The cve-search database will be downloaded and updated!

Check if the cve-search database is already installed.

cve-search database not ready.

The installer is going to populate the database.
Starting redis-server (via systemctl): redis-server.service.
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_mgmt_cpe_dictionary.py", line 27, in
from lib.Sources_process import CPEDownloads
File "/home/kali/emba/external/cve-search/sbin/../lib/Sources_process.py", line 13, in
from pymongo import TEXT, ASCENDING
ModuleNotFoundError: No module named 'pymongo'
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_mgmt_json.py", line 22, in
from lib.Sources_process import CVEDownloads
File "/home/kali/emba/external/cve-search/sbin/../lib/Sources_process.py", line 13, in
from pymongo import TEXT, ASCENDING
ModuleNotFoundError: No module named 'pymongo'
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_updater.py", line 21, in
from lib.DatabaseSchemaChecker import SchemaChecker
File "/home/kali/emba/external/cve-search/sbin/../lib/DatabaseSchemaChecker.py", line 7, in
from lib.DatabaseHandler import DatabaseHandler
File "/home/kali/emba/external/cve-search/sbin/../lib/DatabaseHandler.py", line 1, in
from lib.ApiRequests import JSONApiRequest
File "/home/kali/emba/external/cve-search/sbin/../lib/ApiRequests.py", line 5, in
from nested_lookup import nested_lookup, nested_update
ModuleNotFoundError: No module named 'nested_lookup'

The cron.daily update script for EMBA is located in config/emba_updater
For automatic updates it should be copied to /etc/cron.daily/

Installation notes:

INFO: The cron.daily update script for EMBA is located in config/emba_updater
INFO: For automatic updates it should be copied to /etc/cron.daily/
INFO: For manual updates just start it via sudo ./config/emba_updater

WARNING: If you plan using the emulator (-E switch) your host and your internal network needs to be protected.

INFO: Do not forget to checkout current development of EMBA at https://github.com/e-m-b-a.
EMBA installation finished

and I get back the prompt.
Could you help what other proxy settings required to get the installation go through the entire process?

To Reproduce
Steps to reproduce the behavior:

1. Proxy setup
2. EMBA installation (default mode)
3. See error

Expected behavior
Please verify my proxy settings and if possible please give any advice what to do for population the CVE database by the instalation script as kali user:
$ sudo ./installer.sh -d

Screenshots
I copied the error message above.

Desktop (please complete the following information):

• OS: [Kali Linux 2021.04]

Additional context
Proxy must be set but I am not sure I defined all the needed parameters for the emba installer.

Not totally sure if it's an issue but didn't look right?

└──╼ $sudo ./config/emba_updater
[sudo] password for ovarroadmin:
[*] EMBA update - cve_searchsploit update
./config/emba_updater: 9: cve_searchsploit: not found
[*] EMBA update - cve-search update

This is a tall order but would be nice for the roadmap

In most cases. the discoveries for the CVEs don't actually affect the product. For example, if I'm running a kernel version that has 200 CVE's and 7 exploits. When I look at those findings I notice the CVE's are just a raw version analysis but if you dig down into the CVE it can say stuff like "If IPV6 is enabled" "IF the following flag is enabled in x config". IT would be nice to have the ability to go into the HTML report and maybe toggle stuff off that you know is a false positive.

Kina like this project lets you do https://github.com/Guezone/SECMON.

The toggling could let you generate an XML or something that logs the CVE's that you could apply to your next scan --fpxml

Interesting results appear with -t switch.

See here https://www.kali.org/blog/kali-linux-2022-1-release/

Testcases:

• Default/docker installation working - see #218
  • EMBA run with profile default-scan in strict mode (-S)
• Full installation working
  • EMBA run in dev mode with options -s -z -S -D -E -t -W
  • EMBA run in dev mode with options -s -z -S -D, -c, -E, -t, -W
• Docker build
  • EMBA run with profile default-scan in strict mode (-S)
  • EMBA run with profile full-scan in strict mode (-S)

Describe the bug
The emba installer when uses the
pip3 install -r requirements
command this error comes:

To Reproduce
Steps to reproduce the behavior:

1. EMBA installation (default mode)
2. See error

Expected behavior
pip3 install should be working fine.

Screenshots
Added above.

Desktop (please complete the following information):

• OS: [Kali Linux 2022.01, 2021.4 and 2021.3]
  I have tested 3 different Kali versions including the latest one and I got the same issue.

based on #32 and #28 we have docker support.

Currently our docker integration does not support the pre-checker modules and binwalk

Is your feature request related to a problem? Please describe.

Since PR #213 EMBA supports reporting templates. These templates can be easily created for the different modules.
The better the templates are, the more useful is this feature.

Describe the solution you'd like

Include templates for every module. These templates should be generic and should describe the module, the tasks and the results.
See an example here:

Some example templates can be found here: https://github.com/e-m-b-a/emba/tree/master/config/report_templates

Emba should be able to detect vmdk images e.g. with the file command and then extracts it correctly.
The following documentation works quite nice on our Kali environment: https://unix.stackexchange.com/questions/550569/how-can-i-access-the-files-in-a-vmdk-file
Another documentation: https://serverfault.com/questions/244642/extract-files-from-vmdk

We could mount it and then copy everything to the firmware folder in the log area.

Add a -F or -Y switch to installer.sh to force an install of everything, even if its optional.

Thanks a lot for this great project. I fast forwarded through the wiki but was not able to figure out which embedded Linux operating systems are supported. Probably some distributions are not supported at all. Probably some checks are meant for specific distributions only. There are so many Linux-based embedded OSes like e.g. Android Things, Android Cars, OpenWRT, Yocto Project embedded Linux, Ubuntu Core, uClinux, ... Could you provide more info about distro compatibility?

Multiple issues found by reddit user geirha https://www.reddit.com/r/bash/comments/o2ib89/emba_an_open_source_firmware_analyzer_has/

From the logfile, I conclude that certain files in /etc where not found by the tool. Am I correct?
These files are present in the tested firmware image!

[+] Check users, groups and authentication
=================================================================

==> Users with UID zero (0)
-----------------------------------------------------------------
[*] Searching accounts with UID 0
[-] /etc/passwd not available

==> Non-unique accounts
-----------------------------------------------------------------
[*] Searching non-unique accounts
[-] /etc/passwd not available

==> Unique group IDs
-----------------------------------------------------------------
[*] Searching non-unique group ID's
[-] /etc/group not available

==> Unique group name
-----------------------------------------------------------------
[*] Searching non-unique group names
[-] /etc/group not available

==> Query user accounts
-----------------------------------------------------------------
[*] Reading system users
[-] /etc/passwd not available

==> Query NIS and NIS+ authentication support
-----------------------------------------------------------------
[*] Check nsswitch.conf
[-] /etc/nsswitch.conf not available

Is your feature request related to a problem? Please describe.

I am trying to use emba to generate a report . I am combining the results found here with other tools. Initially, the idea was to go through the CSV files and generate rst or latex report but I am running into small problems. For example

f50_base_aggregator.txt (the text version) is pretty nice in terms of overview but to me it appears the CSV does not contain the same information also while trying to read the CSV it appears that not all the lines in the file have the same amount of fields

architecture_verified;"ARM"
strcpy_bin;"libicui18n.so";"11"
version_details;kernel;4.9.160;CVEs;788;Exploits;197

The main thing I would like in terms of functionality is to be able to query the data but at this point, I don't know if reading the CSV files is the way to go.

Describe the solution you'd like

GIven there is also already a MongoDB database I was thinking perhaps the csv needs to become json and this can be put in the database. Perhaps just converting to json would be enough . I think it would be great to have a shell/ web front-end where you can query the data (on file base perhaps?) and get the combined results of different tests. Does Metasploit not do something similar?

Describe alternatives you've considered

I am currently just including the .txt files in my report (including colors) and emba is really useful to help determine what to perform for a pentest but it would be nice to be able to also use the findings(filter them) in a report.
I Have previously written a plugin to include CVE type data into https://github.com/blacklanternsecurity/writehat but I would currently like to use python to read the data and generate nice graphs/ links etc.

Additional context

I hope we can have a nice discussion on the topic. It might also be possible to rewrite some of the shell scripts to store a bit more context( or store the data into a more computer-friendly format) or perhaps I am missing something

Describe the bug
In case of PROXY the default installation of emba stops at I05_emba_docker_image download:

To Reproduce
Steps to reproduce the behavior:

1. Set PROXY settings
2. EMBA installation (/default mode)
3. See error

Expected behavior
The image download should be working fine.

Screenshots
Added above.

Desktop (please complete the following information):

• OS: [Kali Linux 2022.01]

our cwe-checker integration has some troubles running from dockerized emba

I got docker-compose name error on updated Ubuntu 20.04 while installing.

According to the solution I found on stackoverflow, adding this to the install script will fix the problem:

sudo curl -L https://github.com/docker/compose/releases/download/1.28.5/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

Referance:
https://stackoverflow.com/questions/58155523/unable-to-give-network-name-in-docker-compose

@firmianay found some issues in the installation script that need to be checked.

• Some tools are installed twice
• Dependencies should all be declared (if manual or optional)
• Maybe force as default option?

Previous discussion: #77

Based on #32 and #28 we have docker support.

Currently our setup does not support the cwe module.

�[0;35m�[1mThe cve-search database will be downloaded and updated!�[0m

�[0;35mCheck if the cve-search database is already installed.�[0m

�[0;35mcve-search database not ready.�[0m

�[0;35mThe installer is going to populate the database.�[0m
Starting redis-server (via systemctl): redis-server.service.
Traceback (most recent call last):
File "/opt/emba/external/cve-search/./sbin/db_mgmt_cpe_dictionary.py", line 27, in
from lib.Sources_process import CPEDownloads
File "/opt/emba/external/cve-search/sbin/../lib/Sources_process.py", line 17, in
from lib.DatabaseLayer import (
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseLayer.py", line 35, in
mongo_version = db.command("buildinfo")["versionArray"]
File "/usr/local/lib/python3.9/dist-packages/pymongo/database.py", line 757, in command
with self.__client._socket_for_reads(
File "/usr/lib/python3.9/contextlib.py", line 117, in enter
return next(self.gen)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1387, in _socket_for_reads
server = self._select_server(read_preference, session)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1346, in _select_server
server = topology.select_server(server_selector)
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 244, in select_server
return random.choice(self.select_servers(selector,
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 202, in select_servers
server_descriptions = self._select_servers_loop(
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 218, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 172.36.0.1:27017: timed out, Timeout: 30s, Topology Description: <TopologyDescription id: 61af3eb03c94e0047bb56ec7, topology_type: Single, servers: [<ServerDescription ('172.36.0.1', 27017) server_type: Unknown, rtt: None, error=NetworkTimeout('172.36.0.1:27017: timed out')>]>
Traceback (most recent call last):
File "/opt/emba/external/cve-search/./sbin/db_mgmt_json.py", line 22, in
from lib.Sources_process import CVEDownloads
File "/opt/emba/external/cve-search/sbin/../lib/Sources_process.py", line 17, in
from lib.DatabaseLayer import (
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseLayer.py", line 35, in
mongo_version = db.command("buildinfo")["versionArray"]
File "/usr/local/lib/python3.9/dist-packages/pymongo/database.py", line 757, in command
with self.__client._socket_for_reads(
File "/usr/lib/python3.9/contextlib.py", line 117, in enter
return next(self.gen)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1387, in _socket_for_reads
server = self._select_server(read_preference, session)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1346, in _select_server
server = topology.select_server(server_selector)
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 244, in select_server
return random.choice(self.select_servers(selector,
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 202, in select_servers
server_descriptions = self._select_servers_loop(
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 218, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 172.36.0.1:27017: timed out, Timeout: 30s, Topology Description: <TopologyDescription id: 61af3ed01f494612bdad95e0, topology_type: Single, servers: [<ServerDescription ('172.36.0.1', 27017) server_type: Unknown, rtt: None, error=NetworkTimeout('172.36.0.1:27017: timed out')>]>
Traceback (most recent call last):
File "/opt/emba/external/cve-search/./sbin/db_updater.py", line 21, in
from lib.DatabaseSchemaChecker import SchemaChecker
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseSchemaChecker.py", line 7, in
from lib.DatabaseHandler import DatabaseHandler
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseHandler.py", line 4, in
from lib.DatabasePlugins.config import DatabasePluginLoader
File "/opt/emba/external/cve-search/sbin/../lib/DatabasePlugins/config.py", line 1, in
from lib.DatabasePlugins import *
File "/opt/emba/external/cve-search/sbin/../lib/DatabasePlugins/mongodb.py", line 12, in
from lib.DatabaseLayer import sanitize
File "/opt/emba/external/cve-search/sbin/../lib/DatabaseLayer.py", line 35, in
mongo_version = db.command("buildinfo")["versionArray"]
File "/usr/local/lib/python3.9/dist-packages/pymongo/database.py", line 757, in command
with self.__client._socket_for_reads(
File "/usr/lib/python3.9/contextlib.py", line 117, in enter
return next(self.gen)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1387, in _socket_for_reads
server = self._select_server(read_preference, session)
File "/usr/local/lib/python3.9/dist-packages/pymongo/mongo_client.py", line 1346, in _select_server
server = topology.select_server(server_selector)
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 244, in select_server
return random.choice(self.select_servers(selector,
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 202, in select_servers
server_descriptions = self._select_servers_loop(
File "/usr/local/lib/python3.9/dist-packages/pymongo/topology.py", line 218, in _select_servers_loop
raise ServerSelectionTimeoutError(
pymongo.errors.ServerSelectionTimeoutError: 172.36.0.1:27017: timed out, Timeout: 30s, Topology Description: <TopologyDescription id: 61af3eef37ab0f6687597923, topology_type: Single, servers: [<ServerDescription ('172.36.0.1', 27017) server_type: Unknown, rtt: None, error=NetworkTimeout('172.36.0.1:27017: timed out')>]>

See here https://www.kali.org/blog/kali-linux-2021-3-release/

Testcases:

• Full installation working
  • emba run with profile default-scan
  • emba run with profile full-scan
  • emba run in dev mode with options -D -E -t -W (see #163)
  • emba run in dev mode with options -D, -c, -E, -t, -W (see #163)

I have the Error message which you can see in the screenshot. Im not sure if it came with my changes of my fork. If I remember right it is also appearing in the master branch. It is only in the console output, but not in the logfile.

Is your feature request related to a problem? Please describe.

Kali Linux 2022.2 released here: https://www.kali.org/blog/kali-linux-2022-2-release/
We need to test EMBA on it

Testcases:

• Default/docker installation working
  • EMBA run with profile default-scan in strict mode (-S)
• Full installation working
  • EMBA run in dev mode with options -s -z -S -D -E -t -W
  • EMBA run in dev mode with options -s -z -S -D, -c, -E, -t, -W
• Docker build
  • EMBA run with profile default-scan in strict mode (-S)
  • EMBA run with profile full-scan in strict mode (-S)

As mentioned here: https://github.com/e-m-b-a/emba/wiki/Installation#cve-search-installation the current installer is not able to install all required dependencies.

Describe the bug
cve-search not working with docker-compose. installed it using default switch with sudo

To Reproduce
Steps to reproduce the behavior:

1. EMBA installation using -d switch
2. all firmwares
3. Start EMBA with the default parameters from wiki
4. See error
   

Linux Debian up to date

@p4cx has completely rewritten the emba html reporter here: #101

With this base we now have great possibilities. This issue is a collection and discussion of some ideas:

• Linking to submodules from aggregator eg.
  • "[+] 1091 files and 91 directories detected." should link to "html-report/s05_firmware_details.html"
  • "[+] Found 182 issues in 78 shell scripts." should link to "html-report/s20_shell_check.html"
  • Kernel version can link to the kernel module
• Linking to vulnerability details from aggregator
  "[+] Found version details: dnsmasq : 2.45 : CVEs: 21 : Exploits: 8 " should link to the details of dnsmasq within "html-report/f19_cve_aggregator.html"
• the current visualized firmware file is not included
• Inter-module linking eg.:
  "[+] Found the following root directory: ../emba_logs/logs_dir300/firmware/DIR300B5_FW214WWB01.bin_binwalk_emba/_DIR300B5_FW214WWB01.bin.extracted/squashfs-root" could link to "html-report/s05_firmware_details.html"
• links to further details
  • shellchecker results
  • deep search results
  • php results
• include emba.log into web reporter
• Parallelize creation of info files (like shellchecker, deep search, php results)

Same issue like #38 - emba isn't able to extract firmware with Fact_extractor if started with -D.

What are you missing in emba? Let us know how you are using emba, what is working and what doesn't work

From our point of view we have the following areas to improve:

• improve threading (more modules should use it) (#107 / #106 / #102 / #97)
• improve the web reporter (see #101 / #121)
• emba needs to get faster again (we lost a lot of performance with this pull request #92)
• some modules need some cleanup (e.g. P07 - #110)
• Cleanup and speedup of F19 and S115 (#112 and #115)
• CSV export of aggregator data (#116)
• Speedup of S105 (#117)
• Include Metasploit exploit information to aggregator (#117)
• Include the ability to create configuration profiles (#120)
• Installation summary with hint to update cve-database and protecting your host
• Create a UI for emba (see https://github.com/e-m-b-a/embark)
• Test emba on RPi v4
  tested it, works, but is really slow (RPI v4 with 8 Gigs Ram) and therefore not usable as a standalone server.
• Include a check for the cve-database in the installer (to not touch an already installed and running database environment) (see #145)
• Provide docker images for emba (https://hub.docker.com/r/embeddedanalyzer/emba)
• TLI installation process (e.g. image of debian installation)
  I did some research, it would be possible, but not worth the developing time (especially because we are advising to use docker and then it has to be non-interactive)
• switch from docker to podman (should we?)

What else would be great to bring to emba?

Describe the bug

I am getting an error during the installation near metasploit-framework

To Reproduce

1. EMBA installation (default mode)
   set -e
   git clone https://github.com/e-m-b-a/emba.git
   cd emba
   sudo ./installer.sh -F

Log;

default: tcllib will be newly installed.
default: 
default: metasploit-framework
default: ------------------------------------------------------------------------------------------------
default: Error detected - status code 100
default: Command:  TOOL_INFO="$(apt show "${1:-}" 2> /dev/null)"
default: Location:  ./installer/helpers.sh, line 47
default: Stack Trace:
default:     [1] print_tool_info(): ./installer/helpers.sh, line 47 -> print_tool_info metasploit-framework 1
default:     [2] I01_default_apps(): ./installer/I01_default_apps.sh, line 33 -> I01_default_apps
default:     [3] main(): ./installer.sh, line 156 -> main -F
default: 
default: Important: Consider filling out a bug report at https://github.com/e-m-b-a/emba/issues
default: 
default: ------------------------------------------------------------------------------------------------
default: bash: line 5: xit: command not found

"xit" command not found -> exit perhaps?

Describe the bug

I found bug in https://github.com/e-m-b-a/emba
=> emba/config/bin_version_strings.cfg/openssl

Results when using an existing regular expression

openssl;;bsd;"OpenSSL\ [0-9](\.[0-9]+)+?\ ";"sed -r 's/OpenSSL\
([0-9](\.[0-9]+)+?).*$/openssl:\1/'";
=> OpenSSL 1.1.1  OpenSSL 1.1.1

Result when using the modified regular expression

openssl;;bsd;"OpenSSL\ [0-9](\.[0-9]+)+?\ ";"sed -r 's/OpenSSL\
([0-9](\.[0-9]+)+?)\ /openssl:\1/'";
 => openssl        1.1.1

It looks like the above regex needs to be modified.

To Reproduce
Information not provided

Expected behavior
Information not provided

Screenshots
Information not provided

Desktop (please complete the following information):
Information not provided

Additional context
I got this bug report via mail. Didn't try to reproduce it.