Description

When running docker pull exodusprivacy/exodus-standalone command I'm getting following error

Using default tag: latest
latest: Pulling from exodusprivacy/exodus-standalone
no matching manifest for linux/arm64/v8 in the manifest list entries

Platform Info

MacOS Apple M2 Pro

Hi,

I tried following install instructions on my Mac and got

$ pip install -r requirements.txt
Collecting https://github.com/Exodus-Privacy/exodus-core/releases/download/v1.0.4/exodus_core-1.0.4.tar.gz (from -r requirements.txt (line 1))
  Downloading https://github.com/Exodus-Privacy/exodus-core/releases/download/v1.0.4/exodus_core-1.0.4.tar.gz (771kB)
    100% |████████████████████████████████| 778kB 717kB/s
    Complete output from command python setup.py egg_info:
    Unfortunately, we do not support your platform darwin

    ----------------------------------------

I guess exodus-standalone only works on Linux? Not sure how difficult it would be to support mac as well? If that's a big deal, what about offering a Docker image or something like this for easy local APK analysis on any system?

Thanks for the work you put in this software!

Feels like the current rule of "everything in the root directory" is not ideal

We could reorganize the repository to follow python best practices (at least separate tests from the rest)

We should show in the output the category and the id of each tracker (and maybe its link to https://reports.exodus-privacy.eu.org/ ?)

The first one would make it easier to know what each tracker is doing.
The second one would make it easier to ignore tracker (as it requires its id).

Here is the log:

python3.8 exodus_analyze.py ~/StudioProjects/Pilldroid/app/product/release/app-product-release.aab 
Missing AndroidManifest.xml. Is this an APK file?
=== Information
- APK path: /home/jacques/StudioProjects/Pilldroid/app/product/release/app-product-release.aab
- APK sum: f978a7ce751609a750341d099fd498f0953865df949d2263548e5848b152e744
Traceback (most recent call last):
  File "exodus_analyze.py", line 78, in <module>
    main()
  File "exodus_analyze.py", line 71, in main
    analysis.print_apk_infos()
  File "/home/jacques/exodus-standalone/test_app_venv/lib/python3.8/site-packages/exodus_core/analysis/static_analysis.py", line 471, in print_apk_infos
    print('- App version: {}'.format(self.get_version()))
  File "/home/jacques/exodus-standalone/test_app_venv/lib/python3.8/site-packages/exodus_core/analysis/static_analysis.py", line 239, in get_version
    return self.apk.get_androidversion_name()
  File "/home/jacques/exodus-standalone/test_app_venv/lib/python3.8/site-packages/androguard/core/bytecodes/apk.py", line 620, in get_androidversion_name
    return self.androidversion["Name"]
KeyError: 'Name'

With unzip -l ~/StudioProjects/Pilldroid/app/product/release/app-product-release.aab I actually can't find the AndroidManifest.xml file.

We should add a CONTRIBUTING.md file to make contributing easier.

Things we should mention in it:

• the flake 8 linter
• the tests (how to add new ones, how to run them)

Currently we only can ignore trackers with the --ignore option which is fairly basic and uses tracker id.

Using a separate file could make this more clear, and especially allow to:

• add comments
• ignore trackers by name (although that's a bit risky as names can change)

Hello,

I am getting the reverse error of #5:

Traceback (most recent call last):
  File "exodus_analyze.py", line 62, in <module>
    report = json.dumps(analysis.create_json_report(), indent = 2)
  File "exodus_analyze.py", line 18, in create_json_report
    'libraries': [l.decode('utf-8') for l in self.get_libraries()],
  File "exodus_analyze.py", line 18, in <listcomp>
    'libraries': [l.decode('utf-8') for l in self.get_libraries()],
AttributeError: 'str' object has no attribute 'decode'

get_libraries() already returns strings so there is no need to call decode().

I just spent quite a while trying to get a privacy report that couldn't be generated by the exodus privacy website (the site couldn't download the APK for some reason).

I ended up manually running some commands on a docker container running in Azure, and got the report. Cool!

However, now I'm the only person that knows the trackers/permissions requested by this app. It seems like there should be an optional mechanism for me to upload the report to the master database, so other users can capitalize on my findings. Maybe a -u flag as part of the analyzer python script?

Currently, exodus-standalone downloads εxodus trackers database for each run.

The database being fairly stable (changing only from time to time), the script should store it locally so that it is not downloaded every time (less load on εxodus servers, faster execution, etc.)

What the script could do :

• if no data locally, download it and store it
• if some data is available, check its timestamp and only refresh it if older than X
• if an option such as --refresh-cache is provided, refresh the case anyway

I installed exodus-standalone within venv and get

`ile "exodus_analyze.py", line 59, in
analysis.print_apk_infos()
File "exodus-standalone/venv/lib/python3.7/site-packages/exodus_core/analysis/static_analysis.py", line 483, in print_apk_infos
print('- App libraries: %s' % len(libraries))
TypeError: object of type 'generator' has no len()``

Commenting the three lines makes it run again, but then of course no libraries are shown ;-)

We were using the Docker Hub autobuild feature but it is not available for free anymore.

We could use GitHub Actions instead (after #25 is merged) to do the following:

• for each commit on master, push to latest
• for each new tag (starting with v), push to <tag_name> (without the v if we keep the current convention)

Image repository: https://hub.docker.com/r/exodusprivacy/exodus-standalone

Hello,

It would be nice to have a GUI allowing to upload an APK in browser and get a graphical report.

Thanks

I do analyze an app, and although everything seems to work, it returns me an error like this:

Requested API level 29 is larger than maximum we have, returning API level 28 instead.

Unfortunately that "error" (is it even an error or only a warning/note etc.?) is not really descriptive and I don't understand what this means now.

I.e. what implications does this have?

The analysis looks fine…

Add an option exit-code to allow the script user to change the returned exit code in case of trackers found.

Could be used for instance to run with --exit-code 0 to avoid failing or --exit-code 1 to avoid the current exit codes (based on the number of found trackers).

Hello,

I was not able to run the pip install requirements at first run, I had to downgrade androguard from 3.1.0 to 3.0.1.
It looks like it might be a typo since https://pypi.python.org/pypi/androguard/3.0? shows that the latest version is actually 3.0.1?

edit: actually androguard v3.1.0 is necessary to run with Python3. The solution is to use:
pip install https://github.com/androguard/androguard/archive/v3.1.0-pre.2.zip
otherwise pip only finds v3.0.1

I install the standalone for testing some apks but trackers list is always empty.
For example

=== Information
- APK path: fr.playsoft.lefigarov3.apk
- APK sum: 51532dd165e38e2d0ec11dc119b152bcd7808d9feb09f15c7ed4542987776e67
- App version: 5.1.3
- App version code: 94
- App UID: FABCC978B5CEFF042A9A462A3922ECA956B2420B
- App name: Le Figaro
- App package: fr.playsoft.lefigarov3
- App permissions: 10
    - android.permission.INTERNET
    - android.permission.ACCESS_NETWORK_STATE
    - android.permission.WRITE_EXTERNAL_STORAGE
    - android.permission.RECEIVE_BOOT_COMPLETED
    - android.permission.VIBRATE
    - com.google.android.c2dm.permission.RECEIVE
    - com.android.vending.BILLING
    - android.permission.WAKE_LOCK
    - fr.playsoft.lefigarov3.permission.C2D_MESSAGE
    - android.permission.ACCESS_WIFI_STATE
- App libraries: 0
- Certificates: 1
    - Issuer: countryName=FR, stateOrProvinceName=Ile de France, localityName=Paris, organizationName=Le Figaro, organizationalUnitName=LeFigaro Nouveaux médias, commonName=Francois Rannou 
Subject: countryName=FR, stateOrProvinceName=Ile de France, localityName=Paris, organizationName=Le Figaro, organizationalUnitName=LeFigaro Nouveaux médias, commonName=Francois Rannou 
Fingerprint: e9cdbbbd1d6a307563368c572e2efe2e106dbf4f 
Serial: 1317228289
=== Found trackers: 0

Is not exactly the same version on https://reports.exodus-privacy.eu.org/reports/2296/
But all 17 trackers seems disappeared ?
I have tested this behavior with 3 different apk from playstore and all of them have zero trackers.

I have install the exodus-core with pip in a venv version 1.0.8.
The md5sum of fr.playsoft.lefigarov3.apk is 7bc323c6ac8a146cff8753175b3ec60d , I can supply it if needed for some test.

My install have certainly something wrong ?

Hello, I am trying to install / run exodus standalone, but with no luck on centos7/local vm.

i have done steps ( per readme)
x) git clone https://github.com/Exodus-Privacy/exodus-standalone.git
x) cd exodus-standalone
x) virtualenv venv -p python3
x) source venv/bin/activate
x) pip install -r requirements.txt

some of it failed so had to do (from some basic troubleshooting/searchengine)
x) pip install ez_setup
x) pip install --upgrade setuptools
x) pip install --upgrade setuptools
and also
x) pip install --upgrade -r requirements.txt

And then it said install successful but when trying to run it it cannot find exodus-core.

(venv) [exodus@localhost exodus-standalone]# ./venv/bin/python exodus_analyze.py test.apk
Traceback (most recent call last):
File "exodus_analyze.py", line 4, in
from exodus_core.analysis.static_analysis import StaticAnalysis
ImportError: No module named 'exodus_core'
(venv) [exodus@localhost exodus-standalone]#

Any help / idea ?
Should i go for another Linux flavor ?
thanks.

There should be a way, for instance with a file called .exodusignore to ignore some trackers so that exodus-standalone can be used in cases where some trackers are expected.

We need to think how to identify trackers (name is not necessarily stable, id is probably a bit more)

Add JUnit XML output format so that exodus-standalone can be integrated more easily in some CI systems

1.4.0 version generates errors in terminal (detected in exodus android app Github actions) :

dexdump W 12-26 08:30:24     7     7 dex_file_verifier.cc:3172] This dex file is invalid and will be rejected in the future. Error is: Interface field is not public final static, Lcom/android/tools/r8/androidapi/h;.$desugar$clinit: 1008(static )
dexdump W 12-26 08:30:26    13    13 dex_file_verifier.cc:3172] This dex file is invalid and will be rejected in the future. Error is: Interface field is not public final static, Lcom/android/tools/r8/utils/structural/k;.$desugar$clinit: 1008(static )
dexdump W 12-26 08:30:26    13    13 dex_file_verifier.cc:3172] This dex file is invalid and will be rejected in the future. Error is: Interface field is not public final static, Lcom/android/tools/r8/utils/structural/s;.$desugar$clinit: 1008(static )
dexdump W 12-26 08:30:26    13    13 dex_file_verifier.cc:3172] This dex file is invalid and will be rejected in the future. Error is: Interface field is not public final static, Lcom/android/tools/r8/internal/oo;.$desugar$clinit: 1008(static )

How can we analyse them?

It does not seem to work when I just analyse one or all APKs I can opull from Android.

Edit: Ok, usually analysing the base.apk may be enough.

We need to display clearly which tracker is ignored after detection (with the --ignore option).

Hello,
Are stable release tags planned for this project? It would make it easier to know when to update.

It's unclear from the installation instructions which commands need to run as root and which run as a user. There also seems to be an underlying assumption that users are familiar with python usage. My guesswork made for a rough experience but it worked out in the end. Here's a walk-through:

Looking ahead, a package is needed (dexdump). So that should be installed first because if that fails there's no point in doing the manual steps.

sudo aptitude install dexdump

It's clear that I must choose a location and it's likely that root should run this since I'm doing a systemwide install. So root does cd /usr/local/src/, followed by:

$ torsocks git clone https://github.com/Exodus-Privacy/exodus-standalone.git
$ cd exodus-standalone

As a user:

nano ~/.config/gplaycli/gplaycli.conf

Then as root:

$ virtualenv venv -p python3
bash: virtualenv: command not found

Oops, missed a package. That should be added to the first step.

$ sudo aptitude install virtualenv
$ virtualenv venv -p python3
$ source venv/bin/activate

This next step is labeled "Install dependencies":

$ pip install -r requirements.txt

I don't think I've used pip before, but I wish I had realized that it would download stuff from the WAN, so that I would have known to prefix torsocks. The instructions should really say "download and install dependencies", to prompt Tor users to make arrangements. I was expecting the git clone to have done the downloading.. I wasn't careful enough to notice how little came from the clone and to then realize that pip would download stuff. Others will likely get stung by that too.

So now that installation is complete, as a user I run:

$ python /usr/local/src/exodus-standalone/exodus_analyze.py -h
Traceback (most recent call last):
  File "/usr/local/src/exodus-standalone/exodus_analyze.py", line 5, in <module>
    from exodus_core.analysis.static_analysis import StaticAnalysis
ImportError: No module named exodus_core.analysis.static_analysis

If root runs that command inside the virtualenv then it works, but root only happened to be in the virtualenv as part of the installation process, which is now over. When root does a control-d to exit that virtualenv, the whole shell is killed off including the parent. That's also astonishing. So something apparently did an exec to avoid forking. Whatever the proper way to exit that environment is, it should be documented.

So I first figured the virtualenv command needs to run every time. But that errors. After doing source venv/bin/activate as a user, it worked. So the activate script should be repeated in the "Analyze an APK file" steps. So this is how the instructions should say to run the tool:

$ source <root pkg dir>/exodus-standalone/venv/bin/activate
$ python <root pkg dir>/exodus-standalone/exodus_analyze.py "$apkfile"

Note that some APK files cause that to barf up this:

Traceback (most recent call last):
  File "/usr/local/src/exodus-standalone/exodus_analyze.py", line 56, in <module>
    analysis = AnalysisHelper(apk_file)
  File "/usr/local/src/exodus-standalone/venv/lib/python3.5/site-packages/exodus_core/analysis/static_analysis.py", line 96, in __init__
    self.load_apk()
  File "/usr/local/src/exodus-standalone/venv/lib/python3.5/site-packages/exodus_core/analysis/static_analysis.py", line 130, in load_apk
    self.apk = APK(self.apk_path)
  File "/usr/local/src/exodus-standalone/venv/lib/python3.5/site-packages/androguard/core/bytecodes/apk.py", line 117, in __init__
    self.zip = zipfile.ZipFile(io.BytesIO(self.__raw), mode="r")
  File "/usr/lib/python3.5/zipfile.py", line 1026, in __init__
    self._RealGetContents()
  File "/usr/lib/python3.5/zipfile.py", line 1094, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

But that's related to the APK.. some APKs work.

Anyway, I think I'm sorted now. But the instructions need to guide people away from the above pitfalls. It would also be good to state whether the activate or exodus_analyze.py scripts need the Internet, so users can firejail and or torsocks it. A firejail profile would also perhaps be useful.

We have planned to migrate exodus to debian 12 and python 3.11.
It's can be good also to update exodus-standalone.
Files need to be updated:

• Workflow Github
• DockerFile