Git Product home page Git Product logo

trueseeing's Introduction

README

Last release Last release date Main branch deploy status Main branch last commit

trueseeing is a fast, accurate and resillient vulnerability scanner for Android apps. We operate on the Dalvik VM level -- i.e. we don't care if the target app is obfuscated or not.

Capability

Currently we can:

  • Automatically scan app for vulnerabilities, reporting in HTML/JSON/text format (see below)
  • Manipulate app for easier analysis: e.g. enabling debug bit, enabling full backup, disabling TLS pinning, manipulating target API level, injecting frida-gadget, etc.
  • Examine app for general information
  • Copy in/out app data through debug interface
  • Search for certain calls/consts/sput/iput
  • Deduce constants/typesets for args of op
  • etc.

Installation

Containers

NOTE:

  • As of 2.1.9, we are on ghcr.io. (Docker Hub is somewhat deprecated)
  • Requires adbd in the host to control devices.

We provide containers so you can use right away as follows; now this is also the recommended way, and the only way if you are on Windows, to run:

$ docker run --rm -v $(pwd):/out -v ts2:/cache ghcr.io/alterakey/trueseeing

If you want to run statelessly you omit mounting volume onto /cache (not recommended for day-to-day use though; also see #254):

$ docker run --rm -v $(pwd):/out ghcr.io/alterakey/trueseeing

With pip

Alternatively, you can install our package with pip as follows. This form of installation might be useful for extensions, as it grants them the greatest freedom. Just remember you need a JRE and Android SDK (optionally; to mess with devices):

$ pip install --user trueseeing
$ trueseeing

Usage

Interactive mode

You can interactively scan/analyze/patch/etc. apps -- making it the ideal choice for manual analysis:

$ trueseeing target.apk
[+] trueseeing 2.2.2
ts[target.apk]> ?
...
ts[target.apk]> i                      # show generic information
...
ts[target.apk]> pf AndroidManifest.xml # show manifest file
...
ts[target.apk]> a                      # analyze resources too
...
ts[target.apk]> /s something           # search text
...
ts[target.apk]> as                     # scan
...
[+] done, found 6403 issues (174.94 sec.)
ts[target.apk]> gh report.html

Batch mode

We accept an inline command (-c) or script file (-i) to run before giving you prompt, as well as quitting right away instead of prompting (-q; we don't require a tty in this mode!).

You can use the features to conduct a batch scan, as follows e.g. to dump findings right onto the stderr:

$ trueseeing -eqc 'as' target.apk

To generate a report file in HTML format:

$ trueseeing -eqc 'as;gh report.html' target.apk

To generate a report file in JSON format:

$ trueseeing -eqc 'as;gj report.json' target.apk

To get report generated in stdout, omit filename from final g* command:

$ trueseeing -eqc 'as;gh' target.apk > report.html
$ trueseeing -eqc 'as;gj' target.apk > report.json

Non-interactive scan mode (deprecated)

Traditionally, you can scan apps with the following command line to get findings listed in stderr:

$ trueseeing --scan target.apk

To generate a report in HTML format:

$ trueseeing --scan --scan-output report.html target.apk
$ trueseeing --scan --scan-report=html --scan-output report.html target.apk

To generate a report in JSON format:

$ trueseeing --scan --scan-report=json --scan-output report.json target.apk

To get report generated in stdout, specify '-' as filename:

$ trueseeing --scan --scan-output - target.apk > report.html
$ trueseeing --scan --scan-report=html --scan-output - target.apk > report.html
$ trueseeing --scan --scan-report=json --scan-output - target.apk > report.json

Advanced Usages

Extensions

You can write your own commands and signatures as extensions. Extensions are placed under /ext (containers) or ~/.trueseeing2/extensions/ (pip) . Alternatively you can distribute your extensions as wheels. We provide type information so you can not only type-check your extensions with mypy but also get a decent assist from IDEs. See the details section for details.

Build

You can build it as follows:

$ docker build -t trueseeing https://github.com/alterakey/trueseeing.git#main

To build wheels you can do with flit, as follows:

$ flit build

To hack it, you need to create a proper build environment. To create one, set up a venv, install flit in there, and have it pull dependencies and validating toolchains; esp. mypy and pflake8. In short, do something like this:

$ git clone https://github.com/alterakey/trueseeing.git wc
$ python3 -m venv wc/.venv
$ source wc/.venv/bin/activate
(.venv) $ pip install flit
(.venv) $ flit install --deps=develop -s
(.venv) $ (... hack ...)
(.venv) $ trueseeing ...                         # to run
(.venv) $ mypy trueseeing && pflake8 trueseeing  # to validate
Success: no issues found in XX source files
(.venv) $ flit build                             # to build (wheel)
(.venv) $ docker build -t trueseeing .           # to build (container)

Details

Vulnerability Classes

Currently we can detect the following class of vulnerabilities, largely ones covered in OWASP Mobile Top 10 - 2016:

  • Improper Platform Usage (M1)

    • Debuggable
    • Inadvent publishing of Activities, Services, ContentProviders, BroadcastReceivers
  • Insecure Data (M2)

    • Backupable (i.e. suspectible to the backup attack)
    • Insecure file permissions
    • Logging
  • Insecure Commnications (M3)

    • Lack of pinning (i.e. suspictible to the TLS interception attack)
    • Use of cleartext HTTP
    • Tamperable WebViews
  • Insufficient Cryptography (M5)

    • Hardcoded passphrase/secret keys
    • Vernum ciphers with static keys
    • Use of the ECB mode
  • Client Code Quality Issues (M7)

    • Reflectable WebViews (i.e. XSSs in such views should be escalatable to remote code executions via JS reflection)
    • Usage of insecure policy on mixed contents
  • Code Tampering (M8)

    • Hardcoded certificates
  • Reverse Engineering (M9)

    • Lack of obfuscation

Extension API

Our extension API lays under the trueseeing.api package. As we provide type information with it, your IDE will assist you when writing your extensions.

Commands

To define new commands, implement trueseeing.api.Command and advertise them.

The following class will provide a sample command as t, for example:

from typing import TYPE_CHECKING
from trueseeing.api import Command
from trueseeing.core.ui import ui
if TYPE_CHECKING:
  from trueseeing.api import CommandMap, CommandPatternMap, ModifierMap, OptionMap, ConfigMap

class MyCommand(Command):
  @staticmethod
  def create() -> Command:
    return MyCommand()

  def get_commands(self) -> CommandMap:
    return {'t':dict(e=self._test, n='t', d='sample command')}

  def get_command_patterns(self) -> CommandPatternMap:
    return dict()

  def get_modifiers(self) -> ModifierMap:
     return dict()

  def get_options(self) -> OptionMap:
    return dict()

  def get_configs(self) -> ConfigMap:
    return dict()

  async def _test(self) -> None:
    ui.info('hello world')

Signatures

To define new signatures, implement trueseeing.api.Signature and advertise them.

The following class will provide a sample detector as my-sig, for example:

from typing import TYPE_CHECKING
from trueseeing.api import Signature
if TYPE_CHECKING:
  from trueseeing.api import SignatureMap, ConfigMap

class MySignature(Signature):
  @staticmethod
  def create() -> Signature:
    return MySignature()

  def get_sigs(self) -> SignatureMap:
    return {'my-sig':dict(e=self._detect, d='sample signature')}

  def get_configs(self) -> ConfigMap:
    return dict()

  async def _detect(self) -> None:
    self._helper.raise_issue(
      self._helper.build_issue(
        sigid='my-sig',
        title='hello world',
        cvss='CVSS:3.0/AV:P/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:N/',
      )
    )

File Formats

To define new file formats, firstly implement Contexts (ABC) for your formats, then implement trueseeing.api.FileFormatHandler to create and return their instances, and advertise them.

The following class will provide APK file support under the type named apk2, for example:

from typing import TYPE_CHECKING
from trueseeing.api import FileFormatHandler
from trueseeing.core.android.context import APKContext

if TYPE_CHECKING:
  from typing import Optional, Set
  from trueseeing.api import FormatMap, ConfigMap
  from trueseeing.core.context import Context, ContextType

class MyAPKContext(APKContext):
  # Use a different context type
  def _get_type(self) -> Set[ContextType]:
    return {'apk2'}

class APKFileFormatHandler(FileFormatHandler):
  @staticmethod
  def create() -> FileFormatHandler:
    return APKFileFormatHandler()

  def get_formats(self) -> FormatMap:
    return {'apk2':dict(e=self._handle, r=r'\.apk$', d='sample file format')}

  def get_configs(self) -> ConfigMap:
    return dict()

  def _handle(self, path: str) -> Optional[Context]:
    return MyAPKContext(path)

Then make sure you check for the type of the context in your signatures, making them ignored on unsupported contexts:

context = self._helper.get_context().require_type('apk2')

Upon successful check, require_type(...) will try to downcast them to appropriate types for your convenience.

But by design it works only for known types (currently, the apk). So if you are defining some detailed interface in your new context classes as we do for the apk type, you need to do a downcast here i.e.:

context: MyAPKContext = self._helper.get_context().require_type('apk2')  # type:ignore[assignment]

It is possible to define multiple formats matching the same pattern. We evaluate patterns in the order of from the most stringent (i.e. long) to the least. You use the -F switch to force some format to use with the target file, e.g.:

$ trueseeing -F apk2 target.apk

Package requirements

Extensions can be either: a) any package placed under /ext (container) or ~/.trueseeing2/extensions (pip), or b) any installed module named with the prefix of trueseeing_ext0_.

Origin of Project Name?

The D&D spell, True Seeing.

trueseeing's People

Contributors

akechishiro avatar alterakey avatar falconws avatar slreynolds avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

trueseeing's Issues

REPL to infer values

$ trueseeing --introspect ./target.apk

[.] analyzing classes... 500... 1000... 1500... done (1788.)
>>> map(print, (cl.name for cl in flow.code.OpMatcher(InvocationPattern(...))))
SomethingClass
Something2Class
Something3Class
>>> 

Adapting to Android N

Seemingly important changes are:

  • Grabbing procedure might be more cumbersome due to SELinux policy changes (e.g. pull vs cat > file)
  • Grabbing logs
  • APK signature scheme v2
  • Declarative TLS certificate pinning

crash case on DataFlows.decoded_registers_of

Traceback (most recent call last):
File "/Users/taky/ve/ts2/bin/trueseeing", line 9, in
load_entry_point('trueseeing==2.0.0', 'console_scripts', 'trueseeing')()
File "/Users/taky/works/trueseeing/wc/trueseeing/shell.py", line 114, in entry
return shell(sys.argv)
File "/Users/taky/works/trueseeing/wc/trueseeing/shell.py", line 85, in shell
for e in processed(f, [v for k,v in signatures.items() if k in signature_selected]):
File "/Users/taky/works/trueseeing/wc/trueseeing/shell.py", line 33, in processed
yield from (formatted(e) for e in c(context).detect())
File "/Users/taky/works/trueseeing/wc/trueseeing/shell.py", line 33, in
yield from (formatted(e) for e in c(context).detect())
File "/Users/taky/works/trueseeing/wc/trueseeing/signature/base.py", line 42, in detect
yield from res
File "/Users/taky/works/trueseeing/wc/trueseeing/signature/security.py", line 175, in do_detect
if p and DataFlows.solved_constant_data_in_invocation(p, 0):
File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 59, in solved_constant_data_in_invocation
graph = DataFlows.analyze(invokation_op)
File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 114, in analyze
return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_load_of(op, k)) for k in DataFlows.decoded_registers_of(op.p[0])}}
File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 114, in
return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_load_of(op, k)) for k in DataFlows.decoded_registers_of(op.p[0])}}
File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 111, in analyze
return DataFlows.analyze(DataFlows.analyze_recent_invocation(op))
File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 114, in analyze
return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_load_of(op, k)) for k in DataFlows.decoded_registers_of(op.p[0])}}
File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 114, in
return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_load_of(op, k)) for k in DataFlows.decoded_registers_of(op.p[0])}}
File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 144, in analyze_recent_load_of
caller_reg = DataFlows.decoded_registers_of(caller.p[0], type_=list)[index]
IndexError: list index out of range

static trace failure case

    load_entry_point('trueseeing==2.0.0', 'console_scripts', 'trueseeing')()
  File "/Users/taky/works/trueseeing/wc/trueseeing/shell.py", line 114, in entry
    return shell(sys.argv)
  File "/Users/taky/works/trueseeing/wc/trueseeing/shell.py", line 85, in shell
    for e in processed(f, [v for k,v in signatures.items() if k in signature_selected]):
  File "/Users/taky/works/trueseeing/wc/trueseeing/shell.py", line 33, in processed
    yield from (formatted(e) for e in c(context).detect())
  File "/Users/taky/works/trueseeing/wc/trueseeing/shell.py", line 33, in <genexpr>
    yield from (formatted(e) for e in c(context).detect())
  File "/Users/taky/works/trueseeing/wc/trueseeing/signature/base.py", line 42, in detect
    yield from res
  File "/Users/taky/works/trueseeing/wc/trueseeing/signature/security.py", line 221, in do_detect
    yield self.issue(IssueSeverity.MAJOR, IssueConfidence.TENTATIVE, '%(name)s#%(method)s' % dict(name=self.context.class_name_of_dalvik_class_type(cl.qualified_name()), method=k.method_.v.v), 'detected logging: %(target_val)s: "%(val)s"' % dict(target_val=k.p[1].v, val=DataFlows.solved_constant_data_in_invocation(k, 1)))
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 59, in solved_constant_data_in_invocation
    graph = DataFlows.analyze(invokation_op)
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 114, in analyze
    return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_load_of(op, k)) for k in DataFlows.decoded_registers_of(op.p[0])}}
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 114, in <dictcomp>
    return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_load_of(op, k)) for k in DataFlows.decoded_registers_of(op.p[0])}}
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 111, in analyze
    return DataFlows.analyze(DataFlows.analyze_recent_invocation(op))
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 114, in analyze
    return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_load_of(op, k)) for k in DataFlows.decoded_registers_of(op.p[0])}}
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 114, in <dictcomp>
    return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_load_of(op, k)) for k in DataFlows.decoded_registers_of(op.p[0])}}
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 106, in analyze
    return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_static_load_of(op)) for k in DataFlows.decoded_registers_of(op.p[0])}}
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 106, in <dictcomp>
    return {op:{k:DataFlows.analyze(DataFlows.analyze_recent_static_load_of(op)) for k in DataFlows.decoded_registers_of(op.p[0])}}
  File "/Users/taky/works/trueseeing/wc/trueseeing/flow/data.py", line 126, in analyze_recent_static_load_of
    raise Exception('failed static trace of: %r' % op)
Exception: failed static trace of: <Op id:sget-object:[<Token reg:v1>, <Token reflike:Ljava/util/Locale;->ROOT:Ljava/util/Locale;>]>

[3] dip woes

check_security_arbitrary_webview_overwrite: guessed_size: guessed_dp: warning: ignoring non-dp suffix (0.0dip)
  • investigate

Automatic binary patches

  • Disable certificate pinning
  • Enable debugging
  • Enable logging
  • Instrumenting certain function calls

etc.

Static-key argument analysis needs to be fixed

These logs are bullshit, need to be fixed

com.facebook.w#c(Landroid/content/Context;)Ljava/lang/String;:0:0:severe{tentative}:insecure cryptography: static keys: "0x0" [3] [-Wcrypto-static-keys]
com.google.android.gms.internal.zzfm#a(Ljava/lang/String;)Ljava/security/PublicKey;:0:0:severe{tentative}:insecure cryptography: static keys: "0x0" [3] [-Wcrypto-static-keys]
com.google.android.gms.internal.zzo#a([BLjava/lang/String;)[B:0:0:severe{tentative}:insecure cryptography: static keys: "AES" [3] [-Wcrypto-static-keys]
dmv#b(Ljava/lang/String;)Ljava/lang/String;:0:0:severe{tentative}:insecure cryptography: static keys: "UTF-8" [5] (base64; "51317c" [3]) [-Wcrypto-static-keys]

Dynamic analysis: real world pentesting

Dynamic analysis are:

  • Fuzzing intents
  • UI fuzzing
  • UI hierarchy analysis
  • Log analysis
  • Filesystem analysis
  • Acquiring memory dumps (in emulation mode?)
  • Exploit deployment

refine obfuscator detection

Currently we are detecting ProGuard by testing presence of short names. While it is effective, we think we should test by packages for accuracy.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.