project-artist / dexterous Goto Github PK
View Code? Open in Web Editor NEWLibrary and standalone CLI tool for apk/dex merging, repackaging and signing. Can also get used as a dex analyzer framework.
License: Other
Library and standalone CLI tool for apk/dex merging, repackaging and signing. Can also get used as a dex analyzer framework.
License: Other
When I try to use the current dexterous version @ fd5a845 to merge a codelib into the systemserver (services.jar
), the following error occurs:
Partially merging CodeLib into services.jar using dexterous
I/Dexterous: Loading DexFile: services.apk:classes.dex
I/Dexterous: Loading DexFile: services.apk:classes2.dex
I/Dexterous: DONE Loading 01 Dexfiles (services.apk)
I/Dexterous: Loading DexFile: app-debug.apk:classes.dex
I/Dexterous: Loading DexFile: app-debug.apk:classes2.dex
I/Dexterous: DONE Loading 01 Dexfiles (app-debug.apk)
I/Dexterously: # MULTIDEX File. DexFile Count: 2
I/Dexterously:
I/Dexterously: # DEXFILE: services.apk:classes.dex #############################################
I/Dexterously:
I/Loggy: ----------------------------------------------------------------------------
I/Loggy: Section |TYPE: null | OFF: -1 | SIZE: 0 |
I/Loggy: Section |TYPE: null | OFF: -1 | SIZE: 0 |
I/Loggy: Section |TYPE: HEADER | OFF: 0 | SIZE: 1 |
I/Loggy: Section |TYPE: STRING_IDS | OFF: 112 | SIZE: 63478 |
I/Loggy: Section |TYPE: TYPE_IDS | OFF: 254024 | SIZE: 4637 |
I/Loggy: Section |TYPE: PROTO_IDS | OFF: 272572 | SIZE: 9860 |
I/Loggy: Section |TYPE: FIELD_IDS | OFF: 390892 | SIZE: 19115 |
I/Loggy: Section |TYPE: METHOD_IDS | OFF: 543812 | SIZE: 34535 |
I/Loggy: Section |TYPE: CLASS_DEFS | OFF: 820092 | SIZE: 2616 |
I/Loggy: Section |TYPE: ANNOTATION_SET_REF_LISTS | OFF: 903804 | SIZE: 352 |
I/Loggy: Section |TYPE: ANNOTATION_SETS | OFF: 908676 | SIZE: 3517 |
I/Loggy: Section |TYPE: CODE_ITEMS | OFF: 945024 | SIZE: 24583 |
I/Loggy: Section |TYPE: ANNOTATION_DIRECTORY_ITEMS | OFF: 4128108 | SIZE: 2239 |
I/Loggy: Section |TYPE: TYPE_LISTS | OFF: 4193532 | SIZE: 6329 |
I/Loggy: Section |TYPE: STRING_DATA_ITEMS | OFF: 4262876 | SIZE: 63478 |
I/Loggy: Section |TYPE: DEBUG_INFO_ITEM | OFF: 5763692 | SIZE: 22550 |
I/Loggy: Section |TYPE: ANNOTATION_ITEM | OFF: 6587462 | SIZE: 3781 |
I/Loggy: Section |TYPE: ENCODED_ARRAY_ITEMS | OFF: 6656830 | SIZE: 539 |
I/Loggy: Section |TYPE: CLASS_DATA_ITEMS | OFF: 6670105 | SIZE: 2604 |
I/Loggy: Section |TYPE: MAP_LIST | OFF: 6880792 | SIZE: 1 |
I/Loggy: ----------------------------------------------------------------------------
I/Dexterously:
I/Dexterously: # DEXFILE: app-debug.apk:classes.dex #############################################
I/Dexterously:
I/Loggy: ----------------------------------------------------------------------------
I/Loggy: Section |TYPE: null | OFF: -1 | SIZE: 0 |
I/Loggy: Section |TYPE: null | OFF: -1 | SIZE: 0 |
I/Loggy: Section |TYPE: HEADER | OFF: 0 | SIZE: 1 |
I/Loggy: Section |TYPE: STRING_IDS | OFF: 112 | SIZE: 21234 |
I/Loggy: Section |TYPE: TYPE_IDS | OFF: 85048 | SIZE: 2352 |
I/Loggy: Section |TYPE: PROTO_IDS | OFF: 94456 | SIZE: 3489 |
I/Loggy: Section |TYPE: FIELD_IDS | OFF: 136324 | SIZE: 10394 |
I/Loggy: Section |TYPE: METHOD_IDS | OFF: 219476 | SIZE: 17438 |
I/Loggy: Section |TYPE: CLASS_DEFS | OFF: 358980 | SIZE: 1626 |
I/Loggy: Section |TYPE: CODE_ITEMS | OFF: 411012 | SIZE: 11730 |
I/Loggy: Section |TYPE: DEBUG_INFO_ITEM | OFF: 1237269 | SIZE: 10561 |
I/Loggy: Section |TYPE: TYPE_LISTS | OFF: 1463400 | SIZE: 2079 |
I/Loggy: Section |TYPE: STRING_DATA_ITEMS | OFF: 1484594 | SIZE: 21234 |
I/Loggy: Section |TYPE: ANNOTATION_ITEM | OFF: 2021194 | SIZE: 2040 |
I/Loggy: Section |TYPE: CLASS_DATA_ITEMS | OFF: 2054433 | SIZE: 1516 |
I/Loggy: Section |TYPE: ENCODED_ARRAY_ITEMS | OFF: 2151369 | SIZE: 291 |
I/Loggy: Section |TYPE: ANNOTATION_SETS | OFF: 2167192 | SIZE: 1916 |
I/Loggy: Section |TYPE: ANNOTATION_SET_REF_LISTS | OFF: 2189004 | SIZE: 144 |
I/Loggy: Section |TYPE: ANNOTATION_DIRECTORY_ITEMS | OFF: 2191632 | SIZE: 1436 |
I/Loggy: Section |TYPE: MAP_LIST | OFF: 2237768 | SIZE: 1 |
I/Loggy: ----------------------------------------------------------------------------
I/MethodFilter: Annotationtype used for whitelisting:Lsaarland/cispa/artist/codelib/CodeLib$Inject;
Exception in thread "main" java.lang.IllegalArgumentException: position=88867372 length=2237988
at comm.android.dex.Dex.open(Dex.java:191)
at comm.android.dx.merge.MethodFilter.getString(MethodFilter.java:136)
at comm.android.dx.merge.MethodFilter.getMethodString(MethodFilter.java:150)
at comm.android.dx.merge.MethodFilter.initializeWhitelist(MethodFilter.java:230)
at comm.android.dx.merge.MethodFilter.<init>(MethodFilter.java:387)
at comm.android.dex.Dex.setWhitelistedAnnotation(Dex.java:358)
at saarland.cispa.dexterous.cli.Dexterously.mergeCodeLib(Dexterously.java:617)
at saarland.cispa.dexterous.cli.Main.main(Main.java:49)
It does NOT occur when I use the dexterous version from before the annotation-based whitelisting patch (tested: 3253508).
@alfink do you have an intuition what is going wrong here?
Given that we already build an Android library, we could as well publish so that it can be included in, e.g., a build.gradle
file.
The fastest way to do so seems to be https://jitpack.io/, which automatically builds a project from a GitHub repo and exposes the result, given there is a certain task available. It should be straightforward to set this up, judging from the log at https://jitpack.io/#Project-ARTist/dexterous
Merging fails when whitelisting class fields.
Logcat:
11-03 21:20:19.688 7541 12227 D Dexterous: Dexterous() Init:
11-03 21:20:19.688 7541 12227 D Dexterous: > App: /data/app/de.alfink.regbug-1/base.apk
11-03 21:20:19.688 7541 12227 D Dexterous: > CodeLib: /data/user/0/saarland.cispa.artist.artistgui/app_modules/saarland.cispa.artist.module.regbug/codelib.apk
11-03 21:20:19.689 7541 12227 D Dexterous: > CodeLibName: codelib.apk
11-03 21:20:19.689 7541 12227 D Dexterous: > CodeLibDexName: codelib.apk:classes.dex
11-03 21:20:19.689 7541 12227 D Dexterous: > DexFile: base.apk:classes.dex
11-03 21:20:19.689 7541 12227 D Dexterous: > DexFile: codelib.apk:classes.dex
11-03 21:20:19.690 7541 12227 I MethodFilter: Annotationtype used for whitelisting:Lsaarland/cispa/artist/codelib/CodeLib$Inject;
11-03 21:20:19.690 7541 12227 D MethodFilter: Field whitelisted:10
11-03 21:20:19.690 7541 12227 D MethodFilter: Field whitelisted:11
11-03 21:20:19.691 7541 12227 D MethodFilter: Field whitelisted:12
11-03 21:20:19.691 7541 12227 D MethodFilter: Field whitelisted:13
11-03 21:20:19.691 7541 12227 D MethodFilter: Field whitelisted:14
11-03 21:20:19.692 7541 12227 D MethodFilter: Field whitelisted:15
11-03 21:20:19.692 7541 12227 D MethodFilter: Field whitelisted:16
11-03 21:20:19.692 7541 12227 D MethodFilter: ColdelibWhitelisting Method annotated:fiveargs
11-03 21:20:19.693 7541 12227 D MethodFilter: ColdelibWhitelisting Method annotated:fourargs
11-03 21:20:19.693 7541 12227 D MethodFilter: ColdelibWhitelisting Method annotated:onearg
11-03 21:20:19.693 7541 12227 D MethodFilter: ColdelibWhitelisting Method annotated:sevenargs
11-03 21:20:19.694 7541 12227 D MethodFilter: ColdelibWhitelisting Method annotated:sixargs
11-03 21:20:19.694 7541 12227 D MethodFilter: ColdelibWhitelisting Method annotated:threeargs
11-03 21:20:19.694 7541 12227 D MethodFilter: ColdelibWhitelisting Method annotated:twoargs
11-03 21:20:19.695 7541 12227 I MethodFilter: CodelibWhitelisting Class whitelisted: 17 (Lsaarland/cispa/artist/codelib/CodeLib;)
11-03 21:20:19.695 7541 12227 I MethodFilter: class_annotations_off: 3720
11-03 21:20:19.695 7541 12227 I MethodFilter: f/m/p:7/7/0
11-03 21:20:19.696 7541 12227 I Dexterous: MERGING DEX codelib.apk:classes.dex into base.apk:classes.dex
11-03 21:20:19.697 7541 12227 D Dexterous: DexMerger mergeStringIds...
11-03 21:20:19.698 7541 12227 W DexMerger: Whitelisted: 20 - LL
11-03 21:20:19.749 7541 12227 W DexMerger: Whitelisted: 21 - LLL
11-03 21:20:19.751 7541 12227 W DexMerger: Whitelisted: 22 - LLLL
11-03 21:20:19.751 7541 12227 W DexMerger: Whitelisted: 23 - LLLLL
11-03 21:20:19.752 7541 12227 W DexMerger: Whitelisted: 24 - LLLLLL
11-03 21:20:19.753 7541 12227 W DexMerger: Whitelisted: 25 - LLLLLLL
11-03 21:20:19.754 7541 12227 W DexMerger: Whitelisted: 26 - LLLLLLLL
11-03 21:20:19.755 7541 12227 W DexMerger: Whitelisted: 30 - Ldalvik/annotation/MemberClasses;
11-03 21:20:19.795 7541 12227 W DexMerger: Whitelisted: 32 - Ljava/lang/Object;
11-03 21:20:19.797 7541 12227 W DexMerger: Whitelisted: 43 - Lsaarland/cispa/artist/codelib/CodeLib;
11-03 21:20:19.803 7541 12227 W DexMerger: Whitelisted: 57 - argument1
11-03 21:20:19.843 7541 12227 W DexMerger: Whitelisted: 58 - argument2
11-03 21:20:19.844 7541 12227 W DexMerger: Whitelisted: 59 - argument3
11-03 21:20:19.845 7541 12227 W DexMerger: Whitelisted: 60 - argument4
11-03 21:20:19.845 7541 12227 W DexMerger: Whitelisted: 61 - argument5
11-03 21:20:19.846 7541 12227 W DexMerger: Whitelisted: 62 - argument6
11-03 21:20:19.847 7541 12227 W DexMerger: Whitelisted: 63 - argument7
11-03 21:20:19.847 7541 12227 W DexMerger: Whitelisted: 71 - fiveargs
11-03 21:20:19.868 7541 12227 W DexMerger: Whitelisted: 72 - fourargs
11-03 21:20:19.870 7541 12227 W DexMerger: Whitelisted: 78 - onearg
11-03 21:20:19.925 7541 12227 W DexMerger: Whitelisted: 85 - sevenargs
11-03 21:20:19.950 7541 12227 W DexMerger: Whitelisted: 86 - sixargs
11-03 21:20:19.953 7541 12227 W DexMerger: Whitelisted: 88 - threeargs
11-03 21:20:19.958 7541 12227 W DexMerger: Whitelisted: 89 - twoargs
11-03 21:20:19.962 7541 12227 W DexMerger: Whitelisted: 91 - value
11-03 21:20:19.968 7541 12227 D DexMerger$IdMerger: Blacklisted: 49
11-03 21:20:19.968 7541 12227 D Dexterous: DexMerger mergeTypeIds...
11-03 21:20:19.980 7541 12227 D DexMerger$IdMerger: Blacklisted: 14
11-03 21:20:19.980 7541 12227 D Dexterous: DexMerger mergeTypeLists...
11-03 21:20:20.001 7541 12227 D Dexterous: DexMerger mergeProtoIds...
11-03 21:20:20.025 7541 12227 D DexMerger$IdMerger: Blacklisted: 6
11-03 21:20:20.025 7541 12227 D Dexterous: DexMerger mergeFieldIds...
11-03 21:20:20.029 2228 2228 D wpa_supplicant: wlan0: Control interface command 'SIGNAL_POLL'
11-03 21:20:20.044 2228 2228 D wpa_supplicant: CTRL-DEBUG: global_ctrl_sock-sendto: sock=9 sndbuf=163840 outq=0 send_len=48
11-03 21:20:20.092 7541 12227 W System.err: java.lang.IllegalArgumentException: Expected an unsigned short: -1
11-03 21:20:20.098 7541 12227 W System.err: at comm.android.dex.Dex$Section.writeUnsignedShort(Dex.java:673)
11-03 21:20:20.098 7541 12227 W System.err: at comm.android.dex.FieldId.writeTo(FieldId.java:59)
11-03 21:20:20.098 7541 12227 W System.err: at comm.android.dx.merge.DexMerger$5.write(DexMerger.java:633)
11-03 21:20:20.098 7541 12227 W System.err: at comm.android.dx.merge.DexMerger$5.write(DexMerger.java:607)
11-03 21:20:20.098 7541 12227 W System.err: at comm.android.dx.merge.DexMerger$IdMerger.mergeSorted(DexMerger.java:353)
11-03 21:20:20.098 7541 12227 W System.err: at comm.android.dx.merge.DexMerger.mergeFieldIds(DexMerger.java:635)
11-03 21:20:20.098 7541 12227 W System.err: at comm.android.dx.merge.DexMerger.mergeDexesMethods(DexMerger.java:201)
11-03 21:20:20.098 7541 12227 W System.err: at comm.android.dx.merge.DexMerger.mergeMethodsOnly(DexMerger.java:244)
11-03 21:20:20.098 7541 12227 W System.err: at saarland.cispa.dexterous.Dexterous.mergeCodeLibReference(Dexterous.java:234)
11-03 21:20:20.098 7541 12227 W System.err: at saarland.cispa.dexterous.Dexterous.mergeCodeLibReference(Dexterous.java:220)
11-03 21:20:20.098 7541 12227 W System.err: at saarland.cispa.dexterous.Dexterous.mergeMethodIds(Dexterous.java:107)
11-03 21:20:20.098 7541 12227 W System.err: at saarland.cispa.dexterous.Dexterous.mergeCodeLib(Dexterous.java:121)
11-03 21:20:20.098 7541 12227 W System.err: at saarland.cispa.artist.artistgui.instrumentation.stages.InstrumentationStagesImpl.mergeCodeLib(InstrumentationStagesImpl.java:182)
11-03 21:20:20.098 7541 12227 W System.err: at saarland.cispa.artist.artistgui.instrumentation.InstrumentationTask.run(InstrumentationTask.java:69)
11-03 21:20:20.098 7541 12227 W System.err: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:428)
11-03 21:20:20.098 7541 12227 W System.err: at java.util.concurrent.FutureTask.run(FutureTask.java:237)
11-03 21:20:20.098 7541 12227 W System.err: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
11-03 21:20:20.098 7541 12227 W System.err: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
11-03 21:20:20.098 7541 12227 W System.err: at java.lang.Thread.run(Thread.java:761)
Instead of merging all classes, types and methods, it would be preferable to allow for selective, fine-grained merging. This avoids wasting IDs on Android helper classes that are not really used and allows for merging codelib packages that feature bigger libraries. With this approach, a codelib class could define a small interface to the large library and only those few class methods are assigned IDs in target dex files (see 64k method ID limit).
This could, e.g., be controlled with blacklists or whitelists provided via commandline arguments.
Exception in thread "main" comm.android.dex.DexIndexOverflowException: Too many field IDs. field ID not in [0, 0xffff]: 65536
I leave this here as a reference, in case errors with filenames occur at a large scale testing later:
Digging through https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT D.3 (APPENDIX D - Language Encoding (EFS)):
It's either CP437 (original encoding), UTF-8 or user defined via the 0x0008 Extra Field
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.