Git Product home page Git Product logo

blog-notes's Introduction

Notes ✍️

This is a collection of my own notes ✍️ (goto Issues). Opinions are also my own 😆. Or, check out to my website.

🏡 Topics

Topics are various, but,

  • programming languages and virtual machines,
  • android and aosp,
  • browsers and web,
  • and other micellaneous ones.

🚦Attention

You'd bettern know before reading them,

  • Some of them are accomplished, but some of are not (they are tagged with todo; for these, I either know little about them, or not well prepared to write them down in a logical and structural manner)
  • Some of them are my personal thoughts, but some of them are learned or borrowed directly from others (but I've put references)
  • Take your own risk

🤝 Welcome

Last but not least, I'm learning as you are. So discussions are very much welcome. Let me know if you have any comments, suggestions, questions, or even corrections. Please directly make comments under the issues that you are interested.

blog-notes's People

Contributors

connglli avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

blog-notes's Issues

Force Stop, and Multi-Process-Architecture App

Prerequisite - Application Class

The Application class (or its subclasses) is created in and by ActivityThread, when an app is started (or another process of the same app is started), in detail,

  • the ActivityThread#main() invokes ActivityThread#attach(false) to attach application the ActivityThread; and ActivityThread#attach(false) delegates the task to ActivityManagerService#handleBindApplication()
  • in ActivityManagerService#handleBindApplication(), the Application is instantiated by LoadedApk#makeApplication(), and in the end, by Instrumentation#newApplication(), wherein, the base context is attached (i.e., Application#attachBaseContext() is invoked)

So for a multiproc-arch app, the Application class (or its subclasses) is instantiated multiple times, and its lifecycle, i.e., Application#{attachBaseContext, onCreate, onLowMemory, onTrimMemory, onTerminate}(), is invoked the same number of times

References

Introduction

One can use the android:process tag to annotate the process a component resides in, and the system will test whether the process exists, and creates one if not, then starts the specified component

Usually, the started component resides in the process assigned by android:process, however, if the tag android:multiproc is set to true, the started component resides in the process which starts it

For a multiproc-arch app, the Application context is created, and lifecycled the same number of time as the number of its processes

Constraints on android:process tag:

  • if starts with a “:”, it is considered as a private process of the app
  • otherwise, it is considered as a standalone general process as others, but it have to contain a “.” in its name, i.e., “abc” is invalid, “abc.def” is valid

References

Yoric

If we have a multi-process-architecture demo app named Yoric, packaged com.example.yoric, with the following processes:

  • main process: com.example.yoric, used by MainActivity
  • remote process: com.example.yoric:remote, used by RemoteService
  • android-remote process: android.remote, used by AndroidRemoteService
  • remote-c process: com.example.yoric:remote-c, forked twice using JNI by the remote process, and left as an orphan
  • android-remote-c process: android.remote-c, forked twice using JNI by the android-remote process, and left as an orphan

Let’s assume when MainActivity is started, the two services are started at the same time

image

The following is result of ps -o USER,PID,PGID,PPID,NAME; one can see from the figure:

  • UID (USER): all processes of Yoric share the same UID, which is the UID of Yoric, i.e., 10129 (u0_a129)
  • PGID: all processes of Yoric forked directly or indirectly by zygote belongs to the process group led by zygote

Conclusion: almost all processes forked directly or indirectly by zygote belongs to the process group led by zygote (unless the app itself sets the process group manually)

How Force-Stop Works?

The workflow (based on android-10):

1. Parse pacakge name of the app to get its UID, say UIDXXX 
   (ActivityManagerService#forceStopPackage())
2. Collect into procs (ArrayList<ProcessRecord>) all processes whose UID is UIDXXX 
   via iterating the ProcessList maintained by ActivityManagerService 
   (ProcessList#killPackageProcessesLocked())
3. For each process P in procs (ProcessList#killPackageProcessesLocked()): 
  2.1 kill P using syscall kill(<pid_of_P>, SIGKILL) (ProcessRecord#kill() 
      -> Process#killProcessQuiet())
  2.2 kill the process members of the process cgroup P resides in (ProcessRecord#kill() 
      -> ProcessList#killProcessGroup() -> Process#killProcessGroup() 
      -> libprocessgroup: KillProcessGroup())
    2.2.1 read /proc/<pid_of_p>/cgroup to know which process cgroup p belongs to, 
          say C (libprocessgroup: KillProcessGroup())
    2.2.2 for each process PP in the cgroup C (by reading /acct/uid_UIDXXX/pid_<pid_of_C>/cgroup.procs):
      a. if PP is a leader of a process group (PID == PGID), kill its process group using 
         syscall kill(-<pid_of_PP>, SIGKILL)
      b. otherwise, kill PP itself (libprocessgroup: DoKillProcessGroupOnce())
    2.2.3 repeat 2.2.2 40 times, and sleep 5ms each time each after
4. clear other components still managered by other managers

Given that almost all processes forked by zygote belongs to the process group led by zygote (unless the app itself sets the process group manually), in practice, when killing a common 3rd-party app, the workflow 2.2.2.a is usually redundant, and

  • no process groups are actually killed (cause no processes are leaders),
  • the processes are kill one by one rather than groupfully

Conclusion: force-stop kills all processes share the same uid as UA, not the process group the app resides in

References

Implement a Float Widget (悬浮组件)

Option 1: WindowManager#addView(view)

  • Via wm = Context#getSystemService(WINDOW_SERVICE)
  • Create a new window, add attach the view to it; the window is typed TYPE_SYSTEM_ALERT (for sdk < 26) or TYPE_APPLICATION_OVERLAY (o.w.)
  • One need
    1. the app to request the SYSTEM_ALERT_WINDOW permission in manifest
    2. the app is permitted to overlay by checking Settings#canDrayOverlay(), and if not start the activity using intent action Settings.ACTION_MANAGE_OVERLAY_PERMISSION
  • The widget won’t disappear even if the activity is destroyed, or the app exits, because it is in

References

Option 2: ViewGroup#addView(view)

  • Via vg = (FrameLayout) (activity.getWindow().getDecorView())
  • The view is added as a child of the DecorView, and brosis of the title, and content parent
  • For the float widget to appear at every activity, one might need to add a activity lifecycle callback using Application#registerActivityLifecycleCallbacks(), and add the float widget when each activity is resumed
  • The float widget disappears when the activity is stopped

References

All About Fragment

Lifecycle (with Activity)

  • See https://github.com/xxv/android-lifecycle
  • There are 3 cycles in the lifecycle of Fragment
    • onStop -> onStart: this cycle is activated when a fragment is half overlapped
    • onDestroyView -> onCreateView: this cycle is activated when a fragment is added to the back stack
    • onDetach -> onAttach: this cycle is activated when a fragment is detached from its contained Activity via FragmentTransaction#detach()
  • onCreateView and onDestroyView
    • onCreateView returns the view of this fragment, or return null indicating this is a non-UI fragment. The result will be added to its container by its attached activity. Hence the flag in LayoutInflator#inflate(,,flag) need to be false, or the fragment will be double added.
    • Do view creating in onCreateView, and others in onActivityCreated
  • onAttach
    • onAttach is the first lifecycle method of Fragment. The attached Activity can be gotten via it, and it is right here that you need to convert the Activity to a listener.

References

BackStack

  • The back stack of the fragment is different from that of Activity
  • Each item in it is not a stateful Fragment, but a transaction (a list of operations), while each in Activity back stack is a stateful Activity. Thus adding one to the back stack needs FragmentTransaction.addToBackStack(null)
  • It is maintained by host of the Fragment, while the Activity back stack is maintained by the system
  • Click back button => the host will intercept it => host pops one transaction from the the back stack, and reverse the transaction to recover last one => the host's back stack will not be popped due to the interception

References

getFragmentManager() vs. getChildFragmentManager()

  • The former gets the FragmentManager who managers itself
  • The latter gets the FragmentManager who manages fragments of itself, e.g.,
    <ActivityA>
        <FragmentA>
            <FragmentAA />
        </FragmentA>
        <FragmentB/>
    </ActivityA>
    
    • When call fragmentA.getFragmentManager(), it gets the FragmentManager that manages FragmentA and FragmentB, i.e., the FragmentManager inside ActivityA
    • When call fragmentA.getChildFragmentManager(), it gets the FragmentManager that manages FragmentAA, i.e., the FragmentManager inside FragmentA

References

FragmentActivity

  • There can be multiple “active” fragments on one Activity, and many many added fragments that are inactive
  • Only activities extending FragmentActivity has getSupportFragmentManager() method (e.g., AppCompatActivity), other activities only have a getFragmentManager() method
  • Except general dumps from the Activity, dumps of FragmentActivity also includes the dump of the support fragment manager

References

<data> of <intent-filter>

  • Multiple <data> elements can be put in one <intent-filter>, but they will in the end be merged by Android. The merging:
    • Multiple: host and scheme will be merged into a list via cartesian product.
    • Single: path/pathPrefix/pathPattern will be merged into a more detailed one.
  • The following example will generate http://example.com/r, http://dcom/r, ex://example.com/r, and ex://dcom/r
    <intent-filter>
      <data scheme=“http” host=“example.com” pathPrefix=“/r”>
      <data scheme=“ex” host=“dcom” pathPrefix=“/.*”>
    </intent-filter>
    

References

<uses-feature>

Android supports a variety of features for users to use, including hardware-based (e.g., compass), software-based (e.g., app widgets), and platform-dependent; however, not all Android devices provide all theses features, so Android defines a feature ID for those features that may or may not be included by some Android devices, e.g., FEATURE_SENSOR_COMPASS

<uses-feature required=“true”> works like 4.4-era <uses-permission>: if the features requested using this tag by an app is not available on this devices, the app cannot be installed

<uses-feature required=“false”> works like modern-era <uses-permissions>: developers need to dynamically check the availability of the features requested at runtime

App Loading (Code+Resource) Mechanism

As long as the process is forked from Zygote, and the ART is running, then how does it know where the class’s APK is ⚠️ TO BE ADDED

Code Loading

⚠️ TO BE ADDED

Resource Loading

⚠️ TO BE ADDED

Framework resource?

  • loaded by Zygote

App resources

  • When loaded? When ActivityThread is created
  • How loaded? AssetManager
  • Who loaded them? AssetManager

Classes

Conf

android.content.res.Configuration: all device configurations

android.os.Environment: provides access to environment variables, e.g., ENV_EXTERNAL_STORAGE, DIR_DATA.

android.app.ResourcesManager

  1. Singleton manager that manages the resources an app uses, including both framework-res, and app-res, and the singleton thereby means ActivityThread.mResourcesManager = ResourcesManager.getInstance(), which is initialized in ActivityThread#<init>()
  2. It applies a lazy loading mechanism, i.e., it loads the required resources the first time an caller invokes #getResouces().

Apk

android.app.LoadedApk

  • an interal memory representation of the loaded APK file
  • one can get almost all information about an loaded apk by instance of this class, e.g., the code and resources, information about the inside components like Activity and Service
  • it can be created by ActivityThread#getPackageInfo()

android.content.pm.PackageParser$Package

  • a representation of a full package parsed from APK files on disk. A package
  • it is created by android.content.pm.PackageParser#parsePackage(File, ...)
    • Parser for package files (APKs) on disk. This supports apps packaged either as a single "monolithic" APK, or apps packaged as a "cluster" of multiple APKs in a single directory. One can generate to get all kinds of info data (see following) of an apk (or package, see following) by instance of this class

Similarities and differences

  • similarity: both are representations of an APK file
  • differences:
    • Package prefers to be the represtation of any APK file on disk either loaded or not; it is created by PackageParser parsing the APK file directly via its disk path of the APK
    • LoadedApk prefers to represent the APKs that are loaded into the app's memory; instead of the disk path, it is created by the ApplicationInfo in ActivityThread, however, the ApplicationInfo is generated by PackageParser#generateApplicationInfo(Package, ...) by PMS#getApplicationInfo(packageName); it can do more than Package

All kinds of info data classes

android.content.pm.PackageInfo

  • overall information about the contents of a package. this corresponds o all of the information collected from AndroidManifest.xml.

android.content.pm.ApplicationInfo

  • information you can retrieve about a particular application. this corresponds to information collected from the AndroidManifest.xml's tag

android.content.pm.InstrumentationInfo

  • information you can retrieve about a particular application instrumentation. this corresponds to information collected from the AndroidManifest.xml's tag

android.content.pm.ActivityInfo

  • information you can retrieve about a particular application activity or receiver. this corresponds to information collected from the AndroidManifest.xml's <activity> tag or <receiver> tag
  • one can get the application info by LoadedApk#getApplicationInfo()

...

References

A First Look of Android Permissions

Why Permissions?

  • The purpose of a permission is to protect the privacy of an Android user.
  • A central design point of the Android security architecture is that no app, by default, has permissions to perform any operations that would adversely impact other apps, the operating system, or the user. (This is also why Node.js is rewritten to deno)
  • Which data is sensitive is defined by app developers, so custom permissions can be defined and used.
  • Actually, the permission system can be fully implemented by app developers without any help of os. However, provided by the os, the permission system obtains the os-level protection and enhancement.

References

Use Permissions

  • If one uses any (no matter normal or dangerous, system or custom) permission, one has to declare it in your manifest using <uses-permission>.
  • One cannot assume at any time one has any permission even if one has declared, and even if the user’s ever granted, because the user can at any time go to System Settings and revoke it.
  • At any time one uses a permission, one has to:
    1. check whether granted,
    2. request ungranted permissions, and,
    3. handle request results (all permission handling codes reside in onRequestPermissionResult, one can separate them using requestCode) (see details in Request app permissions)

References

{Normal, Signature, Dangerous} Permissions

  • Normal: Granted directly by system at install-time.
  • Signature: Granted directly by system at install-time, only if the app is signed the same certificate with the app defining the permissions the app wanting to use.
  • Dangerous: Granted by user at run-time (running on API level > 23, and target API level > 23), or granted by user at install-time.

References

https://developer.android.com/guide/topics/permissions/overview

Permission Group

  • Permissions are grouped. Within one group:
    • Running on API level > 23, and target > 23: all normal ones are granted whatever which is granted; all dangerous ones are granted whatever which dangerous one is granted
    • O.W.: all other permissions are granted whatever which is granted
  • One uses <android:permission> on <activity>/<service>/<broadcast-receiver>/<content-provider> to restrict the access of them. One has to declare and request for permissions if one wants to access them.

References

https://developer.android.com/guide/topics/permissions/overview

Step Into Android Binder (ABD)

Overview

image

ABD is used for RPC and IPC, and it is a layered framework, composed of 4 general layers from bottom to top:

  • Driver Layer
  • C++ Layer
  • Java Layer
  • AIDL Layer

Introduction

⚠️ TO BE ADDED

FAQ

1. A 利用 B 的 proxy 向 B 发送的时候如何知道双方 handle?

B 的 proxy 运行在 A 的进程中,虽然 A 调用 B 是 B.proxy.abc(),但仍然在 A 的进程中,所以陷入内核的时候 binder 根据 binder_proc 就知道 A 自己的 handle,而 B 的 handle 显然是 B 的 proxy 自己就知道的

2. B 的 proxy 为什么知道自己的 handle?

这是 B 的 proxy 构造函数的入参

3. B 的 proxy 是 svcmgr 构造的,svcmgr 是怎么知道 B 的 handle 的?

B 像 svcmgr 注册自己的时候调用的是 BpServiceManager.addService(name, this),其中 this 是 BBinder,且运行在自己的进程,所以 binder 就会知道自己的 handle,同时将自己 BBinder 转化为handle 并发给 svcmgr 了

记住:每个 service 在用户空间是不知道自己的 handle 的,但只有处在自己内核空间的 binder 知道(陷入binder后,binder 根据 current 查找 binder_proc,从而知道自己的 handle)

实际上:

  1. 每个进程 binder_proc 所记录的 binder_ref 的 handle 值是从 1 开始递增的
  2. 所有进程 binder_proc 所记录的 handle=0 的 binder_ref 都指向 svcmgr
  3. 同一服务的 binder_node 在不同进程的 binder_ref 的 handle 值可以不同,所以 binder_node 在用户层不可能知道自己的 handle,只有他的 proxy 知道其在其他 binder_node 中的 handle 值
4. 为什么用了 mmap() 后就只需要一次拷贝?

发送方的数据位于用户空间,binder 利用 copyfromuser 将其复制到内核空间(第一次),并直接写入接收方的 buffer,这一步不需要任何数据拷贝

5. 为何最后一步不需要任何数据拷贝?

对任何调用 binder_mmap 的进程 p,binder 会在 p 的用户空间内寻找一块用户指定大小的内存起始地址为 UA,同时在内核空间的也寻找一块相同大小的内存起始地址为 buffer,然后将它们同时映射到同一段物理地址(通过修改页表的方式),这样的话,无论修改 UA 还是 buffer,都能在不拷贝的情况下同时修改了对方

6. 碎碎念

术语:服务端叫服务svc,代理端叫pxy,客户端仅是代理端的包装叫cli

记住:binder driver是IPC机制,而搭建在上面的cpp和java framework(包括AIDL)才是RPC(同socket和其他rpc机制)

IInterface是所有服务接口都要继承的接口,服务的业务逻辑由这个接口定义

IBinder(类似Socket和SocketService)定义了服务端和客户端交互的接口transact,利用这个接口,客户端和服务端可以完成ipc(IBinder本身就包含了服务端地址,即handle)

因为任何服务都既要
(1)实现自己的业务逻辑,又要
(2)利用binder进行RPC,
因此上述两个接口都要实现

对于一个RPC framework而言,显然代理端是不需要手动实现的,仅需知道服务的IPC地址即可,在binder里就是服务的handle

BBinder是IBinder的服务端实现,BpBinder是IBinder的代理端实现,这两个类定义了利用binder这个ipc实现rpc的基本逻辑和框架,因此BpBinder只需知道一个handle就可以完成与服务端的通信
BpRefBase是客户端,即代理端的包装

BnIntetface和BpInterface基本上是两个鸡肋的类,分别继承自BBinder和BpRefBase,并没有实现任何方法,多加了一个类似回调的方法onAsBinder

实现一个服务Abc,需要
(1) 定义IAbc extends IInterface,即定义业务接口
(2) 定义 Abc extends BnInterface implemebts 同时实现业务逻辑,并附着ipc机制
(3) 其客户端,仅需 BpAbc extends BpInterface,构造时BpAbc(BpBinder(handle))

Stub是代理端,Stub.Proxy其实不是代理端,而是客户端,其代理端是BinderProxy

ProcessState: 每个进程一个单例
IPCThreadState: 每个线程一个单例,通过TLS实现,对ABD api进行了封装,用来实现更方便的IPC

References

What is an Activity?

One should know that, actually, Activity is not the graphical user interface, it is only a handler to the developer which helps

  1. create the graphical user interface i.e. the Window, and,
  2. deal with user actions (inputs)

The Activity creates the Window, i.e., the mWindow field typed PhoneWindow, and attaches the view hierarchy to it for drawing when developer invokes Activity#setContentView() method

Many methods in Activity are delegated to mWindow, e.g., Activity#{findViewById, getLayoutInflator, setContentView,}()

Similar to Activity, the Dialog and PopupWindow are also handlers doing similar things

Activity, Window, and View

Relationships

image

  • Each Activity has a mWindow field, typed PhoneWindow, initialized by Activity#attach(); the Activity does not manage the view hierarchy, it delegates that to mWindow
  • Each PhoneWindow manages only one view hierarchy, which is the DecorView, or mDecor field. mDecor is initialized by PhoneWindow#installDecor()
  • Each DecorView is a FrameLayout, and has a LinearLayout as its children; the LinearLayout is then divided to three parts vertically, of which the first is the status bar (mStatusGuard), and the second is the navigation bar (mNavigationGuard), and the third the app’s view (mContentRoot, including the title view, and the content view) presented to user
    • Title view: action bar is located here
    • Content view: rooted by a ContentFrameLayout, our self-written views are located as children of it (i.e.,inflated in Activity#setContentView())
  • Each DecorView also has a xml layout file, which is our theme

References

PhoneWindow and WindowManager

  • Activity#attach() only instantiates the PhoneWindow, i.e., attach the view hierarchy to it (i.e., to its DecorView), set its WindowManager, etc.; but in this process, the view has not by far added to WindowManager
  • Activity#makeVisible() proactively adds DecorView of PhoneWindow to WindowManager
  • Activity#setVisibility(false) only sets the visibility of DecorView, but not removes it from WindowManager

P.S. Activity#attach() is invoke by ActivityThread#performLaunchActivity(), and Activity#makeVisible() by ActivityThread#handleResumeActivity()

References

ViewStub

  • A ViewStub is a zero-sized invisible view that can be used to lazily inflate layout resources at runtime
  • The ViewStub is inflated and made visible once ViewStub#inflate() or ViewStub#setVisibility is invoked
  • The ViewStub disappears and is replaced by its inflated layout once inflated
    <ViewStub 
        android:id="@+id/stub" 
        android:inflatedId=“@+id/subTree” 
        android:layout=“@layout/mySubTree”
        android:layout_width="120dip" 
        android:layout_height="40dip" />
    
    ViewStub stub = findViewById(R.id.stub); 
    View inflated = stub.inflate();
    

References

Android Compilation Process and Binaries (APK, DEX, OAT, ODEX, VDEX, ART)

Compilation

image

  • Compiler
    • dex
    • d8
  • Optimizer (minimization, shrinking, obfuscation, …)
    • ProGuard & DexGuard
    • R8
  • Runtime
    • Dalvik Virtual Machine (DVM – JIT, Just-In-Time)
    • Android Runtime (ART – AOT, Ahead-Of-Time)

References

ART and DVM

image

At installation time, the installer will separate the .dex, native code and resources.
At the first time of running,

  • DVM will use dexopt to optimize the .dex file to a .odex file, e.g. using some efficient bytecode (opcode) (JIT)
  • ART will compile the .dex file into native code (.oat or .odex), i.e., the ELF file, and directly run it (AOT).

References

APK

On one hand, an .apk file is no more than a general zip, i.e., generally, as long as one wants to, one can put any type of file to it

On the other hand, an .apk file is a specific zip, where, the AndroidManifest.xml describes the semantic of the it, and semantically, the description of AndroidManifest.xml and files in the .apk satisfy the constraint of Android platform; generally, there are three type of file apk

  • manifest: the binary AndroidManifest.xml file
  • code: the dalvik byte code files
    • classes.dex, classes2.dex, ...
  • libraries: dependent libraries
    • lib/x86/: dependent x86 shared objects (ELF .so)
    • lib/x86_64/: dependent x86_64 shared objects (ELF .so)
    • lib/armeabi/: dependent arm32 shared objects (ELF .so)
    • lib/armeabi-v7a/: dependent arm32 (v7a stands for a version) shared objects (ELF .so)
    • lib/arm64/: dependent arm64 shared objects (ELF .so)
    • lib/arm64-v8a/: dependent arm64 (v8a stands for a version) shared objects (ELF .so)
    • lib/mips/: dependent mips shared objects (ELF .so)
  • resources: arsc file, and resources used
    • resources.arsc: a binary file illustrating the resources, including the resource table
    • res/: directory saving all resources used (exactly the res dir in developement)
  • others: other files, explained by the app and its dependent libraries, e.g.,
    • META-INF/: meta files, the de facto jar meta-inf
    • assets/: assets except resources
    • kotlin/: kotlin related files

An apk is not required to be runnable, it can have resources only like the overlays; actually, almost all things inside Android are packed as an apk, e.g., the Instrumentation test, the overlay

DEX, OAT, ODEX, VDEX, ART

  • .dex: DEX bytecode
  • .odex: optimized DEX bytecode, also .dex format but optimized (e.g., replacing some instructions to more-efficient instructions) by the deprecated on-device dexopt tool at installation time to get better interpreter performance
    • on newer versions, this file is an .oat ELF file but suffixed with .odex
  • .vdex: verified verbose DEX bytecode, also .dex format but with some more meta data to speed up verification; it contains a copy of the original .dex file
  • .oat: an ELF file that is generated by dex2oat, this file does not guarantee to contain AOT-compiled code
    • on some versions, .odex format generated by dex2oat may be an alternative to .oat format
    • in older versions, .oat (or .odex) contains a copy of the original .dex file; however, in newer versions (>= Android 8.0), the copy is moved to .vdex, and .oat (.odex) contains only the AOT-compiled code
  • .art: part of the runtime’s image (simply, a variant of heap dump of mirrored C++ objects) to speed up application startup (by avoiding the creation of art::mirror::Class etc.); it especially stores some well-known classes (e.g., java.lang.*)

References

The Android App Sandbox Mechanism (安卓应用沙盒机制)

Each app, when installed, is assigned a unique UID (android), and the UID is remained untouched until the app is uninstalled; for common user apps, the GID of each app is usually equal to its UID; and the same UID and GID also are assigned to files of the app

In other words, the file permissions of all files of the app is rwxrwx--x, so benefiting from the file access control of Linux, one cannot access files of others

And this is the sandbox mechanism of Android.

Try adb shell ls -l /data/dada with root permission (click to see output)
root@generic_x86:/data/data # ls -l
drwxr-x--x u0_a0    u0_a0             2020-06-02 13:35 com.android.backupconfirm
drwxr-x--x u0_a18   u0_a18            2020-06-02 13:35 com.android.backuptester
drwxr-x--x u0_a20   u0_a20            2020-06-02 13:36 com.android.browser
drwxr-x--x u0_a21   u0_a21            2020-06-02 13:35 com.android.calculator2
drwxr-x--x u0_a36   u0_a36            2020-06-03 20:10 com.android.camera
drwxr-x--x u0_a23   u0_a23            2020-06-02 13:35 com.android.captiveportallogin
drwxr-x--x u0_a3    u0_a3             2020-06-02 13:35 com.android.carrierconfig
drwxr-x--x u0_a24   u0_a24            2020-06-02 13:35 com.android.certinstaller
drwxr-x--x u0_a2    u0_a2             2020-06-02 13:35 com.android.contacts
drwxr-x--x u0_a25   u0_a25            2020-06-02 13:35 com.android.customlocale2
drwxr-x--x u0_a4    u0_a4             2020-06-03 20:10 com.android.defcontainer
drwxr-x--x u0_a26   u0_a26            2020-06-02 13:36 com.android.deskclock
drwxr-x--x u0_a27   u0_a27            2020-06-02 13:35 com.android.development
drwxr-x--x u0_a28   u0_a28            2020-06-02 13:35 com.android.development_settings
drwxr-x--x u0_a5    u0_a5             2020-06-02 13:36 com.android.dialer
drwxr-x--x u0_a29   u0_a29            2020-06-02 13:35 com.android.documentsui
drwxr-x--x u0_a19   u0_a19            2020-06-02 13:35 com.android.dreams.basic
drwxr-x--x u0_a30   u0_a30            2020-06-02 13:35 com.android.emulator.smoketests
drwxr-x--x u0_a31   u0_a31            2020-06-02 13:35 com.android.exchange
drwxr-x--x u0_a7    u0_a7             2020-06-02 13:35 com.android.externalstorage
drwxr-x--x u0_a32   u0_a32            2020-06-02 13:35 com.android.fallback
drwxr-x--x u0_a6    u0_a6             2020-06-02 13:35 com.android.gallery
drwxr-x--x u0_a56   u0_a56            2020-06-02 13:35 com.android.gesture.builder
drwxr-x--x u0_a34   u0_a34            2020-06-02 13:35 com.android.htmlviewer
drwxr-x--x system   system            2020-06-02 13:35 com.android.inputdevices
drwxr-x--x u0_a35   u0_a35            2020-06-03 20:10 com.android.inputmethod.latin
drwxr-x--x system   system            2020-06-02 13:36 com.android.keychain
drwxr-x--x u0_a9    u0_a9             2020-06-02 13:36 com.android.launcher3
drwxr-x--x system   system            2020-06-02 13:35 com.android.location.fused
drwxr-x--x u0_a10   u0_a10            2020-06-02 13:35 com.android.managedprovisioning
drwxr-x--x u0_a52   u0_a52            2020-06-02 13:36 com.android.messaging
drwxr-x--x radio    radio             2020-06-02 13:35 com.android.mms.service
drwxr-x--x u0_a40   u0_a40            2020-06-02 13:35 com.android.music
drwxr-x--x u0_a41   u0_a41            2020-06-02 13:35 com.android.netspeed
drwxr-x--x u0_a11   u0_a11            2020-06-02 13:35 com.android.packageinstaller
drwxr-x--x u0_a43   u0_a43            2020-06-03 20:10 com.android.pacprocessor
drwxr-x--x radio    radio             2020-06-02 13:35 com.android.phone
drwxr-x--x u0_a48   u0_a48            2020-06-03 20:10 com.android.printspooler
drwxr-x--x u0_a49   u0_a49            2020-06-02 13:35 com.android.protips
drwxr-x--x u0_a1    u0_a1             2020-06-02 13:36 com.android.providers.calendar
drwxr-x--x u0_a2    u0_a2             2020-06-02 13:35 com.android.providers.calllogbackup
drwxr-x--x u0_a2    u0_a2             2020-06-02 13:35 com.android.providers.contacts
drwxr-x--x u0_a6    u0_a6             2020-06-02 13:35 com.android.providers.downloads
drwxr-x--x u0_a6    u0_a6             2020-06-02 13:35 com.android.providers.downloads.ui
drwxr-x--x u0_a6    u0_a6             2020-06-02 13:35 com.android.providers.media
drwxr-x--x system   system            2020-06-02 13:35 com.android.providers.settings
drwxr-x--x radio    radio             2020-06-02 13:35 com.android.providers.telephony
drwxr-x--x u0_a2    u0_a2             2020-06-02 13:35 com.android.providers.userdictionary
drwxr-x--x u0_a12   u0_a12            2020-06-02 13:35 com.android.proxyhandler
drwxr-x--x u0_a50   u0_a50            2020-06-02 13:36 com.android.sdksetup
drwxr-x--x system   system            2020-06-02 13:35 com.android.server.telecom
drwxr-x--x system   system            2020-06-02 15:24 com.android.settings
drwxr-x--x u0_a13   u0_a13            2020-06-02 13:35 com.android.sharedstoragebackup
drwxr-x--x shell    shell             2020-06-02 13:35 com.android.shell
drwxr-x--x u0_a58   u0_a58            2020-06-02 13:35 com.android.smoketest
drwxr-x--x u0_a57   u0_a57            2020-06-02 13:35 com.android.smoketest.tests
drwxr-x--x u0_a51   u0_a51            2020-06-02 13:35 com.android.soundrecorder
drwxr-x--x u0_a14   u0_a14            2020-06-02 13:35 com.android.statementservice
drwxr-x--x u0_a15   u0_a15            2020-06-02 13:35 com.android.systemui
drwxr-x--x u0_a37   u0_a37            2020-06-02 13:35 com.android.vending
drwxr-x--x u0_a17   u0_a17            2020-06-02 13:35 com.android.vpndialogs
drwxr-x--x u0_a38   u0_a38            2020-06-02 13:35 com.android.wallpaper.livepicker
drwxr-x--x u0_a53   u0_a53            2020-06-03 20:10 com.android.webview
drwxr-x--x u0_a60   u0_a60            2020-06-02 13:35 com.android.widgetpreview
drwxr-x--x u0_a54   u0_a54            2020-06-02 13:35 com.example.android.apis
drwxr-x--x u0_a55   u0_a55            2020-06-02 13:35 com.example.android.livecubes
drwxr-x--x u0_a59   u0_a59            2020-06-02 13:35 com.example.android.softkeyboard
drwxr-x--x u0_a39   u0_a39            2020-06-03 20:10 com.google.android.apps.maps
drwxr-x--x u0_a44   u0_a44            2020-06-03 20:10 com.google.android.apps.photos
drwxr-x--x u0_a22   u0_a22            2020-06-02 13:36 com.google.android.calendar
drwxr-x--x u0_a47   u0_a47            2020-06-02 13:36 com.google.android.gm
drwxr-x--x u0_a8    u0_a8             2020-06-03 20:12 com.google.android.gms
drwxr-x--x u0_a16   u0_a16            2020-06-03 20:10 com.google.android.googlequicksearchbox
drwxr-x--x u0_a8    u0_a8             2020-06-02 13:36 com.google.android.gsf
drwxr-x--x u0_a8    u0_a8             2020-06-02 13:35 com.google.android.gsf.login
drwxr-x--x u0_a46   u0_a46            2020-06-03 20:10 com.google.android.play.games
drwxr-x--x u0_a33   u0_a33            2020-06-02 13:35 com.google.android.syncadapters.contacts
drwxr-x--x u0_a45   u0_a45            2020-06-03 20:10 com.svox.pico
drwxr-x--x u0_a42   u0_a42            2020-06-03 20:10 jp.co.omronsoft.openwnn

The same also happens to the permission control of Android

  • Resources of different permissions have different GIDs
  • For app which would like to access a specific permission, Android adds the app to that group

One can see apps’ metadata via adb shell cat /data/system/packages.xml

And for a specific app (process), one use adb shell cat /proc/<pid>/status, where the GIDS field is the GIDs of permission this app asking

References

Text Input, IMF and IME

Overview

  • Text input is the result of the synergy of two essential components: an Input Method Engine (IME), and an editor.
  • An IME can be a soft keyboard, a hard keyboard, a speech-to-text engine, a handwritten-recognizer, etc. In implementation, It is a service extending InputMethodService, who allows users to generate text, and send them to the editor.
  • The editor is the component that receives text, and displays them. Typically, the editor is EditText. But there are cases developers use customized editors. Editor receives commands through InputConnection interface, and sending commands through InputMethodManager. Customized editors should start by implementing View#onCreateInputConnection() to return its own InputConnection.
  • Input Method Framework (IMF) is the architecture in Android to handle text input from user

References

IME Workflow

In Android, an IME is an application that contains a special IME service, and IME is in essence a service that (1) running in background, and at the same time (2) presenting an UI to user for interacting with the service

IME works in following,

  1. IME presents user its UI, and user interacts with it
  2. the interaction is passed to the IME service
  3. the service takes the interactions, translates them (e.g., a handwritten recognizer), and passes the translation to the editor

IME Services

The IME service is a plain service extending InputMethodService, but one has to config it with special intent filters (i.e., android.view.InputMethod) and tags (e.g., BIND_INPUT_METHOD permission) in the AndroidManifest.xml to help the system recognize it

Lifecycle:
❓❓❓❓❓❓❓❓

References

IME UI

The IME UI resides in a specific window which is different from the window where the activity resides.

In implementation, IME UI is in essence a Dialog (named SoftInputWindow extending Dialog) which is created by InputMethodService#onCreate(), residing in a standalone window, and the UI is the content view of this Dialog

References

ViewParent, ViewRootImpl

  • ViewParent: Defines the responsibilities for a class that will be a parent (not the parent concept of a view layout tree 🤔 Is this true?) of a View. This is the API that a view sees when it wants to interact with its parent.
  • ViewRootImpl: View is only responsible for presenting itself to users, and WindowManager is only responsible for managing a window, and it is the ViewRootImpl’s responsibility to communicate between the two, i.e., ViewRootImpl implements the communication protocol between the root View of a hierarchy (the mView field of ViewRootImpl, usually, it is the DecorView) and WindowManager

References

Resource Qualifier

A good app should provide different resources (e.g., layout, drawable) on different devices, and Android loads them according to device configurations automatically and correctly

For such purposes, app developers name resources by appending a qualifier after primary name (e.g., layout-en-GB-sw360dp-port.xml), and Android loads them by a best match of the qualifier and device configs

image

The above shows the resource qualifier matching algorithm, and the insight behind the qualifier best matching algo is eliminating all non-matching or less-matching resources (see the figure and reference links for more details)

References

Zygote

Overview

  • A special process handling the forking of each new Android application process
  • Cold boot a VM, and load all shared Java classes and resources into memory
  • When a new application needs starting, fork a new process to handle it (the VM now is warmed up, so it is fast)
  • Use CoW mechanism

To start Zygote:

/system/bin/app_process -Xzygote /system/bin --zygote --start-system-server

Zogote Start Process

image

  • How init.rc is configured?
  • How to start it?

❓❓❓❓❓❓❓❓
TO BE ADDED
❓❓❓❓❓❓❓❓

Zygote and System Server

System server is started by Zygote; it is the --start-system-server option as one can see from the previous start command.

  • Why do we need Zygote to start System Server? Why not System Server start itself?
    ❓❓❓❓❓❓❓❓
    TO BE ADDED
    ❓❓❓❓❓❓❓❓

References

Fastboot, Sideload, and Flashing ROMs

Read also #36 and the official document

The Sophisticated Three-"System" Device

Informally, one can treat an Android devices as a three-"system" device, i.e., the bootloader, the recovery, and the normal Android system

  • bootloader: bootloader is the first "system" that typically presents to ROM users, and is the loader program which helps to load recovery or Android; it is mainly responsible for managing all data and bootable partitions on the device, e.g., flash a new partition, flash a new recovery, etc. And one can use adb reboot bootloader to boot into it (on some devices, one can hold volume up + volume down + power to boot into it)

    for bootloader to function normally, one need to unlock it firstly by
    (1) in developer options, check the "OEM lock"
    (2) boot into bootloader via abd reboot bootloader
    (3) after bootloader is started, use fastboot flashing unlock to unlock the bootloader (or fastboot oem unlock for old devices)
    (4) after (3), if one see the "Device Status" becomes "unlocked", it succeeded;
    but note,
    (1) the first time one unlocks the bootloader, everything on data partition will be wiped
    (2) not all bootloaders can be unlocked, some OEM (e.g., MeiZu) locks their bootloader and forbit anyone from unlocking

  • recovery: recovery is the second "system" that typically presents to ROM users, and is the program which is responsible to backup/recover data, or renew/reinstall an Android ROM (see #51 for more information); one can use adb reboot recovery to boot into it, or firstly boot into bootloader, then ask bootloader to load recovery (on some devices, one can hold volume down + power to boot into it)

    users who like to play devices often replace the default recovery to TWRP or LineageOS Recovery

  • Android: this is the last and actual system that present to normal users, providing all kinds of functionalities; it presents to user because the bootloader by default load it instead of recovery

    users who like to play devices often install new ROMs to replace the original OEM ROM, e.g., LineageOS

Normally, the device boots into the Android system either when use the power button or type adb reboot. However, one can also reboot into the other two systems:

  • bootloader: adb reboot bootloader
  • recovery: adb reboot recovery
  • Android: adb reboot

In real world, the partitions and booting sequences of Android is much more complicate than aforementioned. Read also #36 and Android booting shenanigans for further information.

fastboot, sideload

fastboot

fastboot is a protocol and tool that is used to interact with bootloader via USB cable; it can do anything that bootloader allows, e.g., lock/unlock bootloader, erase/format/flash partitions, boot from a ramdisk (see #52 to know more about image disks), etc. So the device need to reboot to bootloader first (someone also use the term "reboot into fastboot mode").

  • to list all bootloader devices (-l list the device paths)
$ fastboot devices -l

Note, this command shows nothing when you haven't rebooted into fastboot mode. All following commands also wait for a device to reboot into bootloader before do anything.

  • to lock/unlock bootloader
$ fastboot flashing lock
$ fastboot flashing unlock

# for old devices
$ fastboot oem lock
$ fastboot oem unlock
  • to erase, format a partition
# PARTITION can be boot, data, cache, ...
$ fastboot erase PARTITION
$ fastboot format[:FS_TYPE[:SIZE]]
  • to flash new images to a partition
# PARTITION can be boot, and RAMDISK can be for example 
$ fastboot flash PARTITION RAMDISK

Typically on A/B update devices (see #51), flash new images to boot send the images to boot_b slot, the following is the output of fastboot flash boot lineage-17.1-20200602-recovery-fajita.img on OnePlus6T

$ fastboot flash boot lineage-17.1-20200602-recovery-fajita.img
Sending 'boot_b' (65536 KB)                        OKAY [  1.820s]
Writing 'boot_b'                                   OKAY [  0.288s]
Finished. Total time: 2.116s
  • to boot from a ramdisk
# RAMDISK can be any ramdisk made, e.g., twrp recovery ramdisk
$ fastboot boot RAMDISK

sideload

Sideloading is a mechanism of recovery that makes it easy to manually install a new ROM.zip to the device from PC instead of via the over-the-air system or SD card. This mechanism is by default disabled, and can be enabled in recovery (but disabled immediately the ROM is sideloaded)

Also sideload is a subcommand of adb, but a separate mode of adb, note:

  1. sideload works only by USB cable
  2. when such mode is enabled from recovery, common adb commands (logcat, reboot, shell, push, pull) don't work

To use sideload, one have to
(1) boot to recovery via adb reboot recovery
(2) in recovery, enable adb-sideload. For example, in TWRP goto "Advanced" then "ADB Sideload", in LineageOS goto "Apply Updates" then "Apply from ADB"
(3) use adb sideload ROMZIP to sideload the rom (copying the zip from PC to device), and the updater script in ROMZIP (META-INF/com/google/android/updater-script) will be executed

the updater script is not a bash script, but an edify script language, see references for how to write it

References

Steps to Flash New ROMs

A typical steps to flash and install new ROMs involves the following steps:

  1. unlock the bootloader if locked
  2. if necessary, flash a temporary recovery
  3. reboot to recovery
  4. wipe data and cache (双清)
  5. sideload a new ROM
  6. reboot to the new system

See references for a detailed step of "Installing LineageOS 17 to OnePlus 6T"

References

ABD - Driver Layer

mmap()

How ordinary mmap() works 🤔 TODO: Learn more

Why mmap()?

However, there are many other IPC implementations in Linux, then why mmap() instead of others e.g., pipe, shared memory, signal, semaphore, message queue?

TLDR: for performance, mmap() enforces only 1 copy (across kernel and user space), while others 2

  • When a process, say A, communicates with the other say B, the first thing A to do is to trap into the kernel, and copy the data in the user space to kernel space using copy_from_user(), this is where the only cross-spce copy happens
  • For other mechanisms, the binder driver has to copy the data from the kernel space to user space of B using copy_to_user(), and finishes the communication, however for mmap(), because both A and B has mapped a user-space memory block

Android Binder Driver Arch

image

ABD is implemented with help of binder driver and service manager

The driver here serves as a router, that router a specific request from the client to a specific service using the service handle

The service manager here serves as a name server, that

  1. registers and unregisters services using the service name and the service handle
  2. translates the service name to the service handle

binder driver

Binder driver runs in kernel space, and it knows all services that registered to it (know their addresses, not names)

References

service manager

The service manager manages the translation of all system services, not custom services

Custom services implemented by AIDL are fetched via Context#bindService()

ServiceManager(Java) is a client of servicemanager (cpp), and additionally, it provides a cache, for reducing requests to servicemanager

For easy of use, we use in following

  • servicemanager for that of the cpp level
  • ServiceManager for that of the Java level
  • Service Manager for the concept

⚠️ TO BE ADDED

References

Android App Link

Overview

Android App Link is a specific type of Deep Link that is verified by app developers and Android.

Each Android App Link is associated with a http/https domain, which is the official website of this app (One can specify multiple http(s) domains that are associated to the same app, and one can even set custom schemes, but a http(s) is mandatory when custom schemes are used).

References

Remote App Invocation

⚠️ TO BE ADDED

References

Class Loaders on Android

Read also #26

The class loading mechanism (when and how) on Android is same as on Java, but uses different class loaders

image

Above figure shows the inheritance hierarchy of class loaders on Android

  • The BootClassLoader is responsible for loading boot classes under BOOTCLASSPATH. For example classes like View/Context in framework.jar, and others under /system/framework/.
  • The BaseDexClassLoader can load classes from any apk/dex/jar files.
  • The PathClassLoader is the default system class loader, it restricts BaseDexClassLoader to only load classes from installed apk/dex/jar files (inside /data/app, your app classes are loaded by it). For example, classes user defined (either self-defined data classes, or classes extending from framework like CustomApplication extends Application), classes user used in 3rd-party libraries like okhttp. It is a customized DexClassLoader. One can verify that via getClass().getClassLoader() in his application class
  • The DexClassLoader faithfully inherits functions from BaseDexClassLoader.

Actually, both BootClassLoader and BaseDexClassLoader are FAKE class loaders since they do not actually find classes from somewhere else and define the class using defineClass(). Instead, they call ART's ClassLinker to find and load that, where the latter manages all class loaders as well as all classes. Each Java-level ClassLoader has a associated mirror::ClassLoader in the Native-level --- it has a class_table_ field, saving all classes of the class loader loaded. However, BootClassLoader does not have a corresponding mirror since it is FAKE; all boot classes are saved in ClassLinker.boot_class_table_.

  • BootClassLoader#findClass() -> Class.classForName() -> ClassLinker#FindClass() -> ClassLinker#DefineClass()
  • BaseDexClassLoader#findClass() -> DexPathList#findClass() -> DexFille#loadClassBinaryName() -> ClassLinker#DefineClass()

where ClassLinker#DefineClass() defines an overall class loading process: load, resolve, verify, and initialize.

In Java, any ClassLoader should rewrite their own findClass() to find class bytes from elsewhere, and call defineClass() to let JVM like HotSpot register that class. However in Android, since classes are saved in .dex as DEX and class bytecode are not supported, a general correct workflow is to extend from BaseDexClassLoader/PathClassLoader/DexClassLoader whose findClass() is redirected to ART. ClassLoader#defineClass() would generally throw UnsupportedOperationException(“cannot load this type of class file”).

References

Example: Xposed

One can only find a class in its own class loader, and cannot find it or can only find a different in other class loaders. Xposed is a hooking framework in Android, it loads each of its module in separate DexClassLoaders, and thereby

  • modules are sandboxed
  • different modules can use the same library with different versions, i.e., okhttp and okhttp3, with no confusions
  • modules and apps are separated

Therefore, hooking modules can hook methods in framework classes in a easier way like:

XposedHelpers.findAndHookMethod(View.class, "toString", new XC_MethodHook() {
    @Override
    protected void afterHookedMethod(MethodHookParam param) {
        // hooking code
    }
});

However, hooking modules, if would like to hook user-defined or 3rd party library classes of the app, modules have to get the PathClassLoader firstly, then uses this class loader to find the hooked method, for example:

XposedHelpers.findAndHookMethod(Application.class, "attach", Context.class, new XC_MethodHook() {
    @Override
    protected void afterHookedMethod(MethodHookParam param) {
        Context context = (Context) param.args[0];
        // context is the user-defined application class, so the 
        // class loader is the default PathClassLoader
        ClassLoader pathClassLoader = context.getClassLoader();

        Class hookedMethod = pathClassLoader.loadClass("com.example.Example");
        Class parameter1 = pathClassLoader.loadClass("com.example.Parameter1");
        Class parameter2 = pathClassLoader.loadClass("com.example.Parameter2");
        XposedHelpers.findAndHookMethod(hookedMethod, "methodName", parameter1, parameter2, new XC_MethodHook() {
            @Override
            protected void afterHookedMethod(MethodHookParam param) {
                // hooking code
            }
        });
    }
});

The following example checks the process/thread xposed module resides in (actually in app process and ui thread)

Log.d(TAG, "mod-process {")
Log.d(TAG, "  .pid = ${Process.myPid()}")
Log.d(TAG, "  .tid = ${Process.myTid()}")
Log.d(TAG, "  .uid = ${Process.myUid()}")
Log.d(TAG, "  .loader = ${Process::class.java.classLoader}")
Log.d(TAG, "}")
Log.d(TAG, "mod-thread {")
Log.d(TAG, "  .id = ${Thread.currentThread().id}")
Log.d(TAG, "  .name = ${Thread.currentThread().name}")
Log.d(TAG, "  .loader = ${Process::class.java.classLoader}")
Log.d(TAG, "}")

XposedHelpers.findAndHookMethod(Application::class.java, "attach", Context::class.java, object : XC_MethodHook() {
    override fun beforeHookedMethod(param: MethodHookParam?) {
        val app = param?.thisObject as Application
        val context = param.args[0] as Context
        val pathClassLoader = context.classLoader

        val looperThread = context.mainLooper.thread
        Log.d(TAG, "looper-thread {")
        Log.d(TAG, "  .id = ${looperThread.id}")
        Log.d(TAG, "  .name = ${looperThread.name}")
        Log.d(TAG, "  .loader = ${looperThread::class.java.classLoader}")
        Log.d(TAG, "}")

        val processClass = pathClassLoader.loadClass("android.os.Process")
        val myPid = processClass.getDeclaredMethod("myPid")
        val myTid = processClass.getDeclaredMethod("myTid")
        val myUid = processClass.getDeclaredMethod("myUid")
        val pid = myPid.invoke(null) as Int
        val tid = myTid.invoke(null) as Int
        val uid = myUid.invoke(null) as Int
        Log.d(TAG, "app-process {")
        Log.d(TAG, "  .pid = $pid")
        Log.d(TAG, "  .tid = $tid")
        Log.d(TAG, "  .uid = $uid")
        Log.d(TAG, "  .loader = ${processClass.classLoader}")
        Log.d(TAG, "}")

        val threadClass = pathClassLoader.loadClass("java.lang.Thread")
        val currentThread = threadClass.getDeclaredMethod("currentThread")
        val mainThread = currentThread.invoke(null) as Thread
        Log.d(TAG, "app-thread {")
        Log.d(TAG, "  .id = ${mainThread.id}")
        Log.d(TAG, "  .name = ${mainThread.name}")
        Log.d(TAG, "  .loader = ${threadClass.classLoader}")
        Log.d(TAG, "}")
    }
})

Logging of ANR, Crash, and Native Crash

ANR

An ANR will be triggered for your app when one of the following conditions occur:

  • While your activity is in the foreground, your app has not responded to an input event or BroadcastReceiver (such as key press or screen touch events) within 5 seconds.
  • While you do not have an activity in the foreground, your BroadcastReceiver hasn't finished executing within a considerable amount of time.

Shown in file system on device: /data/anr/<filename>

References

Crash

image

Shown in logcat

A stack trace shows two pieces of information that are critical to debugging a crash:

  • The type of exception thrown.
  • The section of your of the code where the exception is thrown.

References

Native Crash

image

Shown in tombstone files in file system on device: /data/tombstones/

References

Runtime Resource Overlay (RRO) Mechanism (运行时资源替换机制)

RRO Overview

Each android app can have multiple set of resources; each set is packaged into a single apk, which is called an overlay, and overlay has priority

image

Above shows how the Android system looks for a resource. Specifically, when finding a resource via its id (i.e., R.id.xxx), the system will search the overlays according to the overlay priority, and return the first searched one (i.e., if multiple resources has the same id, the one with largest priority is chosen)

To see all overlays of a same app, use

$ adb shell cmd overlay list <pkgName>

To enable an overlay of an app, use

$ adb shell cmd overlay enable <overlayPkg>

References

Overlay Management

OverlayManager and OverlayManagerService

❓❓❓❓❓❓❓❓❓❓

View Hierarchy Layout Mechanism

Overview

After an activity is launched, created, and started, the view hierarchy inside is created in memory, and ready to be drawn; the drawing starts very from ActivityThread#handleResumeActivity()

The drawing of the hierarchy is sequentialized to three passes:

  • measure: this pass measures the size (width, height) of each view in hierarchy according to the each view’s layout params and parent measurement specs; in this pass, View#onMeasure() is invoked; after this pass, getMeasured{Width, Height}() of View (along with all its children) should return correctly
  • layout: after measured, the view is calculated to where it should be put on the screen (left, top, width, height), according to the device configuration, like screen size, orientation, etc.; in the process, View#onLayout() is invoked; after this pass, get{Left, Top, Width, Height, Right, Bottom}() (along all its children) should return correctly
  • draw: after layout, the view is actual ready to be drawn to the screen; but according to whether the actual drawing is accomplished via GPU or not, this pass is classified to software drawing (swdraw) and hardware accelerated drawing (hwdraw), and after Android 4.+, hwdraw is set as the default; in this pass, View#onDraw() is invoked
    • Swdraw is performed by Skia (actually, skia can do hwdraw, but Google only use it to do swdraw) directly draws the view to the screen synchronously, in implementation, it directly invokes View#draw() recursively
    • Hwdraw is performed via OpenGL (actually, Google prepared to abandon OpenGL and use Skia to do hwdraw). Hwdraw only requires the hierarchy to emit its draw operations one by one. In detail, hwdraw abstracts each view in the hierarchy as a RenderNode, and the drawing performed on them are DrawOps they emit; the hierarchy is then formed to a render node tree, and operations a draw operation list; therefore, in this pass, View#onDraw() is invoked only to emit draw operations but does not actual draw, and it is the render thread's responsibility to sync these operations to OpenGL, and OpenGL will render them via GPU. The following image presents the pass.

image

In implementation, ViewRootImpl#performTraversal() performs all the three passes recursively by traversing the hierarchy top-down

References

ViewRootImpl#performMeasure()

image

The measure pass measures the measured height and width of each view in the hierarchy, therefore after measurement, the measured height and width (View#getMeasured{Width, Height}())of each view should be set correctly, or an IllegalStateException is thrown

Actually, performMeasure() only invokes the measure() method of the view it manages (in practice, it is the decor view), therefore, for a successful measurement, all ViewGroups must invoke measure() of its children recursively

View#measure() is called to find out how big a view should be, but the actual measurement is performed by View#onMeasure(), and measure() does caching, and checks whether measured height and width are set

Generally, every ViewGroup if it has children must override onMeasure() to recursively invoke its children’s measure() method

View#onMeasure()

Height and width of each view comes from both (1) restrictions from its parent (MeasureSpec), and (2) wishes from its users (LayoutParams)

  • MeasureSpec: it is restrictions from its parent in the layout tree, this spec tells the view which restrictions should be satisfied (e.g., the parent may tell its children the most width it can be)
  • LayoutParams: it is the wishes from its user (set in xml or by LayoutParams), e.g., the user may tell that he wants the view to match the width of its parent, or to be a fixed value like 100px
    Therefore, View#onMeasure(widthMeasureSpec, heightMeasureSpec) always judges the users wishes, and adjusts them to satisfy the parent’s restriction; and the values after adjusted is the measured values
    • e.g., the onMeasure() method of LinearLayout (vertical) should divide the height and generate the MeasureSpecs for each of its children

See View#{onMeasure, getDefaultSize}() for more details

Root Spec

The root MeasureSpec set for the view managed by ViewRootImpl comes from the LayoutParams of the window it communicates to

ViewRootImpl#getRootMeasureSpec() generates that, in detail

  • it generates an EXACTLY MeasureSpec (equals to width of the window) for a MATCH_PARENT LayoutParams
  • it generates a fixed-value MeasureSpec (equals to width of the window) for a fixed-value LayoutParams
  • it generates an AT_MOST MeasureSpec (at most width of the window) for a WRAP_CONTENT LayoutParams

See ViewRootImpl#getRootMeasureSpec() for more details

ViewRootImpl#performLayout()

image

The measure pass sets a size and position to each view in the hierarchy, therefore after layout, the left, top, height and width (View#get{Left, Top, Width, Height, Right, Bottom}())of each view should be set correctly

Actually, performLayout() only invokes the layout(t, l, r, b) method of the view it manages (in practice, it is the decor view), therefore, for a successful layout, all ViewGroups must invoke layout() of its children recursively

View#layout() is where the actual layout is performed, it is called to set position and size of a view. layout() additionally checks whether a relayout is needed to improve performance, and if so, callback onLayout() to inform the view

Generally, every ViewGroup if it has children must override onLayout() to recursively invoke its children’s layout() method

View#layout()

layout() is where the actual layout is performed, it is called to find out to where a view is put and how big a view actually be.

layout() by default directly sets the measured width/height as the width and height of the view

Additionally, layout() checks whether a relayout is needed to improve performance, and if so, callback View#onLayout() to notify the view (group to relayout its children)

Root Left/Top and Width/Height

The root left/top and width/height comes from the DecorView, it is the 0/0 and decorView.getMeasured{Width, Height}()

The reason for left/top being 0/0 is that the mLeft/mTop/mRight/mBottom of a View is a relative value to its parent, and DecorView located at (0, 0) of the window

ViewRootImpl#performDraw() and View#draw()

image

ViewRootImpl#performDraw() invokes the draw() of its managed view

The View#draw() method does many works in a specific order:

  1. draw the background (the android:background in xml)
  2. if necessary, draw the canvas’ layers to prepare for fading (animation)
  3. draw the view’s content by callback to View#onDraw(), this is where the draw is actually happened
  4. draw its children; one can see from here, that ViewGroup draw its children in draw, and the draw method is not supposed to override
  5. if necessary, draw the fading edges (animation)
  6. draw decorations, e.g., the scrollbar

View#onDraw(canvas) is where draw actually happened

Generally, every View if it has something special should override onDraw() to emit draw commands

Summary

To implement a custom ViewGroup:

  1. It is responsible to override onMeasure() and onLayout() to manage the measure and layout of its children, by recursively invoking its children’s measure() or layout() method, for them to show at the correct position, and have the right size;
  2. it does not need to recursively call anything for its children to draw, because the draw() method itself already does that; if it has something special to draw, can override onDraw() and draw that

To implement a custom View:

  1. if it has something special to draw, override onDraw() and draw that
  2. it usually does not need to override onMeasure() and onLayout(), the default version is sufficient for them; but if it has some special positions, it needs that

The default View#onMeasure() (actual measurement) sets the measured values to be the minimum one, however, the default View#onLayout() (just notifier, but can also do layout) and View#onDraw() (actual draw) does nothing because there’s nothing to be notified and drawn

References

Binder in Computer Science

Concept

Binder, in computer science, is a design pattern that is used for dynamic binding (动态绑定) mechanism

  • The static binding directly hard codes the address for example IP address into the code, however, dynamic binding writes an human readable and meaningful alias of the address into the code, and translates the alias to its address at runtime
  • The address is not restricted to IP addresses

Usually Binder pattern needs a manager (or name server) as a mediator for managing the translation process

Examples of Binder pattern:

  • URL and IP, the DNS server is the manager
  • Large distributed systems, the name servers is the manager
  • Android Binder, the ServiceManager is the manager

Why Binder Patterns?

  • Flexibility
    • Servers with same interfaces can be registered using same names, and managers can help load balancing
    • Servers can be added and removed dynamically without impacting the client
  • Fault tolerance
    • If a service server is not good enough, and always crashes, the manager can help restarts it, which eases the client
  • Others
    • As a load balancer, for balancing the queries to servers with same interfaces
    • As a validator, for validating the version consistency between client and server

The Binder Workflow

The workflows consists of 4 steps:

  1. Register: the server starts itself, and registers itself to manager using its name and address
  2. Translate: the client requests manager for the address of a service by the service name; then manager translates the name to its address, and returns to client
  3. Communicate: the client communicates with the service using the address
  4. Unregister: the server unregisters itself to manager

Input Events

  • Motion events (MotionEvent)

  • Key events (KeyEvent)

    • Physical: event from hardware, e.g., power button
    • Soft: event from soft keyboard (see create an input method)
  • Sensor events (SensorEvent, or specific event)

    • Low-level: SensorEvent managed by SensorManager (see sensors overview)
    • Hight-level: e.g. LocationManager, CameraManager has specific handler

System Server

Overview

System server is the host process of almost all system services; any system service inside lives either in the main thread of system server (in following, we user system server for the concept, and SystemServer for the java class), or in a standalone thread

Service categories:

  • Bootstrap services: AMS, PMS, …
  • Core services: Battery, UserStats, …
  • Other services: IMS, Watchdog, Clipboard…

References

Starting Process

How system server is started?

  • by zygote, and why 🤔 TODO: Learn more

Android Directories and Partitions

Directory Layout

/data/ ...................... user data files, some (un)system commands save and read data from here; the mount point of userdata.img
 + anr/ ..................... anr traces generated
 + app/ ..................... once installed, apk related files (apk, dex, oat) are copied here
 + data/ .................... this is where apps put their data via Context#getDir(), such as shared_pref, cache; and this is how the Android Sandbox mechanism implements, dirs here has a unique UID and GID, which is their apps' UID, GID, and cannot accessed by others
 + system/
   - overlay.xml ............ overlay metadata, list overlays in the system
   - packages.{xml, list} ... package (app) metadata, including permissions, etc.
 + vendor/ .................. this is used to save vendor-specific data
   + overlay/ ............... vendor overlay apks, including system ui overlay
 + local/
   + tmp/ ................... a temp directory, public to all
 + tombstones/ .............. native crash tombstones
 + user/ .................... internal directories of each app under each user
   + <userid>/ .............. internal directories of apps belonging to user with <userid>, starting from 0

/system/ .................... the operating system where installed; the mount point of system.img
 - build.prop ............... system build properties, consumed by getprop or setprop
 + etc/ ..................... system misc configurations
 + lib/ ..................... system dependent libraries (.so)
 + bin/ ..................... system binaries, e.g., toolbox, app_process, dalvikvm
 + xbin/ .................... system special-use binaries, usually for administrators, e.g., tcpdump, dexdump, daemonize
 + app/ ..................... system apps, usually the firmware-bound 2nd- or 3rd-party apps, e.g., Calculator, Calendor
 + priv-app/ ................ privileged system apps that are required for each phone, they are either official apps, or apps from firmware (e.g., Samsung), cannot be uninstalled, e.g., Contacts, Settings
 + framework/ ............... system level and framework dependent jars, dexes, the default CLASSPATH
   - framework-res.apk ...... framework default resources
   - framework.jar .......... framework java archive
   - monkey.jar ............. monkey tool

/sbin ....................... critical binaries, statically linked, and can be used even when /system is not mounted
 - adbd
 - watchdogd
 - healthd
 - ueventd 

/storage/
 + emulated/ ................ external storage
   + <userid>/
     + Android/ ............. app's external storage, shared by all users, with 0777 permission for each (no sandbox)


/sdcard/ .................... sdcard storage

Important Directories

/system/bin/ .......................... Binaries
/system/framework/ .................... Framework and dependent jars, dexes, the default CLASSPATH
/data/app/ ............................ Installed apps (including APKs, OATs, ODEXes, VDEXes, ARTs), sandboxed
/data/data/ ........................... Internal private app storage directories, sandboxed
/storage/emulator/<userid>/Android/ ... External public app storage directories, no sandboxing

Partitions

For data separation and security, one always divides the full disk to multiple partitions, they are either used as a data partition, or a bootable partition that can install a OS. Therefore, to manage the partitions, a Disk Partition Table (DPT) is require to record each partition's (1) type, (2) start and end sectors in disk, and (3) other meta information like rw permissions. And different design of the table leads to different partition standards. Currently there are two partition standards that are commonly used in today: MBR (Master Boot Record), and GPT (Globally Unique Identifier Partition Table)

MBR

MBR

MBR allocates 64 bytes to DPT, and 16 bytes to each entry of the table, therefore, MBR can only divide the disk to 4 partitions, and can manage at most 2T (2^16 * 512) disk space

GPT

GPT

GPT designs a flexible DPT and makes it capable of dividing the disk to as many partitions as possible. Also, to ensure MBR compatibility, it reserves a PMBR part, bootloaders that only support MBR can use the PMBR to boot the OS, and others will dump directly to GPT header by ignoring PMBR

References

Android Partitions

Android selects GPT

  • the chief concerns of mobile devices is that they must always be repairable, and so some type of "recovery mode" must be enabled on them --- Android Internals: The Power User's View

Partitions of Android

  • boot: this partition is flashed the AOSP Kernel Layer; it includes all the required to boot the phone, e.g., ramdisk.img; this partition can never be wiped until one prepares to install a new ROM; this partition is not mounted, thereby invisible to users
  • system: this partition is flashed all other AOSP layers except Kernel; this partition is mounted to /system on device
  • userdata: this partition is created for users to put their data, including data generated by frameworks, tools, apps, etc; this partition is mounted to /data on device
  • recovery: this partition can be seen as a second boot partition that can be booted into; it is used to perform recovery; actually, this partition maintains a minimal linux that run a special program called recovery; one often rewrites this partition by popular recovery systems like TWRP, and uses this partition to do OTA (on-the-air) update, flash new ROMs, install Magisk, etc; this partition is not mounted, thereby invisible to users

    note, for devices that use A/B System Update, e.g., OnePlus 6T, the recovery partition maybe merged into the boot partition; see more about system update mechanism in #51

  • bootloader: this partition includes the bootloader; the primary bootloader firstly checks the hardware, and then starts secondary bootloader to select which system to boot into (the recovery? the Android? or any others?); this partition is not mounted, thereby invisible to users

    also, one can treat bootloader as a meta-recovery; Android defines a fastboot protocol with bootloader to help bootloader operate all other partitions; this protocol is accomplished with help of fastboot tool, and the USB cable (for data transmission); and to ensure the security, fastboot can be used only when the bootloader is unlocked (see #50)

  • other partitions:
    • sdcard, cache, radio (or modem), ...

Difference between bootloader (fastboot) and recovery

copied from https://stackoverflow.com/questions/31158021/difference-between-fastboot-and-recoverymode-in-android

  • Recovery mode is small linux operating system where you can factory reset your deivce or update your device with vendors images.
  • Fastboot is a protocol and client tool of bootloader, it uses USB cable for data transmission; it's a tool which comes with the android sdk and you can use it to re-flash partitions of your device. Because fastboot starts before android and even when android isn't installed you can use it as an alternative of recovery mode in case recovery mode partition is corrupted. Every phone usually has fastboot,but some vendors has choose to replace fastboot with their tool. Like samsung, instead of fastboot it has Odin.

References

Binder Basics

Overview

image

Binder is an implementation of RPC and IPC in Android for better performance, it uses shared memory as its IPC mechanism:

  • Intent
    • The highest abstraction of Binder
  • AIDL
    • Android Interface Definition Language
    • An abstraction of Binder for ease of use
  • Binder
    • The java Layer, cpp layer, and the driver layer
    • The kernel driver is implemented as a misc device under /dev/binder, controlled via ioctl()

References

Binder RPC

image

The java and cpp layer of binder implements RPC:

  • AIDL defines interfaces and data that Proxy and Stub uses
  • Caller, as client-side, called methods defines by callee who is server-side
  • the two binder layers marshals, transmits to callee, and unmarshals the parcelable data
  • Each callee has a poll of binder thread to receive and handle the transmitted data (just like a web-server)
  • The above process is by default synchronous, but one can use ONEWAY_FLAG to set the communication to be asynchronous

References

Use AIDL

Server-Side

  • Write an IExampleManager.aidl to define the interface (IExampleManager) and parcelables (data, possibly be used in data transmission) that are used during binder RPC
  • Create a class ExampleManagerService extending IExampleManager.Stub, implement the methods
  • Use it
    • For system-wide services, register it when system starts, by service manager in system server
      // implementing ExampleManagerService as a singleton
      ServiceManager.addService(“example”, ExampleManagerService.getInstance())
      
    • For general services, implement ExampleService extending Service, and return the instance of ExampleManagerService in Service#onBind()

Client-Side

  • For system-wide services like am/wm/pm, use it by
    IExampleManager em = IExampleManager.Stub.asInterface(ServiceManager.getService(“exmaple”))
    
  • For general services, by
    IExampleManage em = null
    ServiceConnection conn = new ServiceConnection(binder) {
       em = IExampleManager.Stub.asInterface(emBinder)
    }
    Intent intent = Intent(XxxxActivity.this, ExampleService.class)
    bindService(intent, conn)
    

References

Event Dispatching Mechanism

Overview

The dispatching of each touch event is started from the dispatching of an ACTION_DOWN MotionEvent, and ended by the dispatching of an ACTION_UP MotionEvent or an ACTION_CANCEL MotionEvent

In the dispatching process, once,

  • a view decides to consume the touch event (by returning true), the dispatching process stops, and all events following the very beginning ACTION_DOWN MotionEvent will be dispatched to it
  • a view decides not to consume the touch event (by returning false), the dispatching process continues, and all events following the very beginning ACTION_DOWN MotionEvent will never be dispatched to it any longer; in other words, only the very beginning ACTION_DOWN is passed to it

In other words, a ViewGroup cannot (1) intercept all MotionEvents, but (2) not consume them (i.e., returning false)

References

Dispatch Workflow

image

References

Web vs Android

  • In web, a touch event can be consumed more than once (either in the CAPTURE process, the HANDLE process, or the BUBBLE process), because the dispatching process never stopped until the method HTMLElement#stopPropagation() has been invoked

  • However, in Android, a touch event can only be consumed once, and once consumed (by returning true), the dispatching process stops

  • The root cause of this difference is the composition of touch event. In web, the touch event is an atomic event, however in Android, it is composed of several MotionEvents

  • In web, each view can have multiple CAPTURE event listeners, and multiple BUBBLE event listeners for each type of event; in the CAPTURE process, the CAPTURE listeners are invoked one by one in order of registration time, and the BUBBLE listeners act similarly in the BUBBLE process; the invocation of these listeners does not

  • However in Android, each view have only one event listener for each type of event, and the listener is invoked only when none of its parents consumed the event

References

View Hierarchy

  • The view hierarchy tree is not composed as what it looks like graphically
  • Graphically overlapped views may either
    • be siblings in the tree (children of a same view, e.g., a FrameLayout), or
    • have nothing relationships across windows (e.g., a popup window and an activity)
  • One can never pass events from one view hierarchy to another, but can pass events from one view to its siblings, as long as the former view does not consume the events

References

Intent

  • andriod.intent.action.MAIN: Indicates that this activity is a main entry of this app. Each app can have multiple entries, i.e., multiple activities tagged with MAIN
  • android.intent.category.LAUNCHER: Indicates that this main entry should be shown in Launcher. Each app can have multiple launcher activities, i.e., multiple activities tagged with LAUNCHER

References

/system/bin/app_process

Introduction

  • app_process executes a interpreted runtime environment for dalvik bytecodes in an application-alike environment
    • if together with --application option, System.out is redirected to logcat.
    • otherwise, System.out remains untouched, i.e., the output is the stdout.
  • app_process becomes zygote if starts with --zygote,
    • if together with --start-system-server option, zygote also starts system server by forking itself.
  • app_process can be used as a general program besides starting zygote.

Usage

/system/bin/app_process [java-options] cmd-dir [--application | --zygote [--start-system-server]] start-class-name [options]
  • java-options: Options of dalvik/libart
  • cmd-dir: Root path of the running process (e.g., root path of file operations)
  • start-class-name: Class to run
  • options: Options of the running class

Example

Monkey Example

CLASSPATH=/system/framework/monkey.jar app_process –Xms5G /data/local/tmp com.android.commands.monkey.Monkey –s 1 –v 100

HelloWorld Example

CLASSPATH=/data/local/tmp/HelloWorld.jar app_process /data/local/tmp com.example.HelloWorld

or

app_process -cp /data/local/tmp/HelloWorld.jar /data/local/tmp com.example.HelloWorld

Note, the above HelloWorld example uses /data/local/tmp as cmd-dir.

If cmd-dir is left as empty such as app_process -cp /data/local/tmp/HelloWorld.jar com.example.HelloWorld, the output (stdout, stderr) and all thrown exceptions would be eaten by app_process, resulting in nothing output.

Zygote Example

app_process -Xzygote /system/bin --zygote --start-system-server

Note:

  • The jar file here is not a java jar, but a dex jar, and thereby should be compiled by dx or d8, e.g., d8 --release --output example.jar example.jar

Differences from dalvikvm

As name suggests, app_process is related to an app, which simulates an app's running environment.

  • When app_process starts ART, it reads many startup options from system properties, and registers many JNI functions (class AndroidRuntime::start())
  • The main entry of app_process is RuntimeInit (or ZygoteInit if --zygote). It is a bootstrap class that does some initializations for the start class, then starts the class by invoking the class's main method (class app_main.cc::main(), AppRuntime::onVmCreated(), AppRuntime::onStarted())

However, dalvikvm is much more pure an virtual machine that does not relates to any app/android settings

  • it does not load system properties, and
  • it does not have bootstrap classes
  • it directly start the class by invoking its main method.

Use app_process if you like your class to run in an app-like environment, otherwise dalvikvm.

References

How to Write a System-Level Android (or Java) App (e.g. Monkey)

Introduction

  • Different from a general Android GUI application, a system-level Android (or Java) application running in command line, using (or not using) Android frameworks, is started by developers using app_process or dalvikvm directly instead of by Android framework itself
  • To develop such an application, one need to
    1. if an Android app, prepare a framework.jar for compiling (implementation of framework.jar can be empty or throwing RuntimeException, doesn’t matter); the framework.jar can be obtained from Android SDK directory (the android.jar), or compiling AOSP directly, or pulled from emulators or devices (located at /system/framework/framework.jar), or using a gradle/maven dependency built by robolectric team; but remember to use compileOnly keyword instead of implementation if you'd like to package then into a uber jar (see here on the differences of these keywords)
    2. compiling the compared jar to dex using dx or d8
    3. push the recompiled dex to Android devices
    4. start it using app_process or dalvikvm

Example

  1. Write a HelloWorld.java in directory src/com/example
  2. Compile to class in src: $ javac –cp /path/to/framework.jar com/example/HelloWorld.java
  3. Compile to jar: $ jar cvfe HelloWorld-jar.jar com.example.HelloWorld com/example/HelloWorld.class
  4. Compile to dex: $ d8 --output HelloWorld.dex HelloWorld-jar.jar
  5. Push to device: $ adb push HelloWorld.dex /data/local/tmp
  6. Run it in devices shell by either app_process or dalvikvm
    # app_process
    $ CLASSPATH=/data/local/tmp/HelloWorld.dex app_process /data/local/tmp com.example.HelloWorld
    
    # dalvikvm
    $ CLASSPATH=/data/local/tmp/HelloWorld.dex dalvikvm com.example.HelloWorld

see also #3

The Simplest Way

As long as a system-level app has no differences from a common app (except running in command line). The simplest way is no more easier than reusing the Android Studio tool chain.

  1. just create an Android project without any activities
  2. write your own codes, for example HelloWorld
  3. build the project like a normal project (AS would automatically compile them using javac and dx, and pack them into a classes.dex inside an apk)
  4. push the apk to the device
  5. use app_process/dalvikvm to run
$ dalvikvm -cp /data/local/tmp/hello.apk com.example.HelloWorld

Debugging A System-Level App

JDWP provides the ability of debugging to JVM, which also works in dalvik/ART. Like CDP (chrome devtools protocol), JDWP defines a debugging protocol of any debugger frontend and the backend. So any client/server that implements the protocol can connect to each other.

Step1: Start debugging server

On devices with Android 8 and/or lower, JDWP is implemented using the java agent, i.e., the -agentlib option. To enable JDWP:

$ dalvikvm -cp /data/local/tmp/hello.apk -agentlib:jdwp=transport=dt_socket,suspend=y,server=y,address=5005 com.example.HelloWorld

Above options "transport=dt_socket,suspend=y,server=y,address=5005" tells JVM to enable JDWP as a TCP server at port 5005, and suspend the app once start.

On devices with Android 8+, use:

$ dalvikvm -cp /data/local/tmp/hello.apk -XjdwpProvider:internal -XjdwpOptions:transport=dt_socket,suspend=y,server=y,address=5005 com.example.HelloWorld

Step2: Forward the debugger

Forward that port to the desktop:

$ adb forward tcp:5005 tcp:5005

Step3: Set breakpoints and connect to the debugger

Set your breakpoints in Android Studio.

Then in the menu: Run > Debug > Edit configurations....
Click on + > Remote, and fill the following:

  • Host: localhost
  • Port: 5005

Then click Debug.

WMS: Window Manager Service

Overview

Window manager service is a system service, which is responsible for managing the z-ordered list of windows, which windows are visible, and how they are laid out on a screen

WindowManager

  • WindowManager is implemented by WindowManagerImpl, which delegates to WindowMangerGlobal
  • In detail, there is no explicit Window instance in WindowMangerGlobal; instead, because the view hierarchy and Window forms a 1-1 mapping, WindowMangerGlobal maintains all windows by a list of views, along with their LayoutParams (for where this window is located on the screen), and ViewRootImpl (for bridging the communication between the hierarchy and window manager)
  • The addView(view, params) method creates a ViewRootImpl root, and adds the view to mView list, root to mRoot list, and params to mParams list
  • Actually, there is a Window instance in each ViewRootImpl, i.e., its mWindow field, typed ViewRootImpl$W; and it is this instance that ViewRootImpl uses to interact with window manager service

WindowManagerService

⚠️ TO BE ADDED

Properties of android.view.View

View Properties

  • When adb shell dumpsys activity, a view hierarchy of current Activity is dumped, and the view in the hierarchy is in format of
    android.view.ViewStub{1f1460d G.E...... ......I. 0,0-0,0 #1020187 android:id/action_mode_bar_stub}
    
  • To see the correct meaning of above line, see View#toString() method
  • Either dumper of ui-automator (AccessibilityNodeInfoDumper#dumpNodeRec(), and AccessibilityNodeInfoDumper#dumpWindowToFile()), or dumper of dumpsys (Activity#dump(), ViewRootImpl#dump(), ViewRootImpl#dumpViewHierarchy(), and View#toString()) only dumps subset of overall properties of a view
  • Also, the View#encodeProperties() method provides more properties of a view
  • ui-automator only dumps views that are visible to user (VISIBLE, not INVISIBLE, not GONE), however dumpsys dumps all (including VISIBLE, INVISIBLE, and GONE) => see AccessibilityNodeInfoDumper#dumpNodeRec(), and ViewRootImpl#dumpViewHierarchy()
  • ui-automator dumps the absolute bound of the view, however, dumpsys the relative bound to its parent

References

Instrumentation, and Testing

Android Instrumentation

To make things simple, Android Instrumentation are hooks that Android exposes to developer to monitor and even control the interaction between the system and the app.

For example, let's dive into Instrumentation#callOnActivityCreate(activity, bundle) method.

package android.app;

public class Instrumentation {
    // ...
    public void callActivityOnCreate(Activity activity, Bundle icicle) {
        prePerformCreate(activity);
        activity.performCreate(icicle);
        postPerformCreate(activity);
    }
}

This instrumentation (or hook interchangablely) makes developers capable of controlling how to create the activity. The default behavior of this hook is just directly creating the activity by invoking activity.performCreate() which internally invokes activity#onCreate().

A good developer can control its behavior by,

package com.example.myapp;

public class MyInstrumentation extends Instrumentation {
    // ...
    @override
    public void callActivityOnCreate(Activity activity, Bundle icicle) {
        Log.d("Do what ever you want before the creation");
        super.callActivityOnCreate(activity, icicle);
        Log.d("Do what ever you want after the creation");
    }
}

However, an evil developer may control its behavior by

package com.example.fakeapp;

public class FakeInstrumentation extends Instrumentation {
    // ...
    @override
    public void callActivityOnCreate(Activity activity, Bundle icicle) {
        // don't create, destroy it, ahahahaha
        super.callActivityOnDestroy(activity);
    }
}

This class are usually used by testers and test libraries, but it can used for other purposes as well, and

  • To use instrumentation, use adb shell am instrument
  • One have to write a class extends Instrumentation, add a <instrument> tag in AndroidManifest.xml, and add name attribute pointing to that class
  • Instance of Instrumentation is instantiated before any of the application, if running with instrumentation turned on
  • Instrumentation runs on a standalone thread other than UI thread

References

Test Runner of Android

  • Each test runner is an instance of Instrumentation, and subclass of Instrumentation, e.g.,
    InstrumentationTestRunner, AndroidJUnitRunner

References

Unit Testing and Instrumentation Testing

  • Unit test (test) runs on a JVM, and all methods with Android SDK is provided by a modified version of android.jar, in which, all methods are implemented with a stub (throw RuntimeException)
  • Instrumentation test (androidTest) runs on real device or emulator. Each androidTest is built to an APK, and installed to the device. This APK contains no LAUNCHER activities, so one cannot see it in the Launcher app, but can see it in the app management page or adb shell pm list packages. Manifest of this APK is auto-generated, and tagged with <instrument>

References

Looper, MessageQueue, and Handler

Summaries

image

  • Looper arms general thread with the functionality of message loop and message queue
  • Looper uses ThreadLocal (JAVA Implementation of TLS, Thread Local Storage) to preserve a copy of Looper for each different thread
  • Handler is a message consumer. Handler is used to send messages to message queue of the associated Looper. Looper then fetch messages from queue and dispatch it to its handler for consuming it

References

Looper

  • Looper is not a thread. Thread wanting to use Looper should, in its run method, firstly prepare Looper, and then loops it.
    public void run() { 
        Looper.prepare(); 
        …; 
        Looper.loop(); 
    }
  • Looper.prepare() sets the thread local to a new Looper instance
  • Looper.loop() opens the loop, and starts to receive messages from handler associated with it
  • Code after Looper.loop() are never reached
  • Looper does not maintain a handler list for handlers associated with it. Handler to consume a message is tagged with the message using message.target

References

Handler

  • Each Handler must be associate with a Looper, or exceptions will be thrown
    • Default constructor of Handler, i.e., Handler() will use Looper.myLooper() to associate self with current thread’s looper
    • Constructor Handler(Looper) can construct a handler with a specific looper
  • Use Handler#sendMessage() or #post() or #postDelay() to send a message to the associated looper with current time as when
  • Use Handler#postDelayed() to send a message to the associated looper with current time adding delay as when
  • Any thread can define different handlers associate with the same looper. Although these handlers’ message handling code (Handler#handleMessage() method) are defined within each thread, but all of them are run in this looper’s thread, because Handler#handleMessage() is invoked by the looper

References

MessageQueue

  • MessageQueue is a priority queue, where the priority is the time (e.g., the when filed of Message), and it organizes the messages in it in a linked list
  • MessageQueue#enqueueMessage(msg, time) enqueues a message; it does so by, from the head, checking time and when of each msg in the linked list, and inserting it at the right position
  • MessageQueue#next() will retrieve the head message msg, check msg.when, wait until msg.when if necessary, and return msg
  • Looper will invoke MessageQueue#next() to retrieve next message msg, and directly pass it to msg.target (Handler sending the message) by invoke Handler#dispatchMessage()

References

How Handler Dispatches Messages?

  • Each Handler has a handleMessage() method, and has a mCallback field
  • Each Message msg is either a non-runnable message (with msg.callback == null), or a runnable message (with msg.callback == runnable)
  • Handler#dispatchMessage(msg), according to type of msg (msg.callback == null or not), will either
    • consume the message using the callback (i.e. msg.callback.run()), or
    • consume the message itself: using mCallback.handleMessage(msg) to consume the message, if not consumed (mCallback returning false), then handle it itself
  • So a common message can be handled twice (like the event consuming), by creating a Handler with a Handler$Callback, who returns false

References

HandlerThread

  • HandlerThread is a wrapper of Thread, equipped with a Looper
  • Usage:
    thread = HandlerThread(“thread-name”); 
    thread.start()
    handler = Handler(thread.looper) { 
        handleMessage(msg) { … } 
    }
    handler.post { xxxx }
    

Does Looper.loop() Waist CPU Cycles?

Looper.loop() is an infinite loop, but does not consume CPU when queue is empty. In detail,

  • MessageQueue#next() will invoke nativePollOnce() to wait when empty
  • MessageQueue#enqueueMessage() will invoke nativeWake() to wake up next() when necessary

References

IdleHandler

  • IdleHandler is not a Handler, they are completely different things
  • IdleHandler is a handler that will be invoked if MessageQueue is empty; it can be added by MessageQueue#addIdleHandler()
  • IdleHandler has only one method, i.e., queueIdle(), and MessageQueue will invoke it each time the queue is empty
  • Every IdleHandler is removed the time immediately after queueIdle() returns false
  • So IdleHandler can be used to hook the queue when its empty

References

App Directories

Each app has 2 known directories, i.e., internal directory, external directory

  • Internal Directory
    • Internal directory is private for each apps
    • Located in /data/user/<userId>/<app_package> (can be returned by Context#getDataDir())
    • Accessed by Context#getDir(), Context#getCacheDir(), Context#getFilesDir(), and Context#getDataDir()
  • External Directory
    • External directory is public shared among all apps, any app can access any location
    • Located either in the storage /storage/emulated/<userId>/Android/<type>/<app_package> or /sdcard/storage/<sdcard-id>/Android/<type>/<app_package>
    • Accessed by Context#getExternalXxxDir() or Context#getExternalXxxDirs()

Java Class Loading Mechanism

When to load classes

  • JVM loads class progressively (and lazily), and the class loading can happen either implicitly or explicitly
    • implicitly: whenever JVM sees an unknown class (e.g., when new an instance, when a static method/field is touched, etc.), it a class loader of to load the class
    • explicitly: explicitly called by developer (e.g., via ClassLoader#loadClass(), Class#forName(), etc.)
  • Important: JVM itself does not know which class should be loaded by which class loader (only the class loader itself knows which classes it can load), see following on which class loader to use

Where to load classes

  • JVM knows nothing about classloader, it delegates all of classloading to the class loader
  • Only the class loader itself knows where the classes located, which classes it can load, and which can not
    • URLClassLoader load classes from a given URL, and the URL is given by developers
    • AppClassLoader gives URLClassLoader class path to load classes from CLASSPATH, i.e., the property System.getProperty(”java.class.path”) (see sun.misc.Launcher$AppClassLoader$getAppClassLoader())
    • Other class loaders load classes from elsewhere (developer defined)

How to load classes - Parent-Delegation Model 双亲委派机制

image

  • The class loading mechanism uses a parent-delegation model to load classes

  • In detail, when a class loader is to load a class, it firstly asks its parent to load it, and only when its parent is incapable of loading the class, it loads the class (see this figure); the parent-child relationship is maintained not by is-a (继承), but by has-a (聚集)

  • Specifically, in code XxxClassLoader#loadClass(clsname, resolve):

    klass = findLoadedClass(clsname); // check whether the class is initiating loaded or not by this class loader
    if (klass != null) { // already initiating loaded, i.e., current loader is the initiating loader of clsname
      return klass; 
    } 
    if (parent != null) { // ask parent to recursively check loaded or not and load clsname if not loaded
      klass = parent.loadClass(clsname, false);
    } else { // use the built-in class loader if there isn't parent
      klass = findBootstrapClassLoaderForLoading(clsname, false)
    }
    if (klass == null) { // parent cannot load this class, so it loads it
      klass = findClass(clsname); // one often needs to override this method to find the class from somewhere
      // mark this as klass's initiating loader, and this as klass's defining loader
    } else {
      // mark parent as klass's initiating loader
    }
    if (resolve) {
      resolveClass(klass); // link the class
    }
  • ClassLoader#findLoadedClass(clsname): it only checks whether clsname is initiating loaded by current class loader (i.e., it only checks whether current class loader is the initiating loader of clsname); it means this method does not check recursively following parent-delegation model; the model is enforced by loadClass() method since parent.loadClass(clsname) recursively invokes findLoadedClass()

  • XxxClassLoader#findClass(clsname): this is often the only method one needs to override to find the class from somewhere (e.g., file system, jar, or apk), read the class bytecodes as byte[], and invoke XxxClassLoader#defineClass(bytes, off, len) to return the class

  • NEVER override loadClass() if you don't like to break the parent delegation model

    From Java SDK: Subclasses of ClassLoader are encouraged to override findClass(String), rather than this method

  • When clsname is loading into the virtual machine, there exists a method call stack; along the stack, there is only one class loader invoking findClass(clsname) and defineClass(clsname) but multiple class loaders invoking loadClass(clsname) due to the parent-delegation model; all class loaders invoking loadClass(clsname) along the stack are called the initiating loaders of kclass, and the only class loader invoking findClass(clsname) and defineClass(clsname) is called the defining loader of klass.

    From JLS:

    The Java Virtual Machine invokes loadClass(N) on L. The value returned by the invocation is the created class or interface C. The Java Virtual Machine then records that L is an initiating loader of C.

    When one class loader delegates to another class loader, the loader that initiates the loading is not necessarily the same loader that completes the loading and defines the class. If L creates C, either by defining it directly or by delegation, we say that L initiates loading of C or, equivalently, that L is an initiating loader of C.

    From HotSpot - Class Loader Delegation

    When a class loader is asked to find and load a class, it can ask another class loader to do the actual loading. This is called class loader delegation. The first loader is an initiating loader, and the class loading that ultimately defines the class is called the defining loader. In the case of bytecode resolution, the initiating loader is the class loader for the class whose constant pool symbol we are resolving.

  • However in the explicit loading cases (e.g., invoke loadClass() manually), HotSpot does not record all initiating loaders. It only records the defining loader as initiating loader. See this StackOverflow question.

References

ClassLoaders and ClassLoader Hierarchy

image

  • The above figure shows the parent-child chain, and the responsibility of class loaders
  • Usually, a custom class loader should recognize the AppClassLoader as its parent class loader, for the loading of other general class files

image

  • The above figure shows the inheritance chain
  • The URLClassLoader is responsible for, as its name shows, load classes according to the url (usually from jar file or the file system, for example file://path/to/A.class)
  • One can write his own custom class loader for loading classes from somewhere else, for example, from a zip file, from internet, and if your custom class loader cannot load the class specified, a ClassNotFoundException should be thrown
  • The system class loader of JVM is an AppClassLoader by default, which is ClassLoader#getSystemClassLoader()
  • Each thread has its own context class loader Thread#getContextClassLoader(), and if not set manually by Thread#setContextClassLoader(classLoader), the default context class loader of each thread is that of its parent thread. Typically, the default context class loader of main thread is the system class loader
  • Actually all boot classes (classes of boot class path) are directly loaded by the JVM itself and thereby their ClassLoader is null, i.e., java.lang.String.class.getClassLoader() == null. But we often say they have a bootstrap class loader.

References

Which ClassLoader to use

Implicit Class Loading

For implicit class loading, the class loader used to load a class C is the class loader of the caller class, i.e., caller.getClass().getClassLoader()

class B {
  void b() {
    C c = new C();
    // c.getClass().getClassLoader() == this.getClass().getClassLoader(), if C is never loaded before
  }
}

Explicit Class Loading

Explicit class loading includes

  • ClassLoader#loadClass()/Class.forName(name, init, classLoader): using the specified class loader
  • Class.forName(name): same as implicit loading, using class loader of its caller, which is equivelant to Class.forName(name, false, caller.getClass().getClassLoader())
  • Entry class: the entry class passed to jvm is loaded by the system class loader (i.e., the AppClassLoader)

Class Lifecycle

image

Please goto Execution for a detailed and complete example.

A class is born right from the loading of the class, and dies when the class is unloaded. The overall lifecycle includes Loading (ClassLoader#loadClass(className, false)), Linking (ClassLoader#resolveClass(), i.e., Verification, Preparation, and Resolution), Initializing, Using, and Unloading.

public class Example {
  public static int a = 2;
  public static final int b = 3;
}
  • Loading: a class is loaded either implicitely or explicitely, and the loading follows the delegation model (see above sections about when/where/how to load classes, and the classloader hierarchy). In this step, the classloader finds the class, gets its binary representation (the .class file), loads the stream to the virtual machine, and creates a data structure (i.e., Class<?>) for it. ClassLoader#loadClass(className, false) is responsible for loading the classes.
  • Linking: considering each .class file is compiled without linking (all references to other classes or fields are symbolic references to the constant pool, however, we have no idea whether the symbolic references in the constant pool actually exist or not in the method area), so after the binary stream is loaded, the class is then verified, prepared, and resoluted to be linked. ClassLoader#resolveClass() is responsible for linking the classes.
    • Verification: not all classes can be loaded to the virtual machine, and verification ensures the loaded class is structually correct, well-formed, and can be loaded safely. for example, the verifier may check that the class file format is correct, that every instruction has a valid operation code, that every branch goes to a correct location.
    • Preparation: preparation allocates memories for the Class<?> in the method area, creates its static fields, and sets zero values (or developer assigned non-zero values for a contant) for them (e.g., Example#a is set to 0, however Example#b to 3 because it is a constant)
    • (Optional) Resolution: some bytecode instructions make symbolic references to other classes/interfaces/... using their binary names (e.g., Lcom/example/Example2). resolution actually links the classes, i.e., it often converts each symbolic reference to a real direct reference in the method area (it also ensures each symbolic reference actually exists, and this thereby may need to cause loading of other classes, but need not cause them to be verified or prepared)
  • Initializing: after linking, the classes is well prepared in the method area, and then it is time to initialize it, i.e., invokes its <clinit> method (the <clinit> method is generated if needed (if there are no static fields, no static blocks, it is not needed) by the compiler, e.g., javac), and Example#a is now set to 2. remeber that initializing is thread safe and lazy, it is initialized only once for each thread of each classloader, and it is delayed to be initialized at the time the class is firstly used
  • Using: a class is ready to be used after all previous step finished, one can use it normally or by reflection
  • Unloading: like normal object, a class is unloaded when there are no references to it and thereby garbage collected, however, classes loaded by bootstrap classloader (e.g., java.lang.*) are always reachable and never unloaded

However, the JLS allows an implementation flexibility as to when the Linking take place and how the Linking is implemented. It not only allows the verification, preparation, and resolution to be out of order (e.g., resolution first then verification as in ART), but also allows both Static Linking and Dynamic Lazy Linking. A static linking means that the Linking happens right after Loading, and before Initializing, it also means that the three steps of Linking happens sequentially. However, a dynamic lazy linking is very much flexiable: one can choose to link the class only before the class is initialized, and used; one can also choose to split the Linking up, and insert them as part of Initializing and Using, which means one can link the classes on-the-fly as the class is initialized and used; also, one can choose resolute the class as the class is verified (so it is possible that other classes may be loaded when a class is verified).

Thus, Linking and "Initializing and Using" can be intertwined with each other in many many implementations, and implementation of resolveClass() is thereby flexible. The following gives the implementation of ClassLoader#resolveClass() in OpenJDK 8 && 13. One can see that it does nothing Linking work, and the Linking work is done as Initializing and Using.

// OpenJDK 8
protected final void resolveClass(Class<?> c) {
  resolveClass0(c);
}

JNIEXPORT void JNICALL
Java_java_lang_ClassLoader_resolveClass0(JNIEnv *env, jobject this, jclass cls) {
  if (cls == NULL) {
    JNU_ThrowNullPointerException(env, 0);
    return;
  }

  JVM_ResolveClass(env, cls);
}

JVM_ENTRY(void, JVM_ResolveClass(JNIEnv* env, jclass cls))
  JVMWrapper("JVM_ResolveClass");
  if (PrintJVMWarnings) warning("JVM_ResolveClass not implemented");
JVM_END

// OpenJDK 13
protected final void resolveClass(Class<?> c) {
  if (c == null) {
    throw new NullPointerException();
  }
}

The following presents a lifecycle example of Example

public void foo() {
  {
    ClassLoader loader = new MyCustomLoader();
    // Example is loaded and linked, because we set `resolve` to true, 
    // `Example#a` is 0, and `b` is 3 by far
    Class<Example> ExampleClass = loader.loadClass("com.example.Example", true);
    // Example is initialized and used, because we get its field
    // `Example#a` is 2 by far
    int a = Example.getDeclaredField("a").get(null);
    System.out.println(a);
  }
  // Example will be unloaded in future, because there are no references to loader and ExampleClass, they will be garbage collected in future
}

References

Resources

  • Besides classes, the class loaders should also know all other resources (images, audio, text, etc.)
  • One can get the URL of a resource/resources by
    • ClassLoader#getResource(name)
    • ClassLoader#getResources(name)
  • The resource loading mechanism also obeys parent-delegation model

What is a Context?

As name suggest, context means the current environment of an object. It lets newly-created objects understand what has been going on.

image

As above figure shows, Application, Activity, and Service are all Context, and all of them are ContextWrappers, i.e., they have a base context mBase. The base context of a context c is the context where c resides. For example, the most top-level context is the context where the application is, and the context of an Activity is maybe the Application

References

Differences

The first difference among these Context is that they (1) have different lifecycles, and (2) provides different functionalities.

Lifecycle

Apparently Activity, Service, and Application have different lifecycles

Functionalities

In functionality, only Activity extends ContentThemeWrapper, and can serve UI functionalities. However, Application and Service extends directly from ContextWrapper, and cannot provide such functionalities.

References

ContextImpl

The actual implementation of Context is ContextImpl. A base context is attach to a wrapper in onAttachBaseContext(), and onCreate() happens after that, so it is safe to use Context in onCreate().

When does ContextImpl is initialized?

⚠️ TO BE ADDED

References

ABD - C++ Layer

⚠️ TO BE ADDED

Built-In Transactions

Each IBinder has several built-in transactions that can be used for specific tasks

  • PING transaction, server can handle this transaction for clients to check their liveness, override BBinder::pingBinder() to handle it
  • SHELL_COMMAND transaction, server can handle this transaction for clients to execute some shell command, override BBinder::shellCommand() to handle it; shell commands like am and cmd use this
  • DUMP transaction, server can handle this transaction for clients to query their internal states, override BBinder::dump() to handle it; shell command dumpsys uses this
  • INTERFACE transaction
  • SYSPROPS transaction

References

References

What is a Window?

  • A Window is a rectangle area of the screen
  • It is the Window who manages the view hierarchy
  • Different windows can overlap
  • The depths of different windows are attributed by z-order
  • Each Window can be attached only one view hierarchy
  • WindowManager is responsible for managing the z-ordered list of windows, which windows are visible, and how they are laid out on screen
  • To see all windows of current screen, use
    adb shell dumpsys window w
    

References

Android System Services

Overview

System services in Android are like service daemons in Linux. Each of them focuses on a specific, and runs in the background, e.g.:

  • AMS: Activity Manager Service, manages all activity stuff, including activity lifecycle, activity stack, etc.
  • PMS: Package Manager Service, manages all packages installed, including (un)install a package, etc.
  • WMS: Window Manager Service, manages all z-ordered windows on the screen, including layouting the window, etc.

Android services (including both system and custom) accept a C/S architecture. They expose their functionalities by RPC APIs. And thereby to use them, one should firstly get their RPC clients, and then interact with the clients using the RPC APIs just like they are local objects.

Usage

Take AMS as an example, to use them, one often firstly gets its RPC client ActivityManager by

ActivityManager am = (ActivityManager) context.getSystemService(Context.ACTIVITY_SERVICE);

and then interact with it for example,

am.startActivity(...);

System Server (#47)

Custom Android services often runs in a standalone process, and managed by the service developer itself. However, different custom ones, Android system services reside in one process named System Server, and are managed by a process (also a system service) called Service Manager

Android system services are registered to Service Manager and started as System Server starts, it is exactly when the Android system starts. System services in System Server _run either in the main thread, or in a standalone thread

In System Server, these services are classified to three categories:

  • Bootstrap services: AMS, PMS, …
  • Core services: Battery, UserStats, …
  • Other services: IMS, Watchdog, Clipboard…

Implementation of system services are located at /frameworks/base/services/

Service Manager (#99)

Service Manager manages all services in the system. It acts as a name server. And Context#getSystemService() interacts with it to get the RPC client

AccountManager am = (AccountManager) context.getSystemService(Context.ACCOUNT_SERVICE);

Let's see how Context#getSystemService() work

public Object Context#getSystemService(String name) {
  return SystemServiceRegistry.getSystemService(name);
}

getSystemService directly delegates the invocation to SystemServiceRegistry#getSystemService ()

public static String SystemServiceRegistry#getSystemServiceName(Class<?> serviceClass) {
    return SYSTEM_SERVICE_NAMES.get(serviceClass);
}

SystemServiceRegistry, for each system service, defines a service fetcher to query Service Manager and return its RPC client. It manages all fetchers by a map SYSTEM_SERVICE_FETCHERS, and registers them to the map when class of it is loaded

registerService(Context.ACCOUNT_SERVICE, AccountManager.class,
        new CachedServiceFetcher<AccountManager>() {
    @Override
    public AccountManager createService(ContextImpl ctx) {
        IBinder b = ServiceManager.getService(Context.ACCOUNT_SERVICE);
        IAccountManager service = IAccountManager.Stub.asInterface(b);
        return new AccountManager(ctx, service);
    }});

Fetcher of Account Manager Service will firstly query Service Manager for the remote object, and inject it the the RPC client (AccountManager)

Then developers can use AccountManager to interact with Account Manager Service

References

AOSP Directories

Layers and Directories

see also #124

image

/platform/ ..................... Layers above Linux Kernel
 + manifest/ ................... platform's repo manifest file
 + build/ ...................... the AOSP's build system
   + soong/ .................... the Soong build system
   + tools/
     + docker/ ................. an Ubuntu 14.04 AOSP build image's Dockerfile

 + packages/ ................... Application Layer – Built-in apps

 + frameworks/ ................. Framework Layer – SDKs and framework level commands
   + base/
     + cmds/ ................... e.g., app_process, am, input, svc
     + core/ ................... actual framework SDK
     + services/ ............... implementation of system services, e.g., system server, ams, pms
       + java/ ................. system server implementation
       + accessibility/ ........ accessibility manager service implementation
       + core/ ................. implementations of ams/pms/...
   + native/
     + cmds/ ................... e.g., servicemanager, dumpsys, service, cmd, atrace, installd

 + art/ ........................ Android Runtime Layer – Android runtime and commands, e.g., dalvikvm, dex2oat
   + cmdline/ .................. art cmdline processing tool
   + dalvikvm/ ................. dalvikvm command
   + dex2oat/ .................. dex2oat command
   + dexdump/ .................. dexdump command
   + oatdump/ .................. oatdump command
   + runtime/ .................. the actual art runtime
   + tests/ .................... ART's tests and testing infra

 + dalvik/ ..................... Android Runtime Layer – (Legacy) Dalvik virtual machine, e.g., dx
   + dx/ ....................... the dx tool which compiles Java bytecode to Dex bytecode

 + libcore/ .................... Native Libraries Layer – Java core libs
   + dalvik/ ................... dalvik.system classes such as DexClassLoader/PathClassLoader implementation
   + ojluni/ ................... openjdk implementation
   + libart/ ................... art related Java classes such as VMRuntime/VMStack implementation

 + libnativehelper/ ............ Native Libraries Layer – JNI basics

 + system/ ..................... Native Libraries Layer – System-level libs, daemons, and commands
   + core/ ..................... system level libs, commands, and daemons
     + adb/ .................... adb
     + logcat/ ................. logcat, client tool
     + liblog/ ................. liblog, a logging protocol that connects log client and server
     + logd/ ................... log daemon, the server
     + toolbox/ ................ toolbox
     + libnativebridge/ ........ native bridge infrastructure
   + tools/
     + aidl/ ................... aidl implementation
     + hidl/ ................... hidl implementation

 + bionic/ ..................... Native Libraries Layer – Core C libs, e.g., libc, libm, libstdc++

 + hardware/ ................... Hardware Abstraction Layer

 + kernel/ ..................... AOSP kernel configurations

 + external/ ................... AOSP dependent third-party libs
   + qemu/ ..................... Android emulator (AVD)

 + build/ ...................... AOSP build system
   + tools/
     + docker/ ................. a docker build environment

 + development/ ................ Development tools, docs, especially testing tools
 + developer/ .................. Development tools, docs, especially testing tools
 + sdk/ ........................ Development tools, especially app building tools
 + ndk/ ........................ Development tools, especially app building tools
 + tools/ ...................... Development tools, especially Android Studio and Gradle debugging tools
   + adt/ ...................... android IDE
     + idea/ ................... android studio
     + eclipse/ ................ eclipse
   + dexter/ ................... dex manipulate tool slicer and command line tool dexter
   + apksig/ ................... apk signer
   + base/ ..................... common tools shared by Android Studios and Gradle plugins
     + ddmlib/ ................. ddmlib
     + build-system/ ........... android gradle plugin (AGP)
     + profiler/ ............... android studio profiler
 + prebuilts/ .................. as name says, prebuilt binaries or libs

/kernel/ ....................... Linux Kernel Layer – Linux kernel
 + drivers/ .................... linux drivers, android-specific drivers e.g., binder and ashmem, are under staging/android sub-directory
   + staging/ .................. android specific drivers
 + kernel/ ..................... linux kernel

Afterwards, for simplicity, in this blog, directories or files starting with /kernel/xxx represent directories or files within linux kernel, /platform/kernel/xxx representing those in android platform, and the rest representing those in android platform

Important Directories

/platform/manifest/ .......................... AOSP's repo manifest
/platform/build/ ............................. AOSP's build system
/platform/build/soong/ ....................... AOSP's Soong build system
/platform/framework/base/core/ ............... Framework SDK
/platform/framework/base/services/ ........... System Services
/platform/framework/base/cmd/ ................ Framework-Level Commands, e.g., am, pm, app_process
/platform/system/core/ ....................... System-Level Commands, e.g., adb, logcat, native bridge
/platform/libcore/ojluni ..................... Openjdk
/platform/tools/adt/idea/ .................... Android Studio
/platform/tools/base/ ........................ Shared Debugging Tools (studio-master-dev branch), e.g., ddmlib, layout inspector
/platform/external/qemu ...................... Android emulator
/platform/build/tools/docker ................. Build docker environemnt
/kernel/drivers/staging/android/ ............. Android Drivers, e.g., binder, ashmem

References

ABD - Java Layer

⚠️ TO BE ADDED

Built-In Transactions

Each IBinder has several built-in transactions that can be used for specific tasks

  • PING transaction, server can handle this transaction for clients to check their liveness
  • SHELL_COMMAND transaction, server can handle this transaction for clients to execute some shell command, override Binder#onShellCommand() to handle it; shell command cmd and am uses this
  • DUMP transaction, server can handle this transaction for clients to query their internal states , override Binder::dump() to handle it; shell command dumpsys uses this
  • INTERFACE transaction
  • SYSPROPS transaction

References

References

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.