privacy-tech-lab / gpc-android Goto Github PK

@bella-tassone, @kalicki1, and I discussed possibly using OKHttp Interceptor to intercept and rewrite outgoing HTTP requests. A possible next step we could take is implementing and testing this. Questions include:

• Does it stop all requests?
• Can it "see" requests made by APIs?

@kalicki1, can you take the lead on this? And please contribute here as well, @bella-tassone and @ekuller.

Today is the official release date of iOS 14.5 with the App Tracking Transparency Framework. Apple made a short video about it that is quite good. But it stops short of explaining how it works. It is not 100% clear to me how it is ensured technically that a user is not being tracked. Sure, as a user, I can select not being tracked, but then what?

As a developer, how do I know that a user has enabled App Tracking Transparency? How is this information made available to me, if at all? Apart from implementing the Framework in my app's codebase, do I need to set up something on my backend? Is my app automatically blocked by the Framework to send out HTTP requests to ad networks if a user enables it?

In short, from a technical standpoint, how does Apple's App Transparency Framework work? How does it technically ensure that a user who turned it on is not tracked.

The answer to that question will be informative for our own approach.

We would like to have an SDK (Framework) that, when integrated in an app, is testing which permissions the app is using. Here is a strategy to perform such tests:

if AVCaptureDevice.authorizationStatus(for: .video) ==  .authorized {
    //already authorized
} else {
    AVCaptureDevice.requestAccess(for: .video, completionHandler: { (granted: Bool) in
        if granted {
            //access allowed
        } else {
            //access denied
        }
    })
}

For Swift 4
Make sure that you add the AVFoundation Framework in the Link Binary section of build phases
You should write import AVFoundation on your class for importing AVFoundation

To get a sense of whether/what ad networks would be willing to respect a SharedPrefernce for GPC, we should look into which ones support IAB SharedPreference.

@n-aggarwal brought up this good point.

• Is it already implemented?
• Can people opt out?
• How does it work?

Here is a starting point.

As we discussed today, we want to start implementing GPC for Android. The following is not final and just a work in progress. We probably need to break this issue down into multiple issues once we made initial progress.

Basic Idea

The basic idea is to provide the developer with:

1. A GPC Android library that developers can integrate in their Android app for users to send GPC signals.
2. A GPC server implementation that developers can use to implement their own server for receiving GPC signals.

Roadmap

1. Install Android Studio. It may be that the setup takes some time as there are sometimes problems with Gradle, which is the build tool for compiling Java/Kotling code into the Android APK that runs on Android devices.
2. Develop a basic Android app that we can use as a sample for integrating our library. The app could be an app with a single view per Android Studio's template. Let's use org.privacytechlab as package name (reverse domain name notation to keep the codebase organized). Here is a tutorial for creating an Android app. There are others. This is just a random selection. @bella-tassone and @ekuller, ideally you work on the same and one of you pushes the code to the repo. Let's use Java.
3. Develop a library that developers can use to integrate GPC in their apps. The idea is to create functionality for a settings page that will allow users upon installation of the app to select whether or not they want to send GPC signals. If they do, upon each start of the app a signal is sent to the developer's server. All libraries and frameworks made in Java can be moved and run in a Kotlin project. Both Kotlin and Java compile to Bytecode that can be run on the Java Virtual Machine. So, let us use Java for the library as well. The library includes functionality for the developer to specify the server URL to which the GPC signals should be sent. In addition, to the GPC signal, we also need to identify a particular user. Otherwise, it would not be clear who is sending the signal. One choice for such identifier could be Google's Advertising Identifier (AdID). Then, using some HTTP request library (e.g., OkHTTP), our library is sending a GPC signal with the AdID of a particular user to the server specified by the developer. Here is a tutorial for creating a library and here is another one.
4. Create developer documentation on how to integrate our library into apps. Test the integration with our app.
5. Develop server sample implementations for the developer to demonstrate how they can implement their own server for receiving GPC signals. In principle, our server implementation for websites (in Express.js) is usable for apps as well. However, mobile developers may decide to use some mobile-specific backend, such as Firebase. So, we should look into providing such implementations as well (need to inquire with Dave Harbage, Google, and others to see what is commonly used).

User App

In addition, we could also think of developing a user app that allows users to set GPC settings for multiple apps. However this aggregated GPC settings app is less relevant in the app ecosystem than for the web. Website-by-website settings are burdensome because web users visit many different sites. However, mobile app usage is much more concentrated towards a handful of app. Thus, setting GPC app-by-app is less burdensome. Should we go that route, here are some references:

• Overview of shared storage
• SharedPreferences
• Cross-app uninstall apps

If a user sets their Android device such that tracking is no longer allowed (i.e., apps are not able to use the ad id anymore), what effect does that have on apps?

• The ad ID is zeroed out in all web traffic?
• What is the effect on browser apps vs non-browser apps? The former can still set cookies?
• Are there any limitations in addition to the ad id?
• How much of the limitation is based on the technical enforcement and how much is left to policy enforcement (#47)?

What does the device setting on the current Android version look like anyways? Can someone post a screenshot?

Some testing observing web traffic with pcap may lead to some insightful results here.

Possibly useful starting points:

• Get a user-resettable advertising ID
• API reference android.adservices.adid
• API reference WebView

Fiddler does not seem to capture http requests made by Android emulator unless a proxy is set up. HTTP requests fail when proxy is set up. Debugging mechanism (log) records that the HTTP request is successful without the proxy, but it is unsuccessful when the proxy is set up.

In our SDK, which is integrated in an app, we would like to detect the third parties that are included in the app. Here is the strategy:

Pick a public class and check if that is available or not.

Following is an example for Google Analytics. Google Analytics has a public class called GAI (API Reference). If GAI class is available we can assume app is using Google Analytics SDK.

if NSClassFromString("GAI") != nil {
    print("App is using Google Analytics SDK")
}

This strategy works well. I have tested it.

The easiest way to stop any and all tracking through browsers is by attaching a GPC signal. Unfortunately, doing so without root access can be extremely difficult to implement, if at all possible. An easier solution would be to instruct users to download a browser that has built in GPC support.

The first step is to check which browsers have built-in GPC support; some potential browsers are: Bromite, DuckDuckGo, Brave, Firefox, Tor.

The second step would be to explain to the user that they need to download one of these browsers and use that one instead of the system browser; the explanation should be short, and easy to understand in order to increase usability.

Finally, the third step would be to integrate a button that redirects the user to the play store to the page where they can download the browser.

As discussed in our meeting earlier today, I did a quick Google search regarding whether or not modifying HTTP headers at some level is possible via Android apps or libraries and found that there do exist some discussions about this.

I am opening this issue to save these links so that when the time comes, we are able to revisit these ideas. I found these discussions having looked up android api to intercept request headers.

• Custom Headers in Retrofit
• Use OkHttp to implement a WebSocket client in Android App
• How to build an HTTP interceptor for an Android app with OkHttp3
• Retrofit 2 — Manage Request Headers in OkHttp Interceptor

I'm not sure whether or not OKHTTP is what we are looking for, but they do have a GitHub here. This could also turn out to be not the direction we want to head in.

Feel free to post other things you all find here! This is not meant to be resolved in the coming weeks.

Per @bella-tassone's question in today's discussion. As the HTTP interceptor idea is hard to implement (issue #36), we now need to rethink our design. If anyone has ideas, let's discuss here ...

If we want to implement GPC into the LineageOS operating system, we need a better understanding of LineageOS in general. Let's use this issue to document questions and/or answers we have about LineageOS.

It is possible to build HTTP interceptors (e.g., for Android). However, they only allow changing of HTTP requests of the particular app that implements the interceptor.

A Dialog asking for the user to set a GPC preference pops up when the app is first installed. However, if the user turns the preference on via the dialog, the GPC header is not added to http requests until after a new tab is created or the activity is restarted.

Our purpose is to allow users to voice to apps whether they would like to be tracked or not. As such, it's crucial that our library would implement a dialog in whatever app uses it. This dialog would pop up when the app is first opened, and ask users their preference, collecting the information that would be sent to developers and ad-providers. Here is a tutorial on how to implement a dialog. Let's work on developing one for our library.

Explore use of mitmproxy. Example: mitmproxy/mitmproxy#5155

I will look into that.

As discussed today, @n-aggarwal and @wesley-tan, can you look into the various Google developer policies so that we understand exactly what is allowed and what is prohibited? This analysis will also inform us to which extent declining tracking is similar to enabling GPC, as you brought up this comparison today, @n-aggarwal. Here is an initial starting point; there may be more documents. So, take a look around.

• Google Play Policies start page
• Google Play content policy
• Google Play Developer API Terms of Service
• Google Play Developer Distribution Agreement
• Developer Program Policy
• Privacy, Deception and Device Abuse - User Data
• Monetization and Ads - Ads

We discussed in #56 that apps may use their own identifiers. This raises the question whether and how apps use alternative identifiers for user tracking, e.g. a Facebook ID.

We are following the approach of having different opt out mechanisms for different types of apps.

1. Non-browser apps

For non-browser apps (at least to the extent they are not using cookies), opting out via the AdID should be sufficient. Thus, simply surfacing the OS setting in our app to the user may be sufficient as well.

2. Browser apps (and WebViews)

For browser apps (and WebViews), to which extent can we access cookies and modify web requests? Is our problem easier now since we do not have to intercept all web traffic but only that of a browser?

If it does not get easier, are there browser-level controls that would prevent tracking? So, for example, a simple solution could be that our app simply surfaces a browser-level settings page for turning of third party cookies.

Deleting AdId means that developers cannot query that value anymore; however, there is a possibility that the apps store the AdId in storage and still continue using it. To check whether this happens or not, I will download 10 apps from different categories, apply apk-mitm on them and then observe the traffic.

The process will be as follows: Initially. I will leave the AdId enabled and browse through the apps. Then after sufficient time has been spent, I will delete the Adid and then continue using the app, observing the network data using PCAPdroid to see if the app still passes the Adid.

Given below is the list of 10 apps I plan to install for this experiment:

Potential Apps to download:

If you have any suggestions let me know.

public boolean isLimitAdTrackingEnabled ()

Retrieves whether the user has limit ad tracking enabled or not. When the returned value is true, the returned value of [getId()](https://developers.google.com/android/reference/com/google/android/gms/ads/identifier/AdvertisingIdClient.Info#getId()) will always be 00000000-0000-0000-0000-000000000000 starting with Android 12 (but older devices may be problematic). This shows to me that Integration with previous devices may be an issue. Naive AdID solution will only be able to work on certain phones where the getId function has that certain functonality (https://www.deccanherald.com/business/technology/android-12-list-of-devices-eligible-for-the-latest-google-mobile-os-1031367.html)

Does removing the AdId also set this flag isLimitAdTrackingEnabled() to true in Android 12+?

This flag used to indicate to users whether they allow tracking but was removed in the UI as Google changed the UI and now allows users to remove the AdId.

As a result, it's unclear whether removing the AdId actually encourages apps not to track across apps anymore. This is not 100% clear from Google's policies, as discussed in #48

Todo: Build sample app, connect to AdId API, and show outputs (i.e. isLimitAdTrackingEnabled and retrieved AdId).

API documentation: https://developer.android.com/training/articles/ad-id

Originally posted in #48 (comment)

Let's look into how Firebase databases work, and how we would use them in the development of our library.

As discussed today, let's try to find out whether it is possible to send a do-not-sell header from an iOS app to a service.

The situation is similar as for the browser extension. However, because there is no way to inject a header from our app into another app (at least not without jailbreaking the phone), the idea is to have a curated list of the top 100 apps and send do-not-sell headers to their server URLs that we identified beforehand.

In order to identify the headers, @rgoldstein01 will look into intercepting app traffic via Fiddler. Fiddler Everywhere works on the Mac, but does not yet have all features of the Windows version. But maybe that is not necessary for our purposes. Here are the instructions how to decrypt HTTPS traffic. I would say, give it a try as well, @rgoldstein01. It will give us some greater insight.

(cc'ing @davebaraka, @kalicki1, and @pakaelbling)

An important part of this study will likely be a usability study with app users and developers (possibly, also SDK developers). Otherwise, apart from the Platform, what would be the contribution?

The develop guide for OkHttp Interceptors is here.

As we discussed today, to get started with iOS development @bella-tassone and @ekuller will be creating a basic iOS app written in Swift and push the code to this repo. The steps are:

1. Getting the tools setup, in particular, Xcode
2. Use an Xcode template to write a basic app per below. Maybe, this tutorial is helpful (I have not checked in detail but at cursorily glance it could be what you want)
3. Once the app is finished, integrate a library. AdMob (via the CocoaPods package manager is probably the easiest; once we create our library we can also distribute it via CocoaPods)

One important part is also to get familiar with Git and GitHub. I am including a few slides from my class on using GitHub. Ideally, one of you, @bella-tassone or @ekuller, would push a first start here and we go from there ...

@kalicki1 will be helping out a bit here as well ...

Uploading 2_Command_Line_and_Version_Control.pdf…

Today the California Attorney General released revised regulations on how to implement the CCPA. In particular, the AG said how the opt out button for opting out from the sale of information should be designed.

(cc'ing @rgoldstein01, @pakaelbling, @kalicki1)

In my discussion with Robin Berjon, he mentioned that for an Android GPC implementation above the app-level, shared storage may be an option. I am not sure whether it helps us, but we should look into it. Here in the context of iOS, we should check whether a similar API exists for iOS. I do not believe so, but who knows ...

Along the same lines Ashkan Soltani mentioned:

on android, there are cross-app ad libraries that rely on shared files on the /sdcard to sync things like identifiers or state. wouldn't be hard to do the same for a library that stores a GPC setting

Again, does something like that exist for iOS?

Ultimately, I do not think these are decisive points. We can simply provide a GPC library to the developer for integration into their apps and that library sends GPC signals. So, while it would be nice, it is not absolutely necessary to cover all apps with one library. And ultimately, GPC should be part of the OS. So whatever we do is somewhat provisional either way.

(cc'ing @kalicki1)

Change the send signal behavior inside our dummy app from being sent on the app launch to just send when our button is pressed. Also, let's remove the message reflecting the 'sent' message.

Find the minimum sdk version that our app would be compatible with to maximize the percentage of audience that we can reach.

Possibly use firebase to create a backend infrastructure that a GPC signal can be sent to.

As previously discussed, google is introducing the Privacy Sandbox. It turns out that google plans to phase out Android ID and instead use Topics API to "help preserve users' privacy" over the next 2 years. This would mean that our AdId solutions would fail to work in the long run, so we may need to rethink our implementation approaches.

"Our goal with the Privacy Sandbox on Android is to develop effective and privacy enhancing advertising solutions, where users know their privacy is protected, and developers and businesses have the tools to succeed on mobile. While we design, build and test these new solutions, we plan to support existing ads platform features—including advertising ID—for at least two years, and will provide substantial notice ahead of any future changes."

Reference:
https://developer.android.com/design-for-safety/privacy-sandbox/introduction

This bug occurs regardless of where preference is toggled (i.e. via dialog or settings menu) and is also present with the DNT preference when running the original code in Lineage's Jelly repo which we are modifying.

Our goal is to bring GPC to Android. Our current approach is to advise users of app settings that will effectively enable GPC on Android. In a broader sense, we provide a privacy advice app (at least as ad tracking is concerned). This is an interesting concept. Would users like such an app, find it useful, ...? That is a direction we could go into.

So, the question is, are there already other privacy advice apps? If so, which functionalities do they provide? As @wesley-tan mentioned in our call today, we should look into user-facing apps and also consider SDKs, notably, those provided by consent management platforms (CMPs) like OneTrust. We may not be able to get their code, but still it would be important to understand as much as we can. Other CMPs are Complianz, TrustArc, SourcePoint, Didomi, Wirewheel, ... And then, are there user-facing privacy advice apps on the Play Store?

@wesley-tan will take the lead on this issue and @n-aggarwal will help out as necessary.

I started creating a list of the top 100 free iOS apps per App Annie's ranking of March 18, 2020. @pakaelbling, for our app can you start filling out the list? I already included two examples. Do another handful and see how it goes.

Some details on the list.

• The column Data Sale per CCPA means whether or not the service is selling data. You can look into their privacy policy. If they are selling data they are also required to have a "Do not sell my info" or similar link on their homepage. Maybe, they also have a California-specific policy or opt-out section on their website. Even if you do not find such info, for some services it is pretty clear that they are selling data. For example, I find it hard to believe that TikTok is not selling data (then again, who knows?). They currently do not have anything on their site to that effect. For those services we may at a later point use Fiddler or some other tool to check some of their traffic.

• URL is the URL of the privacy policy or other document that indicates whether or not a service is selling data.

• User Verification Data means what the service currently requires to verify user requests to opt out. For example, maybe they have web form that requires entering the ad id. When I looked at Zynga, they seem to require that. Maybe, services also require additional data to verify that a user is entitled to make that request or to verify the user.

• Opt Out URL means the URL of their website for opting out from the sale.

So, take a look at a few cases and see how it goes.

(cc'ing @rgoldstein01, @davebaraka, @kalicki1)

As we discussed, @kalicki1 will meet with @bella-tassone and @ekuller to continue the tutorial on GitHub, especially, branching. Please feel free to create some test branches in this repo.

As discussed, we are going to start developing our app and pushing the code to this repo. @n-aggarwal will fill in more details.

In response to the phase out of the AAID, apps must surely come up with alternative. Facebook's Advanced Matching and Google's Enhanced Conversions fall into this category. What other approaches do apps use?

I imagine, for example, that some apps may use the same intent-based approach that I used in my early GPC for Android prototype, but for tracking.

cc @SebastianZimmeck

Also, @ekuller brought up the question whether URL schemes are helpful for our purposes. Per Apple's documentation:

A URL scheme is the part of a link that specifies the type of application your device uses to open a URL.

So, basically, a URL scheme tells the device to use the pdf app for pdf URLs, etc. As such this functionality does not seem to relate to our question. However, @ekuller, maybe you had something different in mind?

Originally posted by @SebastianZimmeck in #14 (comment)

Within the broad category of HTTP requests, there are different request methods that have different advantages and disadvantages to them, depending on what you want to accomplish. After doing a bit of research on the different types of request methods, I've decided that a POST or PUT request may be appropriate. Here is some information on them, and the differences between them.

Much of the functionality of our library is related to intercepting HTTP requests of ad-providers (issue #36), and modifying those requests to reflect users' opinion on being tracked. In order to make our 'dummy' app more realistic/similar to those of app developers, we should integrate some Ad API into our own app! This will better showcase the functionality of the library.

We discussed today switching from our mobile work to a browser extension (still to be finally decided). I created a repo with a copy of the current OptMeowt extension as a starting point.

If we decide to make that switch, let's push first remaining local code to this repo and document the status of open issues and anything else that is relevant. That way, we can more easily pick it up again at a later point if we decide to do so. At this point, everything is still fresh in our minds.

Let's use this issue to look into a potential solution for issue #19. The Volley library may allow us to send an http request with a modified header that includes the GPC signal.

Implement a custom request with Volley -- Android developer guide discussing how to customize requests

Customize headers -- Stack Overflow post discussing how to modify headers

@rgoldstein01 will look into which mobile technologies to use.

Here is a video that seems like a good overview: Flutter vs React Native vs NativeScript vs Ionic vs PWAs | Maximilian Schwarzmüller, Udemy

Especially, the web app (PWA) approach might be worthwhile. Wrap a website into an app (in which case we may be able to reuse what @davebaraka is doing at the frontend) and still run the app natively, for example, Cordova.

(cc'ing @davebaraka, @kalicki1, and @pakaelbling)