seattletestbed / seattleonandroid Goto Github PK

I have a Samsung Galaxy Tab P3110 with stock firmware that exhibits benchmark problems during install (see also #1093, #1200, plus #1061 and wiki:SeattleOnOpenWrt for possible reasons). What's more problematic than choosing to continue despite of benchmarking errors is that the native app does not log the Python benchmark and installer outputs.

We don't display that info to the user during installation, but write it to files. In my case, the files exist but are empty. A potential workaround during development would be to also dump the required info to the Android log so we can review it with adb logcat.

(Eventually, we will want to make sure we know what procinfo/df etc. formats to expect based on the Android version, and treat the data we gather during benchmarking accordingly.)

Trunk contains many referenes to blackbox.cs.washington.edu. Many of them must be updated in order to function correctly. Of the rest, they are part of previous published research so we shouldn't change those.

On a tablet, with a bad network connection, sensibility makes the device unusable: when tapping on things, the device was unresponsive. For example, the settings menu took a few seconds to respond to clicks.

Once the user clicked to stop the sensibility testbed app, it was back to normal in terms of speed.

We use Intents to pass around information in SeattleOnAndroid (such as the amount of resource to donate during install, or flags to notify threads to shut down). Stack traces on Google Play evidence that sometimes, getExtras() into a Bundle b dereferences an Intent that is null.

Find out why these Intents can be null, and also catch the issue if it can't be avoided.

When installing SeattleOnAndroid, we also install Python via unzipping a tarball that contains the required binaries, libraries, etc. It happens that during install, the "Unpacking Python..." message is followed by "Unpack successful", but when that message is dismissed, the app returns to "Unpacking".

This might be a race between the the pre-installer (trying to detect the magic files it needs to decide whether Python is already installed) and the filesystem (not actually syncing); I haven't seen the loop ever terminate though.

What's unfortunate is that force-stopping the app and then restarting it seemingly makes the installation source Intent disappear. Thus, the install might go through then, but not credit the planned donor or download a custom installer.

We have a few references to several disk-related API calls that are deprecated. Specifically, getAvailableBlocks and getBlockSize. We should update these to use the new, Long version.

See: http://developer.android.com/reference/android/os/StatFs.html#getAvailableBlocksLong()

Example:
https://seattle.poly.edu/browser/seattle/trunk/dist/sensibility-testbed/SensibilityTestbed/src/com/sensibilitytestbed/InstallerService.java#L434

After SeattleOnAndroid was uninstalled, /data/data/com.seattletestbed was removed:

/data/data # ls com.seattletestbed
com.seattletestbed: No such file or directory

But files still exist in /mnt/sdcard/

/mnt/sdcard # ls -l
d---rwxr-x system   sdcard_rw          2013-11-18 09:38 com.seattletestbed

When I use a TCP port as the source port on an outgoing connection, i.e. a connection I initiated, I can still listen for a (different) incoming connection on the same port. Things absolutely are supposed to work like this.

But why does getresources() have that port in the "used" dictionary if I only use it as outgoing (but not listening) port? What can I learn about network ports from the "used" dict that I can't from try:/except:ing my way through the list of ports that are assigned to my vessel?

Dennis Mirante reports that after restarting his Samsung Galaxy Note 2, the Sensibility app fails to restart. I can replicate the problem on a Samsung GT-P3110 running Android 4.1.2. From my logcat output (cf. Dennis' Google Play crash report):
E/AndroidRuntime( 1951): FATAL EXCEPTION: AsyncTask #1
E/AndroidRuntime( 1951): java.lang.RuntimeException: An error occured while executing doInBackground()
E/AndroidRuntime( 1951): at android.os.AsyncTask$3.done(AsyncTask.java:299)
E/AndroidRuntime( 1951): at java.util.concurrent.FutureTask$Sync.innerSetException(FutureTask.java:273)
E/AndroidRuntime( 1951): at java.util.concurrent.FutureTask.setException(FutureTask.java:124)
E/AndroidRuntime( 1951): at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:307)
E/AndroidRuntime( 1951): at java.util.concurrent.FutureTask.run(FutureTask.java:137)
E/AndroidRuntime( 1951): at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:230)
E/AndroidRuntime( 1951): at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1076)
E/AndroidRuntime( 1951): at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:569)
E/AndroidRuntime( 1951): at java.lang.Thread.run(Thread.java:856)
E/AndroidRuntime( 1951): Caused by: java.lang.NullPointerException
E/AndroidRuntime( 1951): at com.sensibilitytestbed.ScriptActivity.getSeattlePath(ScriptActivity.java:127)
E/AndroidRuntime( 1951): at com.sensibilitytestbed.AutostartListener$AsyncStart.doInBackground(AutostartListener.java:31)
E/AndroidRuntime( 1951): at com.sensibilitytestbed.AutostartListener$AsyncStart.doInBackground(AutostartListener.java:1)
E/AndroidRuntime( 1951): at android.os.AsyncTask$2.call(AsyncTask.java:287)
E/AndroidRuntime( 1951): at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:305)
E/AndroidRuntime( 1951): ... 5 more
Looks like some checking and waiting around ScriptActivity.getSeattlePath is required.

After wondering why the transition scripts were not working properly on android devices and digging through the log files, we discovered that the initial vessel V1 could not be split due to lack of resources.

After digging into it a little further, we found that the benchmarking done at install time was not providing accurate measurement of the diskspace. For example, on a tablet that has a sdcard of 24GB, the benchmark reported that a total diskused that was available was 866308096 (20% of that being ~173MB).

Similarly on a phone that had an SD card that had 16GB free space, the benchmarking showed that there was 232812544 available space.

SeattleOnAndroid assumes it's running on devices that will have (physical or other) menu buttons. These buttons are required to reach some of its functions, e.g. displaying the logs. On newer devices and in recent Android revisions, we should be using an ''!ActionBar'' instead, see http://developer.android.com/guide/topics/ui/actionbar.html for details.

We'll want some additional safety nets for donors running on Android.

1. Let's not deplete the battery once its level goes below a threshold. I for one get a warning message when the charge drops below 15%, so this might be a good point when to pause Seattle.
2. We should also pause if we detect we're on a "foreign" (whatever this means) cellular networks, because roaming plus data plan usually equals high cost for the poor donor.

We could put this functionality right into the Seattle APK; another way were to use the new Sensor interface and come up with separate little apps that check resources and send signals to the node manager. We could even suggest these apps at install time, or declare them dependencies of the main package.

It should be possible for researchers using local Android devices running their customized installer to make sure the devices stay on WiFi and not drop into power-saving mode. This would ease the development cycle.

Technically, this would involve making the SeattleOnAndroid app take the WiFi Lock when requested by the user, e.g. by tapping a button in the GUI.

Google Play notifies us that

Your app com.seattletestbed is running an outdated version of OpenSSL,
which has multiple security vulnerabilities. You should update OpenSSL as
soon as possible.
The vulnerabilities were addressed in OpenSSL versions beginning with
1.0.1h, 1.0.0m, and 0.9.8za. To confirm your OpenSSL version, you can do a
grep via ("$ unzip -p YourApp.apk | strings | grep "OpenSSL""). For more
information about the vulnerability, please consult
http://www.openssl.org/news/secadv_20140605.txt

File ./SeattleOnAndroid/src/com/seattletestbed/InstallerService.java includes SSL via javax.net.ssl.SSLContext. I suppose rebuilding the APK with all local libs updated should fix the problem.

There is a note in the main GUI of SeattleOnAndroid inviting users to add sensors to their install. The link points at wiki:AvailableAndroidSensors. The note isn't helpful, and the page pointed at isn't quite up to date. Both should go away.

Given that we have Sensibility Testbed now, we should also come up with a strategy on what to do with existing SeattleOnAndroid installs, whether we can (and want to!) transition them to run Sensibility Testbed or keep both around, what to do with attempted parallel installs (see Sensibility issue #4), etc.

When installing without Internet connectivity (required for downloading the actual Seattle tarball), our app complains that "SeattleOnAndroid could not be installed". The installerDebug log from the 'Show log' menu does include some hints, but I would suspect the problem is common enough to warrant a more explicit error message.

At the moment SeattleOnAndroid leaves files in at least two directories on sdcard, com.seattletestbed and sl4a/seattle. The latter folder seemingly cannot be deleted by Android's on-board uninstall means; I'm not sure about the former. The problem is that leftovers from an old install can trick a new install into not constructing a new vesseldict etc.; also, we don't play nice with packages that are based on ours (such as Sensibility Testbed).

We should install in a way that our package can be cleanly and completely removed by system means.

Python libraries like platform require specific low-level POSIX command line tools such as uname to be in place in order to work properly. On Android, those tools might or might not exist. The resulting error messages like "uname: not found" might affect the platform detection of Repy V1 and V2; at least they are confusing to the user trying to run Repy from the command line on their Android devices.

We should review which of the platform detection methods is best suitable for us, and possibly develop some kind of fallback. Following this stackoverflow thread, the platform.* methods perform runtime checks using POSIX tools, whereas the related sys.* methods use build-time information.

Line 353 of Linux_resources.py tries to figure out the available disk space during install:

 statresult = commands.getstatusoutput('df -P .')

The problem is, the standard Android "toolbox" (single-binary replacement for small command-line tools) version of df doesn't work with the -P option.

On Florian's Sony Tablet S running Android 4.0.3:

shell@android:/ $ which df
/system/bin/df
shell@android:/ $ ls -l /system/bin/df
lrwxr-xr-x root     shell             2012-04-26 20:54 df -> toolbox
shell@android:/ $ cd sdcard2
shell@android:/sdcard2 $ df -P
Filesystem             Size   Used   Free   Blksize
-P: No such file or directory
shell@android:/sdcard2 $ df .
Filesystem             Size   Used   Free   Blksize
.                       30G     4G    25G   32768

For comparison, on our office smartphone, a ZTE Blade running Android 2.3.7 (!CyanogenMod-7.1.0-Blade), we have "busybox" instead of "toolbox", which works:

localhost / # which df
/system/xbin/df
localhost / # ls -l /system/xbin/df
lrwxrwxrwx    1 root     root             7 Oct 14  2011 /system/xbin/df -> busybox
localhost / # df --help
BusyBox v1.19.1-cm71 bionic (2011-09-01 04:24 +0200) multi-call binary.

(...some output omitted...)

localhost / # cd sdcard
localhost ~ # df -P .
Filesystem           1024-blocks    Used Available Capacity Mounted on
/dev/block/vold/179:1    1932960   1662144    270816  86% /mnt/sdcard
localhost ~ # df .
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/block/vold/179:1
                      1932960   1662144    270816  86% /mnt/sdcard

The installer log shows this error:

New installation, beginning benchmark.
Total resources measured by the script for Linux OS: 
{'diskused': 'unable to read disk partition size', 
'fileread': 186833, 'filewrite': 186833, 'loopsend': None, 'lograte': None, 
'netrecv': None, 'random': 127100, 'insockets': 341, 'filesopened': 341, 
'looprecv': None, 'cpu': 2L, 'memory': 758160000L, 'netsend': None, 
'outsockets': 341, 'events': None}