I connected this app with CK repository. Still need to fix table visualization and graph there.
However, I already see that there can be various further improvements to be done:

• when starting application, do not start detecting platform, exchanging info, etc -> only read scenario files and show in the list!
• when pressing 'update', detect platform info as now, but then record it to the {sdcard}/openscience/platform-features.json - then, when you start application again, pre-load it to memory without any more need for exchange with CK server ...
• when pressing 'recognize' button, we should not again detect platform features -> they should be preloaded as above -> just make global JSONObject plat-feat ...
• we should record pictures from camera to ${sdcard}/openscience/tmp , otherwise it's not clean at the moment (we record it in ${sdcard}). When starting application, clean this directory
• send correct height and width to CK

By the way, just a convention that instead of using vars with the following names "crowdUID", we should use "crowd_uid", i.e. always lower case and '_' ...

I just noticed that GPU is not anymore detected (at least not passed in platform features). Is it possible to check what happened, please?
Thanks!

Some times application log required for details.
It better to have option share log file (for example using standard android share functional).
Or send log to authors.

Hi all,
I have added 4 more scenarios to this app (they may not work due to classification quite hardwired to AlexNet with mean file, but we need to at least make app not fail).

Here I have two major problems:

• When pressing on Update, actually new scenarios are not loaded. When checking out app (the behavior of pipeline is now becoming quite complex and not very clear, while I think it can be quite simple), I noticed that in fact, whenever we get new scenarios for the first time, they will never be updated. That's wrong. When we press 'Update', we should simply go through the original process or testing CK server, detecting features, saving them, getting scenarios, recording them, and showing them (WITHOUT pre-loading or running any default scenario!).

• When I manually cleaned scenarios, I managed to get all scenarios. Now, when I run any scenario, including the original one, I see files being downloaded, but then app simply crashes. Even after files are preloaded, app not still crashes even on the original scenario ...

Therefore, may I ask to slightly simplify up, so that I can get all scenarios from the web and update platform features, each time I press 'Update', and then confirm and check that app now crashes on all scenarios...

Thanks a lot!!!

I've implemented patch
at patchOpenCL()
https://github.com/dividiti/crowdsource-video-experiments-on-android/pull/79/files

and it works at least for my Galaxy Note 4 witch does not have libOpenCL.so lib at system paths and
there is only /system/vendor/lib/egl/libGLES_mali.so
and it compatible with libOpenCL.so

but Galaxy S7 has
libOpenCL.so 64 version
and /system/vendor/lib/egl/libGLES_mali.so 32 as well
and patch applies for Galaxy S7 as well but shouldn't so should be another way
to support both version of phones.

Now patch for old phone is disabled at
app/src/main/java/openscience/crowdsource/video/experiments/MainActivity.java (933)

Images are taken from front camera rotated to wrong direction.
Image opened from file should be rotated.
Make sure image witch sends to remote repo has correction orientation.

Looks like camera preview alwais in free focus mode and auto focus does not work

if there is no scenarios for device scren with scenarios looks like:

Need to disable top link to this screen if now scenarios found.

@gfursin will add full URL to the return JSON (on first request with results)
I'll add option to view results on the web immediatelly (@anton was suggesting that for some time now)

Instead of having time1,time2 and time3 (temporal hack), we should create a loop, aggregate results in a 'time' list, and send it to server ...

I noticed a strange bug - when I was downloading a very big model and in the middle I switched Internet from mobile to WiFi, download continued as normal, but then started showing me 290Mb of 271Mb completed. Looking at log, it seems that app restarted download, but the counter of the downloaded bytes did not reset at the same time ...

Info from @psyhtest:

FYI: on the X20 boards, libOpenCL.so is a symbolic link to:

lrwxrwxrwx root     root              2010-01-01 00:05 libOpenCL.so -> /system/lib/egl/libGLES_mali.so

At the moment we search for libOpenCL.so only in /system/vendor/lib/libOpenCL.so,
so we should add '/system/vendor/lib' path to search too. In fact, we should create a list
of possible paths where to search. 

All travis builds fails
https://travis-ci.org/dividiti/crowdsource-video-experiments-on-android/builds/207979380

after Commit
6b0b447

Looks like some re configuration required at .travis.yml

Fix recognition result parser for opencl scenarios:
skip extra GPU info

The new versions of the app may be adding detection of new platform features. Hence, I suggest to record somewhere in config current version of the Android app, and if it's updated, call platform feature detection automatically at the beginning of the program ... Discussion is welcome ...

We currently use Build.CPU_ABI to detect processor ABI (to specialize our scenarios). However it is now deprecated and it seems that some latest devices return "armeabi-v7a" instead of "arm64-v8a" ...
Daniil, you have this problem on your device (see stats at cKnowledge.org/repo) - do you mind to check what is the better solution? For example, here ( http://stackoverflow.com/questions/11989629/api-call-to-get-processor-architecture ) someone suggests to use:

public static final String CPU_ABI = getString("ro.product.cpu.abi");

Maybe it's the right solution ...

It's quite urgent to fix it!
Thanks a lot!

We will need to rebuild Caffe with OpenCL and run it on Samsung S7. I will need to check that we can compile caffe for Android via CK pipeline. Then we will add it as a scenario to this crowd-benchmarking app.

Now layout is the same for portrete and landscape orientation for result screen:

It better to implement landscape orientation for result screen the same as for main screen.

App does not work on some devices

blackberry z30
blackberry os 10.3.2.2836

Just crushes:

Need to add global exception handling
Thread.setDefaultUncaughtExceptionHandler(new ExceptionHandler(this));

and add to handler logging to own app log reason of problem and allow app to start and view main screen with problem displayed at mini log view at main screen

I had to remove "autofocus" requirement from Manifesto, to be able to install app on cheaper Android platforms that do not have autofocus (however they still can recognize pre-recorded images). However, on such devices camera is not working at all. Maybe there is a way to check whether camera has autofocus or not and still support it even without autofocus (if it's the problem) ... Thanks!

Hi Daniil,
While testing latest version found a minor glitch. My email got erased when I installed new version and instead there is a 'acknowledge your contributions' sent to server. So
a) email should not be removed (there is a likely glitch)
b) if email is empty, you should send empty string, i..e 'acknowledge your contributions' should NOT be a default text, but there is a way to make it as a semi-transparent text which is shown in the field when there is no text...
Thanks!

1. Add states for Scenario

• NEW - not downloaded yet, Download button available
• DOWNLOADING_IN_PROGRESS - size look like 10 of 260 M, download button switched to green circle means in progress
• DOWNLOADED - size look like 260 MB (downloaded), download button and green circle not shown

2. Add on click handler for Button

3. Subscribe UI controls on updates from async task to see real time progress at drop down

4. Apply refactoring to mooving out every work with Scenariouse preloading and file downloading to separate RecognitionScenarioService and ReloadScenarioAsyncTask

platform_features does not exist at request after recognition now

http://cknowledge.org/repo ...

Some times at the end of the recognition process
alert window is not displayed. Case to reproduce

1. Start recognize
2. Let smartphone to turn off the screen (screen auto lock)
3. Unlock the screen
4. Finally dialog should be displayed but in this case it does not because of unhanded error:

 E/WindowManager: android.view.WindowLeaked: Activity openscience.crowdsource.video.experiments.MainActivity has leaked window com.android.internal.policy.impl.PhoneWindow$DecorView{1e7b6232 V.E..... R....... 0,0-1336,1173} that was originally added here
                     at android.view.ViewRootImpl.<init>(ViewRootImpl.java:475)
                     at android.view.WindowManagerGlobal.addView(WindowManagerGlobal.java:277)
                     at android.view.WindowManagerImpl.addView(WindowManagerImpl.java:69)
                     at android.app.Dialog.show(Dialog.java:312)
                     at openscience.crowdsource.video.experiments.MainActivity$RunCodeAsync$4.run(MainActivity.java:2449)

At the moment I hardwired path in scenario CMD such as '$#local_path#$/openscience/data/ba4.../topology/deploy.prototxt'

Instead, we should use '$#ck_local_path#$/$#SCENARIO_KEY_TOPOLOGY#$'

Android app should substitute $#ck_local_path#$ with {sdcard}/openscience
and then check all files to find a key SCENARIO_KEY_TOPOLOGY (not done yet - I have to add it to meta) and substitute $#SCENARIO_KEY_TOPOLOGY#$ with data/ba4.../topology...

Then we will not have any hardwired path in scenarios any more.
It's not urgent and we need to do it at the same time on Android side and CK side ...

It's not really related to this Android app but to the scenarios, but I added it here for now:

I successfully compiled Caffe libs for Android, and 'classification' program works, but 'caffe' fails on 'caffe time --model=xyz.proto --iterations=1' with seg fault ... Is it possible to check it via adb shell, why it fails?

Now travis runs build but there is no link to apk.

Should have a small button to flip camera (front/back).
Should have a small button to open image (just pictogramm).

At the end of recognition open an alert window saying the top recognition in red bold, and then with smaller other ones with confidence and ask correct or not. If not correct, ask for a correct title and ask if it's ok to send image, wrong and right prediction to CK. Then prepare dict with file in MIME64 + right and wrong prediction and send it to server (I will add function) ...

I have a strange issue on a tablet ACER B1-780 (Android 6.0, ARM64). Recognition is fine on 3 first models (GoogleNet, SqueezeNet), but fails on CaffeNet and AlexNet. In fact it returns the following error:

__bionic_open_tzdata_path: ANDROID_DATA not set!
__bionic_open_tzdata_path: ANDROID_ROOT not set!

and empty recognition. I don't know if it will be easy to reproduce such issue, because all models work fine on my S7 ...

Layout is not supported for tablet now. Too big log aria..

App just crashes while starting app on T-Firefly dev board.
Android app log:

E/AndroidRuntime( 1799): java.lang.RuntimeException: Unable to start activity ComponentInfo{openscience.crowdsource.video.experiments/openscience.crowdsource.video.experiments.MainActivity}: java.lang.Nul
lPointerException: Attempt to invoke virtual method 'void android.widget.Button.setOnClickListener(android.view.View$OnClickListener)' on a null object reference
E/AndroidRuntime( 1799):        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2345)
E/AndroidRuntime( 1799):        at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2407)
E/AndroidRuntime( 1799):        at android.app.ActivityThread.access$800(ActivityThread.java:151)
E/AndroidRuntime( 1799):        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1319)
E/AndroidRuntime( 1799):        at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime( 1799):        at android.os.Looper.loop(Looper.java:135)
E/AndroidRuntime( 1799):        at android.app.ActivityThread.main(ActivityThread.java:5280)
E/AndroidRuntime( 1799):        at java.lang.reflect.Method.invoke(Native Method)
E/AndroidRuntime( 1799):        at java.lang.reflect.Method.invoke(Method.java:372)
E/AndroidRuntime( 1799):        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:963)
E/AndroidRuntime( 1799):        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:758)
E/AndroidRuntime( 1799): Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'void android.widget.Button.setOnClickListener(android.view.View$OnClickListener)' on a null object ref
erence
E/AndroidRuntime( 1799):        at openscience.crowdsource.video.experiments.MainActivity.onCreate(MainActivity.java:332)
E/AndroidRuntime( 1799):        at android.app.Activity.performCreate(Activity.java:6222)
E/AndroidRuntime( 1799):        at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1106)
E/AndroidRuntime( 1799):        at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2295)
E/AndroidRuntime( 1799):        ... 10 more
W/ActivityManagerService(  456):   Force finishing activity 1 openscience.crowdsource.video.experiments/.MainActivity
W/AppOps  (  456): Bad call: specified package media under uid 1000 but it is really 1013

First time need to ask question before first model downloading where to store files.

I have a minor issue with some buttons hidden on my tablet. Please, see attached screenshot. Will it be possible to solve it?

Thanks!

First of all describe test plan here to cover most cases

GPU is not detected correctly - it can be done via OpenGL, but I found it quite tricky. If it's possible to figure out how to do it and add it (maybe adding invisible GLSurfaceView), that would be very useful when we will be adding OpenCL-version of Caffe ...

Step to reproduce

• Install and run App
• load new scenarios
• Download files for first scenario from the list ("Caffe CPU: Squeezenet 1.1 img (26 MB)")
• Run "Recognize"

Actual result :
Error at log
CANNOT LINK EXECUTABLE: could not load library "libcaffe.so" needed by "./classification"; caused by library "libcaffe.so" not found

Expected result :
should recognize fine as before

We should add the following (in case scenario binary has xOpenME instrumentation):

• clean tmp-ck-timer.json before running a scenario (in a working directory)
• check if tmp-ck-timer.json exists and if yes, load it and add it as key 'tmp-ck-timer' when sending results to server

Steps to reproduce

• Start app
• Clean up scenarios
• Start downloading any big scenario
• Emulate connection reset - for example turn off Wi-Fi

Actual result

• Scenario with incomplete downloaded files looks like ready to use for recognition, but if you select and start recognition it will resume downloading except the just recognizing

Expected result

• Catch connection timeout exception and and try to reconnect for example 3 times if it still connection timeout exception mark scenario as NOT DOWNLOADED and print sorry message to log or alert dialog

The new version is now working quite well, so before big layout refactoring I suggest to fix the following bugs and make a few improvements:

• BUG: after I capture an image from webcam and recognize it, I press recognize again. However, now the default image is recognized instead of the one which was captured. We should always remember last image and recognize it. For example, we want to recognize the same captured image with different neural networks - currently it's impossible. I think it should be easy to fix, but it's very important for comparison!

• BUG: At the moment, there is no check for stderr (ret[2]). So even if stdout is empty, results will be sent to server. We should be checking that stdout is empty, print stderr, and stop recognition/statistical repetitions without sending info to server. It's easy to emulate - just remove libcaffe.so from LD_LIBRARY_PATH ...

• We should make 4 statisitcal. iterations instead of 3, and ignore the first iteration (mobile usually warms up if it was in a low freq state)! We should also print recognition results to log after this first iteration and last (so that if it's long, at least users see what was recognized)...

• I added a few keys to "files" in scenario meta:

• "library":"yes" - tells you that you should add a given file to LD_LIBRARY_PATH (currently you search for .so)

• "default_image":"yes" tells you that this is a default image (instead of hardwiring it - then we may have several images and change them from time to time). If the recognized image is default, we should add a key "default_recognized_image":"{name of the image}" along time when submitting results to server. Because if we change it in the future, we will not know which one was recognized.

• Do you mind to collect all cpu_freqs (there is already a function) just after first warm up run, and also after all iterations and add it along with time when sending to server using keys "cpu_freqs_before" and "cpu_freqs_after".

This will make this app even more usable. In the mean time, I will try to add OpenME instrumentation to extract time of model init and recognition ...

Thanks a lot!

Implement new design accoriding to provided mockups available at
https://app.zeplin.io/project.html#pid=58206d680d55b47443f12d55&dashboard
for subscribed users (only main contributors now).

Actual issue state:
Done:

• navigations (from secondary activities use back button to return to recognize main activity now)
• scenarios dropdown style (basicaly)
• Suggest screen style
• fix captured image data receiving (now catture but do nothing)
• remove from logger some errors related to empty app config values (it's ok)
• display currently selected scenario on result page
• set selected scenario on return back
• toolbar design margins, remove rectangles around the buttons
• display result page even if we've switched to other activities (log, about) and then returned back to home see recognition result
• some other design / style issue like margins corners and so on ...
  ** gray style for Recognize button and scenraio selection if not available - not actual. Mini logs under image view is more useful
• scenarios dropdown (separate issue #38)
  • download / downloaded ico
  • replace hardcoded volume on recieved
• confirmation dialogs (email, correct answer, clean up confirmation) according to common design and style
• recognition result preview as faster as we can on top of the image view over the transparent bar except the capture and open image buttons while recognition is cumming

In complete

• optimize logging using in-memory structure (ConcurentHashMap)

• not actual at this moment, problem was in big log file. Now it clears up at start up and works file.
  all optimisations will be applied at separate issue.

This branch continued at issue-38, so finally all items from this issue and issue #38 will be merged to master at PR #39

• add currect application version at the top of the log
• add copy logs to buffer button
• add send logs by email button

Add standard app menu and move all low priorities buttons like:

• Update
• Users
• view crowd results
• Clean log
• Get open-source SDK
• Edit email
• About

Add OpenME to classification to dump timing in JSON of loading a model, initializing it, loading image, recognizing, storing results, etc ...

New classification has XOpenME included to expose fine-grain timers. We now need to remove tmp-ck-timer.json, and then if it is created after executing classification on a mobile device, we need to add this dictionary to "raw_results":{"xopenme": [{xopenme json from stat repetition1}, {xopenme json from stat repeittion2}, ...] and then send it back to server besides global time ...

Crushes on start app if no scenarios found for CPU architecture

Add such 'highlight_by_user=EMAIL" to the URL at "view results"
to highlight crowd result rows at
http://cknowledge.org/repo

Landscape orientation now is not properly supported.
Need to adopt current layout for landscape orientation as well.

change about link to http://cKnowledge.org