dkentw / 0xbench Goto Github PK

This task aims to improve description of web service:
(1) Add descriptive text to welcome page during loading uploaded results
    (0xbench page URL or brief introduction)
(2) marker -> manufacturer
(3) model -> hardware model

This task is to improve the descriptions in wiki page:
    http://code.google.com/p/0xbench/wiki/Source

At least, it should mention the following:
(1) Java part
(2) JNI
(3) standalone C/C++ programs

as Linaro mentioned ( https://bugs.launchpad.net/linaro-android/+bug/803566 )

To provide a configurable path to 0xbench is necessary.

What steps will reproduce the problem?
1. adb shell am start -n org.zeroxlab.zeroxbenchmark/.Benchmark --ez misc true 
--ez autorun true

What is the expected output? What do you see instead?
 Expected behaviour is that 0xbench ui start and begin to run misc category testes.
 Current behaviour is that 0xbench ui start but doesn't run misc category testes.

What version of the product are you using? On what operating system?
 clone of git repository. On android.

Please provide any additional information below.
 I attach a patch that fix the problem.

What steps will reproduce the problem?
1. Linpack benchmark
2. Scimark benchmark
3.

What is the expected output? What do you see instead?

Expected 90 for single thread or 332.676 for multithread via linpack app in 
market.

0xbenchmark gives linpack result of 15

I did not test scimark, but it is way off


What version of the product are you using? On what operating system?

1.1.5

Please provide any additional information below.

you can see 0xbenchmark result: 

http://0xbenchmark.appspot.com/run/[email protected]/PublicPage


Am I just using the application wrong or did I miss a setting somewhere?

Thanks

What steps will reproduce the problem?
1. Start program
2. Select all benchmarks
3. Click run

What is the expected output? What do you see instead?
Infinite loop

What version of the product are you using? On what operating system?
TAZ Pad with 832M memory available
Android 4.0.4

Please provide any additional information below.


Problem is due to activity state not being saved.
So on memory constrained systems the initial activity which contains the "Case" 
objects is destroyed then recreated when the first test activity ends. The Case 
object state used to determine which test to run is thus lost and the first 
test is rerun (even if not selected).

Attached patch fixes this

What steps will reproduce the problem?
1. Initiate 0xBenchmark on android platform;
2. Check the Native from batch select;
3. Press Run button;
4. Check the output after test done;

What is the expected output? What do you see instead?
A report should be auto generated, but nothing happens after test done.

What version of the product are you using? On what operating system?
0xbench 1.1.1, Android 2.1

Please provide any additional information below.
Other options works well such as Math, 2D, 3D, VM.

Every submission should attach extra system information for better reference.

We can import the existing implementation from system-information project:
    http://code.google.com/p/system-information/

What steps will reproduce the problem?
1. Start Canvas Redraw and Draw Circle

What is the expected output? What do you see instead?
 Expected test are visually ok.
 But a code inspection show that they have both the same logic flaw. They both did not test what they
should be which should be their onDraw() method speed.
 For example for DrawCircle:
After a new view invalidate in DrawCircle.java:oneRound() we return to 
Tester.java:lazyLoop() polling
for end of test and calling again oneRound() until the thread is preempted and 
ui main thread is
scheduled. At this point we execute DrawCircle.java:onDraw() method and 
decrease loop counter. Then
we return to Tester.java:lazyLoop() to invalidate again view and looping until 
new preemption.
 So the test spent the main part of it's time into Tester.java:lazyLoop() and not into DrawCircle.java:onDraw()
as it should do.
 Note that in case we have a dual processor core then test correctly measure DrawCircle.java:onDraw() execution
time.

What version of the product are you using? On what operating system?
 git version on Android

Please provide any additional information below.

What steps will reproduce the problem?
1. Run a 2D benchmark on the phone.
2. Login to get API key on the phone.
3. Upload the results to a new page with specific name such as 
new-result-2d-run1
4. Visit http://0xbenchmark.appspot.com and login.  The page exists with name 
new-result-2d-run1 but when you open the page, no data is shown and the 
following message is shown - "No data available."

What is the expected output? What do you see instead?
I expect to see the benchmark results but see a page with "No data available" 
message.

What version of the product are you using? On what operating system?
0xBenchmark 1.1.5 on Android 2.3.6

Please provide any additional information below.

What steps will reproduce the problem?
1. run oxbench on froyo
2. in Native execution emulator is crashing.
3.

What is the expected output? What do you see instead?


What version of the product are you using? On what operating system?


Please provide any additional information below.

Issue #15 initially merged SunSpider JavaScript benchmark suite into 0xbench.  
However, we should conserve one copy of that and split it from directory src/ 
to somewhere like directory external/ .

This task to to integrate existing file system benchmark suite like IOzone[1] 
or Bonnie++[2].

[1] IOzone: http://www.iozone.org/
[2] Bonnie++: http://www.coker.com.au/bonnie++/

This task is to measure Dalvik Virtual Machine (JVM) performance on mobile 
devices by Caffeine Mark.

For native benchmark items such as libmicro, we execute the programs and 
collect the results through UNIX pipe.  However, we should always perform 
sanity checks in advance.

At least, we need to avoid executing native benchmarks when programs are not 
present.

Inspired by Issue #9, 0xbench should implement accurate graphics calculation 
methods to fit more devices with different resolutions.  We would like to call 
it after "strict mode" to reflect the fine-grained calculation methods for 
Android graphics.

for those, that cannot or do not wish to usu google. market

0xbench is indeed a framework for Android benchmarking purpose, and it would be 
great if we could integrate external system benchmark suite like 
byte-unixbench[1] or freebench[2]

[1] byte-unixbench: http://code.google.com/p/byte-unixbench/
[2] freebench: http://code.google.com/p/freebench/

Reported by George Yuan:

the graphics tests seem to be hitting vsync on the more modern devices, such as 
the new Droids and Galaxy phones. Basically, the frame rate of all your tests 
are artifically capped at 55-60fps, making it difficult to discern the 
performance differences between higher-end devices.

Two potential workarounds are to either increase the number of
iterations until you're running very low frame rates or modify the
benchmark to increase the amount of work drawn every frame until it
starts missing vsync, then use the “amount of work” as the benchmark
result.

Hi, 

I tried to run the UnixBench tests of 0xbench. However, I didn't get any result 
for that because I don't have the binaries for the tests in the /system/bin 
directory. Hence, I tried to build the apk and binaries using SDK, eclipse, and 
NDK as described in the Wiki. However, when I create the Benchmark project 
using eclipse, I always get the following errors:

1. Benchmark cannot be resolved to a 
variable    Case.java   /Benchmark/src/org/zeroxlab/zeroxbenchmark  line 42 Java 
Problem

2. Case cannot be 
resolved    Tester.java /Benchmark/src/org/zeroxlab/benchmark   line 60 Java Problem

and so on.

I guess the above problem is for the following line in Case.java

abstract public ArrayList<Scenario> getScenarios (); 

here the compiler cannot resolve what is Scenario.

I would be very much grateful if you would please help me in this regard.

Thanks.
Syed Rafiul Hussain

0xbench is expected to be useful to devices without GPU for the sake of cost 
and popularity.  It is not sufficient to perform benchmarking over Canvas in 
Android framework.  This task is to improve 0xbench's 2D coverage.

Hopefully, it could cover the existing/corresponding 2D items in Rowboat 
General Runner.

We want to integrate it in the FOSS-Market f-droid. Unfortunately it doesn't 
build. 

http://f-droid.org/forums/topic/0xbench-open-source-benchmark-suite-for-android/

Please give instructions how to get it run. 

2D and 3D bench items are not run in full screen mode right now.  there may be 
status bar affecting the bench result.

Run rendering bench inclusive of 2D and 3D items in full screen.

SunSpider is a JavaScript benchmark. This benchmark tests the core JavaScript 
language only, not the DOM or other browser APIs.  This task is using WebView 
to launch SunSpider tests and then obtain results.

At present, the native benchmark is implemented as standalone applications, 
which are not limited by Dalvik VM and optimized by platform-specific 
configurations (such as armv7-a and NEON).  However, it would be nice if there 
are some corresponding JNI items as baseline.  Thus, contributors could put the 
results to web service without "root".

What steps will reproduce the problem?
1. Open 0xbench on a device in landscape mode
2. It appears in portrait mode

What is the expected output? What do you see instead?

Run 0xbench in landscape mode

What version of the product are you using? On what operating system?

1.1.1 on android 2.2

Please provide any additional information below.

We need landscape mode on Android-x86.