taikoo / mobitest-agent Goto Github PK

What steps will reproduce the problem?
1. Run a test on news.bbc.co.uk
2. View the HAR file
3. BBC logo is included as a dataURI and this is listed as a separate in HAR

What is the expected output? What do you see instead?

dataURIs should not be included in HAR as they are not HTTP traffic (included 
in other HTTP traffic)


What version of the product are you using? On what operating system?


Please provide any additional information below.

I got a crash in BZAgentController.mm method

- (void)jobCompleted:(BZJob*)job withResult:(BZResult*)result

at line

result.screenShotImageQuality = job.screenShotImageQuality;

This is because "job" is owned by the BZWebViewController, which is dealloced 
after

[self dismissModalViewControllerAnimated:NO];

As a result "job" is released, which leads to a exception when trying to access 
the poperty "screenShotImageQuality".


Solution:
Just move the line 

result.screenShotImageQuality = job.screenShotImageQuality;

before the line 


[self dismissModalViewControllerAnimated:NO];

and everything works fine.

Thanks,
Oliver

P.S.: The resulting code after the fix:

- (void)jobCompleted:(BZJob*)job withResult:(BZResult*)result
{   
#if BZ_DEBUG_PRINT_HAR
    NSLog(@"Job completed");
#endif

    result.screenShotImageQuality = job.screenShotImageQuality;

    //Dismiss the web view
    [self dismissModalViewControllerAnimated:NO];

    [idleView showUploading:@"Job succeeded"];

    // Compress screenshots with the image quality setting of the job.

    busy = NO;

    [self restartIfRequired];

    [self processNextJob:YES];
}

See also the fixed source attached.

TouchJSON is no longer at the URL provided in 
iOS/BZAgent/Depencies/README-DEPENDENCIES. The current URL would appear to be 
https://github.com/TouchCode/TouchJSON.git 

clang: error: no such file or directory: 
'~/mobitest-agent-read-only/mobitest-agent/iOS/BZAgent/Dependencies/TouchJSON/CD
ataScanner.m'
clang: error: no input files

The android agent uses the nearest video frame to an event for its screen 
capture.  This can be off by a substantial amount.  Investigate capturing on 
the event callback.

What steps will reproduce the problem?
1. http://mobitest.akamai.com/m/results.cgi?testid=130204_CQ_DD
2.
3.

What is the expected output? What do you see instead?
the load time should be about 9 secs

What version of the product are you using? On what operating system?


Please provide any additional information below.
Once this is fixed, it should apply to the mobitest app as well.

What steps will reproduce the problem?
1.  We see a very noticeable time difference between the first test and the 
second/third runs for any test/any device.  Wondering what is causing the time 
difference.  
2.  Tried on multiple tests/devices/locations.
3.  Tried on different days.

What is the expected output? What do you see instead?

I wouldn't expect to consistently see the first request be the slowest of the 
three test runs.

What version of the product are you using? On what operating system?

Using http://mobitest.akamai.com

Please provide any additional information below.

Here are some of the test results:

http://mobitest.akamai.com/m/results.cgi?testid=121024_X4_CS&video=0&src=blaze

http://mobitest.akamai.com/m/results.cgi?testid=121024_WK_CT&video=0&src=blaze

What steps will reproduce the problem?
1. Start a test on android
2. wait for the result

What is the expected output? What do you see instead?

The results of the test are expected, "The test completed but there were no 
successful results." and "First View: Test Data Missing" are what i get instead.

What version of the product are you using? On what operating system?

I'm using the version downloadable from the repository of the project, compiled 
on windows 8 with eclipse and installed both on an android avd emulator with 
cyanogenmod 7.1 (API level9) installed on it and on a LG P350 with android 
2.2.2 rooted. The private instance of webpagetest is installed on a windows 
server 2003 r2.

Please provide any additional information below.

This is the situation on the server: 
http://img18.imageshack.us/img18/3244/cattura1d.png

I've checked if blaze agent appears on the list of allowed application on 
superuser and it does. I also have tried to capture some traffic with tcpdump 
maually using the terminal and it works. Finally I've tried to execute blaze 
agent with the debugger of eclipse enabled and both from emulator and from a 
physical device i obtain the same error. I've pasted it here: 
http://pastebin.com/b3RY6xrP and, just in case, i've pasted the entire capture 
of a test here: http://pastebin.com/tmYhkRZ3

On the server i can find the screenshots of the test but the har files are 
empty.

What steps will reproduce the problem?
1. Flash a device or emulator with the latest JB MR
2. Install Superuser and BZAgent as usual
3. Run BZAgent and click "Poll Now"

*** What is the expected output? ***

The initial BZAgent poll should trigger a Superuser prompt to allow/deny su 
access.  The run should succeed.

*** What do you see instead? ***

The Superuser prompt never occurs.  The ProcessManager testRootPermissions "su":
    echo blah > /system/sd/temporary.txt
returns an exitValue of 1, which is incorrectly declared success (since it's != 
255).  The "su" tcpdump also silently fails.  However, the end-of-run zip fails 
because the pcap dir is empty.

The above "/system/sd/" path doesn't exist on JB or ICS.  If I change it to a 
valid path, it still fails.

*** Please use labels and text to provide additional information. ***

The Eclipse log contains:
    01-08 20:40:02.957: I/ProcessManager(1700): Testing root permissions
    01-08 20:40:02.965: E/su(1813): sudb - Opening database
    01-08 20:40:02.965: E/su(1813): sudb - Database opened
    01-08 20:40:02.965: E/su(1813): sudb - Database closed
    ...
    01-08 20:40:03.293: W/ActivityManager(402): Permission Denial: broadcast asks to run as user -1 but is calling from user 0; this requires android.permission.INTERACT_ACROSS_USERS_FULL or android.permission.INTERACT_ACROSS_USERS
    ...
   01-08 20:40:23.668: W/su(1813): request rejected (10063->0 /system/bin/sh)

A web search found the above error:
    http://forum.xda-developers.com/showthread.php?t=1989261&page=2#postmenu_34063138
The proposed workaround is to use SuperSU instead of Superuser.  I tried that 
via:
    adb install SuperSU_0_99.apk
which installed, but when I used the app to update the su binary, it failed 
with:
    Installation Failed!
This might be because I installed via "adb install" instead of the playstore:  
(untested)
    http://forum.xda-developers.com/showthread.php?t=1989261&page=4#postmenu_34154143

*** Proposal ***

testRootPermissions should detect the above "su" error.

Warn users re: JB.  Recommend ICS until we have a fix.

BZAgent should work without a Superuser-like app on rooted images, provided 
tcpdump is chmod'd appropriately. The validation should be tweaked -- make sure 
tcpdump works, as opposed to indirectly checking that su works. It's still OK 
to run via su, it's just a no op without the superuser app, but tcpdump itself 
will work if chmod'd.

The desktop agents on WebPageTest support overriding the IP a certain hostname 
would resolve to in two ways:

setDns allows specifying a specific IP address a host should map to: 
https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting#TOC-
setDns

setDnsName allows specifying an alternate hostname that should be resolved 
instead of the original hostname: 
https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting#TOC-
setDNSName

We need to have such support in Mobitest as well. 
Support for setDnsName is more important than support for setDns, but 
presumably both can be added with roughly the same level of effort.

Note that we want to support this *without* requiring a jailbroken device. If 
we determine that's not doable, even with private APIs, we can reconsider.

See if we can make the iOS screenshots a bit faster, to support more frequent 
screenshots for fast-loading pages.

Note: the default "bz-fps" is 1 and the "Settings.bundle/Root.plist" only 
allows the developer to chose from 1, 2, 5, 10; not sub-second resolutions.  If 
we can optimize the screenshots then we should allow smaller values.

The raw screenshot work is in "UIGetScreenImage()".  We likely can't do 
anything to optimize that.

However, the post-capture work could perhaps be optimized, assuming that it's 
significant relative to the screenshot time.  The "TODO" code comments in 
BZWebViewController.mm captureScreen suggest some ideas:

1) Instead of converting the image to a JPEG via:
       NSData *imageData = UIImageJPEGRepresentation(image, 1.0);

  a) Use at a lower quality than 1.0, especially because the BZModel+ZIP.m upload will scale down these video images anyways based on the "bz-vid-jpg-quality" setting, which defaults to 0.3  (0.7 for non-video "bz-chkpoint-jpg-quality" checkpoint screenshots).

  b) Similarly, scale the images here instead of in BZModel+ZIP.m, 'though the default "bz-jpg-size" scale factor is 1.0 (i.e. don't resize).

  c) Do this conversion later


2) Instead of writing to the disk after every capture, keep them in memory and 
batch-flush as necessary (e.g. at end of each run),

Today, Mobitest only supports WebPageTest tests that provide a URL to test. 
The desktop agents, however, support running a job using a WPT script as well 
(as detailed here: 
https://sites.google.com/a/webpagetest.org/docs/using-webpagetest/scripting). 

We need to add support for scripting in Mobitest as well. 

As a first step, the only script command that needs to be supported is 
"navigate", which is practically the same as getting a URL to test. However, 
the code should make it easy to add additional commands over time.

The android agent builds the .har of network timing info on the device.  We can 
do this on the server, and avoid duplicating the har manipulation code on both 
client and server.

What steps will reproduce the problem?
1. Run Android BZAgent on phone and begin polling a local instance of 
WebPagetest.
2. Once a result has been picked up, notice how the URL is navigated to in the 
browser succesfully.
3. Program seems to halt and throw an exception due to a zip file issue.

What is the expected output? What do you see instead?
I expect there to be a zip greater than the size of 0 bytes in the folder 
'/sdcard/blaze', I also expect these results to be sent to WebPagetest 
successfully and have them displayed in the browser.

What version of the product are you using? On what operating system?
Using Mac OS X, running the agent from Eclipse. Running a private instance of 
WebPagetest 2.6. Tried the agent on an emulator and Nexus S with the same 
problem. Devices use Android 4.0.

Please provide any additional information below.
Note, originally no data at all was being sent back, I then renamed the 'lib' 
folder to 'libs' and noticed a screenshot was sent back.
If I run adb shell, I can see a file which is 0 bytes named: 
"120419_VY_2_1_0.pcap.zip" (I also see a results.har file)

Error Messages in LogCat:
E/BZZipUtil(15010): java.util.zip.ZipException: No entries
E/BZZipUtil(15010):     at 
java.util.zip.ZipOutputStream.finish(ZipOutputStream.java:231)

The android agent can build a HAR today.  It also has an option to upload pcaps 
and let the server do HAR construction.  When the server does HAR construction, 
the agent should upload any data it has outside the pcap.  Today it sends 
doccomplete and fully loaded time.  It shoudl also send times of resource loads 
as seen from webview callbacks.  

Looking at the client side har creation code should show what data can be 
uplaoded.

What steps will reproduce the problem?
1. Keep the mobitest agent running for  more than 24hours
2. Observe the logs when it crashes


What is the expected output? What do you see instead?
Expected to run mobitest agent seemlessly

What version of the product are you using? On what operating system?
built of latest mobitest code base on IOS-5 and IOS-7

Please provide any additional information below.
Attaching the Crashlogs

Periodic screenshots are captured relative to the run start, not the first 
paint callback, so sometimes we miss the first paint screen.  This is 
especially bad if our page loads quickly (<2 seconds) but our screenshots are 
infrequent (e.g. once per second).

This also causes some unrelated bugs, e.g. screenshot before first paint, a 
long delay until the first screenshot, etc.

Instead of capturing periodic screenshots relative to the run start time, the 
iOS BZAgent should capture a screenshot on the first paint callback and then 
periodically relative to the first paint time.

The agent should still capture the usual marker screens as before:
  1) on "startSession", to capture the initial "blank" screen
  2) on "startRender" (same as first paint callback?)
  3) on "docComplete"

Let's fix iOS first, then the other agents (Android/etc).

What steps will reproduce the problem?
1. Install the BZAgent on iphone/ipad
2. run performance test
3. check the waterfall

What is the expected output? What do you see instead?
we should see time to first byte, dns lookup numbers should be shown for each 
object/resource in the page

What version of the product are you using? On what operating system?
Latest version installed in mobitest UI 

Please provide any additional information below.
check the waterfall on 
http://mobitest.akamai.com/m/results.cgi?testid=131212_DP_3Y

What steps will reproduce the problem?
1. Create a HAR file using Mobitest
2. View the raw HAR file 

What is the expected output? What do you see instead?

As per HAR spec (http://www.softwareishard.com/blog/har-12-spec/#log) expect to 
see:

{
  "log": {
    "version":"1.1",  or 1.2
    "creator": {

Actually see:

{
  "log": {
    "version":"1.8",
    "creator": {


What version of the product are you using? On what operating system?

Mobitest Agent v1.8 on iOS


Please provide any additional information below.

Will fix this as part of the changes to support landscape