Is it possible to grab the developer email address from the API?

I'm new to google play crawler and java, my server is on Amazon cloud, US Region.
I update /home/ubuntu/google-play-crawler-master/googleplay/crawler.conf with my gmail account and password.

root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf checkin
Your account succesfully checkined!
AndroidID : 33dd2d49569f3f23
SecurityToken : 4eb656111550c479

I update /home/ubuntu/google-play-crawler-master/googleplay/crawler.conf with above AndroidID and SecurityToken

root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf register
A device is registered to your account! You can see it at "https://play.google.com/store/account" after a few downloads!

root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf search games
Title;Package;Creator;Price;Installation Size;Number Of Downloads
Index: 0
root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf search facebook
Title;Package;Creator;Price;Installation Size;Number Of Downloads
Index: 0
root@ubuntupc:/home/ubuntu/google-play-crawler-master/googleplay# java -jar googleplay.jar -f crawler.conf search play
Title;Package;Creator;Price;Installation Size;Number Of Downloads
Index: 0

Any suggestion on why I always get Index:0 ?

Hi sir,
I have been trying to build the code following the steps specified. My development platform is osX and other sbt and maven details are following:

Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 19:21:28+0530)
Maven home: /Users/tushar/Downloads/apache-maven-3.0.5
Java version: 1.6.0_65, vendor: Apple Inc.
Java home: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
Default locale: en_US, platform encoding: MacRoman
OS name: "mac os x", version: "10.9", arch: "x86_64", family: "mac"

The sbt step runs fine but the maven step is giving the following problem:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project googleplaycrawler: Compilation failure -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project googleplaycrawler: Compilation failure
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:290)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:230)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute(AbstractCompilerMojo.java:862)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute(CompilerMojo.java:129)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
    ... 19 more
[ERROR] 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

I would be glad if you could help me solve this issue.

How to determine where is logo vs screenshots in returned ImageList ?

List imageList = docV2.getImageList();

I got this error:
Exception in thread "main" com.akdeniz.googleplaycrawler.GooglePlayException:

Forbidden

Error 403

at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeHttpRequest(GooglePlayAPI.java:522)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeGet(GooglePlayAPI.java:513)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeDownload(GooglePlayAPI.java:356)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.download(GooglePlayAPI.java:320)
at com.akdeniz.googleplaycrawler.TestGooglePlayCrawler.shouldDownload(TestGooglePlayCrawler.java:149)
at com.akdeniz.googleplaycrawler.TestGooglePlayCrawler.main(TestGooglePlayCrawler.java:163)

Can you help me with this ? Thank you!

I would like to know if i can get what kind of architecture is supported by apk?.
is x86 supported?.
is optimized for tablets?.

Hi

First, thank you for great job.
I want to ask, how i can change country of google play from which google return category application list. Changing only localization return the same list of applications. I found in googlePlayApi.java only one country string - device_contry in checkin. Is that it? If yes, should I "rechekin" for each country which i want to crawl?

Thank for reply

In order to parse apps and comments in differents languages we need to provide different locales per request. How it could be implemented with this library ? Thanks.

In TestGooglePlayCrawler.java, login() will be failed by using the android-id generated after run checkin().
Listcommand also failed.

Hi, Akdeniz
Really grateful for your google-play-crawler, I can checkin and download apps with a new gmail account without 'Android Id' now.
But I got a problem when access https://play.google.com from the browser, after login with the same account just checkin, it did not display my account was binded with the Galaxy S3, it say "You don't have any devices", and then I can not download any apps from the browser.
So can you help me, waitting for your reply....

Hi,

I tried to figure out how to download paid apps, but i was not able to find the way to do it.

It would be helpful if you can tell me how this flow works

Thanks

I am just wondering how it works without android id ? I'm really surpised about that. I just provided google username an password and that's it. Is it normal to work without valid android id ? Thanks.

Tried installing the project several times.

At first, thought it was a scala/sbt version problem but after several tries, I always end up with this error

| private Queue filteredMessages = new ConcurrentLinkedDeque<>()

Any suggestions?

OS: Ubuntu 12.04

Is it possible to get developer's information, for example name and email via service.bulkDetails(packageNames).

I tried following code and it always empty (except packageName):

BulkDetailsResponse bulkDetails = service.bulkDetails(packageNames);
List bulkDetailsEntryList = bulkDetails.getEntryList();
for (BulkDetailsEntry bulkDetailsEntry : bulkDetailsEntryList) {
DocV2 docV2 = bulkDetailsEntry.getDoc();
DocumentDetails docDetails = docV2.getDetails();
AppDetails appDetails = docDetails.getAppDetails();
System.out.println(appDetails.getPackageName());
System.out.println(appDetails.getDeveloperName());
System.out.println(appDetails.getDeveloperEmail());
}

but with a single call like this:

    DetailsResponse details = service.details(packageNames.get(0));
    AppDetails appDetails = details.getDocV2().getDetails().getAppDetails();
    System.out.println(appDetails.getDeveloperName());
    System.out.println(appDetails.getDeveloperEmail());

everything works fine.

Is it possible to get developers bulk details ?
Thanks.

This is not an issue or a bug, I just want to know that how much application can we download via this library?

BTW thanks to @Akdeniz for this awesome open source library.

It perfectly works without this method call. What is the purpose of this method ? Thanks.

Hey, this is a great project, thanks.
I have tried to get description for applications with bulk query but the response did not contain this field.

We've been using and reading your googleplay java api, and it works perfectly well in downloading an app from the play store. We were trying to go a step further and see if those downloads could also lead to an increase in install numbers on the play store. But that didn't happen, and in order to make that happen:

We observed the following requests when Gplay market downloads an app:

1. https://android.clients.google.com/auth
2. https://android.clients.google.com/fdfe/details
3. https://android.clients.google.com/fdfe/rev
4. https://android.clients.google.com/fdfe/rec
5. https://android.clients.google.com/fdfe/purchase
6. https://android.clients.google.com/fdfe/delivery
7. https://android.clients.google.com/fdfe/log

Apart from these, there was another request, the following, which was being sent at random times. This contained a Base64 encoded string which, as far as we could apprehend had Authtoken, logging id(constant for a device), device_id(gsf_id for the device), and a string that had list of package names installed which we think are protobuffed and we couldn't figure out.

1. https://android.clients.google.com/market/api/ApiRequest

   Here is the trace for one such request :

     POST https://android.clients.google.com/market/api/ApiRequest HTTP/1.1
   
     Content-Type: application/x-www-form-urlencoded; charset=UTF-8
   
     User-Agent: Android-Market/2
   
     X-Public-Android-Id: 6532b095d0193d17
   
     Content-Length: 7183
   
     Host: android.clients.google.com
   
     Connection: Keep-Alive
   

version=2&request=CuMCCosCRFFBQUFMb0FBQURjQnUxX1JmM3pfYzhhMnpLbmhMamh3UmVpVUxiUDl1YTNheEEwalVyWTBwSW9ieTZNTFNmdlc1MUNnTkhReGN2cTBVek1tZkI0M1R5UnFCSWtwLXZ4UDhRYkFnLTVXTFhYZTVDS0I4U2xRa1Q2QXlPVFQyQlpaTk4zRWJlZTB6SGdlODlYX1JqamtSWkRJMklROFJMSEdQTi10WmNwMmhGZGZVWVY5Wkl2UzIzYjA4RHR4Z09PWXdvZkFCTjJRYU53UW9oN2NsdFlVQkVuTktLMlVLT0RTTGVVZ3E4V2kxYXBnZEFlTzIydElubmgyRGNWRldpaGVGVXlCZG94cnFnEAEYlbuhJiIQMzdhYWVjZDk0YmExNWVjNSoLdmJveDg2dHA6MTYyAmVuOgJVU0IASgBSAFoAYgphbS11bmtub3duahEtNjg2ZTVjZWMyZmZjMjM3MhNKkicTGiR2Mjpjb20ubGlvbWlpLndhdGVyLnB1enpsZS5sb2dpYzoxOjMgAiizvp3Spyg6HWNvbS5saW9taWkud2F0ZXIucHV6emxlLmxvZ2ljQAMUExoddjI6Y29tLnRlc3RhcHAuaGFzb2ZmZXJzNjoxOjMgAiizvp3Spyg6FmNvbS50ZXN0YXBwLmhhc29mZmVyczZAAxQTGh52MjptZS5zb3VuZHdhdmUuc291bmR3YXZlOjE6OTAgAiizvp3Spyg6Fm1lLnNvdW5kd2F2ZS5zb3VuZHdhdmVAWhRTWgdhbmRyb2lkYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhRjb20uYW5kcm9WTS52bWNvbmZpZ2ABahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oZY29tLmFuZHJvaWQuYmFja3VwY29uZmlybWAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oVY29tLmFuZHJvaWQuYmx1ZXRvb3RoYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhNjb20uYW5kcm9pZC5icm93c2VyYBBqG1llMDNmb1hUaHFqZjdtdUdTOWhiQ19xbHI0RVRTWhdjb20uYW5kcm9pZC5jYWxjdWxhdG9yMmAQahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oUY29tLmFuZHJvaWQuY2FsZW5kYXJgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaEmNvbS5hbmRyb2lkLmNhbWVyYWABahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oZY29tLmFuZHJvaWQuY2VydGluc3RhbGxlcmAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oUY29tLmFuZHJvaWQuY29udGFjdHNgEGobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaGGNvbS5hbmRyb2lkLmRlZmNvbnRhaW5lcmAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oVY29tLmFuZHJvaWQuZGVza2Nsb2NrYMsBahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oRY29tLmFuZHJvaWQuZW1haWxgkIMZahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oUY29tLmFuZHJvaWQuZXhjaGFuZ2VgoMIeahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oTY29tLmFuZHJvaWQuZ2FsYXh5NGABahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1oTY29tLmFuZHJvaWQuZ2FsbGVyeWAQaht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1oWY29tLmFuZHJvaWQuaHRtbHZpZXdlcmAQahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oYY29tLmFuZHJvaWQuaW5wdXRkZXZpY2VzYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWh1jb20uYW5kcm9pZC5pbnB1dG1ldGhvZC5sYXRpbmAQahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1oeY29tLmFuZHJvaWQuaW5wdXRtZXRob2QucGlueWluYBBqG1d6YU1feTJpYUdtV3ZKWHF3WkRxcFBWakQtVVRTWhRjb20uYW5kcm9pZC5rZXljaGFpbmAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oUY29tLmFuZHJvaWQubGF1bmNoZXJgEGobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaJGNvbS5hbmRyb2lkLmxpdmV3YWxscGFwZXIubWljcm9iZXNnbGAPahtuRUZ5bEhHeWtQeUxJQXZQWXRLVWQtc1lHdWtUU1oWY29tLmFuZHJvaWQubWFnaWNzbW9rZWAQahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1oPY29tLmFuZHJvaWQubW1zYBBqG1llMDNmb1hUaHFqZjdtdUdTOWhiQ19xbHI0RVRTWhFjb20uYW5kcm9pZC5tdXNpY2AQahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oTY29tLmFuZHJvaWQubXVzaWNmeGCgUWobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaFGNvbS5hbmRyb2lkLm11c2ljdmlzYBBqG1d6YU1feTJpYUdtV3ZKWHF3WkRxcFBWakQtVVRTWhZjb20uYW5kcm9pZC5ub2lzZWZpZWxkYAFqG1d6YU1feTJpYUdtV3ZKWHF3WkRxcFBWakQtVVRTWhxjb20uYW5kcm9pZC5wYWNrYWdlaW5zdGFsbGVyYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhVjb20uYW5kcm9pZC5waGFzZWJlYW1gAWobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaEWNvbS5hbmRyb2lkLnBob25lYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWiJjb20uYW5kcm9pZC5wcm92aWRlcnMuYXBwbGljYXRpb25zYBBqG1d6YU1feTJpYUdtV3ZKWHF3WkRxcFBWakQtVVRTWh5jb20uYW5kcm9pZC5wcm92aWRlcnMuY2FsZW5kYXJgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaHmNvbS5hbmRyb2lkLnByb3ZpZGVycy5jb250YWN0c2AQahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1ofY29tLmFuZHJvaWQucHJvdmlkZXJzLmRvd25sb2Fkc2AQaht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1oiY29tLmFuZHJvaWQucHJvdmlkZXJzLmRvd25sb2Fkcy51aWAQaht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1oZY29tLmFuZHJvaWQucHJvdmlkZXJzLmRybWAQaht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1obY29tLmFuZHJvaWQucHJvdmlkZXJzLm1lZGlhYP0Daht0NTMwcUM2UXRYNm5aU1dyY0Rlckk0cEM5ZE1UU1oeY29tLmFuZHJvaWQucHJvdmlkZXJzLnNldHRpbmdzYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWh9jb20uYW5kcm9pZC5wcm92aWRlcnMudGVsZXBob255YBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWiRjb20uYW5kcm9pZC5wcm92aWRlcnMudXNlcmRpY3Rpb25hcnlgEGobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaFWNvbS5hbmRyb2lkLnByb3Zpc2lvbmAQahtKeGx1T0d1SFhuYXQ5d0RuNm9Ua3h1N2pQZm9UU1oUY29tLmFuZHJvaWQuc2V0dGluZ3NgEGobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaH2NvbS5hbmRyb2lkLnNoYXJlZHN0b3JhZ2ViYWNrdXBgEGobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaE2NvbS5hbmRyb2lkLnNtc3B1c2hgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaGWNvbS5hbmRyb2lkLnNvdW5kcmVjb3JkZXJgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaFGNvbS5hbmRyb2lkLnN5c3RlbXVpYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhNjb20uYW5kcm9pZC52ZW5kaW5nYJW7oSZqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhdjb20uYW5kcm9pZC52aWRlb2VkaXRvcmALahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUU1oXY29tLmFuZHJvaWQudm9pY2VkaWFsZXJgEGobWWUwM2ZvWFRocWpmN211R1M5aGJDX3FscjRFVFNaFmNvbS5hbmRyb2lkLnZwbmRpYWxvZ3NgEGobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaFWNvbS5hbmRyb2lkLndhbGxwYXBlcmAQahtXemFNX3kyaWFHbVd2Slhxd1pEcXBQVmpELVVUU1ogY29tLmFuZHJvaWQud2FsbHBhcGVyLmhvbG9zcGlyYWxgEGobV3phTV95MmlhR21XdkpYcXdaRHFwUFZqRC1VVFNaIGNvbS5hbmRyb2lkLndhbGxwYXBlci5saXZlcGlja2VyYBBqG0p4bHVPR3VIWG5hdDl3RG42b1RreHU3alBmb1RTWhtjb20uY3lhbm9nZW5tb2QuZmlsZW1hbmFnZXJgZWobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaHWNvbS5leGFtcGxlLmFuZHJvaWQubGl2ZWN1YmVzYBBqG1llMDNmb1hUaHFqZjdtdUdTOWhiQ19xbHI0RVRTWiljb20uZ29vZ2xlLmFuZHJvaWQuYXBwcy5nZW5pZS5nZW5pZXdpZGdldGCfCmobSkxza3dGNUg0Szc2YUtXS2RtRjUyYllUcGdBVFNaIGNvbS5nb29nbGUuYW5kcm9pZC5hcHBzLnVwbG9hZGVyYMC4AmobT0pHS1JUMEhHWk5VLUxHYThGN0dWaXp0VjRnVFNaGWNvbS5nb29nbGUuYW5kcm9pZC5iYWNrdXBgEGobSnhsdU9HdUhYbmF0OXdEbjZvVGt4dTdqUGZvVFNaG2NvbS5nb29nbGUuYW5kcm9pZC5mZWVkYmFja2AQahtPSkdLUlQwSEdaTlUtTEdhOEY3R1ZpenRWNGdUU1onY29tLmdvb2dsZS5hbmRyb2lkLmdvb2dsZXF1aWNrc2VhcmNoYm94YIC5x19qG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhZjb20uZ29vZ2xlLmFuZHJvaWQuZ3NmYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhxjb20uZ29vZ2xlLmFuZHJvaWQuZ3NmLmxvZ2luYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhtjb20uZ29vZ2xlLmFuZHJvaWQubG9jYXRpb25g1ghqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWiJjb20uZ29vZ2xlLmFuZHJvaWQubWFydmluLnRhbGtiYWNrYERqG20wSk1MU2V0VWFRcU0zNEx0cGtjZHV5a1JHRVRTWiVjb20uZ29vZ2xlLmFuZHJvaWQub25ldGltZWluaXRpYWxpemVyYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWh9jb20uZ29vZ2xlLmFuZHJvaWQucGFydG5lcnNldHVwYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWh5jb20uZ29vZ2xlLmFuZHJvaWQuc2V0dXB3aXphcmRgggFqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWiljb20uZ29vZ2xlLmFuZHJvaWQuc3luY2FkYXB0ZXJzLmJvb2ttYXJrc2AQahtPSkdLUlQwSEdaTlUtTEdhOEY3R1ZpenRWNGdUU1ooY29tLmdvb2dsZS5hbmRyb2lkLnN5bmNhZGFwdGVycy5jYWxlbmRhcmAPahtPSkdLUlQwSEdaTlUtTEdhOEY3R1ZpenRWNGdUU1ooY29tLmdvb2dsZS5hbmRyb2lkLnN5bmNhZGFwdGVycy5jb250YWN0c2AQahtPSkdLUlQwSEdaTlUtTEdhOEY3R1ZpenRWNGdUU1oXY29tLmdvb2dsZS5hbmRyb2lkLnRhbGtgygJqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWhZjb20uZ29vZ2xlLmFuZHJvaWQudHRzYBBqG09KR0tSVDBIR1pOVS1MR2E4RjdHVml6dFY0Z1RTWh5jb20uZ29vZ2xlLmFuZHJvaWQudm9pY2VzZWFyY2hggJL0AWobT0pHS1JUMEhHWk5VLUxHYThGN0dWaXp0VjRnVFNaF2NvbS5ub3NodWZvdS5hbmRyb2lkLnN1YC9qG1h4RV93c0lLZkp2WktCa2lhaktoa0V0MTc0c1RTWg1jb20uc3ZveC5waWNvYAFqG1llMDNmb1hUaHFqZjdtdUdTOWhiQ19xbHI0RVRTWhRjb20udGYudGhpbmtkcm9pZC5zZ2DnswpqGzl2UVlPS0QyS3ozMTViZVJfXy1iYlZ4OG1aY1RTWhdqcC5jby5vbXJvbnNvZnQub3BlbndubmAQahtZZTAzZm9YVGhxamY3bXVHUzloYkNfcWxyNEVUcAQU&

Your code simulates the following 5 requests:

1. https://android.clients.google.com/auth
2. https://android.clients.google.com/fdfe/details
3. https://android.clients.google.com/fdfe/rev
4. https://android.clients.google.com/fdfe/rec
5. https://android.clients.google.com/fdfe/purchase

After using your functions for the above requests we tried to simulate the following requests ourselves:

-https://android.clients.google.com/fdfe/log

   This request contains a protobuffed string(as per our understanding) which seems to be some kind of a confirmation for downloading the app to the device.

   This is how the string looks: = ( 4confirmFreeDownload?doc=com.google.android.apps.maps

-https://android.clients.google.com/market/api/ApiRequest

   We tried to construct the Base64 encoded string, and the protobuffed string for apps that are installed on a particular device, but recieved a 400 response 

   We replicated a request sent by a Device without changes but we got 400 response.

   (We could not figure out how to get the logging_id as well.)

We need help in reconstructing the same request, and understanding how the the data being sent in the request is formed so that we can figure out the last three requests and complete the install cycle.

Any help would be highly appreciated since you're the ultimate authority on this. :-)

bad security token
Error 401

Could you added supporting of Google Play Movies?

It will be a great bunch of tools.
As I understand need only protobuf for that?

Is there any way to tell if an app is not available due to country restrictions? I have an array of ~10 devices to try downloading an app. I know I could check for checkoutRequired to see if its a price restriction, but I don't know how to account for connectivity issues vs country availability.

Is there a way to list apps in collections like these?

https://play.google.com/store/apps/collection/promotion_3000147_play_picks_phone

Both the Google Play app and site show those but I don't see to be able to fetch those with either browse or list methods.

Also if there is is there a way to list all the available collections

Thanks!

Recently I could not download any apps through this library, It's always return null for getDownloadAuthCookie(0) and download get failed!

BTW DocV2 is no longer working and I have to use DocV1!

any ideas?

Thanks a lot Akdeniz for the great work!

As some other people already mentioned, checkin no longer works now. So if I want to run usegcm, how do I find out the security token? service.getSecurityToken() returns null after login().

BTW, if I want to get the APK files for paid APPs, is usegcm the only feasible option? service.download() fails for paid APPs.

Hello,

Thank you very much for the good work first.

I am trying to get a list of apps, which is in large scale. My scritp is to get 50 apps of each category in a time to the total number 2000.

The problem is that as the offset in the search is larger than 500. The search will come across a failure. I do not think it is caused by the blocking of my IP address. In my manual test, the successful search (offset less than 500) follows tightly with the search with failure. Also, in my script, it sleeps for 10 mins between two searchs.

I appreciate your efforts on reading this problem.

Best regards,
Zhengyang

java -jar googleplaycrawler-0.3.jar -f crawler.conf list BOOKS_AND_REFERENCE -s apps_topselling_free -n 50 -o 200
[success]

java -jar googleplaycrawler-0.3.jar -f crawler.conf list BOOKS_AND_REFERENCE -s apps_topselling_free -n 50 -o 500
[failure]

I have been trying this out, but I'm consistently getting the same response. A bulk detail request for any number of packages (2 and above) will result in a response that has the correct number of documents returned, but only the first one is actually filled with data.

First of all, thanks for your library. It is extremely useful and easy to use.

Is it possible to add a new recommendation type - similar?

Thanks.

Hi,

I am able to Checkin, Register with the Credentials but when i try to download an app i get error.

Error retrieving information from server. [RPC:S-5:AEC-0]·♣

I have a Samsung Galaxy S3 running Android 4.2.2 (CyanogenMod 10.1). I'm using the android ID from the that device, with the Google Apps account login and password associated with it. I can search for apps, but the problem is that it is only showing apps that are available for Android 1.1, and not all the apps available for my device. Here is an example:

➜  googleplay git:(master) ✗ java -jar googleplaycrawler-0.2.jar -i myandroididhere -e [email protected] -p mypasswordhere search gmail    
Title;Package;Creator;Price;Installation Size;Number Of Downloads
Backup to Gmail;net.cpedia.backup2gmail;cpedia Mobile;$1.99;1296011;50,000+
Gmail Unread Count OLD;se.alexanderblom.gmailunreadcount;Alexander Blom;Free;321781;500,000+
Cloud Print My Gmail;com.khymaera.android.cpmg;Khymaera;Free;579057;10,000+
backup SMS MMS Photo to Gmail;com.xmobileapp.xarchive;ianXian;$1.99;419239;1,000+
SMS Backup (Gmail);mobi.eapps.SMSBackupFree;WooBoo;Free;263897;10,000+
SMS Backup or Save to Gmail;com.jeff.app.smsforgmail;Think Studio;Free;326301;10,000+
Gmail Tasks;funwithapps.gmailtasks;Fun With Apps;Free;14207;10,000+
Contacts Backup(Gmail);com.startrek.ContactsBackupGmail;tarzanapp;$0.99;292540;500+
Gmail Spam Checker;net.jundu.android.spamchecker;JUNDU;Free;83999;5,000+
SMS to GMAIL;com.mpgroupslovakia.smstogmail;M&P Group Slovakia;Free;277915;1,000+
Fast Email and Gmail;com.tenromans.fastemail;Top Download;$0.99;45389;500+
GTD GMail Collect;net.ronincoders.automail;Salvamaine.com;Free;264365;1,000+
SMS Backup(Gmail);com.tarzanapp.SMSBackupGmail;tarzanapp;$0.99;271003;100+
Gmail Priority;com.netcompss.gpri.gmail3;NetComps LTD;$1.99;227903;100+
eNotify Gmail;com.netcompss.enotify3.gmail3;JY Team NetComps LTD;$1.99;350772;50+
FreeNote, note everything;com.suishouxie.freenote;flyable;Free;629838;1,000,000+
LED me know BETA;com.p0w4.ledMeKnow;p0w4p0ty;Free;379387;100,000+
Facebook;com.facebook.katana;Facebook;Free;3091271;100,000,000+
Pandora® internet radio;com.pandora.android;Pandora;Free;2124677;50,000,000+
Documents To Go 3.0 Main App;com.dataviz.docstogo;DataViz, Inc.;Free;4591508;10,000,000+

Gmail itself doesn't even show up in my search results. All of these apps are for Android 1.1. I appreciate any advice!

Hi, Akdeniz
Thanks for your goolge play api.
I try to use it to crawl user reviews of apps.
I find that the method getAuthorName() for retrieving the author name of user reviews seems not work. I'm not sure if there are some bugs.

Best,
Ning

I have today responses

java -jar googleplaycrawler-0.2.jar -f crawler.conf search games

<HTML>
<HEAD>
<TITLE>Not Found</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Not Found</H1>
<H2>Error 404</H2>
</BODY>
</HTML>

i used https://code.google.com/p/android-market-api/ but it's old and sometimes buggy when trying to download some apps.

this api is great, but missing search for free apps.

Hi,

first of all great job!

I was wondering how can we parse the category of an app and any other available details, after the search result returns?

I tried appDetails.getAppCategory() by randomly guessing that there should be something like that but it just prints Index= Size=0.

Where can I find the API for AppDetails and DocV2?

Thanks!

The following request

POST /auth HTTP/1.1
Content-Length: 278
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Host: android.clients.google.com
Connection: Keep-Alive
User-Agent: Apache-HttpClient/4.2.2 (java 1.5)

Does not return a response anymore, Is this happening for anyone else?

When I tried checkin or usegcm command, it showed the following message:
This is supposed to be overridden by subclasses.

My shell command is:
java -jar google-play-crawler/googleplay/googleplaycrawler-0.3.jar -f conf/crawler.conf checkin

Hi Akdeniz,
You are awesome!

I am trying to get all the reviews for an app. However, it gives me an error "Too many results requested.". Could you please tell me how could I avoid this? For example this app "de.lotum.whatsinthefoto.us" has over 200K reviews but I can get only some of them :(

TestGooglePlay.java code

ReviewResponse reviews = service.reviews("de.lotum.whatsinthefoto.us", null, 500, 20);
GetReviewsResponse response = reviews.getGetResponse();

Error msg:

Exception in thread "main" com.akdeniz.googleplaycrawler.GooglePlayException: ����Too many results requested.*��»�
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeHttpRequest(GooglePlayAPI.java:490)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeGet(GooglePlayAPI.java:481)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeGETRequest(GooglePlayAPI.java:385)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.reviews(GooglePlayAPI.java:352)
at com.akdeniz.googleplaycrawler.test.TestGooglePlay.reviewsCommand(TestGooglePlay.java:79)
at com.akdeniz.googleplaycrawler.test.TestGooglePlay.main(TestGooglePlay.java:62)

Hi,
Thank you for your api, it worked great.
But recently, I have found an error when checkin, please check the message shown below. What is the reason? Thank you very much.

Exception in thread "main" com.akdeniz.googleplaycrawler.GooglePlayException:

Not Found

Error 404

at com.akdeniz.googleplaycrawler.GooglePlayAPI.executeHttpRequest(GooglePlayAPI.java:489)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executePost(GooglePlayAPI.java:449)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.executePost(GooglePlayAPI.java:429)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.loginAC2DM(GooglePlayAPI.java:163)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.checkin(GooglePlayAPI.java:148)
at com.akdeniz.googleplaycrawler.test.TestGooglePlay.main(TestGooglePlay.java:38)

Hi,

I am using the search function in the TestGooglePlay.java file. When I search for any tag, I can fetch maximum 20 results. Is that a restriction from Google Play?

SearchResponse searchResponse = service.search("calendar");

Hi,

I´ve problems with the use of "java -jar googleplay.jar list".
I don´t know how to use it right.

I want to have (for example) the top 50 apps of each category (free apps).

Here´s my command:
"java -jar gooleplay.jar -f crawler.conf list -s tools category"
=> I get: "↕6↕4Error retrieving information from server. [DF-LA-02]↨☻"
But I can download app without probelems...

Can anyone give me an example?
Thanks :-)

Hi,

how can I set a proxy in the "TestGooglePlay.java" class?
I set it in the crawler.conf, but I think this class don´t gets anxy information from the conf-file, right?

Thanks

I am using default code and in TestGooglePlay.java file, there is one method testBulkDetails() which downloads details for following packages.

"com.mobulasoft.criticker",
"com.cricbuzz.android",
"com.sticksports.stickcricket",
"com.indiagames.cricketfever".

But the problem is it only downloads information about first two packages, not for the last two ones. I have tried to debug the code what could have been a problem. The reason I found was that

service.details() and service.bulkdetails() methods cannot fetch version and installation size information about the packages from Google Play. Any idea, what should I try else? Thanks.

I continuosly get empty apps Title(Name)

DetailsResponse detailsResponse = service.details(packageName);
DocV2 docV2 = detailsResponse.getDocV2();
DocumentDetails details = docV2.getDetails();
AppDetails appDetails = details.getAppDetails();

           appDetails.getTitle() // is always empty..

What is wrong with my code ? Thanks.

Hello,

First of all I'd like to say thank you for this awesome library !

I want to work with this stuf in multithreading environment with a few concurrent GooglePlayAPI clients. Is it possible to setup different proxy hosts and ports to GooglePlayAPI clients ? If so, how it could be achieved ?

Thanks!

HI!

Firstly, thank you for a great lib!

After update a version I have the error
I work with library in jruby

After import com.akdeniz.googleplaycrawler.GooglePlayAPI

I have error in log

LoadError: load error: ./lib/google_play -- java.lang.VerifyError: class com.akdeniz.googleplaycrawler.GooglePlay$ListResponse overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
require at org/jruby/RubyKernel.java:1027
(root) at script/google_play.rb:8

May be do you know how it resolve?

I need Google Play data(for example, reviews) in multiple localizations. I want to parse reviews in multiple threads, so in each thread I

• set localization
• get reviews.

When runned in one thread, code works perfectly, but problems appears in multithreaded mode - reviews are requested from incorrect locale. I don't want to synchronize these two actions by mutex or smth else - it destroys the idea of using threads. There was an idea about cloning instance of GooglePlayAPI object for each thread, but it is unclonable.

How can I use library in multithreaded mode? Am I missing something? Maybe it's possible to set localization for each request?

My environment: jruby, sidekiq as threads manager. Yep, I use library from ruby.

Trying to wrap my head around the whole Android_ID, autogeneration etc...
We have a server with the google play crawler where I can successfully serve all the operations via REST endpoints to be consumed by clients connecting to the REST API.

Obviously the user / password constructor then checkin() is not an option for this case.
We want to use the clients access to google accounts to provide our Android id, secretToken and cd2m auth token so each client has it's own GooglePlayAPI instance associated to the account that use for the requests.

What is the recommended flow approach to use the GooglePlayAPI class without a user and pass?

The checkin() call implies you have a password instance variable.

Any ideas on how to implement such a flow? thanks in advance and great job.

Unauthorized

Error 401

Hi,

my test google account for this api was disabled by google.
I think because I used this API to often? Is that possible?

How can I avoid this in future and should I just reactivate my account by let sending me a verification code via sms?

thanks

[info] Compiling 20 Java sources to /Users/ahikmat/Documents/kh/crawler/google-play-crawler-master/target/scala-2.9.2/classes...
[error] /Users/ahikmat/Documents/kh/crawler/google-play-crawler-master/src/main/java/com/akdeniz/googleplaycrawler/gsf/MessageFilter.java:18: illegal start of type
[error] private Queue filteredMessages = new ConcurrentLinkedDeque<>();
[error] ^
[error] 1 error
error javac returned nonzero exit code
[error] Total time: 135 s, completed May 27, 2014 10:54:06 AM

I used your code before to build a play crawler but it failed several days before because of the checkin problem. And then I updated with the newest one you supplied. I got this error message from TestGooglePlayCrawler. Do you know why? Thanks!!

[TestNG] Running:
/tmp/testng-eclipse--1719814961/testng-customsuite.xml

FAILED: shouldCheckin
java.lang.VerifyError: class com.akdeniz.googleplaycrawler.GooglePlay$AndroidCheckinRequest overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:792)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.akdeniz.googleplaycrawler.Utils.generateAndroidCheckinRequest(Utils.java:167)
at com.akdeniz.googleplaycrawler.GooglePlayAPI.checkin(GooglePlayAPI.java:156)
at com.akdeniz.googleplaycrawler.TestGooglePlayCrawler.shouldCheckin(TestGooglePlayCrawler.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:335)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:330)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)