google-play-scraper-kotlin

Library for scraping of the application data from the Google Play Store.

Related Projects

The library was inspired by the following projects:

google-play-scraper

Node.js module to scrape application data from the Google Play Store.

Installation

First, make sure that you've added the mavenCentral() repository to your top-level build.gradle file:

allprojects {
    //...
    repositories {
        //...
        mavenCentral()
        //...
    }
    //...
}

Then, add the desired version of the google-play-scraper as a dependency:

Latest version:

dependencies {
    implementation("com.arthurivanets:google-play-scraper:x.y.z")
}

Usage

Initialization

// default configuration
val scraper = GooglePlayScraper()

//...

// custom configuration
val scraper = GooglePlayScraper(
    GooglePlayScraper.Config(
        throttler = HumanBehaviorRequestThrottler(),
        //...
    )
)

Available methods

App Details - retrieves the detailed information about a specified app.

Request Parameters:

appId - the exact id of the application (e.g. com.myapp).
language - language code (ISO 639-1) to define the language of the information to be retrieved (optional, defaults to EN).
contry - country code (ISO 3166) to define the origin of the information to be retrieved (optional, detauls to US).

val params = GetAppDetailsParams(
    appId = "com.some.app",
    language = "EN",
    country = "US"
)
val response = scraper.getAppDetails(params).execute()

if (response.isSuccess) {
    val appDetails = response.requireResult()
    // do something with the obtained app details
} else {
    val error = response.requireError()
    // do something with the error
}

Developer Apps - retrieves the apps published by a specified developer.

Request Parameters:

devId - the exact id of the developer.
language - language code (ISO 639-1) to define the language of the information to be retrieved (optional, defaults to EN).
contry - country code (ISO 3166) to define the origin of the information to be retrieved (optional, detauls to US).
limit - maximum number of apps to be retrieved (optional, defaults to 100).

val params = GetDeveloperAppsParams(
    devId = "Super+Useful+Apps",
    language = "EN",
    country = "US",
    limit = 150
)
val response = scraper.getDeveloperApps(params).execute()

if (response.isSuccess) {
    val apps = response.requireResult()
    // do something with the obtained apps
} else {
    val error = response.requireError()
    // do something with the error
}

Similar Apps - retrieves the apps that are deemed similar to a specified app.

Request Parameters:

appId - the exact id of the application (e.g. com.myapp).
language - language code (ISO 639-1) to define the language of the information to be retrieved (optional, defaults to EN).
contry - country code (ISO 3166) to define the origin of the information to be retrieved (optional, detauls to US).
limit - maximum number of apps to be retrieved (optional, defaults to 100).

val params = GetSimilarAppsParams(
    appId = "com.myapp",
    language = "EN",
    country = "US",
    limit = 150
)
val response = scraper.getSimilarApps(params).execute()

if (response.isSuccess) {
    val similarApps = response.requireResult()
    // do something with the obtained apps
} else {
    val error = response.requireError()
    // do something with the error
}

Apps List - retrieves the apps that are associated with a specified category/collection.

Request Parameters:

category - the exact category of the apps to be retrieved (optional, defaults to null).
collection - the exact collection of the apps to be retrieved (optional, defaults to Collection.TOP_FREE).
language - language code (ISO 639-1) to define the language of the information to be retrieved (optional, defaults to EN).
contry - country code (ISO 3166) to define the origin of the information to be retrieved (optional, detauls to US).
limit - maximum number of apps to be retrieved (optional, defaults to 100).

val params = GetAppsParams(
    collection = Collection.TOP_PAID,
    language = "EN",
    country = "US",
    limit = 150
)
val response = scraper.getApps(params).execute()

if (response.isSuccess) {
    val apps = response.requireResult()
    // do something with the obtained apps
} else {
    val error = response.requireError()
    // do something with the error
}

App Search - performs an app search.

Request Parameters:

query - app search query (e.g. todo list).
language - language code (ISO 639-1) to define the language of the information to be retrieved (optional, defaults to EN).
contry - country code (ISO 3166) to define the origin of the information to be retrieved (optional, detauls to US).
limit - maximum number of apps to be retrieved (optional, defaults to 100).

val params = SearchAppsParams(
    query = "todo list",
    language = "EN",
    country = "US",
    limit = 150
)
val response = scraper.searchApps(params).execute()

if (response.isSuccess) {
    val apps = response.requireResult()
    // do something with the obtained apps
} else {
    val error = response.requireError()
    // do something with the error
}

App Permissions - retrieves the list of permissions of a specified app.

Request Parameters:

appId - the exact id of the application (e.g. com.myapp).
language - language code (ISO 639-1) to define the language of the information to be retrieved (optional, defaults to EN).

val params = GetAppPermissionsParams(
    appId = "com.myapp",
    language = "EN",
)
val response = scraper.getAppPermissions(params).execute()

if (response.isSuccess) {
    val appPermissions = response.requireResult()
    // do something with the obtained permissions
} else {
    val error = response.requireError()
    // do something with the error
}

App Reviews - retrieves the list of reviews for a specified app.

Request Parameters:

appId - the exact id of the application (e.g. com.myapp).
language - language code (ISO 639-1) to define the language of the information to be retrieved (optional, defaults to EN).
contry - country code (ISO 3166) to define the origin of the information to be retrieved (optional, detauls to US).
limit - maximum number of apps to be retrieved (optional, defaults to 100).
sortingOrder - sorting order of the app reviews (optional, defaults to ReviewSortingOrder.NEWEST).

val params = GetAppReviewsParams(
    appId = "com.myapp",
    language = "EN",
    country = "US",
    limit = 150
)
val response = scraper.getAppReviews(params).execute()

if (response.isSuccess) {
    val reviews = response.requireResult()
    // do something with the obtained reviews
} else {
    val error = response.requireError()
    // do something with the error
}

Throttling

All scraper methods interact with Google Play services in one way or another, which means that all of them get impacted by the request throttling policies imposed by the Google Play itself. It is quite easy to exhaust the request quota and start getting the 503 responses with requesting entitiy verification captchas which, if not completed properly, might lead to a temporary ban of the requesting IP address (usually lasts for about an hour). So, it's a good idea to configure the client-side throttling in order to reduce the risk of running into the aforementioned problems.

Throttling can be configured by providing an appropriate implementation of the RequestThrottler during the initialization of the GooglePlayScraper.

Available RequestThrottler implementations:

HumanBehaviorRequestThrottler - applies an artificial delay to each scraper-specific request (can be configured).
NoRequestThrottling - no request throttling (default configuration).

arn-arm-1 / google-play-scraper-kotlin Goto Github PK