brsanthu / google-analytics-java Goto Github PK

I'm having some trouble seeing my requests come through in GA. I can see the data in real time just fine - about one request per second.

But even after leaving it 72 hours the number of requests in the standard report section is much reduced from previous (I was using the JGoogleAnalytics lib previously). For example, one page view that was recording around 25k hits per day is now recording around 300-400.

I was using trackAsynchronously in JGoogleAnalytics which simply creates a new thread, but it looks like the thread pool in this library is bounded only to Integer.MAX_VALUE items so it seems unlikely requests are being discarded.

I'll see if I can find out more and update this thread.

Currently the GoogleAnalyticsStats exposes separate counters for different hit types, but the most useful total hit counter is missing. Please add it.

While we are at this, consider using micrometer instead of creating your own metrics implementation.

There is no way of knowing what happened if a post() fails. I think you should throw the error and let the caller deal with it. Other option is to set the Exception in a new field on the HttpResponse (which will always be empty in case of a failure).

Check:

• googleapis/services/AbstractGoogleClientRequest.java#L415
• google/api/client/http/HttpRequest.java#L843

are there any path i could include in the android studio?

Hits typed as T extends GoogleAnalyticsRequest<T> are difficult to process. It creates situations where you have to cast U extends GoogleAnalyticsRequest<U> to T extends GoogleAnalyticsRequest<T>. If I try to use just GoogleAnalyticsRequest<?>, then the fluent setters return Object, which prevents setter chaining even when using only general parameters defined on GoogleAnalyticsRequest.

Consider dropping all classes derived from GoogleAnalyticsRequest and making GoogleAnalyticsRequest non-generic. Just put all parameter setters in this single class. It's less clean from OOP perspective, but it makes it much easier to implement generic hit processing code while keeping actual hit code as clean as it was before.

Should be "dimension"... see it in EventHit.customDimention for example. From what I can see, "dimention" is not a valid spelling but I could be wrong about that.

Instead of doing minimum of 1/config, code is doing max of 1/config.

The code for the GoogleAnalytics.java will create an Unbounded LinkedBlockingDeque<Runnable>() with an five minutes timeout for each call and a default 0 to config.getMaxThreads poolSize.

protected synchronized ThreadPoolExecutor createExecutor(GoogleAnalyticsConfig config) {
   return new ThreadPoolExecutor(0, config.getMaxThreads(), 5, TimeUnit.MINUTES, new LinkedBlockingDeque<Runnable>(), createThreadFactory());
}

We should at least make that configurable with sensible defaults. It also uses the wrong constructor with the defaultHandler which is private static final RejectedExecutionHandler defaultHandler = new AbortPolicy(); and will cause events to be descarded.

The feature "exclude bot traffic" of Google Analytics filters out the GA events posted from the server via the google-analytics-java plugin (v1.1.1). Our application is deployed on Heroku and based on Play.

Any idea how to solve this? Did we miss to configure the plugin correctly / passing the right parameters to GA in the event posts?

Does this library support desktop applications?

I tried using your library in an Android application but I get the following error at runtime:

Caused by: java.lang.NoClassDefFoundError: java.awt.Toolkit
            at com.brsanthu.googleanalytics.GoogleAnalytics.deriveSystemParameters(GoogleAnalytics.java:284)
            at com.brsanthu.googleanalytics.GoogleAnalytics.<init>(GoogleAnalytics.java:96)
            at com.brsanthu.googleanalytics.GoogleAnalytics.<init>(GoogleAnalytics.java:79)

Is this problem related to Android or I am doing something wrong?

My hit processing pipeline sometimes needs to send the same or slightly modified hit to another property. I could use correct public implementation of the clone() method on GoogleAnalyticsRequest that would return T.

I'm trying to use the customDimention(int index, String value) method of EventHit and the custom dimensions are not being delivered to google analytics. The event comes through but no custom dimension data.

I verified that I can send custom dimensions with the google's native JS api.

Hi,
very nice work!!!!!!
I've some customized dimensions and I'm sending "post()" and "postAsync()" methods to GoogleAnalytics.
Now, the impression is that not all sends arrive to GA. Many sends get lost.
Why?
Is it possible?
May you help me?
Thanks for all.
Onofrio

I'm using this code:

GoogleAnalytics ga = new GoogleAnalytics("UA-12345678-1");
PageViewHit request = new PageViewHit("http://customurl.com", "Search");
request.customDimention(1, "first-column, 11-13(1)");
request.customDimention(2, "second-column, 11-13(2)");
request.customDimention(3, "third column, 11-13(3)");
ga.post(request);

OR

GoogleAnalytics ga = new GoogleAnalytics("UA-12345678-1");
PageViewHit request = new PageViewHit("http://customurl.com", "Search");
request.customDimention(1, "first-column, 11-13(1)");
request.customDimention(2, "second-column, 11-13(2)");
request.customDimention(3, "third column, 11-13(3)");
ga.postAsync(request);

OR

ga.postAsync(new RequestProvider() {
public GoogleAnalyticsRequest getRequest() {
PageViewHit request = new PageViewHit("http://customurl.com", Search");
request.customDimention(1, "prima-colonna-nunc, 11-13(1)");
request.customDimention(2, "seconda-colonna-nunc, 11-13(2)");
request.customDimention(3, "terza-colonna-nunc, 11-13(3)");
return request;
}
});

https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#uid

UserId parameter is missing

Can I please check the licence for you excellent looking library? Apache 2.0 appears in the source code, but not in the pom.xml or in a LICENSE file.

Eclipse project files are committed in the repo.

.classpath
.project
.settings/org.eclipse.jdt.core.prefs

Hey @brsanthu, I've been using your API for a while but the design of it keeps bothering me, I'd like to know if you're interested in pushing a redesign of it on a Fluent interface way, so instead of:

GoogleAnalytics ga = new GoogleAnalytics("UA-12345678-1");
ga.post(new PageViewHit("https://www.google.com", "Google Search"));

We would have:

GoogleAnalytics ga = GoogleAnalyticsBuilder.withUA("UA-12345678-1").build();
ga.trackPageView("https://www.google.com", "Google Search").post();

Or

GoogleAnalytics ga = GoogleAnalyticsBuilder.withUA("UA-12345678-1").build();
ga.trackPageView("https://www.google.com", "Google Search").postAsync();

Or

GoogleAnalytics ga = GoogleAnalyticsBuilder.withUA("UA-12345678-1").build();
ga.trackPageView("https://www.google.com", "Google Search").withRequestProvider(new RequestProvider() {
    public GoogleAnalyticsRequest getRequest() {
        return new PageViewHit("https://www.google.com", "Google Search");
    }
}).postAsync();

Would you like to talk about it?

Can you add them please?

After some experimentation with batching enabled, it seems that all hits come in at the same time point (the time of the batch request), when I really want the hits to appear at the time they were posted. This seems like a general issue with using batched requests on GA, but I have been unable to find anything about it. Any pointers would be appreciated.

Anyway, as a solution, I am considering adding the Queue Time parameter to each hit, but I can see no other way than deriving the qt value at the time when the batch request is made, which means it has to be done when building the batch request in the GoogleAnalyticsImpl.submitBatch(boolean) method. Only then is the time offset of each hit known. This requires keeping time of the batched hits.

Any thoughts or maybe consideration of implementing the option of keeping time of the hits and setting the queue time parameter automatically, when in batch mode, at the time of posting the batch request?

Thanks!

Documentation on the Queue Time parameter: https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#qt

Is now available a new endpoint to validate the Google Analytics Measurement Protocol requests:
https://developers.google.com/analytics/devguides/collection/protocol/v1/validating-hits.

In the GoogleAnalyticsConfig one can set the http url, eg:

GoogleAnalyticsConfig()
                .setHttpUrl("http://www.google-analytics.com/debug/collect")
// ...

but then the http response from the server is mapped to GoogleAnalyticsResponse, that only keeps the status code and the request params. So atm it is impossible to check the response payload and thus validating the hit.

A really easy modification to the code would be just adding a field to GoogleAnalyticsResponse containing the original org.apache.http.HttpResponse.

I am using this library server-side, proxying analytics data from the client. I want to anonymize IP addresses before sending them to GA. I am currently doing this:

/*
 * Perform local IP address anonymization.
 * We are also instructing GA to anonymize IPs on GA end,
 * but it is technically and legally safer to anonymize locally.
 * Anonymization rules are the same as those used by GA,
 * i.e. zero last 80 bits of IPv6 and last 8 bits of IPv4.
 */
InetAddress ip = InetAddress.getByName("12.34.56.78");
byte[] address = ip.getAddress();
int anonymizedBytes = ip instanceof Inet6Address ? 10 : 1;
for (int i = 0; i < anonymizedBytes; ++i)
	address[address.length - i - 1] = 0;
return InetAddress.getByAddress(address).getHostAddress();

I guess many people could use IP anonymization for compliance reasons. It would give your library a privacy edge. Consider adding this as an option. Thanks.

I'm using this library on App Engine. The implementation in GoogleAnalytics.java doesn't work on App Engine as (a) you can't use threading and (b) you have to use the URL service which doesn't work out of the box with the Apache components.

I've rewritten GoogleAnalytics.java (code here) but for it to work the visibility of some methods needs to change. Specifically, getUrl in GoogleAnalyticsConfig.java, and the *Hit methods in GoogleAnalyticsStats.java.

If you're interested I'll create a pull request.

Failed to load class org.slf4j.impl.StaticLoggerBinder

This warning message is reported when the org.slf4j.impl.StaticLoggerBinder class could not be loaded into memory. This happens when no appropriate SLF4J binding could be found on the class path. Placing one (and only one) of slf4j-nop.jar slf4j-simple.jar, slf4j-log4j12.jar, slf4j-jdk14.jar or logback-classic.jar on the class path should solve the problem.
SINCE 1.6.0 As of SLF4J version 1.6, in the absence of a binding, SLF4J will default to a no-operation (NOP) logger implementation.
If you are responsible for packaging an application and do not care about logging, then placing slf4j-nop.jar on the class path of your application will get rid of this warning message. Note that embedded components such as libraries or frameworks should not declare a dependency on any SLF4J binding but only depend on slf4j-api. When a library declares a compile-time dependency on a SLF4J binding, it imposes that binding on the end-user, thus negating SLF4J's purpose.

Hi,

I was looking at the GA for E-Com related and found you lib useful.
With the new Enhancement E-Comm support by Google, it would be great if you could update it with the new things supported by GA.

https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#pr_id

Thanks,
Vinay

i am not able to set country code and city.

How to set country and city through brsanthu/googleanalytics API

GoogleAnalyticsConfig config = new GoogleAnalyticsConfig();
config.setDiscoverRequestParameters(true);
config.setEnabled(true);

    if(sGoogleAnalytics == null) sGoogleAnalytics = new GoogleAnalytics(config, ID, APP_NAME, VER);

I am sending event which i am able to see in User Explorer view according to UUID.
But in end when i am calling GoogleAnalyticsObject.close(); to end my session, that session is not closing.

session count remains 1 , irrespective to start and terminate of my application.

my google tracking ID is of WebApp type.
session count should be equal to how many times i have launched my application. Correct ?
How to manage session ?

Method name contains a typo. It should be customMetrics().

In GoogleAnalyticsRequest, customDimentions is misspelled. Should be customDimensions.

I am using this library server-side and I could use some easy way to gather basic information from HTTP headers. I am currently doing this:

HttpServletRequest request = ...;
hit.userIp(ip(request.getHeader("X-Forwarded-For")));
hit.anonymizeIp(true);
hit.userLanguage(language(request.getHeader("Accept-Language")));
hit.userAgent(request.getHeader("User-Agent"));
hit.documentUrl(request.getRequestURL().toString());
hit.documentTitle(request.getRequestURI());

This uses two helper methods. Method ip() decodes IP address from X-Forwarded-For if available:

public static String ip(String xff) {
	if (xff == null)
		return null;
	/*
	 * X-Forwarded-For may contain multiple IP addresses.
	 * In this case the first recognizable address is the real one.
	 */
	for (String candidate : Arrays.stream(xff.split(",")).map(String::trim).collect(toList())) {
		try {
			/*
			 * We want to skip junk IP addresses, which might have found their way
			 * into X-Forwarded-For via various localhost, company, or ISP proxies.
			 * We will be weeding out IP addresses that cannot be parsed
			 * or that belong in some special address range.
			 */
			InetAddress ip = InetAddress.getByName(candidate);
			if (ip.isSiteLocalAddress() || ip.isLinkLocalAddress() || ip.isLoopbackAddress() || ip.isMulticastAddress())
				continue;
			if (ip instanceof Inet6Address && ip.getAddress()[0] == (byte)0xfd)
				continue;
			byte[] address = ip.getAddress();
			if (IntStream.rangeClosed(0, address.length).allMatch(i -> address[i] == 0))
				continue;
			/*
			 * Perform local IP address anonymization.
			 * We are also instructing GA to anonymize IPs on GA end,
			 * but it is technically and legally safer to anonymize locally.
			 * Anonymization rules are the same as those used by GA,
			 * i.e. zero last 80 bits of IPv6 and last 8 bits of IPv4.
			 */
			int anonymizedBytes = ip instanceof Inet6Address ? 10 : 1;
			for (int i = 0; i < anonymizedBytes; ++i)
				address[address.length - i - 1] = 0;
			return InetAddress.getByAddress(address).getHostAddress();
		} catch (Throwable t) {
		}
	}
	return null;
}

Method language parses language from request's Accept-Language header:

/*
 * This helper method takes contents of Accept-Language header and
 * returns user's language in the 'en-US' or 'en' format.
 */
private static String language(String accepts) {
	if (accepts == null)
		return null;
	/*
	 * The header can list multiple languages, so we just pick the first one.
	 */
	String[] alternatives = accepts.split(",");
	if (alternatives.length == 0)
		return null;
	String first = alternatives[0];
	/*
	 * If there's priority attached to the language (e.g. 'en-US;q=0.5'),
	 * we strip the priority field.
	 */
	String[] parts = first.split(";");
	if (parts.length == 0)
		return null;
	return parts[0].trim().toLowerCase();
}

Please consider adding this functionality into the library, so that people don't have to reimplement it for every project. Thanks!

I started testing the stable version (v1.1.2) but I think this issue is still present on 2.0.

When sending a post to GA the userAgent in my case is:

java/1.8.0_112-b15/Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/Windows 10/10.0/amd64

It includes the OS data but I guess the format is not correct or at least is not the one that GA expects.

User Agent Override

Example value: Opera/9.80 (Windows NT 6.0) Presto/2.12.388 Version/12.14
Example usage: ua=Opera%2F9.80%20%28Windows%20NT%206.0%29%20Presto%2F2.12.388%20Version%2F12.14

The first part is correctly gathered by GA:
Browser: java
Browser Version: 1.8.0_112-b15

but then the rest is empty:
Operating System: (not set)

I just made a test overriding the user agent:

defaultRequest.userAgent("Java/1.8.0_112-b15 (Windows NT 10.0; Win64; x64)");

Now I have to wait until GA audience reports are available, but I will share if that format works.

Hi,

Thank you so much for your effort. I just found your project and it's really useful!

I was trying to implement a back-end test using multi-armed bandit approach. However I noticed that your library doesn't support Experiment ID and Experiment Variant parameter definitions (xid and xvar respectively). Is this by design or is it a missing feature?

This doesn't seem to work. I tried the following:

  GoogleAnalytics ga = new GoogleAnalytics("UA-4XXXXX4-1");
  ga.post(new PageViewHit("https://www.google.com", "Google Search"));

also tried sending events as in:

EventHit eventHit  = new EventHit();
eventHit.clientId("12345");
eventHit.anonymizeIp(true);
eventHit.applicationName("My app");
eventHit.eventCategory("Click");
eventHit.eventAction("Do it!");

GoogleAnalytics ga = new GoogleAnalytics("UA-4XXXXXX4-1");
ga.post(eventHit);

Neither throws any exception however, they don't make it to google analytics.
I've stepped through the code and confirm the HTTPS request is going out and google is returning a response but no events are coming up in Google analytics. Confirmed my UA tracking id is correct.

I can`t get events to appear in google analytics. Everything works fine when I send requests from Postman.

Here is my configuration
No error logs nor exceptions

	ga = GoogleAnalytics.builder()
			.withTrackingId("UA-XXXXXXXX-1")
			.withAppName("S******r")
			.withAppVersion("0.1.1")
			.build();

ga.event()
	.eventAction(event.getAction().name())
	.eventCategory(event.getCategory().name())
	.eventLabel(event.getValue())
	.clientId(clientId)
	.send();

Maven dependency

<dependency>
  <groupId>com.brsanthu</groupId>
  <artifactId>google-analytics-java</artifactId>
  <version>2.0.0</version>
</dependency>

I have just checked out your project and It builds perfectly. It seems that I am missing something because I can not see any data at real time. Is there any mandatory configuration parameter about it?

Couldn't see help here so looked at the code, I think things that I'll use should be in separate namespace/directory (like Ga, GaConfig, different request types...etc)

I can't find anything like this https://github.com/brsanthu/google-analytics-java/wiki/V1-Readme one for v1. Everything is changed and I can't work it out. Could you give me a basic example of the client initialization, a page view hit and an event hit?

class GoogleAnalyticsConfig

private String httpsUrl = "https://www.google-analytics.com/collect";

url for SSL should be "https://ssl.google-analytics.com/collect" according to google documentation

Google analytics user the IP address to get GEO information. This is an issue when sending events from a server since the server's IP is what get's passed to google. The same is true of USER-AGENT which is an HTTP header.

Google recently introduced two new params to override IP and user-agent respectively.

IP Override
parameter: &uip Should be a valid IP address. This will always be anonymized just as though &aip (anonymize IP) had been used. example: &uip=1.2.3.4
User Agent Override
parameter: &ua Should be a User Agent reported by the browser (don't forget to URL encode the value!). Note: We have libraries to identify real user agents. Hand crafting your own agent could break at any time. example: &ua=Opera%2F9.80%20(Windows%20NT%206.0)%20Presto%2F2.12.388%20Version%2F12.14

More on this topic can be found in this google group topic:
https://groups.google.com/forum/#!msg/google-analytics-measurement-protocol/8TAp7_I1uTk/KNjI5IGwT58J

Hi, first of all nice job for the lib =)

I found a problem with EventHit that in the constuctor the eventAction parameter is being set to eventLabel field.

Please expose GoogleAnalyticsExecutor interface on GoogleAnalytics. I can now implement my own GoogleAnalyticsExecutor to do some custom hit processing, but then I have no easy way to forward the hit to the appropriate GoogleAnalytics instance.

This does not count new users vs. returning users because you use random UUID everytime.

Hi brsanthu,

I'm very new with GA and I found this resource. I got it into my local and can build with Netbean (it take 2 hours for netbean downloaded all jar files). But now I don't know how to use for get data or post data to GA. Can you give me some url that link to example code (sorry for my basic request).

Thanks.