maxmind / geoip2-java Goto Github PK

We are using paid maxmind account, GeoIP2-Country binary db, DatabaseReader (not webservice) api.

Operation used: country search
Ip being searched: 65.208.151.118
Exception is thrown:
com.maxmind.db.InvalidDatabaseException: The MaxMind DB file's data section contains bad data: pointer larger than the database.
at com.maxmind.db.Decoder.decode(Decoder.java:80)
at com.maxmind.db.Decoder.decode(Decoder.java:101)
at com.maxmind.db.Reader.resolveDataPointer(Reader.java:205)
at com.maxmind.db.Reader.get(Reader.java:103)
at com.maxmind.geoip2.DatabaseReader.get(DatabaseReader.java:148)
at com.maxmind.geoip2.DatabaseReader.country(DatabaseReader.java:193)

Max mind api version in maven:
<dependency> <groupId>com.maxmind.geoip2</groupId> <artifactId>geoip2</artifactId> <version>2.1.0</version> </dependency>

Code for country search used:
InetAddress ipAddress = InetAddress.getByName(ip);
CountryResponse response = database.country(ipAddress);
com.maxmind.geoip2.record.Country country = response.getCountry();
String result = country.getIsoCode();

Code for database loading:
InputStream data = ExternalIpMappingServiceImpl.class.getClassLoader().getResourceAsStream("GeoIP2-Country.mmdb");
database = new DatabaseReader.Builder(data).build();

In GeoIP2-Java documentation (http://maxmind.github.io/GeoIP2-java/) the requirements states:
"MaxMind has tested this API with Java 6 and above."

But versions >= 2.8 cant run on Java 6.
It would be helpful if that page shows a version summary of GeoIP versions x Java supported versions.

you are saying to use and create pom.xml file
where do i store it ?

<dependency>
    <groupId>com.maxmind.geoip2</groupId>
    <artifactId>geoip2</artifactId>
    <version>2.2.0</version>
</dependency>

there is library class conflict or something if you are using a different version of jackson already in the developer's app?

java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonParser.getValueAsString()Ljava/lang/String;
at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:29)
at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:432)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:314)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449)
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449)
at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:2860)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1569)
at com.fasterxml.jackson.databind.ObjectMapper.treeToValue(ObjectMapper.java:1902)
at com.maxmind.geoip2.DatabaseReader.get(DatabaseReader.java:96)
at com.maxmind.geoip2.DatabaseReader.city(DatabaseReader.java:118)

Hi,
In our project we are using older version of following jars and unfortunately we cannot replace with the jar which comes with maxmind geoip2-2.8.0 libs.
We are using
commons-codec-1.7
jackson-annotations-2.4.2
jackson-core-2.4.2
jackson-databind-2.4.2

maxmind geoip2-2.8.0
commons-codec-1.9
jackson-annotations-2.8.0
jackson-core-2.8.2
jackson-databind-2.8.2

Due to this I am always below getting run time error

java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
	com.maxmind.db.Decoder.decodeArray(Decoder.java:272)
	com.maxmind.db.Decoder.decodeByType(Decoder.java:156)
	com.maxmind.db.Decoder.decode(Decoder.java:147)
	com.maxmind.db.Decoder.decodeMap(Decoder.java:281)
	com.maxmind.db.Decoder.decodeByType(Decoder.java:154)
	com.maxmind.db.Decoder.decode(Decoder.java:147)
	com.maxmind.db.Decoder.decode(Decoder.java:87)
	com.maxmind.db.Reader.<init>(Reader.java:132)
	com.maxmind.db.Reader.<init>(Reader.java:116)
	com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:35)
	com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:23)
	com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:129)

Any suggestion ?
thanks in advance

The below issue was not there before two days.

Failed to execute goal on project ABC: Could not resolve dependencies for project com:war:1.0.0: Failed to collect dependencies at com.maxmind.geoip2:geoip2:jar:2.7.0 -> com.maxmind.db:maxmind-db:jar:1.2.1 -> com.fasterxml.jackson.core:jackson-databind:jar:2.9.0-SNAPSHOT: Failed to read artifact descriptor for com.fasterxml.jackson.core:jackson-databind:jar:2.9.0-SNAPSHOT: Could not find artifact com.fasterxml:oss-parent:pom:28

I had a look at:
http://dev.maxmind.com/geoip/legacy/geolite/

The GeoLite databases are our free IP geolocation databases. They are updated on the first Tuesday of each month. These databases are offered in the same binary and csv formats as our subscription databases. Any code which can read the subscription databases can also read the GeoLite databases.

So I downloaded http://geolite.maxmind.com/download/geoip/database/GeoIPv6.dat.gz
and unzipped into a .dat file.

Have the following code:

        File file = new File("C:\\GeoIPv61.dat");
        DatabaseReader reader = new DatabaseReader.Builder(file)
                .withCache(new CHMCache(100)).build();

        InetAddress ipAddress = InetAddress.getByName("128.101.101.101");
        CityResponse response = reader.city(ipAddress);
        Country country = response.getCountry();
        System.out.println(country.getIsoCode());
        System.out.println(country.getName());
        System.out.println(country.getNames().get("zh-CN"));

Result:

Exception in thread "main" com.maxmind.db.InvalidDatabaseException: Could not find a MaxMind DB metadata marker in this file (GeoIPv61.dat). Is this a valid MaxMind DB file?

How should one use the java api here to access the legacy databases?

When database file is read into ByteArrayInputstream and used in Reader. Getting below exception.

DatabaseReader reader = new DatabaseReader.Builder(stream).withCache(new CHMCache()).build();

error
Could not find a MaxMind DB metadata marker in this file (). Is this a valid MaxMind DB file?

Can you please provide a JAR file for those of us who do not use Maven?

Thanks.

Hi Folks,
There are newever versions of the Apache httpclient artifact which this library should consider updating to.
Clients consuming geoip2 have classloading issues due to geoip2 using an old httpclient version.
I'll see if I can get something together and submit a PR.

Where i can find this File GeoIP2-City.mmdb

Hi,

When I try to create a DatabaseReader from an InputStream instead of a File (something like):

InputStream is = getClass().getClassLoader().getResourceAsStream(filename);
cities = new DatabaseReader.Builder(is).build();

I get the following:

Caused by: java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/node/FloatNode
    at com.maxmind.db.Reader.<init>(Reader.java:89)
    at com.maxmind.db.Reader.<init>(Reader.java:65)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:33)
    at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:122)

PS: This is using version 0.8.0:

        <dependency>
            <groupId>com.maxmind.geoip2</groupId>
            <artifactId>geoip2</artifactId>
            <version>0.8.0</version>
        </dependency>

When using FileMode.MEMORY_MAPPED, the .mmdb file on the file system is not available for manipulation (renaming, moving, deleting) even after the reader is closed.

Using FileMode.MEMORY doesn't suffer from this problem. See sample unit test below. Could be OS-related (using JDK7 on Win7, MaxMindDB: v.0.3.0, GeoIP2: v.0.6.0).
Ran into the issue while working on automatically updating the .mmdb file once a month in a long-running process.

import java.io.File;
import java.io.IOException;
import org.junit.Assert;
import org.junit.Test;
import com.maxmind.db.Reader.FileMode;
import com.maxmind.geoip2.DatabaseReader;
import com.maxmind.geoip2.DatabaseReader.Builder;

public class _GeolocationFileUpdate
{
    private static final String GEO_LITE2_CITY_MMDB = "GeoLite2-City.mmdb";
    private static final String GEO_LITE2_CITY2_MMDB = "GeoLite2-City2.mmdb";
    private static final String DIR = "c://Temp/"; //adjust if on a diff. OS. Place the .mmdb file here
    private static final File f1 = new File( DIR + GEO_LITE2_CITY_MMDB );
    private static final File f2 = new File( DIR + GEO_LITE2_CITY2_MMDB );

    @Test
    public void testSwitchingMem()
    {
        try
        {
            // make sure the file name is reverted after renaming:
            f2.renameTo( f1 );

            Builder builder = new DatabaseReader.Builder( f1 );
            builder.fileMode( FileMode.MEMORY );
            DatabaseReader reader = builder.build();
            reader.close();

            // make sure file is not in use by a process:
            Assert.assertTrue( f1.renameTo( f2 ) );
        }
        catch ( IOException e )  {  e.printStackTrace();  }
    }

    @Test
    public void testSwitchingMemMapped()
    {
        try
        {
            // make sure the file name is reverted after renaming:
            f2.renameTo( f1 );

            Builder builder = new DatabaseReader.Builder( f1 );
            builder.fileMode( FileMode.MEMORY_MAPPED );
            DatabaseReader reader = builder.build();
            reader.close();

            // make sure file is not in use by a process:
            // ! test fails on the line bellow, file is in use !:
            Assert.assertTrue( f1.renameTo( f2 ) );
        }
        catch ( IOException e ) { e.printStackTrace(); }
    }
}

This is one thing that has been bugging me for years as a user of GeoIP City.

For some inexplicable reason the Country codes and country names were not part of the data file, but instead hard coded in an array in the API lookup code. This of course meant a code upgrade every time a new country appears. For a while it even meant array index out of bounds exceptions if we were too late updating our client library.

These days I see the same folly repeated for time zone names and region names. Why on earth can those not be placed in the data file? At 40+ MB the added cost of a few hundred extra names should be negligible.

If the concern is speed it should be easy enough to read all the lookup tables into memory on init.

If the concern is compatibility there is still no reason to hard code more lookups into the API code. Without knowing the full details of the data file layout I still imagine there would be room to add slots for lookup tables without breaking existing clients.

I was making use of legacy Geolite City api. But for every IP I am not able to get region. This was very frustrating for me. Of course its a free, It should return country name and region code at least.

Does this API support taking a root level object and serializing the full graph to JSON (and back and forth)? If not, could it be added? This would be useful when taking output from your library and dumping into a document store such as mongo

Hi

I have ran a simple test to check the performance of the library when working with several threads. I am working currently with the GeoLite2-City.mmdb database.
What I did was running 4 threads, each thread doing 1 million requests to the API (DatabaseReader.city). Test ran ~80 seconds. I did the similar test with 1 thread - test ran ~30 seconds. In order to make sure this is not a machine resource issue I ran the first test again (4 threads) - only this time I created a different instance of DatabaseReader per thread. The test ran 30 seconds (same result as 1 thread). In this case the memory consumption was obviously 4 times higher.
By the way, I have set the FileMode to both MEMORY and MEMORY_MAPPED... no real difference between the two in all cases.

I tried to look into the code a bit. I found one suspected issue - I don't think that the method BufferHolder.get() should be synchronized... all it does, I think, is just duplicating a "prototype" (read only). However, I tried changing it and it did not help, so I do not believe this is the problem.

When working with high performance services, this issue is really important... I would appreciate your response..:)

Thanks!
Doron.

Hi

We are trying to synchronize a DatabaseReader update (after mmdb file changed) with reads happening a-synchronically without using locks (code MUST be clean of locks).
We are working in MEMORY mode (not MEMORY_MAPPED).
The only way to do that, as I see it, is if we first load the new file into a new DatabaseReader - and then replace the reader we are using in the reads (as long as the reader is a volatile member, this should be a safe operation). The problem is that in such flow we cannot call close() on the DatabaseReader we are dumping, because we may close a reader being currently used (simple test created such scenario and exceptions).
How important is the close() operation when using MEMORY mode? I tried profiling a bit and did not see any "memory leak" when I am not closing. Do I miss anything? It seems like the "set null" done there is meant for garbage collection that is important only in MEMORY_MAPPED mode.

Thanksl

I need to load the DB from within a Jar classpath, so i don't have a File reference. Can you add a constructor that takes an InputStream please?

Hello, I can not integrate GeoIP2 with Wowza, what could be happening? The GeoIP version (1) worked perfectly.

Wowza Code:

import com.maxmind.geoip2.*;
import com.maxmind.geoip2.exception.GeoIp2Exception;
import com.maxmind.geoip2.model.CityResponse;
import com.maxmind.geoip2.record.City;
import com.maxmind.geoip2.record.Country;
import com.maxmind.geoip2.record.Location;
import com.maxmind.geoip2.record.Postal;
import com.maxmind.geoip2.record.Subdivision;
import com.maxmind.db.*;

    public static final String DATABASE_CITY_PATH = "C:/wowza/GeoIP2-City.mmdb";

    public void onAppStart(IApplicationInstance appInstance) {
        appInstance.setStreamNameAliasProvider(this);
        this.applicationInstance = appInstance;
        this.vHostName = appInstance.getVHost();

        try {
            Class.forName("com.mysql.jdbc.Driver").newInstance(); 
            // A File object pointing to your GeoLite2 database
               File dbFile = new File(DATABASE_CITY_PATH);
               getLogger().info("DATABASE_CITY_PATH: "+DATABASE_CITY_PATH);
               // This creates the DatabaseReader object,
               // which should be reused across lookups.

               DatabaseReader reader = new DatabaseReader.Builder(dbFile).build();

               // A IP Address
               InetAddress ipAddress = InetAddress.getByName("128.101.101.101");

               // Get City info
               CityResponse response = reader.city(ipAddress);

               // Country Info
               Country country = response.getCountry();
               System.out.println("Country IsoCode: "+ country.getIsoCode()); // 'US'
               System.out.println("Country Name: "+ country.getName()); // 'United States'
               System.out.println(country.getNames().get("zh-CN")); //

               Subdivision subdivision = response.getMostSpecificSubdivision();
               System.out.println("Subdivision Name: " +subdivision.getName()); // 'Minnesota'
               System.out.println("Subdivision IsoCode: "+subdivision.getIsoCode()); // 'MN'

               // City Info.
               City city = response.getCity();
               System.out.println("City Name: "+ city.getName()); // 'Minneapolis'

               // Postal info
               Postal postal = response.getPostal();
               System.out.println(postal.getCode()); // '55455'

               // Geo Location info.
               Location location = response.getLocation();

               // Latitude
               System.out.println("Latitude: "+ location.getLatitude()); // 44.9733

               // Longitude
               System.out.println("Longitude: "+ location.getLongitude()); // -93.2323

        } catch (Exception e) { 
            getLogger().error("Error loading plugins: "+e.toString());
        }
    }

Here is a link with more explanations. http://pt.stackoverflow.com/q/180286/8984
Because the old version works, but the new one, GeoIP2 did not work @oschwald ?

I'm testing GeoIP2 vs GeoIP and getting a very strange result, it seems the new api is way slower.
maxMind1 - Legacy API
maxMind2 - new API, FileMode.MEMORY_MAPPED
maxMind2v2 - new API, FileMode.MEMORY
Caches are on;

Here are my benchmark results:

Benchmark                      Mode     Cnt       Score      Error  Units
MindMaxBenchmark.maxMind1     thrpt      50  176225.276 ± 1603.768  ops/s
MindMaxBenchmark.maxMind2     thrpt      50     825.766 ±   12.428  ops/s
MindMaxBenchmark.maxMind2v2   thrpt      50     767.863 ±   35.938  ops/s
MindMaxBenchmark.maxMind1      avgt      50      ≈ 10⁻⁵              s/op
MindMaxBenchmark.maxMind2      avgt      50       0.001 ±    0.001   s/op
MindMaxBenchmark.maxMind2v2    avgt      50       0.001 ±    0.001   s/op
MindMaxBenchmark.maxMind1    sample  131466      ≈ 10⁻⁵              s/op
MindMaxBenchmark.maxMind2    sample   38187       0.001 ±    0.001   s/op
MindMaxBenchmark.maxMind2v2  sample   34064       0.001 ±    0.001   s/op
MindMaxBenchmark.maxMind1        ss      50      ≈ 10⁻⁴              s/op
MindMaxBenchmark.maxMind2        ss      50       0.028 ±    0.005   s/op
MindMaxBenchmark.maxMind2v2      ss      50       0.026 ±    0.004   s/op

Services are initialized at the test start like that:

final DatabaseReader mindMaxDB = new DatabaseReader.Builder(...)
                    .withCache(new CHMCache())
                    .fileMode(Reader.FileMode.MEMORY)
                    .build();

and the test code is like (some null checks are ommited):

        public LocationData resolve(InetAddress inetAddress) {

                CityResponse cityResponse = mindMaxDB.city(inetAddress);
                Location location = cityResponse.getLocation();
                return new LocationData(
                        inetAddress, cityResponse.getCountry().getIsoCode(), cityResponse.getCity().getName()
                );
            }

The Legacy API:
Service created like that

LookupService lookupService = new LookupService(file, LookupService.GEOIP_MEMORY_CACHE);

and test code is

 com.maxmind.geoip.Location location = lookupService.getLocation(inetAddress);
                            return new LocationData(
                                    inetAddress, location.countryCode,
                                    location.city);

I'd like to know if I'm missing something?

The transitive dependency on com.maxmind.db:maxmind-db contains an open-ended version clause for jackson-databind: [2.7.0,). This causes gradle to pull in the 2.9.0-SNAPSHOT of jackson-databind into the project.

+--- com.maxmind.geoip2:geoip2:2.8.0
|    +--- com.maxmind.db:maxmind-db:1.2.1
|    |    \--- com.fasterxml.jackson.core:jackson-databind:[2.7.0,) -> 2.9.0-SNAPSHOT (*)
|    +--- org.apache.httpcomponents:httpclient:4.5.2 (*)
|    \--- com.fasterxml.jackson.core:jackson-databind:2.8.2 -> 2.9.0-SNAPSHOT (*)

This of course is a big problem -- the runtime now fails when the rest of the project uses a 2.8 release version of jackson.

I'm not sure why this has caused problems in our builds only recently because AFAICT this is not a new thing in maxmind-db. Still, it would be better if maxmind-db declared the version dependency explicitly instead of giving an open-ended range, which I think is the issue.

com.fasterxml.jackson.databind.node was introduced to Jackson with 2.2 but tag 0.4.0 of com.maxmind.geoip2:geoip2 depends on 2.1 (due to com.google.http-client:google-http-client-jackson2:jar:1.15.0-rc:compile), so I am getting java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/node/FloatNode

import com.maxmind.db.Metadata;
import com.maxmind.db.Reader;
import com.maxmind.db.Reader.FileMode;

It can't find these files.
So it can't run - class not found.

From a performance point of view it would be much better to just return null in this case. The overhead that is required to generate the exception object is noticeable when profiling our application.

Hi,

I'm using GeoIP2 in my Android app and I'm getting this crash with my users. I think the crash rate is about 50% although I cannot replicate it. There is no particular version of Android, though all my users are Android 4+. No particular language or device. Can you please help? I don't know what it is they are getting as a response that is causing the casting of a String to Long to fail.

Cheers, Duane.

java.lang.RuntimeException: An error occured while executing doInBackground()
at android.os.AsyncTask$3.done(AsyncTask.java:300)
at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:355)
at java.util.concurrent.FutureTask.setException(FutureTask.java:222)
at java.util.concurrent.FutureTask.run(FutureTask.java:242)
at android.os.AsyncTask$SerialExecutor$1.run(AsyncTask.java:231)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:848)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Long
at com.google.api.client.http.HttpHeaders.java.lang.Long getContentLength()(SourceFile:326)
at com.maxmind.geoip2.WebServiceClient.java.lang.Object responseFor$296315a2(java.lang.String,java.lang.Class)(SourceFile:258)
at com.maxmind.geoip2.WebServiceClient.com.maxmind.geoip2.model.CityResponse city()(SourceFile:225)
at com.enzuredigital.weatherbomb.WeatherActivity$InitPlaceTask.java.lang.String doInBackground$4af589aa()(SourceFile:2459)
at com.enzuredigital.weatherbomb.WeatherActivity$InitPlaceTask.java.lang.Object doInBackground(java.lang.Object[])(SourceFile:2405)
at android.os.AsyncTask$2.call(AsyncTask.java:288)
at java.util.concurrent.FutureTask.run(FutureTask.java:237)
... 4 more

This is the code in my Android app:

WebServiceClient client = new WebServiceClient.Builder(MAXMIND_ID, MAXMIND_KEY).build();
CityResponse response = null;
try {
    response = client.city();  // <---- Offending line.
} catch (IOException e) {
    e.printStackTrace();
} catch (GeoIp2Exception e) {
    e.printStackTrace();
}

Hello,

I'm using your API (v 2.5) through this library and requesting as:

new WebServiceClient.Builder(userId, licenseKey).city(InetAddress.getByName(anIpAddress))

In the documentation I see the field subdivisions and I don't see any indication that it is optional.

Debugging I see that the CityResponse returned by this library has an empty ArrayList as the value for it's subdivisions field but when serialized into JSON the field isn't there which leads me to assume that when a collection is empty the key-value pair is not even serialized (in my opinion this is wrong, since an empty collection would also indicate that there are no subdivisions but wouldn't force the clients to check for the field's existence).

So, is the subdivisions field optional? Where does it say if it is or not?

I'm trying to use this library, but get the error below from the following line:

val file = new File("src/main/resources/GeoLite2-City.mmdb")
val db = new DatabaseReader.Builder(file).withCache(new CHMCache()).build()

yields:

An exception or error caused a run to abort: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V 
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
    at com.maxmind.db.Decoder.decodeArray(Decoder.java:272)
    at com.maxmind.db.Decoder.decodeByType(Decoder.java:156)
    at com.maxmind.db.Decoder.decode(Decoder.java:147)
    at com.maxmind.db.Decoder.decodeMap(Decoder.java:281)
    at com.maxmind.db.Decoder.decodeByType(Decoder.java:154)
    at com.maxmind.db.Decoder.decode(Decoder.java:147)
    at com.maxmind.db.Decoder.decode(Decoder.java:87)
    at com.maxmind.db.Reader.<init>(Reader.java:132)
    at com.maxmind.db.Reader.<init>(Reader.java:116)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:39)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:27)
    at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:133)
    at com.sift.etl.processors.IpToLatLong$.<init>(IpToLatLong.scala:22)

Does this indicate the source is missing the jackson.databind.jar? Is it possible I have two conflicting copies of this dependency?

When we try to pass a valid InetAddress created from a hostname to the DatabaseReader, we receive a IOException back from the library.

// This creates the DatabaseReader object, which should be reused across
// lookups.
DatabaseReader reader = new DatabaseReader.Builder(database).build();

CityResponse response = reader.city(InetAddress.getByName("www.maxmind.com"));
System.out.println(response.getCountry().getIsoCode()); // 'US'

Would it be possible to get also for this cases a valid result, because the InetAddress receives the correct address.

thx Tommy

Hi,

it looks like the DatabaseReader.close() of geoip2-2.0.0 doesn't work as excpeted. Looking deeper it seems that just an AtomaticReference is set to null. The problem is that, closing the reader, moving the db file, and opening it again, fails. This is demonstrated in the JUnit test case below. Moreover a System.gc should not be necessary.

import java.io.File;
import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;

import org.junit.Assert;
import org.junit.Test;

import com.maxmind.db.Reader.FileMode;
import com.maxmind.geoip2.DatabaseReader;
import com.maxmind.geoip2.exception.GeoIp2Exception;

public class TestDatabaseReaderClose {
    @Test
    public void test1() throws Exception {
        executeTest(false, true);
    }
    @Test
    public void test2() throws Exception {
        executeTest(true, false);
    }
    @Test
    public void test3() throws Exception {
        executeTest(false, true);
    }
    @Test
    public void test4() throws Exception {
        executeTest(true, false);
    }
    private void executeTest(boolean memoryMapped, boolean gc) throws IOException, GeoIp2Exception, UnknownHostException {
        String fileName = "C:\\Data\\GeoIP2-City.mmdb";

        File file1 = new File(fileName);
        File file2 = new File(fileName + ".temp");

        DatabaseReader reader1 = new DatabaseReader.Builder(file1).fileMode(memoryMapped ? FileMode.MEMORY_MAPPED : FileMode.MEMORY).build();
        Assert.assertEquals("GB", reader1.city(InetAddress.getByName("80.1.1.1")).getCountry().getIsoCode());
        reader1.close();
        if (gc)
            System.gc();

        Assert.assertTrue(file1.renameTo(file2));
        Assert.assertTrue(file2.renameTo(file1));

        DatabaseReader reader2 = new DatabaseReader.Builder(file1).build();
        Assert.assertEquals("GB", reader2.city(InetAddress.getByName("80.1.1.1")).getCountry().getIsoCode());
        reader2.close();
    }
}

For a scenario with a long-running process doing automatic downloads of the GeoLite2 DB from the MaxMind's web site it would be very helpful to be able to get some metadata of the DB.
Currently the com.maxmind.db.Metadata is not exposed neither through the com.maxmind.geoip2.DatabaseReader nor through the com.maxmind.db.Reader classes. Would be nice to be able to get it through another getter in the com.maxmind.geoip2.DatabaseReader class.

Ideally there should be a way to check the metadata even before downloading (maybe through a GET at a different URL than the DB) so a decision can be made if the DB should be downloaded or not at this time (for example if there is a major revision update, it would become incompatible with the code reading it) but this would involve web site changes.

The documentation found at http://maxmind.github.io/GeoIP2-java/index.html does not reflect the recent changes to the model.

1. In the old API, the Location class provided the dmacode. Is there any API to get the DMA code from the new API ?
2. In the old API, the Location class provided the region which maps to the state. ? Is the multi-level subdivision attributes provides the state info in the new API ?

Thanks,
Yogesh

I put dependency GeoIP2-java in maven project, but i have this error:

2016-10-08 17:20:26,482 ERROR [dispatcherServlet]: Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Handler processing failed; nested exception is java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V] with root cause
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
at com.maxmind.db.Decoder.decodeArray(Decoder.java:272) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decodeByType(Decoder.java:156) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decode(Decoder.java:147) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decodeMap(Decoder.java:281) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decodeByType(Decoder.java:154) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decode(Decoder.java:147) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Decoder.decode(Decoder.java:87) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Reader.(Reader.java:132) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.db.Reader.(Reader.java:116) ~[maxmind-db-1.2.1.jar:na]
at com.maxmind.geoip2.DatabaseReader.(DatabaseReader.java:35) ~[geoip2-2.8.0.jar:2.8.0]
at com.maxmind.geoip2.DatabaseReader.(DatabaseReader.java:23) ~[geoip2-2.8.0.jar:2.8.0]
at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:129) ~[geoip2-2.8.0.jar:2.8.0]

please help

The way jackson-databind is being used, is that it always uses reflection to suppress access checking in order to deserialise into the geoip2 model classes. This doesn't have to be way and jackson-databind can also be used without suppressing access checks. The jackson-databind's ObjectMapper needs be configured to not overwrite access modifiers ObjectMapper#configure(MapperFeature.CAN_OVERRIDE_ACCESS_MODIFIERS, false) and the model classes would then also need to have public getters/setters or public constructors. I'm wondering if using reflection to suppress access checking was chosen for a particular reason. It would be great if geoip2-java can be used without the suppressAccessChecks permission.

We're currently developing a pipeline framework to enrich data before indexing:
elastic/elasticsearch#14049

and are using geoip for geo lookups: https://github.com/elastic/elasticsearch/blob/feature/ingest/plugins/ingest/src/main/java/org/elasticsearch/ingest/processor/geoip/GeoIpProcessor.java#L47

Elasticsearch runs with the security manager enabled and doesn't allow suppressing access checks. Because this pipeline framework is a plugin we can specifically allow it in specifically privilege code blocks, but we prefer to use geoip2 without setting the suppressAccessChecks privilege at all.

Hi am trying to write a UDF for pig and i have using this for getting ip details.

my code is new DatabaseReader.Builder(db).build();

which generate error stack like this.

Error: java.lang.ClassNotFoundException: com.maxmind.geoip2.DatabaseReader$Builder
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.XXXXXX.XXXXXXX.pig.ParseGeolocation.getDBReader(ParseGeolocation.java:86)

I am using maven for dependency.

Java version is 1.8

also if i trying to use withcache() then i always get com/maxmind/db/NodeCache not found. I have no idea why this happening.

Version I am using is 2.7.0

thanks and regards
Mithun Mandal

I am new to GeoIP and am trying to set up a test app to try it out on the free databases. I have imported the DatabaseReader class as well as the DatabaseReader.Builder class, and when I create a DatabaseReader object the I get an error that says:

Exception in thread "main" java.lang.NoClassDefFoundError: com/maxmind/db/Reader$FileMode
at com.maxmind.geoip2.DatabaseReader$Builder.(DatabaseReader.java:76)
at GeoIP.main(GeoIP.java:55)
Caused by: java.lang.ClassNotFoundException: com.maxmind.db.Reader$FileMode
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 2 more
Java Result: 1

I am not sure why this is, I have a File object pointing to my database and have no errors there. And I have properly added the GeoIP2.jar file to my project. Has anyone seen this error before?

Hi there,

I want to use geoip2 to process nging log with Spark Streaming, but I could not find any example about how to use it with Spark Streaming. Do I use it in a wrong way ?

Hi, I was using the old legacy library and it had the region codes and names hard coded in.

Is there any plan to add them back?
Why were they added as a class instead of in the database itself?

Regards

In the Readme

   <dependency>
        <groupId>com.maxmind.geoip2</groupId>
        <artifactId>geoip2</artifactId>
        <version>v2.3.0</version>
    </dependency>

The version bit should be just '2.3.0' and not 'v.2.3.0'

With the version 2.6.0, downloaded at https://github.com/maxmind/GeoIP2-java/releases/download/v2.6.0/geoip2-2.6.0-with-dependencies.zip, I came across this issue:

com/maxmind/db/Decoder.java:272:in `decodeArray': java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)

When trying to instance a DatabaseReader, with jRuby (this is to build a logstash filter plugin):

db_file = JavaIO::File.new(@database)
@parser = DatabaseReader::Builder.new(db_file).build();

It looks like an issue with Jackson lib?

Hi there

Can you give some guidance on how DatabaseReader is intended to be used in combination with weekly scheduled updates via the geoipupdate tool?

Should DatabaseReader "just work" in this case, do you need to close it and open a new one every week or otherwise prod it to check for an updated DB? Something in the docs about this would be good. And of course strong preference for "just works" :)

Cheers!
-Matt

Hi,
I have already used ok City and ISP API, work very well. Thanks so much!
But I'm having issue about getting timezone data. Would you please show me the way to get it?
Thanks!

I tried the following:

File database = new File("/path/to/GeoLite2-City.mmdb");
boolean memoryMapped = true    
    DatabaseReader reader = new DatabaseReader.Builder(file).fileMode(memoryMapped ? FileMode.MEMORY_MAPPED : FileMode.MEMORY).build();
InetAddress ipAddress = InetAddress.getByName("78.54.245.106")
CityResponse cResponse = reader.city(ipAddress)
City city = cResponse.getCity()
println("city2: "+city.getName())

it turns out that city is always null.

Hi,

I have seen that you have the possibility of setup a connection timeout which is working fine. I have opened a service request few days ago about the possibility of adding a read timeout.

I got as a response this branch: https://github.com/maxmind/GeoIP2-java/tree/bz/add-read-timeout which I have used but seems not to work.

Although I setup a read timeout of 50 msecs (just for testing purposes) the call to your web service lasts for few seconds. Using the debugger looks like the read timeout is being completely ignored.

We need this feature as the response time is pretty critical to our company.

Can someone give a look to this?

Thanks

Google's client wraps the Apache client but often lags a few version behind and has been a source of problems. The new minFraud Java API already uses the Apache client API directly.

We may also be able to share code between the GeoIP2 and minFraud APIs.

It would be great to have the possibility to open database in the com.maxmind.db.Reader FileMode.MEMORY mode.

Java implementation of memory-mapped files has some issues (e.g.http://bugs.sun.com/view_bug.do?bug_id=4724038 <http://bugs.sun.com/view_bug.do?bug_id=4724038 ) and in some cases it is necessary to avoid the use of this technology.

I want to use GeoIP2 Java API for location detection. I found a way for detecting location from IP. But it is not a comprehensive way if user is behind of a proxy server. Is there any way exist for detecting location from Longitude and Latitude using GeoIP2 Java API? I want to avoid using Google API for detecting location as it is not fully free.

I am expecting a quick response or resolution regarding this.

Thanks.

This line creates a new transport on each request:

https://github.com/maxmind/GeoIP2-java/blob/master/src/main/java/com/maxmind/geoip2/WebServiceClient.java#L281

As per the comment, this is done because the transport is not thread-safe. However, according to Google the transport is thread-safe:

"Implementation is thread-safe. For maximum efficiency, applications should use a single globally-shared instance of the HTTP transport."

From:

https://developers.google.com/api-client-library/java/google-http-java-client/reference/1.20.0/com/google/api/client/http/javanet/NetHttpTransport