[Feature Request] Supporting IPinfo MMDB databases about locus HOT 6 OPEN

@g-andrade My apologies. I should have been clearer. I wanted to mention that the structure is tabular. You know how in MaxMind, if you want to get the city name of an IP address, you have to go through city → en → iso_code. With us, it is just city.

Here is a GIF of using the MMDB database using the mmdbctl tool.

No rush. This is just a request. I really appreciate your taking a look. Thank you very much.

from locus.

No worries about the IPv4 issue, it was indeed a bug that had been present in locus since forever, only it didn't show up with other databases:
#44

It's fixed in the latest version, which I pushed today.

from locus.

Hi @abdullahdevrel, after a very quick look at your proposal, I think it’s important to clarify that MMDB as supported by locus refers to a particular binary format:
https://maxmind.github.io/MaxMind-DB/

These tabular files (from a first look, either JSON or CSV) are, effectively, in an entirely different format that locus doesn’t know about. Calling them MMDB would be akin to saying an SVG image is just like a PNG one, only in vector format!

Let me know if I got it wrong - as I wrote earlier, I took a very quick look and got alarmed at seeing what may not be MMDB at all. An entirely different format would raise very substantially the technical cost for any kind of support.

from locus.

My bad, I got the wrong thought when I read tabular. As long as it’s MMDB, there should be no issue. I’ll give it a try when I get a chance.

from locus.

Overall compatibility

The free IPinfo ASN, Country and Country+ASN databases load alright from the local filesystem:

Possible IPv4 bug

IPv4 lookups may not be working (tried 93.184.215.14, the address for example.com);
IPv6 lookups are (tried 2606:2800:21f:cb07:6820:80da:af6b:8b2c).

That reminded me of PR #39, closed a few months ago. Given what it appeared to suggest was in need of fixing:

There may be a 6+ years old bug in the code owing to my wrong interpretation of the spec, which I never encountered before out of chance.

Downloading (and updating) using HTTP

The databases can be loaded using HTTP (censor_query will censor the token in logged messages):

Although I very quickly hit the default limit of 10 daily downloads per database, which is not too bad since locus caches the database locally and subsequent requests are conditional (it sends if-modified-since to avoid unnecessary data transfer) but will log errors.

from locus.

@g-andrade Thank you very much for looking into the request. Really appreciate it!

Although I very quickly hit the default limit of 10 daily downloads per database

Can you please let me know the email account or the access token you used to sign up? You can email it to me if you'd like. My email is [email protected]. I will increase the rate limit.

I tried switching the IPv4-in-IPv6 tree prefix from ::ffff:0:0/96 to ::/96, and it... worked? I can now get IPv4 entries from the IPinfo database.

I will look into it. I have faced some similar errors when working with DuckDB's inet data type.

from locus.