maxmind / maxmind-db-reader-python Goto Github PK

View Code? Open in Web Editor NEW

172.0 24.0 36.0 500 KB

Python MaxMind DB reader extension

Home Page: https://maxminddb.readthedocs.org/

License: Apache License 2.0

Python 65.82% C 32.65% Shell 1.53%

geoip mmdb geoip2 maxmind

maxmind-db-reader-python's Introduction

MaxMind DB Python Module

Description

This is a Python module for reading MaxMind DB files. The module includes both a pure Python reader and an optional C extension.

MaxMind DB is a binary file format that stores data indexed by IP address subnets (IPv4 or IPv6).

Installation

To install maxminddb, type:

$ pip install maxminddb

If you are not able to use pip, you may also use easy_install from the source directory:

$ easy_install .

The installer will attempt to build the C extension. If this fails, the module will fall-back to the pure Python implementation.

Usage

To use this module, you must first download or create a MaxMind DB file. We provide free GeoLite2 databases. These files must be decompressed with gunzip.

After you have obtained a database and imported the module, call open_database with a path, or file descriptor (in the case of MODE_FD), to the database as the first argument. Optionally, you may pass a mode as the second argument. The modes are exported from maxminddb. Valid modes are:

MODE_MMAP_EXT - use the C extension with memory map.
MODE_MMAP - read from memory map. Pure Python.
MODE_FILE - read database as standard file. Pure Python.
MODE_MEMORY - load database into memory. Pure Python.
MODE_FD - load database into memory from a file descriptor. Pure Python.
MODE_AUTO - try MODE_MMAP_EXT, MODE_MMAP, MODE_FILE in that order. Default.

NOTE: When using MODE_FD, it is the caller's responsibility to be sure that the file descriptor gets closed properly. The caller may close the file descriptor immediately after the Reader object is created.

The open_database function returns a Reader object. To look up an IP address, use the get method on this object. The method will return the corresponding values for the IP address from the database (e.g., a dictionary for GeoIP2/GeoLite2 databases). If the database does not contain a record for that IP address, the method will return None.

If you wish to also retrieve the prefix length for the record, use the get_with_prefix_len method. This returns a tuple containing the record followed by the network prefix length associated with the record.

You may also iterate over the whole database. The Reader class implements the __iter__ method that returns an iterator. This iterator yields a tuple containing the network and the record.

Example

>>> import maxminddb
>>>
>>> with maxminddb.open_database('GeoLite2-City.mmdb') as reader:
>>>
>>>     reader.get('152.216.7.110')
{'country': ... }
>>>
>>>     reader.get_with_prefix_len('152.216.7.110')
({'country': ... }, 24)
>>>
>>>     for network, record in reader:
>>>         ...

Exceptions

The module will return an InvalidDatabaseError if the database is corrupt or otherwise invalid. A ValueError will be thrown if you look up an invalid IP address or an IPv6 address in an IPv4 database.

Requirements

This code requires Python 3.8+. Older versions are not supported. The C extension requires CPython.

Versioning

The MaxMind DB Python module uses Semantic Versioning.

Support

Please report all issues with this code using the GitHub issue tracker

If you are having an issue with a MaxMind service that is not specific to this API, please contact MaxMind support for assistance.

maxmind-db-reader-python's People

Contributors

Stargazers

Watchers

maxmind-db-reader-python's Issues

New version is breaking Python 2 installs - missing setup.py python_requires update

Hi,

The current version, which is stated to be Python 3 only, actually gets into Python 2 installations because of a missing python_requires update:

MaxMind-DB-Reader-python/setup.py

Line 112 in 30d8bf0

python_requires=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*",

Could you please fix this?

CFFI bindings

Under PyPy performance could be hurt by using the CPython extension.
I know that it's disabled right now but we could replace it with a CFFI extension which might boost performance.
Do you guys have a benchmark suite that you use to test this?

Python aarch64/arm64 whl releases

With increasing popularity of M1/M2/M3 and ARM based cloud computing, it would be immensely helpful to also get aarch64/arm64 whl releases.

Thanks!

Should support the context manager

This API should support using the context manger for opening and closing the database.

ValueError: Unsupported open mode: MODE_MMAP_EXT

reader = maxminddb.open_database(self.GeoLite2_City_file, mode="MODE_MMAP_EXT")

Providing a mode to the open_database function results in the following error:

ValueError: Unsupported open mode: MODE_MMAP_EXT

Can't import maxminddb (v0.3.0)

When I install maxminddb via PyPi, and then try and import maxminddb from the interpreter, I get an import error. This occurs whether the C extension is installed or not.

$ pip install maxminddb
$ python
>>> import maxminddb
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named maxminddb

To resolve this issue locally, I modified the setup.py file in the repo (after cloning it from master) to execute the command:

packages=find_packages('.')

instead of

packages=find_packages('lib')

and then did

$ pip install .

and all was well with the world.

Argument unused during compilation (-mno-used-mad)

Crashing due to -Werror on OS X Mavericks.

...
creating build
creating build/temp.macosx-10.9-intel-2.7
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o -Wall -Werror -Wextra -std=c99 -pedantic
clang: error: argument unused during compilation: '-mno-fused-madd'
error: command 'cc' failed with exit status 1

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

pypi.python.org/pypi/maxminddb/0.2.0 is outdated (still has -Werror)

setup.py was updated to remove -Werror after 0.2.0, however version wasn't bumped.

pypi still has the older 0.2.0, uploaded 2013-10-15
(https://pypi.python.org/packages/source/m/maxminddb/maxminddb-0.2.0.tar.gz)

On OS X, clang: error: argument unused during compilation: '-mno-fused-madd'

Prevents pip installation.

PyPI sdist tarball no longer contains tests/data folder

Hello.

I'm not sure whether this change was intentional but it's no longer possible to run tests with just the content from PyPI:

$ find maxminddb-2.4.0/tests/ | wc -l
52
$ find maxminddb-2.5.0/tests/ | wc -l
3

Feature Request: Expose database mode in maxminddb.reader.Metadata

In production environments, it can be hard to determine which mode (MODE_FILE, MODE_MMAP, MODE_MMAP_EXT, etc.) the reader is using with MODE_AUTO without experimenting (setting the mode explicitly), seeing which modes work, and then assuming MODE_AUTO is selected the mode you want (MODE_MMAP_EXT). Experimenting in production can cause problems.

It would be very helpful if the maxminddb.reader.Metadata object exposed the mode (string value) and whether that mode was auto or explicit (boolean). That way, a simple glance at the metadata could confirm that you've installed and configured everything correctly.

Reading Multiple MMDB Files

Is it possible to read multiple mmdb files at the same time? Is this something that will supported in the future?

Clarification around parsing `int32` from files

I'm reviewing your code for a personal project and on Line 109 of the decoder, you create a parser for int32.

It seems to indicate that the type can be shorter(from the pad=True parameter), however in the actual function it reads exactly 4 bytes(type_size) irrespective of the size passed in. This renders the pad option redundant.

My reading of the function implies line 53 should read new_offset = offset + size. Related to this it might also be wise to assert that if pad: assert size <= type_size.

setuptools is (incorrectly?) listed as a production dependency in pyproject.toml

Hello!

The [dependencies] section of pyproject.toml lists setuptools>=68.2.2. I think this is unnecessary; it's not required in the production code, only to build it. As it is, this means that production builds must ship (or provide its own stub of) setuptools, even though it will never be used in a production deployment.

I think this should be as simple as moving the version constraint part of this dependency up to its entry in requires under [build-system]. I'd gladly open a PR for this because it seems like a trivial fix, but I do not know if there are reasons for how it currently is.

Thanks for the library!

Type hints for mypy

I've created .pyi type hints (for Python 3) for this library and I would like to submit them to python/typeshed.
What do you think about it?

Performance variance between python versions

I'm working on updating an application from Python 3.6 to Python 3.8, using the C extension and MODE_MMAP_EXT explicitly on both versions.

I've however noticed that the reader performs a lot better on Python 3.6 i.e. more consistent performance with 90% of requests under 40 microseconds.

On Python3.8 there is a lot more variance in the readers performance.

Here is my code.

import timeit, decimal

sys.path.insert(1, './lib')

import maxminddb

reader = maxminddb.open_database('./GeoLite2-City.mmdb', maxminddb.MODE_MMAP_EXT)

def timer(f):
  def wrapper(*args, **kwargs):
    start_time = timeit.default_timer()
    result = f(*args, **kwargs)
    elapsed = timeit.default_timer() - start_time
    print('"{name}" took {time} seconds to complete.'.format(name=f.__name__, time=decimal.Decimal(elapsed)))
    return result
  return wrapper

@timer
def get(ip):
  return reader.get(ip)

Null pointer dereference with corrupt database file in MODE_MMAP_EXT

Linux, python 3.7, maxminddb 1.5.2, libmaxminddb 1.4.2.

Given the corrupt database files np00.gz or np01.gz (uncompressed obviously)

And the example code:

>>> reader = maxminddb.open_database("./np00")
>>> reader.get_with_prefix_len("163.254.149.39")

this will produce a null pointer dereference, in the first case when Reader_get_with_prefix_len tries to Py_DECREF a null record. In the second case, from_map tries to PyDict_SetItem at maxminddb.c:538 with a null key, leading to a read from 0x8.

Iterating over the whole database is broken

You may also iterate over the whole database. The Reader class implements the iter method that returns an iterator. This iterator yields a tuple containing the network and the record.

This doesn't work. The Reader class doesn't have __iter__.

Simple test:

import maxminddb

with maxminddb.open_database('GeoLite2-City.mmdb') as reader:
    for network, record in reader:
        print(network, record)

python3 maxmind_processor.py Traceback (most recent call last): File "/Users/mlopez/bin/utils/maxmind_processor.py", line 4, in <module> for network, record in reader: TypeError: 'Reader' object is not iterable

Version:
pip freeze > maxminddb==2.4.0

Is the pip package correctly compiled?

Running in an environment without mmap?

Is there a way to run this is in an environment without mmap any more, given that decoder.py requires it?

Python `whl` releases

For future releases, can we also get pre-compiled .whl uploaded to PyPI?

Use ipv4 and ipv6 geographic libraries to generate mmdb files

Hi：

When I only use the region library of ipv4 to generate a mmdb file, I found that some ipv4 cannot query the region information from mmdb. But when I use the region library of ipv6 and ipv4 to generate a mmdb file, I found that these ipv4 can find the corresponding Geographical location, what causes this. Is this normal？

Thanks！

test failures in Python 3.7

The test suite passes for me with Python 3.6.5, but when run with Python 3.7.0b5 there are two failures.

======================================================================
FAIL: test_no_constructor_args (reader_test.TestAutoReader)
----------------------------------------------------------------------
TypeError: function missing required argument 'database' (pos 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/maxminddb-1.4.0/tests/reader_test.py", line 179, in test_no_constructor_args
    cls)
AssertionError: " 1 required positional argument|\(pos 1\) not found|takes at least 2 arguments" does not match "function missing required argument 'database' (pos 1)"

======================================================================
FAIL: test_no_constructor_args (reader_test.TestExtensionReader)
----------------------------------------------------------------------
TypeError: function missing required argument 'database' (pos 1)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/maxminddb-1.4.0/tests/reader_test.py", line 179, in test_no_constructor_args
    cls)
AssertionError: " 1 required positional argument|\(pos 1\) not found|takes at least 2 arguments" does not match "function missing required argument 'database' (pos 1)"

----------------------------------------------------------------------

Getting around file size limits?

Is there a way to get around file size limits (e.g., Google App Engine has a limit of 32 MB, which the GeoLite2 city database exceeds)?

The best solution I can think of is to allow a buffer to be passed in instead of the filename, so that the file can be loaded either as a .zip file and extracted in memory, or segmented into multiple files and reassembled in memory.

PyPI tarball - some test data missing

I'd like to update RPM package in Fedora but the tests fail because some test files in the tarball from PyPI are missing.

Should I download the test data from the git submodule or you can fix this? I think that it might be a good idea to include all or nothing so I guess that this is a bug.

======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestAutoReader)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
    reader = open_database(
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 44, in open_database
    return maxminddb.extension.Reader(database)
FileNotFoundError: No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'

======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestExtensionReader)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
    reader = open_database(
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 44, in open_database
    return maxminddb.extension.Reader(database)
FileNotFoundError: No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'

======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestExtensionReaderWithIPObjects)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
    reader = open_database(
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 44, in open_database
    return maxminddb.extension.Reader(database)
FileNotFoundError: No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'

======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestFDReader)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
    reader = open_database(
  File "/usr/lib/python3.8/site-packages/mock/mock.py", line 1092, in __call__
    return _mock_self._mock_call(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/mock/mock.py", line 1149, in _mock_call
    result = effect(*args, **kwargs)
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 45, in get_reader_from_file_descriptor
    with open(filepath, "rb") as mmdb_fh:
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'

======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestFileReader)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
    reader = open_database(
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 46, in open_database
    return maxminddb.reader.Reader(database, mode)
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/reader.py", line 56, in __init__
    self._buffer = FileBuffer(database)
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/file.py", line 16, in __init__
    self._handle = open(database, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'

======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestMMAPReader)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
    reader = open_database(
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 46, in open_database
    return maxminddb.reader.Reader(database, mode)
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/reader.py", line 51, in __init__
    with open(database, "rb") as db_file:
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'

======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestMMAPReaderWithIPObjects)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
    reader = open_database(
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 46, in open_database
    return maxminddb.reader.Reader(database, mode)
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/reader.py", line 51, in __init__
    with open(database, "rb") as db_file:
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'

======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestMemoryReader)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
    reader = open_database(
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 46, in open_database
    return maxminddb.reader.Reader(database, mode)
  File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/reader.py", line 60, in __init__
    with open(database, "rb") as db_file:
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'

Scan whole maxmind database

Hi,

I'm currently working on a python project involving geolocalization. Unfortunately, there is no way to scan a whole database to reconstruct it in a custom external format.

The reference perl implementation actually does, via iterate_search_tree (https://github.com/maxmind/MaxMind-DB-Reader-perl/blob/3fade689fa12708981fe70c5419a12a55561508a/lib/MaxMind/DB/Reader/PP.pm).

I tried to implement a python version of this function, but I'm not sure if my results are coherent with what I should get. Here is the code :

def iterate_search_tree(
        self,
        data_callback=lambda: None,
        ip_version=6,
        ):
    if ip_version == 6:
        max_bit_depth = 128
    elif ip_version == 4:
        max_bit_depth = 32
    else:
        raise ValueError('Invalid ip version')

    start_node = self._start_node(max_bit_depth)

    self._iterate_search_tree(
        data_callback=data_callback,
        start_node=start_node,
        bit_depth=1,
        max_bit_depth=max_bit_depth,
        decimal_ip=0,
    )

def _iterate_search_tree(
        self,
        data_callback=lambda: None,
        start_node=0,
        bit_depth=1,
        max_bit_depth=128,
        decimal_ip=0,
        ):

    for bit in [0, 1]:
        node = self._read_node(start_node + bit_depth, bit)

        if bit:
            decimal_ip = decimal_ip | 1 << (max_bit_depth - bit_depth)

        if node < self._metadata.node_count:
            if bit_depth > max_bit_depth:
            self._iterate_search_tree(data_callback=data_callback,
                                      start_node=start_node,
                                      bit_depth=bit_depth+1,
                                      max_bit_depth=max_bit_depth,
                                      decimal_ip=decimal_ip)
        elif node > self._metadata.node_count:
            mask = bit_depth + bit
            data = self._resolve_data_pointer(node)
            data_callback(decimal_ip, mask, data)
        else:
            pass

When I call iterate_search_tree in a trivial script counting entries via data_callback, I just get crazy results : a total of 2M+ ipv4/ipv6 detected, whereas a trivial script counting entries in the reference perl implementation counts 780K of them (both ipv4 and ipv6) for the same database file. I spent time trying to figure out what could go wrong but I don't manage to debug it.

I was wondering if you already thought about including this kind of feature in your package. If so, could you check what's wrong in my code and potentially include it in your reader ?

Thanks,

City name is none

For some IP addresses, the city name is None. What does that mean? Is that a bad sign?

License Change Request

The library is currently LGPL 2.1 licensed. Is there a reason for this? This makes it problematic in certain environments. The license is incompatible with both Apache and any GPLv3.

Would it be an option to relicense it under the MIT license? There is not a lot of code in it, and it's trivially to reimplement it if necessary and the value in the whole thing is in the database files anyways.

Allow accessing the network field in the ASN mmdb file

Follow up to maxmind/mod_maxminddb#81 and maxmind/libmaxminddb#207

ImportWarning from extension folder

Could we consider changing the name of the folder with the extension source to be different than then name of the extension? currently, if there is no extension built, we get an import warning from python from import maxminddb.extension

Feature request: need a Writer in Python

Hello!

It is of-course great that Write is provided at all, not so great that it is implemented in Perl. Would be great to have a MaxMind DB writer in more current language like Python.

Expose netmask

Any plans to expose the netmask field from MMDB_lookup_result_s in the library code? I'm migrating over from GeoIP legacy which provided this data and I see it's not exposed in the new API.

The GH releases have no test db

Even though the 1.5.4 release notes state a test database was added, they are not in the release assets:

$ unzip -l v1.5.4.zip |grep mmdb
$ tar tzf v1.5.4.tar.gz | grep mmdb
$ sha256sum v1.5.4.*
dadfb3c19dae3f12d9cfb8ebec41bc3d506c8825527b661f3f19d6c1413ec076  v1.5.4.tar.gz
1c4b32b6589a9ecbcb9960e77a770a91e08c9533305b78a01e32f84e3c495584  v1.5.4.zip

The 1.5.4 tarball from https://pypi.debian.net/maxminddb/ has them (same as https://pypi.org/project/maxminddb/#modal-close):

$ tar tzf maxminddb-1.5.4.tar.gz | grep mmdb
maxminddb-1.5.4/tests/data/MaxMind-DB-test-metadata-pointers.mmdb
maxminddb-1.5.4/tests/data/bad-data/libmaxminddb/libmaxminddb-offset-integer-overflow.mmdb
maxminddb-1.5.4/tests/data/bad-data/maxminddb-golang/cyclic-data-structure.mmdb
...
$ sha256sum maxminddb-1.5.4.tar.gz
f4d28823d9ca23323d113dc7af8db2087aa4f657fafc64ff8f7a8afda871425b  maxminddb-1.5.4.tar.gz

Linux aarch64/arm64 whl releases

Another whl release request, but I think the release action can be tweaked a little more to also support linux-aarch64.

Here is the example:

      - name: Set up QEMU
        if: runner.os == 'Linux'
        uses: docker/setup-qemu-action@v3
        with:
          platforms: all

      - name: Build wheels
        uses: pypa/[email protected]
        env:
          # configure cibuildwheel to build native archs ('auto'), and some
          # emulated ones
          CIBW_ARCHS_LINUX: auto aarch64 ppc64le s390x

TypeError: 'Reader' object is not iterable

Hello
From the docs it suggests:
You may also iterate over the whole database. The Reader class implements the iter method that returns an iterator. This iterator yields a tuple containing the network and the record.

After opening GeoIP2-ISP.mmdb and attempting to iterate over the reader I am getting
TypeError: 'Reader' object is not iterable

using maxminddb version 2.4.0

Problem with build on OS X 10.9.1

Running setup.py install for maxminddb
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
      warnings.warn(msg)
    building 'maxminddb' extension
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o -Wall -Werror -Wextra -std=c99 -pedantic
    clang: error: argument unused during compilation: '-mno-fused-madd'
    error: command 'cc' failed with exit status 1
    Complete output from command /Users/user/.env/example/bin/python -c "import setuptools;__file__='/Users/user/.pip/build/maxminddb/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/yp/xhcn8dkj4rzcql9tfdcwnsj40000gn/T/pip-ncTA6Z-record/install-record.txt --single-version-externally-managed --install-headers /Users/user/.env/example/include/site/python2.7:
    /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'

  warnings.warn(msg)

running install

running build

running build_ext

building 'maxminddb' extension

creating build

creating build/temp.macosx-10.9-intel-2.7

cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o -Wall -Werror -Wextra -std=c99 -pedantic

clang: error: argument unused during compilation: '-mno-fused-madd'

error: command 'cc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /Users/user/.env/example/bin/python -c "import setuptools;__file__='/Users/user/.pip/build/maxminddb/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/yp/xhcn8dkj4rzcql9tfdcwnsj40000gn/T/pip-ncTA6Z-record/install-record.txt --single-version-externally-managed --install-headers /Users/user/.env/example/include/site/python2.7 failed with error code 1 in /Users/user/.pip/build/maxminddb
Storing complete log in /Users/user/.pip/pip.log

Type annotations with `RecordDict` not satisfying mypy

I'm not able to figure out how to get mypy to be okay w/ the response from this library. The code I have looks something like this:

def ipinfo_db_lookup(ip_address: str) -> dict | None:
    with maxminddb.open_database(ipinfo_db_path) as reader:
        record: dict | None = reader.get(ip_address)
    return record

This is what mypy is saying:

Incompatible types in assignment (expression has type "Any | float | int | RecordList | RecordDict | None", variable has type "dict[Any, Any] | None")  [assignment]mypy(error)

I've even swapped dict for maxminddb.types.RecordDict and I get:

Incompatible types in assignment (expression has type "Any | float | int | RecordList | RecordDict | None", variable has type "RecordDict | None")  [assignment]mypy(error)

Any insight would be appreciated; I don't want to have to leave a type: ignore[assignment] statement in here.

Generated docs include API docs

This would make the docs on Read the Docs a little more complete. We might have to reformat some of them a bit as reStructuredText.

We already do this for geoip2.

__int128 is not supported on this target

Compilation of maxminddb.c fails on OS X Mavericks.

...
creating build
creating build/temp.macosx-10.9-intel-2.7
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o
clang: warning: argument unused during compilation: '-mno-fused-madd'
In file included from maxminddb.c:2:
/usr/local/include/maxminddb.h:64:18: error: __int128 is not supported on this target
typedef unsigned __int128 mmdb_uint128_t;
                 ^
1 error generated.
error: command 'cc' failed with exit status 1

"MODE_MMAP_EXT requires the maxminddb.extension module to be available" though libmaxminddb is installed

I've been trying to use MODE_MMAP_EXT mode but couldn't make it run neither in alpine docker container neither on macOS

bash-5.0# pip list | grep maxminddb
maxminddb  1.5.4
bash-5.0# pip list | grep geoip2
geoip2     3.0.0
bash-5.0# apk list | grep libmaxmind
libmaxminddb-doc-1.4.2-r0 x86_64 {libmaxminddb} (Apache-2.0)
libmaxminddb-dev-1.4.2-r0 x86_64 {libmaxminddb} (Apache-2.0)
libmaxminddb-1.4.2-r0 x86_64 {libmaxminddb} (Apache-2.0) [installed]

Test file:

import sys

from geoip2.database import Reader
from maxminddb import MODE_MMAP_EXT

r = Reader('GeoLite2-Country.mmdb', mode=MODE_MMAP_EXT)
print(r.country(sys.argv[1]).country.iso_code)

ash-5.0# python test.py 1.1.1.1
Traceback (most recent call last):
  File "test.py", line 6, in <module>
    r = Reader('GeoLite2-Country.mmdb.1', mode=MODE_MMAP_EXT)
  File "/usr/local/lib/python3.8/site-packages/geoip2/database.py", line 85, in __init__
    self._db_reader = maxminddb.open_database(fileish, mode)
  File "/usr/local/lib/python3.8/site-packages/maxminddb/__init__.py", line 41, in open_database
    raise ValueError(
ValueError: MODE_MMAP_EXT requires the maxminddb.extension module to be available

P.S. I've installed libmaxminddb before the pip packages as mentioned in documentation.