maxmind / maxmind-db-reader-python Goto Github PK
View Code? Open in Web Editor NEWPython MaxMind DB reader extension
Home Page: https://maxminddb.readthedocs.org/
License: Apache License 2.0
Python MaxMind DB reader extension
Home Page: https://maxminddb.readthedocs.org/
License: Apache License 2.0
Hello!
The [dependencies]
section of pyproject.toml
lists setuptools>=68.2.2
. I think this is unnecessary; it's not required in the production code, only to build it. As it is, this means that production builds must ship (or provide its own stub of) setuptools, even though it will never be used in a production deployment.
I think this should be as simple as moving the version constraint part of this dependency up to its entry in requires
under [build-system]
. I'd gladly open a PR for this because it seems like a trivial fix, but I do not know if there are reasons for how it currently is.
Thanks for the library!
L
Is there a way to run this is in an environment without mmap any more, given that decoder.py requires it?
I'm not able to figure out how to get mypy to be okay w/ the response from this library. The code I have looks something like this:
def ipinfo_db_lookup(ip_address: str) -> dict | None:
with maxminddb.open_database(ipinfo_db_path) as reader:
record: dict | None = reader.get(ip_address)
return record
This is what mypy is saying:
Incompatible types in assignment (expression has type "Any | float | int | RecordList | RecordDict | None", variable has type "dict[Any, Any] | None") [assignment]mypy(error)
I've even swapped dict
for maxminddb.types.RecordDict
and I get:
Incompatible types in assignment (expression has type "Any | float | int | RecordList | RecordDict | None", variable has type "RecordDict | None") [assignment]mypy(error)
Any insight would be appreciated; I don't want to have to leave a type: ignore[assignment]
statement in here.
I've been trying to use MODE_MMAP_EXT mode but couldn't make it run neither in alpine docker container neither on macOS
bash-5.0# pip list | grep maxminddb
maxminddb 1.5.4
bash-5.0# pip list | grep geoip2
geoip2 3.0.0
bash-5.0# apk list | grep libmaxmind
libmaxminddb-doc-1.4.2-r0 x86_64 {libmaxminddb} (Apache-2.0)
libmaxminddb-dev-1.4.2-r0 x86_64 {libmaxminddb} (Apache-2.0)
libmaxminddb-1.4.2-r0 x86_64 {libmaxminddb} (Apache-2.0) [installed]
Test file:
import sys
from geoip2.database import Reader
from maxminddb import MODE_MMAP_EXT
r = Reader('GeoLite2-Country.mmdb', mode=MODE_MMAP_EXT)
print(r.country(sys.argv[1]).country.iso_code)
ash-5.0# python test.py 1.1.1.1
Traceback (most recent call last):
File "test.py", line 6, in <module>
r = Reader('GeoLite2-Country.mmdb.1', mode=MODE_MMAP_EXT)
File "/usr/local/lib/python3.8/site-packages/geoip2/database.py", line 85, in __init__
self._db_reader = maxminddb.open_database(fileish, mode)
File "/usr/local/lib/python3.8/site-packages/maxminddb/__init__.py", line 41, in open_database
raise ValueError(
ValueError: MODE_MMAP_EXT requires the maxminddb.extension module to be available
P.S. I've installed libmaxminddb before the pip packages as mentioned in documentation.
Even though the 1.5.4 release notes state a test database was added, they are not in the release assets:
$ unzip -l v1.5.4.zip |grep mmdb
$ tar tzf v1.5.4.tar.gz | grep mmdb
$ sha256sum v1.5.4.*
dadfb3c19dae3f12d9cfb8ebec41bc3d506c8825527b661f3f19d6c1413ec076 v1.5.4.tar.gz
1c4b32b6589a9ecbcb9960e77a770a91e08c9533305b78a01e32f84e3c495584 v1.5.4.zip
The 1.5.4 tarball from https://pypi.debian.net/maxminddb/ has them (same as https://pypi.org/project/maxminddb/#modal-close):
$ tar tzf maxminddb-1.5.4.tar.gz | grep mmdb
maxminddb-1.5.4/tests/data/MaxMind-DB-test-metadata-pointers.mmdb
maxminddb-1.5.4/tests/data/bad-data/libmaxminddb/libmaxminddb-offset-integer-overflow.mmdb
maxminddb-1.5.4/tests/data/bad-data/maxminddb-golang/cyclic-data-structure.mmdb
...
$ sha256sum maxminddb-1.5.4.tar.gz
f4d28823d9ca23323d113dc7af8db2087aa4f657fafc64ff8f7a8afda871425b maxminddb-1.5.4.tar.gz
setup.py was updated to remove -Werror after 0.2.0, however version wasn't bumped.
pypi still has the older 0.2.0, uploaded 2013-10-15
(https://pypi.python.org/packages/source/m/maxminddb/maxminddb-0.2.0.tar.gz)
On OS X, clang: error: argument unused during compilation: '-mno-fused-madd'
Prevents pip installation.
Follow up to maxmind/mod_maxminddb#81 and maxmind/libmaxminddb#207
Any plans to expose the netmask
field from MMDB_lookup_result_s
in the library code? I'm migrating over from GeoIP legacy which provided this data and I see it's not exposed in the new API.
This would make the docs on Read the Docs a little more complete. We might have to reformat some of them a bit as reStructuredText.
We already do this for geoip2.
I'd like to update RPM package in Fedora but the tests fail because some test files in the tarball from PyPI are missing.
Should I download the test data from the git submodule or you can fix this? I think that it might be a good idea to include all or nothing so I guess that this is a bug.
======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestAutoReader)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
reader = open_database(
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 44, in open_database
return maxminddb.extension.Reader(database)
FileNotFoundError: No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'
======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestExtensionReader)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
reader = open_database(
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 44, in open_database
return maxminddb.extension.Reader(database)
FileNotFoundError: No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'
======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestExtensionReaderWithIPObjects)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
reader = open_database(
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 44, in open_database
return maxminddb.extension.Reader(database)
FileNotFoundError: No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'
======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestFDReader)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
reader = open_database(
File "/usr/lib/python3.8/site-packages/mock/mock.py", line 1092, in __call__
return _mock_self._mock_call(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/mock/mock.py", line 1149, in _mock_call
result = effect(*args, **kwargs)
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 45, in get_reader_from_file_descriptor
with open(filepath, "rb") as mmdb_fh:
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'
======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestFileReader)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
reader = open_database(
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 46, in open_database
return maxminddb.reader.Reader(database, mode)
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/reader.py", line 56, in __init__
self._buffer = FileBuffer(database)
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/file.py", line 16, in __init__
self._handle = open(database, "rb")
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'
======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestMMAPReader)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
reader = open_database(
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 46, in open_database
return maxminddb.reader.Reader(database, mode)
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/reader.py", line 51, in __init__
with open(database, "rb") as db_file:
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'
======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestMMAPReaderWithIPObjects)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
reader = open_database(
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 46, in open_database
return maxminddb.reader.Reader(database, mode)
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/reader.py", line 51, in __init__
with open(database, "rb") as db_file:
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'
======================================================================
ERROR: test_database_with_invalid_utf8_key (reader_test.TestMemoryReader)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/builddir/build/BUILD/maxminddb-1.5.3/tests/reader_test.py", line 291, in test_database_with_invalid_utf8_key
reader = open_database(
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/__init__.py", line 46, in open_database
return maxminddb.reader.Reader(database, mode)
File "/builddir/build/BUILD/maxminddb-1.5.3/maxminddb/reader.py", line 60, in __init__
with open(database, "rb") as db_file:
FileNotFoundError: [Errno 2] No such file or directory: 'tests/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb'
Crashing due to -Werror
on OS X Mavericks.
...
creating build
creating build/temp.macosx-10.9-intel-2.7
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o -Wall -Werror -Wextra -std=c99 -pedantic
clang: error: argument unused during compilation: '-mno-fused-madd'
error: command 'cc' failed with exit status 1
Dependabot couldn't authenticate with https://pypi.python.org/simple/.
You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.
Hello.
I'm not sure whether this change was intentional but it's no longer possible to run tests with just the content from PyPI:
$ find maxminddb-2.4.0/tests/ | wc -l
52
$ find maxminddb-2.5.0/tests/ | wc -l
3
The test suite passes for me with Python 3.6.5, but when run with Python 3.7.0b5 there are two failures.
======================================================================
FAIL: test_no_constructor_args (reader_test.TestAutoReader)
----------------------------------------------------------------------
TypeError: function missing required argument 'database' (pos 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/maxminddb-1.4.0/tests/reader_test.py", line 179, in test_no_constructor_args
cls)
AssertionError: " 1 required positional argument|\(pos 1\) not found|takes at least 2 arguments" does not match "function missing required argument 'database' (pos 1)"
======================================================================
FAIL: test_no_constructor_args (reader_test.TestExtensionReader)
----------------------------------------------------------------------
TypeError: function missing required argument 'database' (pos 1)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/maxminddb-1.4.0/tests/reader_test.py", line 179, in test_no_constructor_args
cls)
AssertionError: " 1 required positional argument|\(pos 1\) not found|takes at least 2 arguments" does not match "function missing required argument 'database' (pos 1)"
----------------------------------------------------------------------
I'm working on updating an application from Python 3.6 to Python 3.8, using the C extension and MODE_MMAP_EXT explicitly on both versions.
I've however noticed that the reader performs a lot better on Python 3.6 i.e. more consistent performance with 90% of requests under 40 microseconds.
On Python3.8 there is a lot more variance in the readers performance.
Here is my code.
import timeit, decimal
sys.path.insert(1, './lib')
import maxminddb
reader = maxminddb.open_database('./GeoLite2-City.mmdb', maxminddb.MODE_MMAP_EXT)
def timer(f):
def wrapper(*args, **kwargs):
start_time = timeit.default_timer()
result = f(*args, **kwargs)
elapsed = timeit.default_timer() - start_time
print('"{name}" took {time} seconds to complete.'.format(name=f.__name__, time=decimal.Decimal(elapsed)))
return result
return wrapper
@timer
def get(ip):
return reader.get(ip)
When I install maxminddb via PyPi, and then try and import maxminddb from the interpreter, I get an import error. This occurs whether the C extension is installed or not.
$ pip install maxminddb
$ python
>>> import maxminddb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named maxminddb
To resolve this issue locally, I modified the setup.py file in the repo (after cloning it from master) to execute the command:
packages=find_packages('.')
instead of
packages=find_packages('lib')
and then did
$ pip install .
and all was well with the world.
I'm reviewing your code for a personal project and on Line 109 of the decoder, you create a parser for int32
.
It seems to indicate that the type can be shorter(from the pad=True
parameter), however in the actual function it reads exactly 4 bytes(type_size
) irrespective of the size
passed in. This renders the pad
option redundant.
My reading of the function implies line 53 should read new_offset = offset + size
. Related to this it might also be wise to assert that if pad: assert size <= type_size
.
For future releases, can we also get pre-compiled .whl
uploaded to PyPI?
This API should support using the context manger for opening and closing the database.
Hi,
I'm currently working on a python project involving geolocalization. Unfortunately, there is no way to scan a whole database to reconstruct it in a custom external format.
The reference perl implementation actually does, via iterate_search_tree (https://github.com/maxmind/MaxMind-DB-Reader-perl/blob/3fade689fa12708981fe70c5419a12a55561508a/lib/MaxMind/DB/Reader/PP.pm).
I tried to implement a python version of this function, but I'm not sure if my results are coherent with what I should get. Here is the code :
def iterate_search_tree(
self,
data_callback=lambda: None,
ip_version=6,
):
if ip_version == 6:
max_bit_depth = 128
elif ip_version == 4:
max_bit_depth = 32
else:
raise ValueError('Invalid ip version')
start_node = self._start_node(max_bit_depth)
self._iterate_search_tree(
data_callback=data_callback,
start_node=start_node,
bit_depth=1,
max_bit_depth=max_bit_depth,
decimal_ip=0,
)
def _iterate_search_tree(
self,
data_callback=lambda: None,
start_node=0,
bit_depth=1,
max_bit_depth=128,
decimal_ip=0,
):
for bit in [0, 1]:
node = self._read_node(start_node + bit_depth, bit)
if bit:
decimal_ip = decimal_ip | 1 << (max_bit_depth - bit_depth)
if node < self._metadata.node_count:
if bit_depth > max_bit_depth:
self._iterate_search_tree(data_callback=data_callback,
start_node=start_node,
bit_depth=bit_depth+1,
max_bit_depth=max_bit_depth,
decimal_ip=decimal_ip)
elif node > self._metadata.node_count:
mask = bit_depth + bit
data = self._resolve_data_pointer(node)
data_callback(decimal_ip, mask, data)
else:
pass
When I call iterate_search_tree
in a trivial script counting entries via data_callback
, I just get crazy results : a total of 2M+ ipv4/ipv6 detected, whereas a trivial script counting entries in the reference perl implementation counts 780K of them (both ipv4 and ipv6) for the same database file. I spent time trying to figure out what could go wrong but I don't manage to debug it.
I was wondering if you already thought about including this kind of feature in your package. If so, could you check what's wrong in my code and potentially include it in your reader ?
Thanks,
reader = maxminddb.open_database(self.GeoLite2_City_file, mode="MODE_MMAP_EXT")
Providing a mode to the open_database function results in the following error:
ValueError: Unsupported open mode: MODE_MMAP_EXT
Compilation of maxminddb.c fails on OS X Mavericks.
...
creating build
creating build/temp.macosx-10.9-intel-2.7
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o
clang: warning: argument unused during compilation: '-mno-fused-madd'
In file included from maxminddb.c:2:
/usr/local/include/maxminddb.h:64:18: error: __int128 is not supported on this target
typedef unsigned __int128 mmdb_uint128_t;
^
1 error generated.
error: command 'cc' failed with exit status 1
Running setup.py install for maxminddb
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
building 'maxminddb' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o -Wall -Werror -Wextra -std=c99 -pedantic
clang: error: argument unused during compilation: '-mno-fused-madd'
error: command 'cc' failed with exit status 1
Complete output from command /Users/user/.env/example/bin/python -c "import setuptools;__file__='/Users/user/.pip/build/maxminddb/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/yp/xhcn8dkj4rzcql9tfdcwnsj40000gn/T/pip-ncTA6Z-record/install-record.txt --single-version-externally-managed --install-headers /Users/user/.env/example/include/site/python2.7:
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'bugtrack_url'
warnings.warn(msg)
running install
running build
running build_ext
building 'maxminddb' extension
creating build
creating build/temp.macosx-10.9-intel-2.7
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c maxminddb.c -o build/temp.macosx-10.9-intel-2.7/maxminddb.o -Wall -Werror -Wextra -std=c99 -pedantic
clang: error: argument unused during compilation: '-mno-fused-madd'
error: command 'cc' failed with exit status 1
----------------------------------------
Cleaning up...
Command /Users/user/.env/example/bin/python -c "import setuptools;__file__='/Users/user/.pip/build/maxminddb/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/yp/xhcn8dkj4rzcql9tfdcwnsj40000gn/T/pip-ncTA6Z-record/install-record.txt --single-version-externally-managed --install-headers /Users/user/.env/example/include/site/python2.7 failed with error code 1 in /Users/user/.pip/build/maxminddb
Storing complete log in /Users/user/.pip/pip.log
I've created .pyi
type hints (for Python 3) for this library and I would like to submit them to python/typeshed
.
What do you think about it?
Is there a way to get around file size limits (e.g., Google App Engine has a limit of 32 MB, which the GeoLite2 city database exceeds)?
The best solution I can think of is to allow a buffer to be passed in instead of the filename, so that the file can be loaded either as a .zip file and extracted in memory, or segmented into multiple files and reassembled in memory.
Could we consider changing the name of the folder with the extension source to be different than then name of the extension? currently, if there is no extension built, we get an import warning from python from import maxminddb.extension
The library is currently LGPL 2.1 licensed. Is there a reason for this? This makes it problematic in certain environments. The license is incompatible with both Apache and any GPLv3.
Would it be an option to relicense it under the MIT license? There is not a lot of code in it, and it's trivially to reimplement it if necessary and the value in the whole thing is in the database files anyways.
Another whl release request, but I think the release action can be tweaked a little more to also support linux-aarch64.
Here is the example:
- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v3
with:
platforms: all
- name: Build wheels
uses: pypa/[email protected]
env:
# configure cibuildwheel to build native archs ('auto'), and some
# emulated ones
CIBW_ARCHS_LINUX: auto aarch64 ppc64le s390x
Under PyPy performance could be hurt by using the CPython extension.
I know that it's disabled right now but we could replace it with a CFFI extension which might boost performance.
Do you guys have a benchmark suite that you use to test this?
In production environments, it can be hard to determine which mode (MODE_FILE
, MODE_MMAP
, MODE_MMAP_EXT
, etc.) the reader is using with MODE_AUTO
without experimenting (setting the mode explicitly), seeing which modes work, and then assuming MODE_AUTO
is selected the mode you want (MODE_MMAP_EXT
). Experimenting in production can cause problems.
It would be very helpful if the maxminddb.reader.Metadata
object exposed the mode (string value) and whether that mode was auto or explicit (boolean). That way, a simple glance at the metadata could confirm that you've installed and configured everything correctly.
Hi,
The current version, which is stated to be Python 3 only, actually gets into Python 2 installations because of a missing python_requires update:
MaxMind-DB-Reader-python/setup.py
Line 112 in 30d8bf0
Could you please fix this?
For some IP addresses, the city name is None. What does that mean? Is that a bad sign?
Hi:
When I only use the region library of ipv4 to generate a mmdb file, I found that some ipv4 cannot query the region information from mmdb. But when I use the region library of ipv6 and ipv4 to generate a mmdb file, I found that these ipv4 can find the corresponding Geographical location, what causes this. Is this normal?
Thanks!
Hello
From the docs it suggests:
You may also iterate over the whole database. The Reader class implements the iter method that returns an iterator. This iterator yields a tuple containing the network and the record.
After opening GeoIP2-ISP.mmdb and attempting to iterate over the reader I am getting
TypeError: 'Reader' object is not iterable
using maxminddb version 2.4.0
Linux, python 3.7, maxminddb 1.5.2, libmaxminddb 1.4.2.
Given the corrupt database files np00.gz or np01.gz (uncompressed obviously)
And the example code:
>>> reader = maxminddb.open_database("./np00")
>>> reader.get_with_prefix_len("163.254.149.39")
this will produce a null pointer dereference, in the first case when Reader_get_with_prefix_len
tries to Py_DECREF
a null record
. In the second case, from_map
tries to PyDict_SetItem
at maxminddb.c:538
with a null key
, leading to a read from 0x8
.
You may also iterate over the whole database. The Reader class implements the iter method that returns an iterator. This iterator yields a tuple containing the network and the record.
This doesn't work. The Reader class doesn't have __iter__
.
Simple test:
import maxminddb
with maxminddb.open_database('GeoLite2-City.mmdb') as reader:
for network, record in reader:
print(network, record)
python3 maxmind_processor.py Traceback (most recent call last): File "/Users/mlopez/bin/utils/maxmind_processor.py", line 4, in <module> for network, record in reader: TypeError: 'Reader' object is not iterable
Version:
pip freeze > maxminddb==2.4.0
Is the pip package correctly compiled?
Is it possible to read multiple mmdb files at the same time? Is this something that will supported in the future?
Hello!
It is of-course great that Write is provided at all, not so great that it is implemented in Perl. Would be great to have a MaxMind DB writer in more current language like Python.
With increasing popularity of M1/M2/M3 and ARM based cloud computing, it would be immensely helpful to also get aarch64/arm64 whl releases.
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.