Git Product home page Git Product logo

h3-py's Introduction

H3 Logo

h3-py: Uber's H3 Hexagonal Hierarchical Geospatial Indexing System in Python

PyPI version PyPI downloads conda version version

Tests codecov


ANNOUNCEMENT: v4.0.0 beta released!

Install the latest beta version with pip install h3 --pre or with pip install 'h3==4.0.0b2'.
Feedback is welcome via Issues, Discussions, and Slack.


Python bindings for the H3 core library.

Installation

From PyPI:

pip install h3

From conda:

conda config --add channels conda-forge
conda install h3-py

Usage

>>> import h3
>>> lat, lng = 37.769377, -122.388903
>>> resolution = 9
>>> h3.latlng_to_cell(lat, lng, resolution)
'89283082e73ffff'

APIs

We provide multiple APIs in h3-py.

Example gallery

Browse a collection of example notebooks, and if you have examples or visualizations of your own, please feel free to contribute!

We also have an introductory walkthrough of the API.

Versioning

h3-py wraps the H3 core library, which is written in C. The C and Python projects each employ semantic versioning, where versions take the form X.Y.Z.

The h3-py version string is guaranteed to match the C library string in both major and minor numbers (X.Y), but may differ on the patch (Z) number. This convention provides users with information on breaking changes and feature additions, while providing downstream bindings (like this one!) with the versioning freedom to fix bugs.

Use h3.versions() to see the version numbers for both h3-py and the C library. For example,

>>> import h3
>>> h3.versions()
{'c': '3.6.3', 'python': '3.6.1'}

h3-py's People

Contributors

ajfriend avatar alejandrohall avatar dfellis avatar henryiii avatar i3lackracer avatar isaacbrodsky avatar karentycoon avatar krosaen avatar kylebarron avatar nastasi-oq avatar nrabinowitz avatar sadikkuzu avatar tenghu avatar wwchen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

h3-py's Issues

pip install h3 fails for version 3.4.2 on mac os

I've tried upgrading to h3-py version 3.4.2 from 3.1.0 using "pip install h3". Installation is reported as successful, but the C compilation does not seem to execute. The installation is not creating an h3/out directory and the dylib files.
I have been able to revert back to version 3.1.0, which successfully installs with the C libraries correctly building.
I'm running on a mac os with Mojave 10.14.3 and anaconda 1.7.2. Some system .h files on my machine needed to be modified to successfully build to protect against the newer __has_include() function in later cc compilers. With revised .h files, the resulting libraries build and run without problems for h3 3.1.0.

I also tried hiding the pip cache directory with no improvement toward getting 3.4.2 to build the libraries.

$ which cmake
/anaconda3/bin/cmake
$ which make
/usr/bin/make
$ which cc
/anaconda3/bin/cc

get_h3_unidirectional_edge_boundary() returning more than 2 coordinates if hex resolution increased

Hi, I am finding that get_h3_unidirectional_edge_boundary() is returning a varying number of coordinates based on the resolution of the hexagon the edge was extracted from.

In the example below, if the hexagon is at resolution 12, the edge boundary (of a chosen edge) has 2 coordinates. However, if the hexagon resolution is set to 13, the edge boundary has 6 coordinates reflecting the boundary of the complete hexagon.

for res in range(12,14):
    hex = h3.geo_to_h3(52.52, 13.405, res)
    print('hex:', hex)
    print('resolution:', res)
    print('hex boundary:', h3.h3_to_geo_boundary(hex))
    print('edges:', h3.get_h3_unidirectional_edges_from_hexagon(hex))
    chosen_edge = h3.get_h3_unidirectional_edges_from_hexagon(hex)[1]
    print('chosen edge:', chosen_edge)
    print('edge boundary:', h3.get_h3_unidirectional_edge_boundary(chosen_edge))
    print()

with output:

hex: 8c1f1d48945d9ff
resolution: 12
hex boundary: [[52.520084680673634, 13.404984869589383], [52.519998091137644, 13.404939650857564], [52.51993761740116, 13.405047905096861], [52.51996373312568, 13.40520137814525], [52.52005032264476, 13.405246597234472], [52.52011079645624, 13.405138342917892]]
edges: ['11c1f1d48945d9ff', '12c1f1d48945d9ff', '13c1f1d48945d9ff', '14c1f1d48945d9ff', '15c1f1d48945d9ff', '16c1f1d48945d9ff']
chosen edge: 12c1f1d48945d9ff
edge boundary: [[52.519998091137644, 13.404939650857564], [52.51993761740116, 13.405047905096861]]

hex: 8d1f1d48945d9bf
resolution: 13
hex boundary: [[52.520031470105884, 13.404937105573747], [52.519998091137644, 13.404939650857564], [52.519984543788084, 13.404992505368952], [52.52000437540045, 13.405042814637705], [52.52003775437357, 13.405040269404356], [52.520051301729445, 13.404987414851757]]
edges: ['11d1f1d48945d9bf', '12d1f1d48945d9bf', '13d1f1d48945d9bf', '14d1f1d48945d9bf', '15d1f1d48945d9bf', '16d1f1d48945d9bf']
chosen edge: 12d1f1d48945d9bf
edge boundary: [[52.520031470105884, 13.404937105573747], [52.519998091137644, 13.404939650857564], [52.519984543788084, 13.404992505368952], [52.52000437540045, 13.405042814637705], [52.52003775437357, 13.405040269404356], [52.520051301729445, 13.404987414851757]]

Is this behavior intentional? Any insight would be most appreciated.

Thanks Z.

binder pangeo fail to build with h3

Hi all,
I am trying to use h3 within a pangeo binder for the following library.

I cannot make sense of the full stack trace.
Would anybody have a clue about what may be going wrong?

Waiting for build to start... Picked Git content provider. Cloning into '/tmp/repo2docker6v_rhp0r'... HEAD is now at b8c9ee6 Update Dockerfile with pangeo 2019.11.15 Using DockerBuildPack builder Step 1/1 : FROM pangeo/pangeo-notebook-onbuild:2019.11.15 # Executing 6 build triggers ---> Running in 7ef2bbd43665 Removing intermediate container 7ef2bbd43665 ---> Running in 64d68c26d27b Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies. Conda may not use the correct pip to install your packages, and they may end up in the wrong place. Please add an explicit pip dependency. I'm adding one for you, but still nagging you. Collecting package metadata (repodata.json): ...working... done Solving environment: ...working... done

Downloading and Extracting Packages
libtiff-4.1.0 | 609 KB | ########## | 100%
gcc_linux-64-7.3.0 | 22 KB | ########## | 100%
owslib-0.19.0 | 125 KB | ########## | 100%
botocore-1.13.19 | 3.3 MB | ########## | 100%
gxx_linux-64-7.3.0 | 21 KB | ########## | 100%
boto3-1.10.19 | 69 KB | ########## | 100%
tiledb-1.7.0 | 2.3 MB | ########## | 100%
binutils_linux-64-2. | 21 KB | ########## | 100%
nodejs-13.0.0 | 14.4 MB | ########## | 100%
zstd-1.4.4 | 989 KB | ########## | 100%
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

==> WARNING: A newer version of conda exists. <==
current version: 4.7.10
latest version: 4.7.12

Please update conda by running

$ conda update -n base conda

CondaEnvException: Pip failed

Ran pip subprocess with arguments:
['/srv/conda/envs/notebook/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/jovyan/.onbuild-child/binder/condaenv.z84tox34.requirements.txt']
Pip subprocess output:
Collecting h3
Downloading https://files.pythonhosted.org/packages/98/2a/704b2db80465ce3a412d6c94f1c1658cffbbb4e76c5c03bb133f588cc131/h3-3.4.3.tar.gz
Building wheels for collected packages: h3
Building wheel for h3 (setup.py): started
Building wheel for h3 (setup.py): finished with status 'error'
Running setup.py clean for h3
Failed to build h3
Installing collected packages: h3
Running setup.py install for h3: started
Running setup.py install for h3: finished with status 'error'

Pip subprocess error:
ERROR: Command errored out with exit status 1:
command: /srv/conda/envs/notebook/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5tso3r2p/h3/setup.py'"'"'; file='"'"'/tmp/pip-install-5tso3r2p/h3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-frzn1l5f --python-tag cp37
cwd: /tmp/pip-install-5tso3r2p/h3/
Complete output (47 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/h3
copying h3/h3.py -> build/lib.linux-x86_64-3.7/h3
copying h3/init.py -> build/lib.linux-x86_64-3.7/h3
running build_ext

  • VERSION=v3.4.2
  • IS_64BITS=True
  • '[' '' == v3.4.2 ']'
  • command -v cmake
  • echo 'cmake required but not found.'
    cmake required but not found.
  • exit 1
    Traceback (most recent call last):
    File "", line 1, in
    File "/tmp/pip-install-5tso3r2p/h3/setup.py", line 65, in
    distclass=BinaryDistribution)
    File "/srv/conda/envs/notebook/lib/python3.7/site-packages/setuptools/init.py", line 145, in setup
    return distutils.core.setup(**attrs)
    File "/srv/conda/envs/notebook/lib/python3.7/distutils/core.py", line 148, in setup
    dist.run_commands()
    File "/srv/conda/envs/notebook/lib/python3.7/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
    File "/srv/conda/envs/notebook/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
    File "/srv/conda/envs/notebook/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 192, in run
    self.run_command('build')
    File "/srv/conda/envs/notebook/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
    File "/srv/conda/envs/notebook/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
    File "/srv/conda/envs/notebook/lib/python3.7/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
    File "/srv/conda/envs/notebook/lib/python3.7/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
    File "/srv/conda/envs/notebook/lib/python3.7/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
    File "/tmp/pip-install-5tso3r2p/h3/setup.py", line 25, in run
    install_h3(h3_version)
    File "/tmp/pip-install-5tso3r2p/h3/setup.py", line 18, in install_h3
    subprocess.check_call('bash ./.install.sh {} {}'.format(h3_version, is_64bits), shell=True)
    File "/srv/conda/envs/notebook/lib/python3.7/subprocess.py", line 347, in check_call
    raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command 'bash ./.install.sh v3.4.2 True' returned non-zero exit status 1.

ERROR: Failed building wheel for h3
ERROR: Command errored out with exit status 1:
command: /srv/conda/envs/notebook/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5tso3r2p/h3/setup.py'"'"'; file='"'"'/tmp/pip-install-5tso3r2p/h3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-2f31e0_2/install-record.txt --single-version-externally-managed --compile
cwd: /tmp/pip-install-5tso3r2p/h3/
Complete output (49 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/h3
copying h3/h3.py -> build/lib.linux-x86_64-3.7/h3
copying h3/init.py -> build/lib.linux-x86_64-3.7/h3
running build_ext
+ VERSION=v3.4.2
+ IS_64BITS=True
+ '[' '' == v3.4.2 ']'
+ command -v cmake
+ echo 'cmake required but not found.'
cmake required but not found.
+ exit 1
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-5tso3r2p/h3/setup.py", line 65, in
distclass=BinaryDistribution)
File "/srv/conda/envs/notebook/lib/python3.7/site-packages/setuptools/init.py",line 145, in setup
return distutils.core.setup(**attrs)
File "/srv/conda/envs/notebook/lib/python3.7/distutils/core.py", line 148, in setup
dist.run_commands()
File "/srv/conda/envs/notebook/lib/python3.7/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/srv/conda/envs/notebook/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/srv/conda/envs/notebook/lib/python3.7/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/srv/conda/envs/notebook/lib/python3.7/distutils/command/install.py", line 545, in run
self.run_command('build')
File "/srv/conda/envs/notebook/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/srv/conda/envs/notebook/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/srv/conda/envs/notebook/lib/python3.7/distutils/command/build.py", line 135,in run
self.run_command(cmd_name)
File "/srv/conda/envs/notebook/lib/python3.7/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/srv/conda/envs/notebook/lib/python3.7/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-install-5tso3r2p/h3/setup.py", line 25, in run
install_h3(h3_version)
File "/tmp/pip-install-5tso3r2p/h3/setup.py", line 18, in install_h3
subprocess.check_call('bash ./.install.sh {} {}'.format(h3_version, is_64bits), shell=True)
File "/srv/conda/envs/notebook/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'bash ./.install.sh v3.4.2 True' returned non-zero exit status 1.
----------------------------------------
ERROR: Command errored out with exit status 1: /srv/conda/envs/notebook/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5tso3r2p/h3/setup.py'"'"'; file='"'"'/tmp/pip-install-5tso3r2p/h3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-2f31e0_2/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.

Traceback (most recent call last):
File "/usr/local/bin/r2d_overlay.py", line 141, in
main()
File "/usr/local/bin/r2d_overlay.py", line 136, in main
build()
File "/usr/local/bin/r2d_overlay.py", line 122, in build
['/bin/bash', '-c', command], preexec_fn=applicator._pre_exec
File "/srv/conda/envs/notebook/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/bin/bash', '-c', 'conda env update -p /srv/conda/envs/notebook -f /home/jovyan/.onbuild-child/binder/environment.yml']' returned non-zero exit status 1.
Removing intermediate container 64d68c26d27b
The command '/bin/sh -c /usr/local/bin/r2d_overlay.py build' returned a non-zero code: 1

Empty `polyfill()` output for large Polygons?

I was working on the cython branch, and created a simple test case for polyfill(). However, this example seems to fail in both the cython branch and the current master branch. Is the issue that the polygon is too large? This seems like an issue with the C library.

Maybe polyfill() fails when we hit a pentagon? Should we return an error code in this case?

Here's the code, the output of polyfill() is the empty set, which doesn't seem correct.

import h3

box = [[
    [-110.61, 42.68],
    [-109.02, 32.17],
    [ -94.26, 31.57],
    [ -89.38, 42.94],
    [-110.61, 42.68]
]]

json = {
    'type': 'Polygon',
    'coordinates': box,
}

h3.polyfill(json, 3)

Here's what the polygon looks like:

Screen Shot 2019-10-05 at 5 29 54 PM

Fix usage example

Looks like and updated method name has been used but without the example being updated.

hex_boundary = h3.h3_to_geo_boundary(h3_address) # new
#hex_boundary = h3.h3_to_geo_cell_bndry(h3_address) # old       

simple fix

geo_to_h3 and h3_to_geo yields wrong result

There is a problem still.
Using the example, I expect h3_to_geo(geo_to_h3(lat,lon), resolution) to return the original lat,lon input, or something quite nearby... If I use the example, this is true, but if I do it in Australia, its not true -- the [-180,180] bounds are respected but the answer is shifted by 180:

Using the Python library --

>>> h3_address = h3.geo_to_h3(37.3615593, -122.0553238, 15)
>>> hex_center_coordinates = h3.h3_to_geo(h3_address)
>>> hex_center_coordinates
[37.36156313952111, -122.05532505846305]
>>> h3_address = h3.geo_to_h3(130.90938479800002, -12.365644588,15)
>>> hex_center_coordinates = h3.h3_to_geo(h3_address)
>>> hex_center_coordinates
[-49.09061440581596, -12.36565153299568]

Originally posted by @gacafe in uber/h3#38 (comment)

Versioning question

Currently, the h3-python bindings version is tied to the underlying h3 version. This makes sense from a consistency point of view. But what if we want to fix a critical bug in the bindings themselves, or cut a new binding-only release against the same version of H3? What version would these new bindings have?

Waiting for a new version number of h3 core may not be feasible -- especially for critical bugs that may make the bindings unusable, for example.

Are there other mechanisms by which we can release new python bindings that make it clear what version of core they're wrapping, but have the flexibility to have their own versioning?

Issue with `subprocess.check_call` in current install script

Speaking with @dgladkov, he pointed out that "subprocess.check_call should be changed as it doesnt assert cmd status" and that "it should do exit(1) and spit out stdout from cmake if it fails".

Let's put up a PR for this?

I'm assuming this will help avoid the silent failures that can happen if a user tries to install the library but does not have cmake.

How to find point (lat, lng) inside a polygon.

I want to use this library to find whether the point is inside a polygon or not.

Please let me know if this is possible using h3 or I need to do something else.
Write now I am using the mongo library to do this but I am trying to find out a faster way of doing it.

python 2.7 support for h3

Currently setup.py is using Python3.5 api subprocess.run, which causes h3 installation to fail on python2 with:

    File "/tmp/vpip-build-h3-0.0.0-py2.py3-none-any.whl/h3/setup.py", line 15, in install_h3
      subprocess.run(
  AttributeError: 'module' object has no attribute 'run'
  Running setup.py bdist_wheel for h3: finished with status 'error'

Is it possible to make h3 python 2.7 compatible(at least until the official Python 2 deprecation in 2020)?

Optional `numpy` dependency

The cython development branch is written in a way as to make numpy an optional dependency. When numpy is available, the interface h3.api.array_int should work correctly, which provides users with no-copy numpy arrays as output, which should hopefully be faster and more compatible with the PyData ecosystem.

We should set up CI tests for both cases of numpy being available and not, and make sure h3 works as expected in each case.

h3 module includes module imports

>>> from h3 import h3
>>> dir(h3)
['GeoBoundary', 'GeoCoord', 'GeoCoordArrayTen', 'GeoJsonLite', 'Geofence', 'LinkedGeoCoord', 'LinkedGeoLoop', 'LinkedGeoPolygon', 'POINTER', 'Structure', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_coord_array_to_geo_coord', '_dirname', '_geo_json_to_geo_json_lite', '_polygon_array_to_geofence', 'absolute_import', 'byref', 'c_double', 'c_int', 'c_long', 'c_void_p', 'cast', 'cdll', 'compact', 'degs_to_rads', 'edge_length', 'geo_to_h3', 'get_destination_h3_index_from_unidirectional_edge', 'get_h3_indexes_from_unidirectional_edge', 'get_h3_unidirectional_edge', 'get_h3_unidirectional_edge_boundary', 'get_h3_unidirectional_edges_from_hexagon', 'get_origin_h3_index_from_unidirectional_edge', 'h3_distance', 'h3_get_base_cell', 'h3_get_resolution', 'h3_indexes_are_neighbors', 'h3_is_pentagon', 'h3_is_res_class_III', 'h3_is_res_class_iii', 'h3_is_valid', 'h3_set_to_multi_polygon', 'h3_to_children', 'h3_to_geo', 'h3_to_geo_boundary', 'h3_to_parent', 'h3_to_string', 'h3_unidirectional_edge_is_valid', 'hex_area', 'hex_range', 'hex_range_distances', 'hex_ranges', 'hex_ring', 'hexagon_c_array_to_set', 'k_ring', 'k_ring_distances', 'libh3', 'libh3_path', 'math', 'mercator_lat', 'mercator_lng', 'num_hexagons', 'os', 'platform', 'polyfill', 'rads_to_degs', 'string_to_h3', 'uncompact']
>>> 

The imports from the h3.py module shouldn't be included in the package. It's missing __all__
os, platform, math, c_long, c_int, ..., etc

Incorrect installation when using pipenv

When user attempts to install h3 using pipenv, no /out directory is generated.

This does not work:
pipenv install h3

This does work:

pipenv run pip install h3

Any idea why the behaviour is different? This is a problem in production environments, whose deployment is performed with pipenv.

The geojson coordinate ordering for h3 inputs doesn't match the spec

Not really an issue but a bit of an inconvenience: the geojson spec indicates that coordinates should be listed as lng/lat, but python-H3 requires a geojson-like format with the opposite order (lat/lng). Other python gis tools, like Shapely, use lng/lat, as does postgis.

The geojson object used in the following example would not be valid by this spec:

geoJson = {'type': 'Polygon',
 'coordinates': [[[37.813318999983238, -122.4089866999972145],
 [ 37.7866302000007224, -122.3805436999997056 ],
 [37.7198061999978478, -122.3544736999993603],
 [ 37.7076131999975672, -122.5123436999983966 ],
 [37.7835871999971715, -122.5247187000021967],
 [37.8151571999998453, -122.4798767000009008]]] }

Install with google appengine

Do you have recommendations for installing cmake/cc/make for app engine? Wanted to run h3 on a flask web app and have been encountering issues related to the environment setup.

Improve dynamic module loading

As mentioned here, it would be nice to find a less-hacky solution for dynamically creating the different APIs.

# this module tries to DRY-up API code which is repeated across
# modules. Not sure if this function closure is the best solution.
# There doesn't seem to be any obvious best-practice for
# programmatically/dynamically creating modules
# another approach: we could also just use `exec()`
def _api_functions(
_in_scalar,
_out_scalar,
_in_collection,
_out_collection,
):

Perhaps using:

Couple of other, potentially helpful references:

API with integers as parameter

I am told (uber/h3#204) that the Core C lib does use 64-bit ints as the data type for H3 indexes.
Wouldn't it be more appropriate to use 64bit int as the standard parameter as well as return type when identifying hexagons in Python as well instead of hexadecimal strings?
This would have made much more sense for faster comparison and database indexing. I know we can convert it but that adds extra computations every time.
Or at least overload each function to take/return both int64 and hexadecimal strings as valid types (wherever possible).
It was suggested that either @TengHu or @karentycoon might be better resources to help here here.

Error when install h3 in Jenkins

Does anyone know this kind of error in Jenkins?
Some answer said install cmake but Jenkins environment cannot get sudo priority.

Running setup.py install for h3: started
Running setup.py install for h3: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /mnt/jenkins/workspace/test-sim-learn/env/bin/python2.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-sx5ESs/h3/setup.py'"'"'; file='"'"'/tmp/pip-install-sx5ESs/h3/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-OyoolU/install-record.txt --single-version-externally-managed --compile --install-headers /mnt/jenkins/workspace/test-sim-learn/env/include/site/python2.7/h3
cwd: /tmp/pip-install-sx5ESs/h3/
Complete output (120 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/h3
copying h3/init.py -> build/lib.linux-x86_64-2.7/h3
copying h3/h3.py -> build/lib.linux-x86_64-2.7/h3
running build_ext
+ VERSION=v3.4.2
+ IS_64BITS=True
+ '[' '' == v3.4.2 ']'
+ command -v cmake
+ mkdir -p h3/out
+ rm -rf h3c
+ git clone https://github.com/uber/h3.git h3c
Cloning into 'h3c'...
+ pushd h3c
/tmp/pip-install-sx5ESs/h3/h3c /tmp/pip-install-sx5ESs/h3
+ git pull origin master --tags
From https://github.com/uber/h3
* branch master -> FETCH_HEAD
Already up to date.
+ git checkout v3.4.2
Note: checking out 'v3.4.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

Bug in `k_ring_distances()`

In the vicinity of the pentagon 870800000ffffff:

>>> h3.k_ring_distances('870800003ffffff', 2)
[{'0', '870800003ffffff'},
 {'870800000ffffff',
  ...},
 {'870800004ffffff',
  ...}]

Note the extra null hex in the 0th set above.

Doesn't show up in h3.k_ring('870800003ffffff', 2) or when k < 2.

Move Cython code to `_cy` folder

I'd like to hide the Cython code from the user by placing all the *.pyx files inside a _cy folder. I couldn't quite get it to build properly when I tried, however. @ankitmehta21, could you help with this?

I'm thinking that it'd be nice to have src/h3 be laid out something like:

.
├── CMakeLists.txt
├── __init__.py
├── _version.py
├── api
│   └── *.py
├── _cy
│   └── *.pyx
└── core.py

Does that seem reasonable?

Problem importing h3

When importing h3 like in the documentation by doing:
from h3 import h3
I get the following error:

OSError: dlopen(/Users/johannesh/anaconda3/envs/on_demand_mobility_nyc_taxi_rides/lib/python3.7/site-packages/h3/out/libh3.1.dylib, 6): image not found

This also happens for a normal pip install --user installation. I looked in the location, and the /h3/out folder does indeed not exist. I'm not sure what this means, or what exactly is going wrong. Seems to be something during installation?

macOS 10.13.6
python 3.7

Thanks

conda release

Hi,

I want to create a conda release for h3-py. But currently, struggle with the configuration of cc.

In case you can help out, please look at:
https://stackoverflow.com/questions/56341092

The recipe is found here: https://github.com/geoHeil/staged-recipes/blob/h3-py/recipes/h3/meta.yaml

When trying to execute a local build using:

/.circleci/run_docker_build.sh
it fails with:

Unsatisfiable dependencies for platform linux-64: {'cc'}
Where the important part of the mata.yml looks like:

```

requirements:
build:
# If your project compiles code (such as a C extension) then add the required compilers as separate entries here.
# Compilers are named 'c', 'cxx' and 'fortran'.
- {{ compiler('c') }}
- cmake
- make
- cc
host:
- python
- pip
run:
- python


How can I get it to work to submit my first conda-forge package?

And when leaving `- cc` out:

+ command -v cc
  + echo 'cc required but not found.'
it fails as `cc` is not found.

Clarify that installation expects git and cmake (version 3)

Hi, it would be nice to clarify in the README that git and cmake (version 3) are needed for installation. I came across installation errors (triggered by the H3 install bash script) when pip-installing h3 in a Linux-based production environment:

  • git not available (though we might reasonably assume git is available in a local dev environment)
  • cmake installed by sudo yum install cmake was version 2, but apparently H3 needs version 3?

My issues were resolved by sudo yum install -y cmake3 git && sudo ln -s /usr/bin/cmake3 /usr/bin/cmake. I realize that whether one encounters these issues, as well as how they can be resolved, depends on the use case, but the need for git and cmake (version 3) seems worth calling out in the documentation.

Thanks for developing H3 and the python wrapper!

h3 python on windows, C library builds, h3-py may not be supported?

  • The H3 C library using VS2017 created all .exes and appears to function.
  • However h3-py appears to be MacOS and LINUX only based on the package_data from setup.py below?
  • Is there support for h3-py on windows or is it likely in the future?

Thank you

    package_data={
        'h-py':
        ['out/*.dylib' if platform.system() == 'Darwin' else 'out/*.so.*']
    },

Cython next steps

With #51 landed, I'll try to collect some thoughts for next steps.

  • Clean up build
    • Can we use a src directory structure to make sure that tests run only from the installed package? (Like here). I've tripped over this a few times when coming back to development after a while.
    • Install package as h3 instead of h3py. @kwmsmith, can you help here? I can give it a shot, but I feel like you have more expertise.
    • Should we use git subtree instead of submodule to package the H3 C library? Should that directory be under src? What's the cleanest approach?
    • @kwmsmith: What else do we need to do to ensure clean and reliable builds?
  • Tests
    • Integrate old test suite
    • Use pytest
    • Test with multiple versions of python
    • Get CI working
    • Remove the current scripts to ensure the correct Python/C version numbers, and just use a normal PyTest test
  • Finish implementation of polyfill and GeoPolygon
  • Testing with hypothesis
  • Memory safety testing? what's the best way to do this? valgrind?

mismatch between `hex_ring()` docstring and implentation

The docstring claims that hex_ring() falls back to a slower algorithm when a pentagon is encountered. However, the current implementation raises an exception instead:

h3-py/h3/h3.py

Lines 490 to 509 in 392fe16

def hex_ring(h3_address, ring_size):
"""
Get a hexagon ring for a given hexagon.
Returns individual rings, unlike `k_ring`.
If a pentagon is reachable, falls back to a
MUCH slower form based on `k_ring`.
"""
# This technically should be defined in the C code,
# but this is much faster
array_len = 6 * ring_size
HexRingArray = H3Index * array_len
hex_rings = HexRingArray()
success = libh3.hexRing(string_to_h3(h3_address), ring_size, hex_rings)
if success != 0:
raise Exception(
'Failed to get hexagon ring for pentagon {}'.format(h3_address))
return hexagon_c_array_to_set(hex_rings)

Which of these do we actually want?

How to use pip install h3 on Windows

Hey,
I´ve tried to install the h3 package by using pip install h3 on my windows machine. I have cmake, make and gcc installed. However cmake defaults to the Visual Studio compiler and therefore doesn´t create any make files. I´ve tried my best to fix this issue but I haven´t found a way to solve it yet.
Do you know any way how to use the pip install on a windows machine?

Many Thanks

Better output when .install.sh fails

When installing the library or building the library for development, if building the native library fails, there should be some feedback that the build failed and what step failed. For example to make it easier to debug #5

Installation on Windows 10 64 bit

I have tried to install h3 on windows 10 64 bit with python 3.6.
I ran the command pip install h3.
After install, when i run the program, i get this error:

    from h3 import h3
  File "C:\Users\Shahroze Tariq\AppData\Local\Programs\Python\Python36\lib\site-packages\h3\h3.py", line 39, in <module>
    libh3 = cdll.LoadLibrary('{}/{}'.format(_dirname, libh3_path))
  File "C:\Users\Shahroze Tariq\AppData\Local\Programs\Python\Python36\lib\ctypes\__init__.py", line 426, in LoadLibrary
    return self._dlltype(name)
  File "C:\Users\Shahroze Tariq\AppData\Local\Programs\Python\Python36\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

Ideas for cython bindings

xref: uber/h3#220

So I spent some time putting together kwmsmith/h3-cython before realizing that y'all have a cython branch. @isaacbrodsky suggested that we merge. It remains to be seen if that merge would improve things much, since it looks like your cython branch is further along. I particularly like the following:

  • letting Cython/Python manage the lifecycles of your arrays automatically -- very clean; this is definitely the right way to do it.
  • scikit-build + better cmake integration than the janky thing I put together.

Some possible improvements would be the following, and were directions I wanted to take the cython bindings; all up for discussion:

  • Decoupling the sets-of-hex-string representation from the arrays-of-hex-int64, and allowing consumers to decide whether they want to use the int64 or string representations -- this has come up in my own uses of h3, and was one of the original reasons I started on the cython bindings.
  • Leveraging cython's array.array support (see here as well) for no-copy integration between C-level arrays-of-hex-int64s and array.array objects, which has many advantages and nice integration with Cython / Python.
  • Building on previous point, we then have seamless, no-copy integration with NumPy arrays for free via the PEP 3118 buffer protocol, which array.array supports.
  • Once we can connect h3 indices to numpy efficiently, we then can easily work within Pandas and the general PyData stack as well as tighter (py)spark integration via pandas UDFs again, for free.

To be clear: none of the above rules out the current sets-of-hex-string representation, it only decouples that representation from the array-of-int64 representation, so end users have the choice / option to suit their usecase. There are some significant performance / memory improvements that would come with the array-of-int64 representation, as well as the seamless Numpy / Pandas / spark integration mentioned above.

Other things:

  • I've started doing some property-based testing of h3 via hypothesis, which I've found to be really good at finding all those inconsistent edge cases and subtle bugs. There's also some hypothesis stuff specifically meant for testing geojson.

Upgrade to newer version and add missing bindings

Currently h3 is at version 3.6.0, while h3-py is slightly falling behind with version 3.4.3.

Also, compared to API reference, a lot of functions are missing, e.g.: maxFaceCount, maxKringSize, maxH3ToChildrenSize, maxUncompactSize, maxPolyfillSize, h3SetToLinkedGeo, destroyLinkedPolygon

I'm willing to contribute with this if you're accepting pull requests?

Unexpected behavior with the `memview_int` interface

import numpy as np
import h3.api.memview_int as h3

h = h3.geo_to_h3(0,0,9)
a = h3.k_ring(h, 5)

# does not work, and probably shouldn't
b = set(a)
h3.compact(b)
## TypeError: a bytes-like object is required, not 'set'

# does not work, and probably shouldn't
b = list(a)
h3.compact(b)
## TypeError: a bytes-like object is required, not 'list'

# should probably work, but doesn't
b = np.asarray(list(a))
h3.compact(b)
# ValueError: Buffer dtype mismatch, expected 'const H3int' but got 'long'

The first two are probably fine, as set and list probably can't implement the buffer protocol.

The third, however, is a bit weird, and probably has to do with handling integer conversion correctly.

invalid hexagon showing up as valid?

I think h = '8a28308280fffff' is an invalid hexagon address. However, h3_is_valid(h) returns True. This is coming up because my Cython implementation claims it is not valid, and I get segfaults when running algorithms like k_ring on this hexagon.

Also, converting to lat/lng and then back to H3 gets a different hexagon, which makes me think the hexagon is actually invalid.

h3.geo_to_h3(*h3.h3_to_geo(h), res=10) != h

Ensure `polyfill` functions can't segfault

It is fairly easy to feed mal-formed data to the current Cython polyfill functions and generate a segfault. We need to add tests to ensure valid inputs to guarantee that segfaults can't happen.

"No matching distribution found for h3"

I know this repo literally just got created, but is this currently being uploaded to pypi?

$ pip install h3
Collecting h3
  Could not find a version that satisfies the requirement h3 (from versions: )
No matching distribution found for h3

windows 10 64bit cmake error

I am trying to install h3 on Windows 10 64bit. I have installed python 3.7.4 (via python.org) and all the other requirements i.e. Git for Windows, CMake, and Visual Studio 2019 with C/C++ component. However when trying to install via 'pip install h3' I get the following:

C:\Python37>pip install h3
Collecting h3
  Using cached https://files.pythonhosted.org/packages/98/2a/704b2db80465ce3a412d6c94f1c1658cffbbb4e76c5c03bb133f588cc131/h3-3.4.3.tar.gz
Building wheels for collected packages: h3
  Building wheel for h3 (setup.py) ... error
  ERROR: Complete output from command 'c:\python37\python.exe' -u -c 'import setuptools, tokenize;__file__='"'"'C:\\Users\\jwalker\\AppData\\Local\\Temp\\pip-install-xav63t6b\\h3\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\jwalker\AppData\Local\Temp\pip-wheel-0rikge7d' --python-tag cp37:
  ERROR: running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.7
  creating build\lib.win-amd64-3.7\h3
  copying h3\h3.py -> build\lib.win-amd64-3.7\h3
  copying h3\__init__.py -> build\lib.win-amd64-3.7\h3
  running build_ext
  + VERSION=v3.4.2
  + IS_64BITS=True
  + '[' '' == v3.4.2 ']'
  + command -v cmake
  + echo 'cmake required but not found.'
  cmake required but not found.
  + exit 1
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:\Users\jwalker\AppData\Local\Temp\pip-install-xav63t6b\h3\setup.py", line 65, in <module>
      distclass=BinaryDistribution)
    File "c:\python37\lib\site-packages\setuptools\__init__.py", line 145, in setup
      return distutils.core.setup(**attrs)
    File "c:\python37\lib\distutils\core.py", line 148, in setup
      dist.run_commands()
    File "c:\python37\lib\distutils\dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "c:\python37\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "c:\python37\lib\site-packages\wheel\bdist_wheel.py", line 192, in run
      self.run_command('build')
    File "c:\python37\lib\distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "c:\python37\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "c:\python37\lib\distutils\command\build.py", line 135, in run
      self.run_command(cmd_name)
    File "c:\python37\lib\distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "c:\python37\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "C:\Users\jwalker\AppData\Local\Temp\pip-install-xav63t6b\h3\setup.py", line 25, in run
      install_h3(h3_version)
    File "C:\Users\jwalker\AppData\Local\Temp\pip-install-xav63t6b\h3\setup.py", line 18, in install_h3
      subprocess.check_call('bash ./.install.sh {} {}'.format(h3_version, is_64bits), shell=True)
    File "c:\python37\lib\subprocess.py", line 347, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command 'bash ./.install.sh v3.4.2 True' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for h3
  Running setup.py clean for h3
Failed to build h3
Installing collected packages: h3
  Running setup.py install for h3 ... error
    ERROR: Complete output from command 'c:\python37\python.exe' -u -c 'import setuptools, tokenize;__file__='"'"'C:\\Users\\jwalker\\AppData\\Local\\Temp\\pip-install-xav63t6b\\h3\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\jwalker\AppData\Local\Temp\pip-record-25qy8tes\install-record.txt' --single-version-externally-managed --compile:
    ERROR: running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.7
    creating build\lib.win-amd64-3.7\h3
    copying h3\h3.py -> build\lib.win-amd64-3.7\h3
    copying h3\__init__.py -> build\lib.win-amd64-3.7\h3
    running build_ext
    + VERSION=v3.4.2
    + IS_64BITS=True
    + '[' '' == v3.4.2 ']'
    + command -v cmake
    + echo 'cmake required but not found.'
    cmake required but not found.
    + exit 1
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\jwalker\AppData\Local\Temp\pip-install-xav63t6b\h3\setup.py", line 65, in <module>
        distclass=BinaryDistribution)
      File "c:\python37\lib\site-packages\setuptools\__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "c:\python37\lib\distutils\core.py", line 148, in setup
        dist.run_commands()
      File "c:\python37\lib\distutils\dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "c:\python37\lib\distutils\dist.py", line 985, in run_command
        cmd_obj.run()
      File "c:\python37\lib\site-packages\setuptools\command\install.py", line 61, in run
        return orig.install.run(self)
      File "c:\python37\lib\distutils\command\install.py", line 545, in run
        self.run_command('build')
      File "c:\python37\lib\distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "c:\python37\lib\distutils\dist.py", line 985, in run_command
        cmd_obj.run()
      File "c:\python37\lib\distutils\command\build.py", line 135, in run
        self.run_command(cmd_name)
      File "c:\python37\lib\distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "c:\python37\lib\distutils\dist.py", line 985, in run_command
        cmd_obj.run()
      File "C:\Users\jwalker\AppData\Local\Temp\pip-install-xav63t6b\h3\setup.py", line 25, in run
        install_h3(h3_version)
      File "C:\Users\jwalker\AppData\Local\Temp\pip-install-xav63t6b\h3\setup.py", line 18, in install_h3
        subprocess.check_call('bash ./.install.sh {} {}'.format(h3_version, is_64bits), shell=True)
      File "c:\python37\lib\subprocess.py", line 347, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command 'bash ./.install.sh v3.4.2 True' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command "'c:\python37\python.exe' -u -c 'import setuptools, tokenize;__file__='"'"'C:\\Users\\jwalker\\AppData\\Local\\Temp\\pip-install-xav63t6b\\h3\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\jwalker\AppData\Local\Temp\pip-record-25qy8tes\install-record.txt' --single-version-externally-managed --compile" failed with error code 1 in C:\Users\jwalker\AppData\Local\Temp\pip-install-xav63t6b\h3\

C:\Python37>cmake
Usage

  cmake [options] <path-to-source>
  cmake [options] <path-to-existing-build>
  cmake [options] -S <path-to-source> -B <path-to-build>

Specify a source directory to (re-)generate a build system for it in the
current working directory.  Specify an existing build directory to
re-generate its build system.

Run 'cmake --help' for more information.


C:\Python37>

I'm not sure why it's complaining with a cmake required but not found as it is clearly in my path as illustrated above. Would anyone know what the issue is?

I also tried installing via pip within a separate miniconda install and get the same error.

Any help appreciated.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.