Hello, dear authors!
Is it possible to get Nominatim by id?
Like here https://nominatim.org/release-docs/latest/api/Lookup/
https://nominatim.openstreetmap.org/lookup?osm_ids=R11920243&format=json
Сan you write an example?
Thanks!

 nominatim = Nominatim(cacheDir=".nominatim")

used to work prior to version 0.3.3 - now it's giving an error message althought the description still describes this parameter.
76c5f30 seems to be the issue. You might want to add Continuuous Integration tests to avoid such issues since the problem showed up in our own CI tests and debugging the situation lead to this ticket.

I'm using the tree-counter example except that I'm using a bounding box instead of the areaID.
for 'Bartberg, Austria' instead of Vienna. The areaID is obviously None.

bbox = [16.0678, 48.1509, 16.1535, 48.2112]  # Bartberg, Austria
query = overpassQueryBuilder(
      bbox=bbox,
      elementType="node",
      selector='"natural"="tree"',
      out="count"
)

result = overpass.query(query)
count = result.countElements()

Do I miss something here, because the result is empty?

I checked the wget dump for this particular link (``https://overpass-api.de/api/map?bbox=16.0678,48.1509,16.1535,48.2112```). There are trees listed in the dump.
The goal is to get the street names in an area defined by the bounding box.

I was trying to find all runways within a bounding box, and noticed that I get every runway (mapped as a way with aeroway=runway) twice. Here's a test code:

from OSMPythonTools import overpass
query = overpass.overpassQueryBuilder(bbox=[-6.05, 144.95, -6, 145], elementType="way",
                                                                 selector='"aeroway"="runway"', out="body", includeCenter=True)
result = overpass.Overpass().query(query).ways()
for e in result:
    print(e, e.tags())

which prints:

[overpass] downloading data: [timeout:25][out:json];(way["aeroway"="runway"](-6.05,144.95,-6,145);); out center; out body;
<OSMPythonTools.element.Element object at 0x7fb7a11704c0> {'aeroway': 'runway', 'ele': '1516', 'length': '1015', 'name': '03/21', 'source': 'Bing', 'surface': 'paved'}
<OSMPythonTools.element.Element object at 0x7fb7a11700d0> {'aeroway': 'runway', 'ele': '1516', 'length': '1015', 'name': '03/21', 'source': 'Bing', 'surface': 'paved'}

You can see that I got two different Elements holding the exact same data. Here's the test area in OSM
And Overpass-turbo shows that there is in fact only one way tagged ´aeroway=runway´ in that area.

Currently, this library declares a dependency on package DateTime from the Python package index.

However, it doesn't even seem to use it: It uses the datetime module from the Python standard library, but the one from PyPI, which would have to be imported using the name DateTime not datetime. (Note that while PyPI package names are not case sensitive, Python module names — like all identifiers within Python — are case sensitive.)

Also, PyPI's DateTime is only useful for interacting with Zope, which this library AFAIK doesn't do. See https://pypi.org/project/DateTime/:

This package provides a DateTime data type, as known from Zope. Unless you need to communicate with Zope APIs, you're probably better off using Python's built-in datetime module.

I am currently using the osm python tools in windows and want to make use of the caching functionality.

First Question. Does the caching also works for reverse nominatim requests?

Where is the cache location in windows? My %USERPROFILE% contains a .cache folder but this does not has any osm data.

api.query('changeset/118474871')

works, but

api.query('changeset/118474871/download')

doesn't works (see https://wiki.openstreetmap.org/wiki/API_v0.6#Download:_GET_.2Fapi.2F0.6.2Fchangeset.2F.23id.2Fdownload)

Expectation: got a list of modified objects

Hi,
I'm getting this error __init__() got an unexpected keyword argument 'cacheDir' when creating an instance of Overpass.
I just upgraded OSMPythonTools=0.3.2 to OSMPythonTools=0.3.3 (before I didn't have this problem).

So, It seems that the newest version has changed the constructor and instead of cacheDir it accepts a cache strategy.
Is this correct?

At the same time here it is mentioned: "Please observe that the constructor of the class Overpass again accepts the parameters endpoint, cacheDir, and waitBetweenQueries. To query historical data, we can easily add a date:"

Could you please clarify this? and if cacheDir does not exist anymore, could you please provide me with an example or pointer to how to specify the directory where I want the cache to be stored?

Thank you!

Doing:

query = overpassQueryBuilder(area=3600007407, elementType='way',    selector='"railway"="rail"', out='body')
a=overpass.query(query)

len(a.ways())
Out[30]: 751

len(a.elements())
Out[28]: 751

but doing on http://overpass-api.de/query_form.html way(area:3600007407)[railway=rail]; out count;

<count id="0">
    <tag k="nodes" v="0"/>
    <tag k="ways" v="764"/>
    <tag k="relations" v="0"/>
    <tag k="areas" v="0"/>
    <tag k="total" v="764"/>
  </count>

overpy also gives 764 elements.

It seems that missing ways were recently modified like 852538015: https://www.openstreetmap.org/way/852538015 edited 3 month ago

Do you have any idea ?

With the following:

from OSMPythonTools.overpass import Overpass

op = Overpass()
q = 'relation(110796); out body;'
r = op.query(q)
e = r.elements()[0]
e.geometry()

I get the following:

Exception: [OSMPythonTools.Element] Cannot build geometry: cannot find outer ring for inner ring. (relation/110796)

If a query results in an error (e.g., a timeout) even re-running the query with changed parameters (e.g., a different timeout value) immediately bails out. I have not checked if timeouts are the only problem but in general the cache should be invalidated in those cases.

(I am building the query strings manually; using 0.3.0 via pip)

Hi,
I think it would be nice to have the possibility to specify a custom cache directory, like

Nominatim(cacheDir='custom_cache_dir')
Overpass(cacheDir='custom_cache_dir')

Hello,
First of all, thank you for creating this great library, it's very convenient and easy to use.
I installed the latest version (0.2.7) with python 3.7.3 and then tried to use Overpass API with examples from the manual:

from OSMPythonTools.overpass import overpassQueryBuilder
from OSMPythonTools.overpass import Overpass
overpass = Overpass()

query = overpassQueryBuilder(bbox=[48.1, 16.3, 48.3, 16.5], elementType='node', selector='"highway"="bus_stop"', out='body')
busStops = overpass.query(query, timeout=2500)

it produced an error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-a5d094898918> in <module>
      1 query = overpassQueryBuilder(bbox=[48.1, 16.3, 48.3, 16.5], elementType='node', selector='"highway"="bus_stop"', out='body')
----> 2 busStops = overpass.query(query, timeout=2500)

~\Anaconda3\lib\site-packages\OSMPythonTools\internal\cacheObject.py in query(self, onlyCached, shallow, *args, **kwargs)
     38             with open(filename, 'w') as file:
     39                 ujson.dump(data, file)
---> 40         result = self._rawToResult(data, queryString, shallow=shallow)
     41         if not self._isValid(result):
     42             raise(Exception('[' + self._prefix + '] error in result (' + filename + '): ' + queryString))

TypeError: _rawToResult() got an unexpected keyword argument 'shallow'

Same error happens with examples from Nominatim manuals.
However, I then tried to install the previous version 0.2.6 and all the examples worked perfectly.

Hi! When I use the polygon coordinates that I get from heidelberg.wkt() into wkt.loads, I get the following error: JSONDecodeError: Expecting value: line 1 column 1 (char 0)
However, when I use a simplified polygon (obtained from polygons.openstreetmap.fr/?id=285864) it does work.
Do you know if there's a way to obtain a simplified polygon using the Nominatim query?

I’m using your OSMPythonTools for a project of mine (currently at https://nlmaps.gorgor.de/, still WIP) and I’m very happy with it! Thanks a lot for making it!

Specifically, I’m using it for querying Overpass and Nominatim. And having read Nominatim’s usage policy, I think that I should use my application’s name as the User-Agent string.

Therefore, I suggest making the User-Agent string customizable. It’s a trivial change and I’m going to submit a pull request for it in a minute.

Dear Franz-Benjamin 
Is there a smart syntax to get all the selector options ?

example:
selector = "building""^(yes|house|warehouse|apartments...)$ which is not exhaustive
should be something like
selector = "building""^(*)$? in order to get all building types

Best regards 
Simon

query = overpassQueryBuilder(area=areaId, elementType='node', selector='"natural"="tree"', out='count')

====== Example 3 ======
File "D:\ProgramData\Anaconda3\envs\geoai\lib\site-packages\OSMPythonTools_init_.py", line 13, in _raiseException
raise(Exception(msgComplete))
Exception: [OSMPythonTools.overpassQueryBuilder] Please provide an area or a bounding box

The library often handles exception itself, e.g., cacheObject's __query(), by excepting, logging, and then raising a newly created generic Exception object. For example very specific exceptions raised by urllib get converted into this generic type. It thus becomes impossible to differentiate between different errors on the user side, e.g., it one cannot even react differently if the user hits ctrl+c while a transfer is in progress.
As is it is completely impossible to do meaningful error handling without inspection or other tricks AFAICT.

It would be much better to leave the respective error handling to the library user instead. You can still add useful information to the exception (and log stuff if you like), cf. https://wiki.python.org/moin/HandlingExceptions

There is a guide on the sintax for overpassQueryBuilder selector values?

when I insert some values that I'm sure that exist at least one result with the corresponding tag i gets HTTP Error 400: Bad Request

from OSMPythonTools.nominatim import Nominatim
from OSMPythonTools.overpass import overpassQueryBuilder, Overpass
import random

nominatim = Nominatim()
overpass = Overpass()


def removeIfNoTags(element):
    if element is None or element.tags() is None:
        return False
    else:
        return True


elementType = ['node']
areaId = nominatim.query('Piemonte').areaId()
query = overpassQueryBuilder(
    area=areaId, elementType=elementType, selector="name", includeGeometry=True)
result = overpass.query(query)
points = result.elements()
points = list(filter(removeIfNoTags, points))
tags = {}

for point in points:
    p_tags = point.tags()
    for key in p_tags:
        if not(key in tags):
            tags[key] = set()
        tags[key].add(str(p_tags[key]))

chosen_tag = random.randrange(0, len(tags))
tag_list_key = list(tags)
tag_list_value = list(tags[tag_list_key[chosen_tag]])
chosen_tag_value = random.randrange(0, len(tag_list_value))

selector = str(tag_list_key[chosen_tag]) + " = " + \
    str(tag_list_value[chosen_tag_value])
print("tag selezionato: " + selector)

query = overpassQueryBuilder(
    area=areaId, elementType=elementType, selector=selector, includeGeometry=True)
result = overpass.query(query)
points = result.elements()

for point in points:
    p_tags = point.tags()
    print(p_tags)

When running this code and the random gets a tag that has ":" in it, Throw that error:

tag selezionato: wikipedia:en = Arduin of Ivrea
[overpass] downloading data: [timeout:25][out:json];area(3600044874)->.searchArea;(node[wikipedia:en = Arduin of Ivrea](area.searchArea);); out body geom;
The requested data could not be downloaded. HTTP Error 400: Bad Request
Traceback (most recent call last):
  File "/home/loris/.local/lib/python3.8/site-packages/OSMPythonTools/internal/cacheObject.py", line 83, in __query
    response = urllib.request.urlopen(request)
  File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request

The output is JSON regardless of the specified format because it gets overwritten:

I would also suggest renaming toJSON() to toDict() because it is confusing.

Example

from OSMPythonTools.nominatim import Nominatim

nominatim = Nominatim()
params = dict(params=dict(format='xml',polygon_geojson=0,limit=1))
data = nominatim.query(queryString,**params).toJSON()
print(data)

Currently the license is stored in LICENSE.md, marking the file as a Markdown file. However, it just contains default plain text version of the GPL 3.

As every text file is a valid Markdown file, but not necessarily a sensible one, this leads to interesting and probably unintended formatting when rendered by the GitHub website or by other Markdown-capable software.

There's two viable options:

• keep the file content as is, but rename the file to just LICENSE (recommended) or LICENSE.txt (not recommended)
  or
• use an actual sensibly Markdown-formatted version of the GPL 3, as available here and keep the current file name.

Hi Franz-Benjamin,

may you please add GeoJSON output capability? I'm already using toJSON() - but your tools outputs ' instead of ", so that osmtogeojson doesn't work.

Is there a way to download all nodes of requested ways during the overpass query?

I have this query:
query = f'''area({area_id})->.searchArea;
(
rel(area.searchArea);
way(area.searchArea);
node(area.searchArea);
)->.result;
.result out meta;
.result >->.recurseresult;
.recurseresult out meta;'''

result = overpass.query(query)
In my understanding, even if all nodes of a way w are in results, the first time I call w.nodes() it will download all of the nodes.

I would like:

1. query() to download all data so that no other http request are made.
2. nodeRefs() or similar to receive node ids only (currently this info is buried into _json)

Is there such functionality or is planned?

Thanks,
Angelo

(Please correct me if I'm wrong, but) AFAICS, you don't send any meaningful user-agent header when requesting Overpass and Nominatim. Especially for these community-provided services it is very important to receive such a header, in order to find out where traffic comes from and how the available resources can be distributed fairly.

Thanks for this cool software.

I'm unable to use a query containing geocodeArea:

from OSMPythonTools.overpass import Overpass

query = """
{{geocodeArea:Berlin}}->.searchArea; ( node[memorial=ghost_bike](area.searchArea); ); out body;
"""

overpass = Overpass()
result = overpass.query(query)
print(result.toJSON())

The requested data could not be downloaded. HTTP Error 400: Bad Request

The same query, prefixed by [timeout:25][out:json]; runs just fine in Overpass Turbo.

Any idea what I'm doing wrong here?

It'd be great if the revision corresponding to a release on PyPI could be found in this repo by an appropriately named Git tag, especially as the source distributions of this package on PyPI don't seem to contain the pytest tests included in the GitHub repo since e07ce69.

When I did the same thing as the author did. An unexpected unboudnLocalError occured. Is there anyone know how to fix this problem? The code and the error message is attached below:

from OSMPythonTools.api import Api
api = Api()
way = api.query('way/108402486')
way.nodes()

Traceback (most recent call last):
File "G:\Spatial IT program\OSM_Py\7_31_trial.py", line 4, in
way.nodes()
File "G:\Python36\lib\site-packages\OSMPythonTools\element.py", line 54, in nodes
api = SingletonApi()
File "G:\Python36\lib\site-packages\OSMPythonTools\element.py", line 9, in init
SingletonApi.__instance = Api(*args, **kwargs)
UnboundLocalError: local variable 'Api' referenced before assignment

I don't know if this is something not implemented, yet, or if I just haven't been able to figure this out, but is it possible to create a union of selectors ?

I've tried something like this:

from OSMPythonTools.nominatim import Nominatim
from OSMPythonTools.overpass import overpassQueryBuilder
from OSMPythonTools.overpass import Overpass

nominatim = Nominatim()
rdc = nominatim.query('République démocratique du Congo')

query = overpassQueryBuilder(area=rdc.areaId(), elementType='node', selector=['"amenity"="hospital"'], out='body')

overpass = Overpass()
result = overpass.query(query)

print("Number of nodes found: {}".format(result.countElements()))

and I get as output: 90.

With

query = overpassQueryBuilder(area=rdc.areaId(), elementType='node', selector=['"amenity"="clinic"'], out='body'

the result is 536.

But with

query = overpassQueryBuilder(area=rdc.areaId(), elementType='node', selector=['"amenity"="hospital"', '"amenity"="clinic"'], out='body')

the result is 0 which gives me the impression that the result is the intersection of the selectors and not the union.

If it is possible to make this into a union, could someone point me to where in the docs I can find info on that. If not, is it envisageable to implement this ?

That's the code:

from OSMPythonTools.nominatim import Nominatim
from OSMPythonTools.overpass import overpassQueryBuilder, Overpass

nominatim = Nominatim()
overpass = Overpass()

elementType = ['node']
areaId = nominatim.query('Novara, Lombardia, Italy').areaId()
query = overpassQueryBuilder(
    area=areaId, elementType=elementType, includeGeometry=True)
result = overpass.query(query)
points = result.elements()
print(len(points))

for point in points:
    print(point.tags())
    point_name = point.tag('name')
    print(point_name)

and when I run it i get this error:

1168
{'amenity': 'fuel', 'brand': 'Esso', 'name': 'Esso', 'operator': 'Aresi di Sironi Fabio', 'ref:mise': '35940'}
Esso
None
Traceback (most recent call last):
  File "example1.py", line 17, in <module>
    point_name = point.tag('name')
  File "/home/loris/.local/lib/python3.8/site-packages/OSMPythonTools/element.py", line 110, in tag
    return tags[key] if key in tags else None
TypeError: argument of type 'NoneType' is not iterable

I don't understand... Why I should get None as an element of the results?

Hello!

I am trying to query for objects (nodes and ways) that are around some location represented as a (lat, lon) pair.
My query is

query_obj = '(node(around:10000,{0},{1});way(around:10000,{0},{1});); out body;'.format(lat, lon)
overpass.query(query_obj, timeout=5)

And while it works fine with some (lat, lon) pairs, e.g. (52.6851, -0.470191), it throws an exception with others, e.g. (54.59298,-1.48307).

This is what I am getting

What could be the reason? And do you have any idea how to avoid that?

Thanks in advance!

Hi, I noticed that using the overpass.Overpass.query method, using inverted bounding boxes (i.e. where for bbox=[a, b, c, d] a > c or b > d) I get an HTTP error 400: Bad request. I suggest exchanging the two coordinates when they would produce that error otherwise - do you think that would be possible ? Because else I'd have to do that in the script that uses the OSMPythonTools, and every other user of this module too, which leads to an incredible amount of code duplication. Something like this in the constructor of OSMPythonTools.overpass.overpassQueryBuilder:

if bbox[0] > bbox[2]:
    bbox[0], bbox[2] = bbox[2], bbox[0]
if bbox[1] > bbox[3]:
    bbox[1], bbox[3] = bbox[3], bbox[1]
´´´

Hi,

I just created a fresh Anaconda environment, installed pip, OSMPythonTools, ipykernel and pandas. But when I try the example in readme.md

from OSMPythonTools.api import Api api = Api() way = api.query('way/5887599')

I get this error which states that the SSL certificate has expired:

The requested data could not be downloaded. Please check whether your internet connection is working.
Traceback (most recent call last):
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1037, in _send_output
self.send(msg)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 975, in send
self.connect()
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\ssl.py", line 1070, in _create
self.do_handshake()
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\site-packages\OSMPythonTools\internal\cacheObject.py", line 95, in __query
response = urllib.request.urlopen(request)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 525, in open
response = meth(req, response)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 634, in http_response
response = self.parent.error(
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 557, in error
result = self._call_chain(*args)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 749, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 519, in open
response = self._open(req, data)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)>
Traceback (most recent call last):
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1037, in _send_output
self.send(msg)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 975, in send
self.connect()
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\http\client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\ssl.py", line 1070, in _create
self.do_handshake()
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\site-packages\OSMPythonTools\internal\cacheObject.py", line 95, in __query
response = urllib.request.urlopen(request)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 525, in open
response = meth(req, response)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 634, in http_response
response = self.parent.error(
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 557, in error
result = self._call_chain(*args)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 749, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 519, in open
response = self._open(req, data)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 1, in
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\site-packages\OSMPythonTools\internal\cacheObject.py", line 48, in query
data = self.__query(queryString, params)
File "C:\Users\fherb\anaconda3\envs\OpenStreetMap\lib\site-packages\OSMPythonTools\internal\cacheObject.py", line 103, in __query
raise Exception(msg, err)
Exception: ('The requested data could not be downloaded. Please check whether your internet connection is working.', URLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)')))

Edit:
Further System information:
Windows 11, Python 3.10.4, conda 4.11.0

Using Overpass Turbo you can add if block statements to query by conditions. For example, if I want all roads that include tunnels, I can query:
way["tunnel"="yes"]["highway"](if: length() > 80)(area); out;

Currently, I don't believe there is any way to do this using the overpassQueryBuilder.
If you add this 'if' block to the selector argument, the 'if' block becomes enveloped in [] in the query string:
way["tunnel"="yes"]["highway"][(if: length() > 80)](area.searchArea););
This causes an error.

Is there currently any functionality to support 'if' block conditions when querying the Overpass API?

Hi Franz

First of all thank you very much for the library you've build. I'm pretty new to OSM (and in general to geographic information) and your library is helping me a lot.
My first goal was to identify all libraries within a city (and later per country and around the world) and later to do the same for other keys.

The first step was to find a city by name + verify the coordinates (I used Vienna like in your examples):

    from OSMPythonTools.nominatim import Nominatim
    nominatim = Nominatim()
    city = nominatim.query('Vienna', wkt=True)
    print('Vienna: ' + str(city.wkt()))

I get the following results: POLYGON((16.181831 48.1711198,16.181902 48.1710313, ...))

If I compare the coordinates (again I'm not an expert on geographic information) I wanted to verfiy the coordinates of Vienna via https://www.wikidata.org/wiki/Q1741
https://geohack.toolforge.org/geohack.php?params=48.208333333333336_N_16.3725_E_globe:earth&language=en
-> 48.208333, 16.3725

Is it intentional that the latitude and longitude are reversed in order or is that a different format?
(latitude=48.208333, longitude=16.3725) <-> the polygons in city.wkt() are containing the longitude first and the latitude second.

After that I started with the overpassQueryBuilder() function, where I used the example from here https://infovis.fh-potsdam.de/tutorials/infovis8geovis.html

library_query = overpassQueryBuilder(
    area=city.areaId(), # the query can be contrained by an area of an item
    elementType='node', # which are points (OSM also has ways and relations)
    # the selector in the next line is really the heart of the query:
    selector='"amenity"="library"', # we're looking for libraries
    out='body', # body indicates that we want the data, not just the count
    includeGeometry=True # and we want the geometric information, too
)

lib_data = overpass.query(library_query)
libraries = [ (lib.tag("name"), lib.geometry() ) for lib in lib_data.nodes()]
for library in libraries:
    name = library[0]
    location = library[1]["coordinates"]
    print('Library: ' + str(name) + ', coordinates: ' + str(location))

I do get the libraries in Vienna, but again it seems the coordinates are in the format (longitude, latitude). Is this on purpose or a different format?

e.g. output: Büchereien Wien, Rabenhof, coordinates: [16.400691, 48.196254]
<-> https://www.openstreetmap.org/node/115181418
it shows the coordinates in the format (48.1962536, 16.4006914)

Also may I ask you, how I could extend the existing query to either e.g. a bunch of cities/a country/worldwide?
If I try the overpassQueryBuilder() without the parameter "area"

library_query = overpassQueryBuilder(
    # area=city.areaId(), # the query can be contrained by an area of an item
    elementType='node', # which are points (OSM also has ways and relations)
    # the selector in the next line is really the heart of the query:
    selector='"amenity"="library"', # we're looking for libraries
    out='body', # body indicates that we want the data, not just the count
    includeGeometry=True # and we want the geometric information, too
)

I get the feedback "Please provide an area or a bounding box". If I try to use the parameter bbox with a huge number for "around" and the coordinates of Vienna (not sure if I should be using (48.208333, 16.3725) like in wiki/OSM or the opposite (16.3725, 48.208333))

library_query = overpassQueryBuilder(
    # area=city.areaId(), # the query can be contrained by an area of an item
    bbox='(around: 100000000000, 48.208333, 16.3725)',
    elementType='node', # which are points (OSM also has ways and relations)
    # the selector in the next line is really the heart of the query:
    selector='"amenity"="library"', # we're looking for libraries
    out='body', # body indicates that we want the data, not just the count
    includeGeometry=True # and we want the geometric information, too
)

I get the following exception: "('The requested data could not be downloaded. HTTP Error 400: Bad Request', <HTTPError 400: 'Bad Request'>)"

May I ask you if you could help me on this?
Thank you very much

Hi! I'd like to ask how to resolve the issue where querying an overpass name that most probably be because of a space in between causes a TypeError.

This is how the code looks like in my python views:

def query_overpass(request):
    query = request.GET['query']
    print(query)
    overpass = Overpass()
    city_query = "rel[name="+ query + "][type=boundary][boundary=administrative];out geom;"
    city_boundary = overpass.query(city_query)
    result = json.dumps(city_boundary.toJSON())
    return render(request, 'plexus/home.html', {'result': result})

and the error it raises:

Quezon City
Internal Server Error: /search/
Traceback (most recent call last):
  File "/home/gridlockdev/Desktop/heroku/grace/env/lib/python3.5/site-packages/django/core/handlers/exception.py", line 41, in inner
    response = get_response(request)
  File "/home/gridlockdev/Desktop/heroku/grace/env/lib/python3.5/site-packages/django/core/handlers/base.py", line 187, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/home/gridlockdev/Desktop/heroku/grace/env/lib/python3.5/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/gridlockdev/Desktop/heroku/grace/network/views.py", line 40, in query_overpass
    city_boundary = overpass.query(city_query)
  File "/home/gridlockdev/Desktop/heroku/grace/env/lib/python3.5/site-packages/OSMPythonTools/internal/cacheObject.py", line 38, in query
    result = self._rawToResult(data, queryString)
  File "/home/gridlockdev/Desktop/heroku/grace/env/lib/python3.5/site-packages/OSMPythonTools/overpass.py", line 44, in _rawToResult
    return OverpassResult(data, queryString)
  File "/home/gridlockdev/Desktop/heroku/grace/env/lib/python3.5/site-packages/OSMPythonTools/overpass.py", line 73, in __init__
    self._elements = list(map(lambda e: Element(json=e), self.__get('elements')))
  File "/home/gridlockdev/Desktop/heroku/grace/env/lib/python3.5/site-packages/OSMPythonTools/overpass.py", line 87, in __get
    return self._json[prop] if prop in self._json else None
TypeError: argument of type 'NoneType' is not iterable

At first I assumed that there really might be no element to return, but I tried it over at Overpass Turbo and it returns exactly what I want. See: http://overpass-turbo.eu/s/tfP

Other than this problem, it works great! 👍

Hope you could help! Thanks in advance!

I would like to query for specific areas & time periods ways meeting a certain criteria and for those ways get count and total length. As an example:

[timeout:60][date:"2020-01-01T00:00:00Z"][out:json];area(3603579531)->.searchArea;(way["mtb:scale"](area.searchArea););make stat number=count(ways),length=sum(length()); out;

I can manually build the query and use .toJSON() on query response to access the desired stats values, however preferred way would be to use overpassQueryBuilder and Data class. Looking at the code it looks like this is not supported?

Hi there,

I am new to OSM and wanted to play around with it in a Jupiter Notebook.

I checked out the readme and wanted to implement the example.
But somehow it does not do what it is supposed to.
Please check out the error messages in the attachment.

Any suggestion is greatly appreciated.

Thanks
Pascal

Thanks for the very useful library!
One tiny feature I would love to see is the possibility to change the overpass API endpoint that is now hardcoded to http://overpass-api.de/api/ - which somehow has quite a big lag right now and I would like to quickly switch to a different server.

First of all, thank you for this very practical package.
I would like to do an overpass query that returns the center of ways as well.
This basically works fine but I can't read the center coordinates from the result element.
The query I performed looks something like this:

(
  node["some"="type"];
  way["some"="type"];
  relation["some"="type"];
);

out center;
out body;

Is there a way which I can query a latitude and longitude?

Hello, I am new to Github and Python, so I apologize if this is a dumb question. (Feel free to tell me if it is).

I tried to run (from your sample code) after installing OSMPythonTools in a cloned ArcGIS Pro environment. I got the following error (and ignored it):

Then I ran the code.

from OSMPythonTools.nominatim import Nominatim
nominatim = Nominatim()
heidelberg = nominatim.query('Heidelberg')

I subsequently got the following JSON DeCode Error. Is the problem the error I got with the installation?

JSONDecodeError Traceback (most recent call last)
~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo_accessor.py in from_layer(layer)
2288 from arcgis.features.geo._io.serviceops import from_layer
-> 2289 return from_layer(layer=layer)
2290 except ImportError:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo_io\serviceops.py in from_layer(layer, query)
104 raise ValueError("Invalid inputs: must be FeatureLayer or Table")
--> 105 sdf = layer.query(where=query, as_df=True)
106 sdf.spatial._meta.source = layer

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in query(self, where, out_fields, time_filter, geometry_filter, return_geometry, return_count_only, return_ids_only, return_distinct_values, return_extent_only, group_by_fields_for_statistics, statistic_filter, result_offset, result_record_count, object_ids, distance, units, max_allowable_offset, out_sr, geometry_precision, gdb_version, order_by_fields, out_statistics, return_z, return_m, multipatch_option, quantization_parameters, return_centroid, return_all_records, result_type, historic_moment, sql_format, return_true_curves, return_exceeded_limit_features, as_df, datum_transformation, **kwargs)
1047 params['returnCountOnly'] = True
-> 1048 record_count = self._query(url, params, raw=as_raw)
1049 if 'maxRecordCount' in self.properties:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in _query(self, url, params, raw)
2047 else:
-> 2048 raise queryException
2049

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\layer.py in _query(self, url, params, raw)
2017 result = self._con.post(path=url,
-> 2018 postdata=params, token=self._token)
2019 except Exception as queryException:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\gis_init_.py in _token(self)
10777 else:

10778 self._hydrate()
10779 return self._lazy_token

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\gis_init_.py in _hydrate(self)
10756 self._lazy_token = None

10757 self._refresh()
10758

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\gis_init_.py in _refresh(self)
10699 else:

10700 raise e
10701

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\gis_init_.py in _refresh(self)
10692 try:

10693 dictdata = self._con.post(self.url, params, token=self._lazy_token)
10694 except Exception as e:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\gis_impl_con_connection.py in post(self, path, params, files, **kwargs)
709 try_json=try_json,
--> 710 force_bytes=kwargs.pop('force_bytes', False))
711 #----------------------------------------------------------------------

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\gis_impl_con_connection.py in _handle_response(self, resp, file_name, out_path, try_json, force_bytes)
496 else:
--> 497 data = resp.json()
498 #if 'error' in data:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\requests\models.py in json(self, **kwargs)
897 pass
--> 898 return complexjson.loads(self.text, **kwargs)
899

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\json_init_.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
353 parse_constant is None and object_pairs_hook is None and not kw):
--> 354 return _default_decoder.decode(s)
355 if cls is None:

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\json\decoder.py in decode(self, s, _w)
338 """
--> 339 obj, end = self.raw_decode(s, idx=_w(s, 0).end())
340 end = _w(s, end).end()

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\json\decoder.py in raw_decode(self, s, idx)
356 except StopIteration as err:
--> 357 raise JSONDecodeError("Expecting value", s, err.value) from None
358 return obj, end

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Exception Traceback (most recent call last)
in
18 # get_URL = input("Paste in the URL for the city boundary layer you want from Open Street Map:")
19 dc_fl = FeatureLayer('https://nominatim.openstreetmap.org/ui/details.html?osmtype=R&osmid=1839150&class=boundary')
---> 20 dc_df = GeoAccessor.from_layer(dc_fl)
21 display(dc_df.head())
22

~\AppData\Local\ESRI\conda\envs\arcgispro-py3-clone\lib\site-packages\arcgis\features\geo_accessor.py in from_layer(layer)
2291 raise ImportError("Could not load from_layer.")
2292 except json.JSONDecodeError as je:
-> 2293 raise Exception("Malformed response from server, could not load the dataset: %s" % str(je))
2294 except Exception as e:
2295 raise Exception("Could not load the dataset: %s" % str(e))

Exception: Malformed response from server, could not load the dataset: Expecting value: line 1 column 1 (char 0)

Hi, first of all thank you very much for all your work on this package.

While insalling this package, I realised that it introduces a dependency on pytest and pytest-sugar (see this line and this line).

Are those dependencies needed for end users, or should they be distributed somewhere else for developer of this library only?
I'm asking this question because installing the package produces a side-effect for me which is modifying my pytest interface.

Thank you for your time.

Hello, thank you for his great job.
My question : is it possible to access the building footprint from overpassQueryBuilder() ?
somathing like overpassQueryBuilder(elementType='area', selector='"building"="yes"')

Simon

Hi there,

first of all, great library, it's been really useful to me!

Now my issue, after experiementing with the public endpoints I've setup a personal overpass endpoint to escape the wait times and not over utilize public goods.
I've followed along the instructions on: https://wiki.openstreetmap.org/wiki/Overpass_API/Installation
Installing Release 0.7.58

Pointing the library to my endpoint via:
Overpass(endpoint='<MyEndpoint>')

I ended up with the following error (happy to share a more detailed trace if needed):
Exception: [overpass] could not fetch or interpret status of the endpoint

After some analysis it seems I'm failing in the libraries _waitForReady function because my status response has more rows and therefore the statusString[] indexes are off.

Status Response
Connected as: 3232235754
Current time: 2022-06-01T15:19:28Z
Announced endpoint: none
Rate limit: 0
Currently running queries (pid, space limit, time limit, start time):

Can the function be changed to work line independent and detect required infos smarter?
I'm just starting out so I don't want to mess with the libraries code.

Cheers!

With the following:

from OSMPythonTools.overpass import Overpass

op = Overpass()
q = 'relation(110796); out body;'
r = op.query(q)
e = r.elements()[0]
e.geometry()

I get the following:

Exception: [OSMPythonTools.Element] Cannot build geometry: cannot find outer ring for inner ring. (relation/110796)

First, let me say that OSMPythonTools is really excellent, the work is much appreciated!

I'm looking for major/minor road intersections per this example:
https://gis.stackexchange.com/questions/296278/overpass-api-find-road-intersection-points-with-defined-line

I manually construct my query as follows:

query = '[bbox:%3.6f, %3.6f, %3.6f, %3.6f]; way[highway~\"^(motorway|trunk|primary|secondary|tertiary|(motorway|trunk|primary|secondary)_link)$\"]->.major; way[highway~\"^(unclassified|residential|living_street|service)$\"]->.minor; node(w.major)(w.minor); out body geom;' % (bbox[0], bbox[1], bbox[2], bbox[3])
logger.info(query)

# Query
self.intersections = self.overpass.query(query, timeout=60)

My constructed query is as follows, and successfully runs in overpass turbo, but I get HTTP Error 400: Bad Request when running overpass.query.

[bbox:40.115150, -105.143300, 40.215150, -105.043300]; way[highway~"^(motorway|trunk|primary|secondary|tertiary|(motorway|trunk|primary|secondary)_link)$"]->.major; way[highway~"^(unclassified|residential|living_street|service)$"]->.minor; 
node(w.major)(w.minor); out body geom;

Thoughts?

Hi everyone,
I am new to this library and tried to run the example code first. For me the example 3 fails with the following information:

[overpass] downloading data: [timeout:25][date:"2013-01-01T00:00:00Z"][out:json];area(3600109166)->.searchArea;(node["natural"="tree"](area.searchArea);); out count;
Exception: [overpass] runtime error: Query run out of memory using about 2048 MB of RAM.
NoneType: None
[overpass] error in result (overpass-b156c1029beee7d4af0cdae8c9ae8c29459dcfd6): [timeout:25][date:"2013-01-01T00:00:00Z"][out:json];area(3600109166)->.searchArea;(node["natural"="tree"](area.searchArea);); out count;
NoneType: None

I tried to run it locally and on google colab with the same results.
Do you know what the problem is?

Thank you in advance,
Sören

Hi,

could you help me with the following problem? I think there's some easy solution that I can't seem to find by myself...

I'm running the following query for several cities. The query works fine for cities with generally easy names like Berlin, Budapest, London, Paris, etc. However, I can't make it work for cities with spaces in the name (New York, Rio de Janeiro), or cities with special characters (Köln, Münich, Rīga). Any ideas?

"""
[out:json][timeout:900];
area[name=New York]->.city;
(node'amenity'='bus_station';node'highway'='bus_stop';
);
(._;>;);
out center;

When running the following query:

area(3600165475);
(
node["amenity"="fast_food"](area);
way["amenity"="fast_food"](area);
relation["amenity"="fast_food"](area);
);
out center;

I get this error:

Exception: [overpass] error in result (cache/overpass-0e7ebc0bbe8cd160b8389a3314ba688ba0f3c47f): [timeout:500][out:json];area(3600165475);
            (
            node["amenity"="fast_food"](area);
            way["amenity"="fast_food"](area);
            relation["amenity"="fast_food"](area);
            );
            out center;

During handling of the above exception, another exception occurred:

Exception: [overpass] runtime error: Query timed out in "print" at line 7 after 26 seconds.
    return search_areas(search_term_type, search_term, areas)
  File "/Users/work/Library/Application Support/JetBrains/PyCharmCE2020.1/scratches/scratch_4.py", line 181, in search_areas
    res = scrape_data(search_query)
  File "/Users/work/Library/Application Support/JetBrains/PyCharmCE2020.1/scratches/scratch_4.py", line 100, in scrape_data
    r = api.query(search_query, timeout=500)
  File "/Users/work/plotproject.scrips/lib/python3.7/site-packages/OSMPythonTools/internal/cacheObject.py", line 42, in query
    raise(Exception('[' + self._prefix + '] error in result (' + filename + '): ' + queryString))
Exception: [overpass] error in result (cache/overpass-0e7ebc0bbe8cd160b8389a3314ba688ba0f3c47f): [timeout:500][out:json];area(3600165475);
            (
            node["amenity"="fast_food"](area);
            way["amenity"="fast_food"](area);
            relation["amenity"="fast_food"](area);
            );
            out center;

I am using the following code where I set the timeout to be 500 seconds:

search_query = f"""(
            node["{search_term_type}"="{search_term}"]({search_box});
            way["{search_term_type}"="{search_term}"]({search_box});
            relation["{search_term_type}"="{search_term}"]({search_box});
            );
            out center;"""
api = Overpass()
r = api.query(search_query, timeout=500)

OSMPythonTools version is 0.2.8

However, when I run the same query in https://overpass-turbo.eu/ it works just fine. It also works fine on Postman