Comments (9)
Dear @stefanct,
You are raising a good point. The overpassQueryBuilder
does not add a timeout on intention, and when you build the query manually, a timeout should not be added manually. Instead of adding the timeout to your query directly, you can (and should) add it as a parameter to the query
function:
overpass.query('...', timeout=25)
The following query, for instance, fails:
zandvoort = nominatim.query('Zandvoort')
query = overpassQueryBuilder(area=zandvoort, elementType='way', selector='"highway"', includeCenter=True)
roads = overpass.query(query, timeout=.1)
Rerunning the query then works as expected, i.e., the data is is downloaded, properly cached, and saved in the variable roads
:
zandvoort = nominatim.query('Zandvoort')
query = overpassQueryBuilder(area=zandvoort, elementType='way', selector='"highway"', includeCenter=True)
roads = overpass.query(query, timeout=30)
Can you confirm this behaviour, or is this broken on your machine?
Thanks for checking.
from osm-python-tools.
I am building the queries myself. However, they don't include the timeout implicitly but I set it with the parameter like you show it. See my code here (the request comes from a function on the top): https://github.com/stefanct/osm_refhistorymeta/blob/main/ref_contributors.py#L79
Your MWE do not work with the version I have. Even without the includeCenter
it doesn't due to "Bad Request" errors from the server (no matter the timeout).
from OSMPythonTools.overpass import Overpass, overpassQueryBuilder
from OSMPythonTools.nominatim import Nominatim
nominatim = Nominatim()
overpass = Overpass()
zandvoort = nominatim.query('Zandvoort')
query = overpassQueryBuilder(area=zandvoort, elementType='way', selector='"highway"')
roads = overpass.query(query, timeout=100)
results in:
[overpass] downloading data: [timeout:100][out:json];area(<OSMPythonTools.nominatim.NominatimResult object at 0x7f5998d7cb00>)->.searchArea;(way["highway"](area.searchArea);); out body;
The requested data could not be downloaded. HTTP Error 400: Bad Request
Traceback (most recent call last):
File "/home/stefanct/.local/lib/python3.7/site-packages/OSMPythonTools/internal/cacheObject.py", line 83, in __query
response = urllib.request.urlopen(request)
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
Traceback (most recent call last):
File "/home/stefanct/.local/lib/python3.7/site-packages/OSMPythonTools/internal/cacheObject.py", line 83, in __query
response = urllib.request.urlopen(request)
File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "foo.py", line 8, in <module>
roads = overpass.query(query, timeout=100)
File "/home/stefanct/.local/lib/python3.7/site-packages/OSMPythonTools/internal/cacheObject.py", line 37, in query
data = self.__query(queryString, params)
File "/home/stefanct/.local/lib/python3.7/site-packages/OSMPythonTools/internal/cacheObject.py", line 87, in __query
raise Exception(msg)
Exception: The requested data could not be downloaded. HTTP Error 400: Bad Request
Best would probably to update but... what's the easiest way to do that?
from osm-python-tools.
This is the expected behaviour if you did not update. If you provide area=zandvoort.areaId()
, it should work.
However, I recommend to update, making this change unnecessary:
pip3 install OSMPythonTools --upgrade
from osm-python-tools.
OK, well, I see the same behavior as in my application: once there is a timeout error it does not try to re-download but instantly fails. Also, any timeouts <1.0 produce Bad Requests - no idea why that would work for you (it's not due to locales). I tried multiple things (1.0, 0.9, 0.1, .1 and even strings) to no avail.
from osm-python-tools.
I have investigated this further. I will provide a fix soon. Stay tuned.
from osm-python-tools.
Ok, I think I have fixed it now. Please update again to the newest version. Then, either run my example to test, or run your example, it does not matter.
Btw: My example was not correct. The parameter timeout
seems to accept only integer values (at least in version of Overpass that runs on the official server). So it should be timeout=1
to provoke a timeout, at least sometimes.
I would be happy to read if it now works as expected.
from osm-python-tools.
Additional remark: If you like the package, feel welcome to "star" it. ;) (If you don't, be honest ;)
from osm-python-tools.
It's not just floats - it works fine with float values >=1.0. No idea what's going on there but I guess it's a server issue/feature. In any case, your fix regarding the errors in cached files works for me too, thanks!
from osm-python-tools.
Perfect, this is nice to hear! Have fun using it.
from osm-python-tools.
Related Issues (20)
- download changeset HOT 1
- Рисование маршрута
- overpass.py - _waitForReady - Hardcoded Status String Positions HOT 16
- querying cities with special characters and spaces HOT 1
- HTTP Error 400: Bad Request for query that is successful in overpass turbo HOT 2
- Coordinates (latitude, longitude) <-> (longitude, latitude) and how to query countries/worldwide with overpassQueryBuilder() HOT 3
- Example 3 fails with runtime error / timeout HOT 1
- SSL Error in fresh Conda environment HOT 1
- HTTP error 400: Bad request when using inverted bounding boxes for the overpass API module HOT 1
- Custom cache directory HOT 1
- Overpass returns every way twice HOT 6
- OSMPythonTools.nominatim output preferred language HOT 1
- Overpass shallow query results in an error
- It does not work on m1 macbook
- Could we get (perhaps computed) center coordinates for all elements? HOT 1
- Significantly reduced performance since version 0.3.3
- WARNING output that does not use the logging module
- does it support the notes API? HOT 1
- Cannot build geometry for relation that is not polygon HOT 1
- ERROR 403: Forbidden Request HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from osm-python-tools.