Comments (6)
What are the tags on the object?
If the same data in a PBF and XML gets different results, there's a bug.
The fact that you're getting an error is not a bug - you're supplying a string literal for a text[] value. There's nothing wrong with doing that and doing the conversion yourself - in fact you must do it because there isn't a built-in conversion but your code is responsible for getting the string literal right, including escaping.
from osm2pgsql.
I found more out about it. But first a correction of the output of osm2pgsql which got lost during copy and past:
My first workaround was:
- Clip and convert the PBF to OSM for my bounding box with osmium
- Import the OSM file with osm2pgsql
What was also working:
- Clip to my bounding box and save as PBF
- Import with osm2pgsql
So I was converting the whole file and searched for 'green:white:green:' and found this relation https://www.openstreetmap.org/relation/1830701#map=14/50.1851/12.2991. That is far away from my boundingbox:
osm2pgsql -O flex -S demo.lua -b 13.662356,50.792525,13.888777,50.89853 sachsen-latest.osm.pbf
The error happens, if I convert the complete file sachsen-latest.osm.pbf to OSM-XML.
from osm2pgsql.
As @pnorman mentioned the problem is in your Lua script. You make no attempt to escape/format the value correctly. The reason why this fails here and not somewhere else is probably the backslash in the tag (which also needs to be corrected in the OSM data, but that's beside the point here). Because you are using sql_type
you have to convert the data you have somehow into the correct format for PostgreSQL to understand. Osm2pgsql can not do this for you, because it doesn't interpret the sql_type
at all but just gives the data "as is" to the database and hopes that the database can interpret it according to the type you defined in sql_type
.
from osm2pgsql.
It sounds like the PBF vs XML bug was from the areas being covered being different, not anything to do with osm2pgsql. In that case, there is no bug on the osm2pgsql side and you will need to fix your code.
from osm2pgsql.
I have added some warning words to the manual about the use of sql_type
. For most use cases using the json
or jsonb
type
is probably a better solution, because osm2pgsql will do the conversion automatically for you.
from osm2pgsql.
Thanks for your help and explanations!
Here is my adjusted configuration
print('osm2pgsql version: ' .. osm2pgsql.version)
-- Variables
local tables = {}
local import_schema = 'osm' -- Defines the import schema
local epsg_code = 32633 -- Defines the projection
local w2r = {}
-- Table defenitions
tables.traffic = osm2pgsql.define_table({
name = 'traffic',
schema = import_schema,
ids = {
type = 'way',
id_column = 'way_id'
},
columns = {{
column = 'fid',
sql_type = 'serial',
create_only = true
}, {
column = 'highway',
type = 'text'
}, {
column = 'name',
type = 'text'
}, {
column = 'osmc_symbols',
type = 'jsonb'
},{
column = 'rel_ids',
sql_type = 'int8[]'
}, {
column = 'geom',
type = 'linestring',
projection = epsg_code
}},
indexes = {{
column = 'geom',
method = 'gist'
}}
})
function osm2pgsql.select_relation_members(relation)
-- Only interested in relations with type=route
if relation.tags.type == 'route' and relation.tags.route == 'hiking' then
return { ways = osm2pgsql.way_member_ids(relation) }
end
end
-- Function which fill the tables
function osm2pgsql.process_node(object)
end
function osm2pgsql.process_way(object)
if object.tags.highway or object.tags.railway then
row = {
name = object.tags.name,
highway = object.tags.highway,
geom = object:as_multilinestring()
}
local d = w2r[object.id]
if d then
local refs = {}
local ids = {}
for rel_id, rel_ref in pairs(d) do
refs[#refs + 1] = rel_ref
ids[#ids + 1] = rel_id
end
table.sort(refs)
table.sort(ids)
row.osmc_symbols = refs
row.rel_ids = '{' .. table.concat(ids, ',') .. '}'
end
tables.traffic:insert(row)
end
end
function osm2pgsql.process_relation(object)
local type = object:grab_tag('type')
if type == 'route' and object.tags.route == 'hiking' then
for _, member in ipairs(object.members) do
if member.type == 'w' then
if not w2r[member.ref] then
w2r[member.ref] = {}
end
w2r[member.ref][object.id] = object.tags['osmc:symbol']
end
end
end
end
from osm2pgsql.
Related Issues (20)
- Handling of schemas HOT 1
- Loading ways in non-slim mode is slower than with --slim.
- osm2pgsql-replication init fails on get_dsn_parameters() HOT 1
- Test failure in bdd-flex with 1.9.0 HOT 2
- Confusing middle pgsql processing. HOT 7
- Problem with query to get changed parent objects HOT 7
- 1.9.2 failed to build on several architectures (error: static assertion failed) HOT 2
- Segmentation fault (core dumped) HOT 2
- Allow more than 32 generalization jobs HOT 2
- Chunky rivers when generalizing water areas HOT 2
- Question: could not extend file "base/361191441/368615606.137": No space left on device
- osm2pgsql should not perform analyze by itself HOT 15
- `object:as_multipolygon()` does not take `object.members` into account. HOT 2
- Some research on middle performance HOT 4
- highway=rest_area treatet as line HOT 2
- free(): invalid size Aborted Core dumped HOT 4
- North America import fails HOT 1
- nlohmann-json is missing from the Alpine build dependencies command
- Deprecating -i,--tablespace-index? HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from osm2pgsql.