Git Product home page Git Product logo

Comments (6)

pnorman avatar pnorman commented on June 20, 2024

What are the tags on the object?

If the same data in a PBF and XML gets different results, there's a bug.

The fact that you're getting an error is not a bug - you're supplying a string literal for a text[] value. There's nothing wrong with doing that and doing the conversion yourself - in fact you must do it because there isn't a built-in conversion but your code is responsible for getting the string literal right, including escaping.

from osm2pgsql.

MathiasGroebe avatar MathiasGroebe commented on June 20, 2024

I found more out about it. But first a correction of the output of osm2pgsql which got lost during copy and past:

grafik

My first workaround was:

  1. Clip and convert the PBF to OSM for my bounding box with osmium
  2. Import the OSM file with osm2pgsql

What was also working:

  1. Clip to my bounding box and save as PBF
  2. Import with osm2pgsql

So I was converting the whole file and searched for 'green:white:green:' and found this relation https://www.openstreetmap.org/relation/1830701#map=14/50.1851/12.2991. That is far away from my boundingbox:

osm2pgsql -O flex -S demo.lua -b 13.662356,50.792525,13.888777,50.89853 sachsen-latest.osm.pbf

The error happens, if I convert the complete file sachsen-latest.osm.pbf to OSM-XML.

from osm2pgsql.

joto avatar joto commented on June 20, 2024

As @pnorman mentioned the problem is in your Lua script. You make no attempt to escape/format the value correctly. The reason why this fails here and not somewhere else is probably the backslash in the tag (which also needs to be corrected in the OSM data, but that's beside the point here). Because you are using sql_type you have to convert the data you have somehow into the correct format for PostgreSQL to understand. Osm2pgsql can not do this for you, because it doesn't interpret the sql_type at all but just gives the data "as is" to the database and hopes that the database can interpret it according to the type you defined in sql_type.

from osm2pgsql.

pnorman avatar pnorman commented on June 20, 2024

It sounds like the PBF vs XML bug was from the areas being covered being different, not anything to do with osm2pgsql. In that case, there is no bug on the osm2pgsql side and you will need to fix your code.

from osm2pgsql.

joto avatar joto commented on June 20, 2024

I have added some warning words to the manual about the use of sql_type. For most use cases using the json or jsonb type is probably a better solution, because osm2pgsql will do the conversion automatically for you.

from osm2pgsql.

MathiasGroebe avatar MathiasGroebe commented on June 20, 2024

Thanks for your help and explanations!

Here is my adjusted configuration

print('osm2pgsql version: ' .. osm2pgsql.version)

-- Variables
local tables = {}
local import_schema = 'osm' -- Defines the import schema
local epsg_code = 32633 -- Defines the projection
local w2r = {}

-- Table defenitions

tables.traffic = osm2pgsql.define_table({
    name = 'traffic',
    schema = import_schema,
    ids = {
        type = 'way',
        id_column = 'way_id'
    },
    columns = {{
        column = 'fid',
        sql_type = 'serial',
        create_only = true
    }, {
        column = 'highway',
        type = 'text'
    }, {
        column = 'name',
        type = 'text'
    }, {
        column = 'osmc_symbols', 
        type = 'jsonb' 
    },{ 
        column = 'rel_ids', 
        sql_type = 'int8[]' 
    }, {
        column = 'geom',
        type = 'linestring',
        projection = epsg_code
    }},
    indexes = {{
        column = 'geom',
        method = 'gist'
    }}
})


function osm2pgsql.select_relation_members(relation)
    -- Only interested in relations with type=route
    if relation.tags.type == 'route' and relation.tags.route == 'hiking' then
        return { ways = osm2pgsql.way_member_ids(relation) }
    end
end


-- Function which fill the tables

function osm2pgsql.process_node(object)

end

function osm2pgsql.process_way(object)

    if object.tags.highway or object.tags.railway then
        row = {
            name = object.tags.name,
            highway = object.tags.highway,
            geom = object:as_multilinestring()
        }

        local d = w2r[object.id]
        if d then
            local refs = {}
            local ids = {}
            for rel_id, rel_ref in pairs(d) do
                refs[#refs + 1] = rel_ref
                ids[#ids + 1] = rel_id
            end
            table.sort(refs)
            table.sort(ids)
            row.osmc_symbols = refs
            row.rel_ids = '{' .. table.concat(ids, ',') .. '}'
        end

        tables.traffic:insert(row)
    end

end

function osm2pgsql.process_relation(object)

    local type = object:grab_tag('type')

    if type == 'route' and object.tags.route == 'hiking' then
        for _, member in ipairs(object.members) do
            if member.type == 'w' then
                if not w2r[member.ref] then
                    w2r[member.ref] = {}
                end
                w2r[member.ref][object.id] = object.tags['osmc:symbol']
            end
        end
    end

end

from osm2pgsql.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.