In the OSM data structures, only Nodes carry a location. Ways and Relations reference a set of nodes - so their location exists implicitly through the location of their nodes.

This is arguably a usefully compact and DRY usage of data structures, but when you're actually trying to do something useful with OSM data, you'll need to get the geometry for each Way / Relation.

Currently osmio mostly leaves this up to the client code to implement, but I think it'd make sense to provide osmio users with a blessed way to do this.

The algorithm tends to look something like this:

1. First, store each Node's location, indexed by the Node's node_id
2. Then, for each Way, take it's node_ids, and use them to look up the stored locations, copying them onto the Way.

This is conceptually simple, but becomes non-trivial when dealing with a range of file sizes. For very small files you probably want to just store everything in some kind of hashtable in RAM, but for very large files (e.g a continent.pbf or planet.pbf) this could require many GB. If you can afford it, you still might want it all in RAM, but for resource constrained environments, there should be a way to output the node locations to file. (nodestore seems to exist to this end).

I don't know exactly what the interface would look like, but I'd propose adding a couple of different strategies and documentation for hydrating Way and Relation locations into osmio.

Hi! Firstly, thanks for the crate!

Currently in osmio, Lat/Lon is represented by a f32, but sometimes this is not precise enough for osm data, which is spec'd to 7 decimal places.

from https://wiki.openstreetmap.org/wiki/Node:

decimal number [...] with 7 decimal places
...
Do not use IEEE 32-bit floating point data type since it is limited to about 5 decimal places for the highest longitude.

My understanding is that the internal representation is a scaled 32bit integer

A 32-bit method used by the Rails port is to use an integer (by multiplying each coordinate in degrees by 1E7 and rounding it

So, as I understand it, 1.234567890 degrees would be stored as 12345678i32

Looks like libosmium also uses this representation internally.

My intuition is that the lon/lat types you've currently defined should become structs backed by an i32, with some in/out methods, including something like lat.to_f64() (but probably not a lat.to_f32()).

And, just to be clear, I expect that this would break a lot of consumers code, who are already coding around these things being f32's. For a lot of end user applications, the precision probably doesn't make a difference, but I think for a parsing library like this (and libosmium) it makes a lot of sense to respect the upstream precision model.

The 7 rounded decimal places for coordinates in degrees define the worst error of longitude to a maximum of ±5.56595 millimeters on the Earth equator, i.e. it allows building maps with centimetric precision. With only 5 decimal places, the precision of map data would be only metric [(within a meter)], causing severe changes of shapes for important objects like buildings, or many zigzags or angular artefacts on roads.

What do you think @amandasaurus?

If you're interested in the concept, but don't have time for the work, this is something I'd be interested in working on.

Thanks for this great piece of software.

I'm currently struggling with converting StringOSMObj to an ArcOSMObj. My intuition is that it shouldn't be a problem, as I already was able to stream objects from an PBFReader to an XMLWriter. And of course there is the OSMObj trait.

My problem boils down to this: I want to write a test for a function show_id that in production only processes ArcOSMObj from an PBFReader. However for the test I'd like to be able to write xml, as it is more convenient. I have no problem with putting the trait OSMObj into show_id's type signature instead of ArcOSMObj.

However this doesn't compile:

use osmio::{OSMObj, ObjId};

fn show_id(obj: dyn OSMObj) -> ObjId {
    obj.id()
}

fn main() {
    println!("Hello, world!");
}

#[cfg(test)]
mod tests {
    use super::*;
    use osmio::xml::XMLReader;
    use osmio::OSMReader;

    #[test]
    fn test_show_version() {
        let xml = r#"
          <?xml version="1.0" encoding="utf-8"?>
          <osm version="0.6">
            <node id="1" lat="0.0" lon="0.0"/>
          </osm>
        "#;
        let reader = XMLReader::new(xml.as_bytes());
        for obj in reader.objects() {
            assert_eq!(1, show_id(obj));
        }
    }
}

$ rustc --version
rustc 1.55.0

$ cargo test
   Compiling cast_test v0.1.0 (/home/kerstin/fh/cityvis/camps/osm-history/cast_test)
error[E0038]: the trait `OSMObj` cannot be made into an object
   --> src/main.rs:3:17
    |
3   | fn show_id(obj: dyn OSMObj) -> ObjId {
    |                 ^^^^^^^^^^ `OSMObj` cannot be made into an object
    |
note: for a trait to be "object safe" it needs to allow building a vtable to allow the call to be resolvable dynamically; for more information visit <https://doc.rust-lang.org/reference/items/traits.html#object-safety>
   --> /home/kerstin/.cargo/registry/src/github.com-1ecc6299db9ec823/osmio-0.6.0/src/lib.rs:285:23
    |
285 | pub trait OSMObjBase: PartialEq + Debug + Clone {
    |                       ^^^^^^^^^ the trait cannot be made into an object because it uses `Self` as a type parameter

For more information about this error, try `rustc --explain E0038`.
error: could not compile `cast_test` due to previous error

Any ideas on how to achieve this?

e.g. the default overpass-turbo script:

/*
This is an example Overpass query.
Try it out by pressing the Run button above!
You can find more examples with the Load tool.
*/
node
  [amenity=drinking_water]
  ({{bbox}});
out;

1. click run
2. click export
3. then download/copy as raw OSM data

You'll get a file like this (though unzipped):
overpass-export-bom.osm.gz

When I try to process it, the osmio parser explodes with:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { pos: 1:1, kind: Syntax("Unexpected characters outside the root element: \u{feff}") }', /Users/mkirk/src/georust/osmio/src/xml/mod.rs:65:25
stack backtrace:
   0: rust_begin_unwind
             at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/panicking.rs:92:14
   2: core::option::expect_none_failed
             at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:1329:5
   3: core::result::Result<T,E>::unwrap
             at /Users/mkirk/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/result.rs:1037:23
   4: <osmio::xml::XMLReader<R> as osmio::OSMReader>::next
             at /Users/mkirk/src/georust/osmio/src/xml/mod.rs:65:22
   5: <osmio::OSMObjectIterator<R> as core::iter::traits::iterator::Iterator>::next
             at /Users/mkirk/src/georust/osmio/src/lib.rs:505:9
   6: osm2fgb::convert_xml
             at ./src/main.rs:63:20
   7: osm2fgb::main
             at ./src/main.rs:46:5
   8: core::ops::function::FnOnce::call_once
             at /Users/mkirk/.rustup/toolchains/stable-x86_64-apple-darwin/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5

If I open in vim, run :set nobomb and save it as: overpass-export-nobom.osm.gz, then osmio can successfully process the input.