Git Product home page Git Product logo

openlr-dereferencer-python's Introduction

openlr-dereferencer

License Coverage Status PyPI - Python Version PyPI Documentation

This is a Python package for decoding OpenLR™ location references on target maps.

It implements Chapter G – Decoder in the OpenLR whitepaper, except "Step 1 – decode physical data".

Its purpose is to give insights into the map-matching process.

Dependencies

  • Python ≥ 3.6
  • geographiclib (PyPi package)
  • shapely (PyPi package)
  • openlr (PyPi package; implements the decoder's step 1 from the whitepaper)
  • For unittests: SpatiaLite

State

  • Decoding line locations
  • Decoding 'point along line' locations
  • Decoding 'POI with access point' locations

Structure

This package is divided into the following submodules:

maps

Contains an abstract map class, which you may want to implement for your target map.

maps.wgs84 provides methods for reckoning with WGS84 coordinates.

example_sqlite_map

Implements the abstract map class for the example map format used in the unittests and examples

decoding

The actual logic for matching references onto a map.

This includes finding candidate lines and scoring them, and assembling a dereferenced location.

observer

Contains the observer class, allowing you to hook onto certain events in the decoding process.

Installation

This project is available on PyPi:

pip3 install openlr-dereferencer

Usage

The decode(reference, mapreader) function will take a location reference and return map objects.

Usage Example

First, implement the MapReader class for your map. The example_sqlite_map module is an implementation you may look at.

Second, construct a location reference. For instance, parse an OpenLR line location string:

from openlr import binary_decode
reference = binary_decode("CwmG8yVjzCq0Dfzr/gErRRs=")

Third, decode the reference on an instance of your map reader class:

from openlr_dereferencer import decode
real_location = decode(reference, mapreader)

real_location.lines # <- A list of map objects

Configuration

The decode function takes in an optional Config object containing decoding settings. With it, you may specify the decoder's behaviour:

from openlr_dereferencer import decode, Config

my_config = Config(
    geo_weight = 0.66,
    frc_height = 0.17,
    fow_height = 0.17,
    bear_weight = 0.0
)

decode(reference, mapreader, config=my_config)

Logging

openlr-dereferencer logs all mapmatching decisions using the standard library logging module.

Use it to turn on debugging:

import logging

logging.basicConfig(level=logging.DEBUG)

Observing

Via implementing the Observer interface, you can hook onto certain events happening while decoding, and inspect the situation:

from openlr_dereferencer import DecoderObserver, SimpleObserver

# Look at SimpleObserver for an example implementation
my_observer = SimpleObserver()
decode(reference, mapreader, observer=my_observer)

Development environment

Firstly create a Python virtual environment for the project.

python3 -m venv .venv

Activate the virtual environment.

source .venv/bin/activate

Install the dependency packages into the virtual environment.

pip install openlr geographiclib shapely

You may need to install the spatialite module for sqlite if this is not already present.

sudo apt install libsqlite3-mod-spatialite

To run the decoding tests.

python3 -m unittest

More Documentation

You are welcomed to read the generated API documentation at https://openlr-dereferencer-python.readthedocs.io.

openlr-dereferencer-python's People

Contributors

aksakalli avatar claudiadieckmann-tomtom avatar gotalite avatar mohammadhassanzahraee-tomtom avatar nadjaseiferth-tomtom avatar rijnb avatar sarahsharifian-tomtom avatar sarahsharifianrazavi-tomtom avatar stephencurran-tomtom avatar ugniljoz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

openlr-dereferencer-python's Issues

Small openlr's dnp(sugesstion)

Some openlr is small, the actual distance about 10 meters( and the total lines about 10 meters too ), but the smallest dnp is 29m(58.6/2).
So when I use the maximum accepted length, the shortest path deviation is too large, then the result is error.

I adjuest some code:
(in candidates.py, about line 99)

if current.dnp <= 30:
    point1 = [current[0], current[1]]
    point2 = [tail[0][0], tail[0][1]]
    curr_dnp = distance_pp(point1, point2)    # new function, count actual distance
else:
    curr_dnp = current.dnp

maxlen = (1 + MAX_DNP_DEVIATION) * curr_dnp

Other case.
Some openlr is small, the actual distance is about 10 meters, but the total lines is about 5 meters.
So the shortest path deviation is too large, then trying other candidate.

I adjuest some code:
(in candidates.py, about line 125)

If path does not match DNP, continue with the next candidate pair

if deviation > MAX_DNP_DEVIATION and current.dnp > 30:
debug(
"Shortest path deviation is too large, trying next candidate")
continue

But the problem is still unresolved, sometimes is OK, sometimes is wrong.
I don't know what to do.

Maybe score_bear have some error

The code in scoring.py.

def score_bearing(wanted: LocationReferencePoint, candidate: Line, is_last_lrp: bool) -> float:
"""Scores the difference between expected and actual bearing angle.

A difference of 0° will result in a 1.0 score, while 180° will cause a score of 0.0."""
point1 = candidate.start_node.coordinates
point2 = get_bearing_point(candidate)
bear = degrees(bearing(point1, point2))
expected_bearing = BEAR_MULTIPLIER * wanted.bear
if is_last_lrp:
    return score_angle_difference(expected_bearing, bear - 180)
return score_angle_difference(expected_bearing, bear)

If the line is last lrp, the point1's coordinates is start_node,maybe is error, the right is end_node.

another quertion.
why use the 'BEAR_MULTIPLIER'

about 'LRDecodeError'

Hi!
At the 'candidates.py', when the openlr didn't decode location references, the program will exit.
But sometimes should have some infomation.

I add some code at the end of the 'candidates.py'.
try:
raise LRDecodeError()
except LRDecodeError:
print('no candidates links.')

or change class LRDecodeError.

The match have more sql

Hi:
I followed a match process. One openlr(about 322 meters), match 3 links. That has about 645 sql total time is about 2 seconds.(I had change the database to postgis)

Then I analyzed these sql.
2:buffer the start or end point;
358: select link's frc, fow, startnode, endnode, length, node's node_id, coord
285: select link's numpoints, then query each node's coord.

2:these buffer sql are must;
285:select each node, I don't known the reason; maybe count length, bearing; but they can record information as a field, then can reduce queries.
358: about frc, fow, startnode, endnode, length... Is it possible to query all links and nodes, then as a data structure or a dictionary, then can reduce queries database.(we keep the current match program, then add another way for large data)

Because I use the match for TMC, that have 100 thousand openlrs in a city, that about 5 thousand openlrs will changed every 2 minutes, then we must quickly.(I can recode openlr math the links relation to solve this problem, but I think this can improve the efficiency of our program.)

I don't know if my description is clear and reasonable.

Thanks.

via point

Hi, guys!

Is the via point useful? Because it is not the best path that shortest path between first node and last node.
The openlr has more than 2 points(include via point).

Thanks.

Encoding

Hi guys,

Do you plan to implement an encoder as well as the decoder already implemented?

If not, any pointers to high quality open source encoding implementations?

about SEARCH_RADIUS

Sometimes the map coordinate is WGS84(epsg:4326).

The code in ./decoding/init.py (about line 23)
SEARCH_RADIUS = 100

So when I use the function 'find_lines_close_to', that will select more rows, then the program is slowly, but sometimes the radius is the distance(meters) at elsewhere.

Is it necessary to set two parameters?

Now, when I use the function 'find_lines_close_to'(my map data, epsg:4326), I set radius / 100000, I use radius at elsewhere.

About p_off and n_off

Hi! guy!
May be the p_off and n_off is offset of the total path, not the dnp.
Because they are always different.

In the 'tools.py'

def build_line_location(lines: List[Line],
reference: LineLocationRef) -> LineLocation:
"""Builds a LineLocation object from the location reference path and the offset values.

The result will be a trimmed list of Line objects, with minimized offset values"""
p_off = reference.poffs * reference.points[0].dnp
print(reference.noffs)
print(reference.points[-2].dnp)
n_off = reference.noffs * reference.points[-2].dnp
print(n_off)
adjusted_lines, p_off, n_off = remove_unnecessary_lines(
    lines, p_off, n_off)
print(n_off)
return LineLocation(adjusted_lines, p_off, n_off)

About the speed

Hi, guys!
Now, It spend about 2 second when I match one openlr.
I have about 40 thousand openlrs, it will spend long time.
How can I rasie the speed.

Others:
I want to change the database from sqlite to postgis, then I will make a plugin into qgis for openlr references on target maps.

Clarify purpose and status of this repository

IMPORTANT PLEASE READ: NO RESPONSE WILL RESULT IN REPO ARCHIVAL AND FINALLY DELETION

Dear maintainer,

One important topic in our TomTom open source engineering community is to make our public organization on GitHub, as well as the repositories hosted on it, more welcoming to contributors. That process starts which making it easy for other to understand the purpose of a codebase is and how people can contribute.

In order to do so we want to learn which repo's are still actively maintained and important to keep available to the open source community and which are not. We have therefore decided to start a clean-up proces which will be done as follows:

  1. Please respond to this issue within 30 days and provide us with the necessary information requested below.
  2. If there is no reply to this issue after these 30 days, the repository will be archived
  3. After your repo has been archived you have an additional 30 days to respond and provide information and discuss with us whether you want to revert, or keep archived status.
  4. IMPORTANT After the above mentioned additional 30 days no response is given the repository will be removed from the TomTom-International organization or will be DELETED

Could you please be so kind to provide us with the following information

Contact details

Repo owner/primary contact: [please provide email address]

Repository details

Describe the purpose/objective of this codebase in 1-5 sentences:
[enter description here]

Please select how this repository contributes to the following (thick all that apply):

  • Internal use: The code is used inside TomTom and there is a vested interest in long term maintenance
  • Employer branding: Project gives an insight into the scale and problem-solving of TomTom tech.
  • Hiring: contributors and users of the project could be attractive job candidates
  • Learning: TomTom employees can collaborate with and learn from external experts on the topic
  • Enhancement: the project could gain new functionality and refinement by external use and contribution
  • Product: The project is needed to make our tech or products integrate with other products or tech stacks
  • License Compliance
  • Fork: Fork needed for for upstream contributions, long-term upstream collaboration or product / CI/CD pipeline dependency or other need. If this is the case you can move your fork to a new GitHub organization https://github.com/tomtom-forks. If you need access to this org reach out at the #ospo channel on slack with your TomTom Github user account.

Additional context information about the purpose of this codebase:
[enter addition info here]

Questions?

Please respond to this issue or drop a line to [email protected] mentioning this issue nr.
Or for TomTom'ers reach out on the #ospo slack channel.

Thanks so much for helping keep our public repositories up to date and welcoming to all contributors.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.