trep / opentrep Goto Github PK

When using queries containing UTF8 quote characters, such as for instance 'Thessaloniki “Macedonia” airport', some calls to the Xapian API trigger exceptions. When using the Web site, no answer is returned due to an internal error.

Proposed solution: remove any Unicode quote, accent, punctuation, thanks to the [http://github.com/trep/opentrep/blob/trunk/opentrep/basic/OTransliterator.hpp](OTranslitor class).

seems like some distances are not correctly computed. example:
CGI-STL should be close to 182 km, but is reported as 165 km
MAZ-SJU should be close to 122 km, but is reported as 112 km

STN redirects to an Australian place (http://search-travel.org/search/?q=stn), namely Coolawanyah, Western Australia, Australia.
It should redirect to Standsted, UK.

When someone searches for common types and/or countries, there is first a full text matching process, retrieving a maximum number of POR, for instance currently 30. Out of that limited list, the one with the highest PageRank value is then returned. That works well for cities (e.g., "paris airport"), but not so well for countries and/or travel types (e.g., "france airport", "uk railway", or just "usa" or "airport").

So, when indexing on the country level and/or travel type, it would be good to weigh with the PageRank value of the POR.

As the Pituffik Air Base is not an airport, the "airport" keyword is currently not used for the indexing. It should.
Hence, the http://www.search-airports.com/search/?q=pituffik+airport request should yield Pituffik Air Base, not Dundas Airport.

Hello.

I'm trying to follow your docs and build opentrep on Fedora rawhide (future 35) where Python 3.10 is the main Python. It seems to me that CMake has some troubles detecting the Python version:

-- The C compiler identification is GNU 11.1.1
-- The CXX compiler identification is GNU 11.1.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Requires Git without specifying any version
-- Current Git revision name: 15ef88fa8d7ec359134d64c2b1857d6264695d58 master
-- Requires Python with version 3.6; however just Python3 is considered here
CMake Error at config/FindPackageHandleStandardArgs.cmake:164 (message):
  Could NOT find Python3 (missing: Python3_LIBRARIES Python3_INCLUDE_DIRS
  Development)
Call Stack (most recent call first):
  config/FindPackageHandleStandardArgs.cmake:445 (_FPHSA_FAILURE_MESSAGE)
  config/FindPython/Support.cmake:2400 (find_package_handle_standard_args)
  config/FindPython3.cmake:311 (include)
  config/project_config_embeddable.cmake:501 (find_package)
  config/project_config_embeddable.cmake:342 (get_python)
  CMakeLists.txt:69 (get_external_libs)

With the following searches, Ankara does not appear:

• ank
• ankara
• ankara city

The only working request is ank city.

world radius should be 12742

Support for UN/LOCODE codes has recently been added (see 7b0e503 for more details).

The Location structure supports currently a single UN/LOCODE.
However, a few POR (points of reference) have several UN/LOCODE codes. For instance, Atlantic City is such a case (referred to as USACX and USAIY). Currently, only USAIY produces the good result on Search Travel web site, whereas USACX should to.

From a C++ source code perspective, it means handling UNLOCode_T structures the same way as CityDetails ones. Indeed, a given travel-related location may serve several distinct cities; the same way, a given location may have several UN/LOCODE codes.

Hi Denis,

I'm getting a ton of absl and protobuf errors. Before blaming somebody else than myself I would like to ask: which version does trep need?

The search algorithm is currently made of a single bloc. It should become modularised, with the full-text search just one of the features.

The search algorithm could be as follows:

• Perform the (Xapian-based) full-text search, as currently done. That full-text search returns:
  • a list of main/idempotent matches, i.e., matches with the same match percentage, usually 100%, for instance when the search is done for a given country such as 'fr';
  • potentially a list of alternative matches, i.e., matches with lower match percentages.
  • All the matches are returned with their full details:
    • Type (airport, city, hotel, train station, etc.).
    • Geographical coordinates.
    • Administrative hierarchy (city, country, continent).
• For every main and alternative match:
  • Assign an overall mark, initialised with the 1 times the matching percentage: 100% => 1.
  • Perform a geographical search on the airports nearby (say, 50 km around). Those airports are sorted according to their importance (https://github.com/opentraveldata/optd/blob/trunk/refdata/geonames/data/por/data/airports_pageranked_2010_06.txt).
  • For every found airport, multiply its importance (page-rank) by the overall mark of the corresponding match.
• Choose a match

Propose close mattching aternatives.
For instance, Asuncion should return ASU.

The Python extension pyopentrep.py script does not contain any Xapian indexing function yet. Hence, Xapian indexing still needs to be performed by the C++ binary (opentrep-indexer).

A Xapian indexing function should therefore be added to pyopentrep.py.

Currently, the opentrep-dbmgr utility reports the progress updates. For instance:

$ opentrep-dbmgr -t mysql
[...]
opentrep> fill_from_por_file 
Indexing the POR file and filling in the SQL database may take a few minutes on some architectures (and a few seconds on fastest ones)...
Number of records inserted into the DB: 1000
[...]
Number of records inserted into the DB: 20000
20039 entries have been processed
opentrep> quit

The actual implementation of that reporting occurs in the DBManager command.
For the indexing process, most probably that actual implementation should occur in the PORParserHelper class.

When seaching for KHI, the city of Jakarta, Indonesia (ID), is returned, whereas KHI is the code of Karachi, Pakistan (PK).

The cause is that:

1. The Hakka Chinese translation of Jakarta is "Ngâ-kâ-tha̍t Sú-tû Thi̍t-khî", including the "khî" keyword, which is therefore part of the indexing keywords for Jakarta city.
2. The respective PageRank values of KHI and JKT/CGK are 8% and 36.5%.
   Hence, KHI will match almost exactly (99.99%) with both Karachi and Jakarta. With the PageRank values, Jakarta comes out with an overall matching weight of ~36% (compared to the ~8% of Karachi).