Create GET /services like it is specified inside https://github.com/IGNF/road2/blob/develop/documentation/apis/administration/1.0.0/api.yaml.

• Code it (see next paragrah)
• Update doc (ex. functionalities, changelog)
• Update tests (new classes if there are; rtest for this route)
• Run tests inside docker (rtest, ctest, utest, itest)

Code

• Add the route inside the router of admin/1.0.0 (no need of an interface I think... only if we want to add some parameters. But we could add this later to me)
• Create a function getServicesConfigurations() inside administrator.js to handle the request and to get all the information. Everything is in this._configuration of the administrator instance (this._configuration is the content of road2.json) :
  • Id is in the config of the admin (aka road2.json)
  • the rest is the content of the config file of the service (aka service.json)

Need

Translate all the documentation directory and readme.md in english.

Tasks

• Create a branch from develop
• Translate readme.md but keep the french file
• Translate all the documentation directory keeping the french part.
• Update the CI for the doc generation

Create a new .github/workflows/docker-publish.yml to add docker publication to CI.

Use https://github.com/marketplace/actions/build-and-push-docker-images to build and push docker image

Docker image must be push in Github Container Registry

• use docker GitHub Actions to build and push

Need

Code the /sources routes of the admin API.

Tasks

• Code it. See next paragraph.
• Update tests
  • utest/itest if modification of a classe
  • rtest if modification of an API
  • ctest if modification of a configuration
• Run tests
  • Inside the docker provided by the project : rtest, ctest, utest, itest
  • Manually if necessary
• Update the documentation
  • changelog.md
  • documentation directory

Code

• Create new request and response classes
• Modify the administrator and/or serviceManager to have a copy of each services administered. Those copies will be used to make some controls before sending requests to a service.
• Add the route inside the router
  • Controls about the conf are made inside the controller
• Create a function computeRequest() inside the administrateur to send the request to its serviceManager
• Modify the service class to handle this new request
  • Modify computeAdminRequest()
  • Create the function computeSourceRequest(), it will call the sourceManager to :
    • create a source
    • delete a source
    • modify a source
• Modify the sourceManager to execute the different unit actions
• Update the getCap

documentation/configuration/pgrouting/constraints.resource.example file describe expected values for constraint definition.

This should be converted to a json-schema (https://json-schema.org/understanding-json-schema/index.html) so we can check resource file for correct constraint.

A /heath route exists but it is a fake one in the sense you just know that the service is alive. Create a more sophisticated one :

• Be sure of the io. See #34
• Choose the protocol of communication between an administrator and its services: direct (sameProcess) or IPC (newProcess)
• Code it (see next paragrah)
• Update doc (ex. functionalities)
• Update tests

Code

• Create requests/healthRequest.js which is the request part of the interface
• Create responses/healthResponse.js which is the response part of the interface
• Update apis/admin/1.0.0/index.js to handle the request and its parameters. This is where an interface is created (see simple/1.0.0). The interface is send to the administrator. The answer will be generated from the interface received in response to the request interface.
• Update administrator/administrator.js to handle the interface. It will only send it to the serviceManager and wait for the answer
• Update service/serviceManager.js to handle the interface. It will send the interface to each administered service (call to administeredService.computeRequest(request)) and wait for the answers.
• Update each administered service to handle the interface (code computeRequest(request) for each one)
  • sameProcess : call to service.computeAdminRequest(request)
  • newProcess : send a message to the child process
• Update service/service.js to handle the interface for each protocol. It reads the request interface and answers the response interface.
  • code service.computeAdminRequest()
  • find a place to code message reception and call to service.computeAdminRequest()
  • code service.health() to call its source.health()
• Update each source to code source.health()

Need

Variables and function' name are in English. But comments, log messages and function' doc are not.

Tasks

• Translate comments
• Translate log message
• Translate function doc

The administration API specification was created but it must be updated :

• Update the specifications @lgrd
• Create a .md for use cases of this API @lgrd
• Review by @IGNF/road2

This spec concerns the management of services, sources and resources inside a Road2 server.

Create a new .github/workflows/linter.yml to add lint to CI.

Need

We need to know available services, and to manage them.

Tasks

• GET /services. See #43
• POST /services
• GET /services/{service}. See #49
• PATCH /services/{service}
• DELETE /services/{service}
• GET /services/{service}/restart. See #45

Add option to define resource for isochrone request:

• new radiobutton to use custom resource
• new lineedit to define resource
• use this new component to define used resource

We have some templates written in markdown but we could use the YAML form when it will be a stable feature of Github.

Tasks

• Wait for the stable version of issue form
• Translate issue templates for Road2
• Translate issue templates for r2gg
• Translate issue templates for Pgr

Il y a déjà de nombreux tests dans le projet. Mais il en manque un certain nombre pour faire le tour du code. Voici la liste des améliorations possibles :

Première étape

• Adapter les tests pour les lancer avec des données OSM
• Finir les tests d'intégration
• Modifier le CONTRIBUTING.md pour donner la procédure des tests

Deuxième étape

• Ajouter des tests sur les requêtes HTTPS (c'est possible mais pas lancés)
• Enrichir les tests de requête et de conf
• Ajouter des tests unitaires qui permettent de vérifier les fonctions des dépendances que l'on utilise
• Ajouter des tests pour voir les admin et service sont bien indépendants et que service peut bien être lancé tout seul
• Modifier world.js pour prendre en compte plusieurs services dans les tests de la config de l'admin
• Tester la partie services de la config admin

Need

The entire configuration is on a file-system. But it could be useful to have it inside an object storage.

Functionnalities

• Have the choice between a file and an object storage. The choice is made for the entire configuration.
• Be able to read the configuration inside an object storage

Tasks

Before coding

• Choose an object storage
• Specify the new IO of the CLI (ROAD2_CONF_FILE will still exist but there will be new ones, like ROAD2_CONF_TYPE = 'file' or 'object')
• Find a solution for all 'directory' attributes inside the service.conf. All the other attributes are object compatible. There are files and directory for OSRM et Valhalla sources but it can't be different. It has to be files and directory locally.

Code it

• Code it. See next paragraph

After or during coding

• Update tests
  • utest/itest if modification of a classe
  • rtest if modification of an API
  • ctest if modification of a configuration
• Run tests
  • Inside the docker provided by the project : rtest, ctest, utest, itest
  • Manually if necessary
• Update the documentation
  • changelog.md
  • documentation directory

Code

• Create a reader using promises/readFile for file, and use it everywhere in the code
• Create getGlobalConfiguration() inside road2.js to handle the different parameters of the CLI. All parameters has to be read via nconf. They could be via -- or inside the ENV.
• To complete...

• add myst_substitutions in documentation/conf.py for repository definition (repo_url, author, ...)
• update some .md files that refers to source code => use of repo_url

Feature souhaitée

Implémenter l'API OGC Route

Todo

• Se renseigner sur l'API et le format en prêtant attention à la feuille de route
• Définir le périmètre
• Analyser et lister les devs

Problème

La reprojection des isochrones ne fonctionnait pas.

Solution

Déjà fait, voir commit 538e971

Exemple de requête

http://localhost:8080/simple/1.0.0/isochrone?resource=bduni-idf-pgr&point=651293,6862531&costType=time&costValue=100&crs=EPSG:2154

Is your feature request related to a problem? Please describe.
When I build the documentation locally, it doesn't work. I think it's because my python version is too old (2.7.x).

Describe the solution you'd like

• Create or modifiy a Dockerfile for the doc building (maybe with python:3.11.x because it works for me)
• Update the documentation on this feature (./documentation/developers/documentation.md)

Describe alternatives you've considered
None

Additional context
None

Need

Code the /resources routes of the admin API.

Tasks

• Code it. See next paragraph.
• Update tests
  • utest/itest if modification of a classe
  • rtest if modification of an API
  • ctest if modification of a configuration
• Run tests
  • Inside the docker provided by the project : rtest, ctest, utest, itest
  • Manually if necessary
• Update the documentation
  • changelog.md
  • documentation directory

Code

• Create new request and response classes
• Modify the administrator and/or serviceManager to have a copy of each services administered. Those copies will be used to make some controls before sending requests to a service.
• Add the route inside the router
  • Controls about the conf are made inside the controller
• Create a function computeRequest() inside the administrateur to send the request to its serviceManager
• Modify the service class to handle this new request
  • Modify computeAdminRequest()
  • Create the function computeResourceRequest(), it will call the resourceManager to :
    • create a resource
    • delete a resource
    • modify a resource
• Modify the resourceManager to execute the different unit actions
• Update the getCap

Need

/configuration route is not implemented in the administration API. This route allow an administrator to get the configuration of road2 administrator and to modify it.

Tasks

• Delete onStart feature. See #48
• Allow verification and creation of an administrator without any services. See #51
• Allow verification and creation and publication of a service without any resources and/or sources. See #47
• Code GET /configuration. See #59
• Code PATCH /configuration
• Update doc (ex. functionalities, changelog)
• Update tests (new classes if there are; rtest for this route)
• Run tests inside docker (rtest, ctest, utest, itest)

We need to define pre-commit rules to add some linters.

I'm not familiar with pre-commit with node.js.

Maybe we could use Husky. See this article https://levelup.gitconnected.com/how-to-run-eslint-using-pre-commit-hook-25984fbce17e

ESLint configuration is outdated. We need to update current configuration.

For example we could use:

{
    "env": {
        "commonjs": true,
        "es2021": true,
        "node": true
    },
    "extends": "eslint:recommended",
    "overrides": [
    ],
    "parserOptions": {
        "ecmaVersion": 2021,
        "sourceType": "script"
    },
    "parser": "espree",
    "rules": {
        "curly": "error",
        "eqeqeq": "error",        
        "no-console": "warn",
        "no-else-return": "error",
        "no-lonely-if": "error",
        "no-unneeded-ternary": "error",
        "no-unused-expressions": "error",
        "no-unused-vars": ["error", { "args": "none" }],
        "no-useless-concat": "error",
        "no-useless-return": "error",
        "no-with": "error",
        "prefer-promise-reject-errors": "error",
        "strict": "error"
    }
}

Need

Today, Road2 can handle route and nearest requests. But OSRM can compute more than these. So, because we have the motor and the data, we could use all the functionalities of OSRM : table, match, trip and tile.

Tasks

• Update the API documentation
• Add table operation
• Add match operation
• Add trip operation
• Add tile operation (maybe in a debug part of Road2)

Situation

When a restart is asked by the admin, servers of the service are closed but the sources are not disconnected. So, some base connections are still there.

Fix

The disconnect functions exist on some classes but they are not used during the restart and in the service.

TODO

• Fix the code
• Update the changelog
• Test the fix manually

Need

This project and its friends don't have a contributing guide. The idea is to add such a guide.

Tasks

• Write a contributing.md
• Validation of the team
• Push to road2
• Push to r2gg
• Push to Pgr

Need

This project and its friends don't have a code of conduct. The idea is to add such a code based on this guide : https://opensource.guide/code-of-conduct/.

Tasks

• Write a code of conduct
• Validation of the team
• Push to road2
• Push to r2gg
• Push to Pgr

We need to check documentation for outdated procedures.

It will be easier to check when the documentation will be available as GitHub pages (waiting #15)

Note in issue description all files that should be updated:

• update test description to remove unavailable test (artest for example)
• remove documentation visualization with web-roa2 image

Documentation should be available on GitHub pages:

• add GitHub action workflow to generate documentation
• step1 : generate documentation with sphinx (waiting #14)
• step2 : deploy generated documentation on GitHub pages

Need

The function administrator.getServicesConfigurations() works and it is used inside the admin API. But sometimes, an emtpy answer is returned.

Tasks

• Fix it
• Run tests
  • inside docker (rtest)
  • outside docker : multiple requests to verifiy

Description

Dans le projet Road2 : Passer à OSRM 5.27.1 et NodeJS 18. Et donc dans le projet Route-graph-generator : python 3.11 sur debian 11 pour générer les données OSRM

Todo

• r2gg : passer sur debian 11 et python 3.11
• r2gg : générer les données
• road2 : passer sur nodeJS 18 et tester
• road2 : passer sur OSRM 5.27.1 et tester

Feature

L'API simple/1.0.0 renvoie des géométries en geojson ou polyline. Mais ces deux formats ne permettent pas de profiter de la reprojection. L'idée est de proposer le WKT pour qu'elle ait du sens.

Todo

• Lire les spécifications
• Analyser et lister les devs en délimitant le périmètre
• Créer un module dans le code et comparer les résultats/performances avec l'ancien
• Si performant, remplacer les usages de smartroutingSource.js
• Ajouter la valeur wkt dans la conf docker de road2 et r2gg
• Modifier les classes du dossier geometry pour utiliser ce nouveau module

It will be nice to have some test coverage from mocha and cucumber tests.

We could check Istanbul https://istanbul.js.org/.

It seems pretty forward to add coverage, we just need to add nyc before running test.

Need

Code the GET /services/{service}/projections/{projection} route of the admin API. This is its second part : return the projection configuration when it is loaded.

Tasks

• Be sure of the IO inside the API yaml
• Code it. See next paragraph.
• Update tests
  • utest/itest if modification/Creation of a class
  • rtest if modification of an API
• Run tests
  • Inside the docker provided by the project : rtest, ctest, utest, itest
  • Manually if necessary
• Update the documentation
  • changelog.md
  • documentation API

Code

• Update request and response classes if necessary
• Update the use of thoses classes
• Modify the projectionManager to store the configuration of each projection and return it when needed
• Update the service.computeProjectionRequest()

Integration test can't be fully run because unavailable code is used:

Error: Cannot find module '../../../../src/js/topology/topologyManager'
Require stack:
- /home/docker/app/test/integration/mocha/resources/integrationResourceManager.js

Tests must be updated to remove reference to unavailable code.

Some changes inside r2gg are very good but break links inside docker-compose uses.

The idea is to update the docker-compose file to have the link, so as to be able to code locally and see the changes inside docker.

Need

Clarify the use of 400/404 and add some 500 inside the doc.

Tasks

• Update the api.yaml
• Update the error status inside the existing routes

Need

See if there are memory issues.

Tasks

• Apply methods described here :
  • https://nodejs.org/en/docs/guides/diagnostics/memory
  • https://nodejs.org/en/docs/guides/diagnostics/memory/using-gc-traces
• Add some tests inside the project
• Update the doc to indicate how to test memory issues

Need

Code the /services/{service}/projections/{projection} route of the admin API. This is its first part : return a 200 if the projection exists, 404 in the other case.

Tasks

• Be sure of the IO inside the API yaml
• Code it. See next paragraph.
• Update tests
  • utest/itest if modification/Creation of a class
  • rtest if modification of an API
• Run tests
  • Inside the docker provided by the project : rtest, ctest, utest, itest
  • Manually if necessary
• Update the documentation
  • changelog.md
  • documentation API

Code

• Create new request and response classes
• Add the route inside the router
  • Controls about the the ids are made here
• Create a function computeRequest() inside the administrateur to send the request to its serviceManager
• Modify the service class to handle this new request
  • Modify computeAdminRequest()
  • Create the function computeProjectionRequest(), it will call the projectionManager.isProjectionLoaded(id)

Need

Actually, there is no restart route for a service. But it could be useful in different cases. For instance, now, it is easy to add and the other admin routes are not available. But in the future, if you have a problem with a specific route, a restart could be useful.

Tasks

• Update API doc to be sure of the IO (ex. GET or POST, responses if ok or not).
• Code it (see next paragrah)
• Update doc (ex. functionalities, changelog)
• Update tests (new classes if there are; rtest for this route)
• Test memory leaks? Tested manually, but an issue where created to investigate it in a more detailed way. See #57
• Run tests inside docker (rtest, ctest, utest, itest)

Code

• Add the route inside the router of the admin API
• Create a function inside the administrator to handle this restart
• Create a function inside the serviceManager to stop and load the service (this is the restart)
• Create the same function inside administeredService and its child classes (serviceProcess and serviceInsider) to stop the service. The function to load the service has already been written.
  • serviceInsider : call to this._serviceInstance.stopAllServers() and this._serviceInstance = null to let the garbage collector take it
  • serviceProcess : send a kill via this._serviceInstance.kill(), and this._serviceInstance = null to let the garbage collector take it
• Modify service.initIPC() to add a process.on('SIGKILL') which call this.stopAllServers()

Add documentation with sphinx:

• add conf.py to define sphinx option
• move all .md documentation files to the documentation directory
• create landing page documentation/index.md

Feature

Expose native OSRM and Valhalla APIs

Is your feature request related to a problem? Please describe.

I would like to develop a web-based application using the maplibre-gl framework, and use the IGN routing APIs. The routing plugins available use native OSRM and Valhalla APIs to work, and I'd want to use them out of the box (https://github.com/maplibre/maplibre-gl-directions and https://github.com/watergis/mapbox-gl-valhalla)

Describe the solution you'd like

I'd like to be able to use the native APIs of OSRM and Valhalla on a road2 service (specifically, the one served by IGN).
For example :

• osrm : {baseRoad2Url}/simple/1.0.0/osrm/driving/13.388860,52.517037;13.385983,52.496891?steps=true
• valhalla : {baseRoad2Url}/simple/1.0.0/valhalla/isochrone?json={"locations":[{"lat":-1.0825252603106321,"lon":35.47230915879868}],"costing":"pedestrian","contours":[{"time":10,"color":"0000ff"}],"polygons":true}

Other suggestion :

• osrm : {baseRoad2Url}/osrm/driving/13.388860,52.517037;13.385983,52.496891?steps=true
• valhalla : {baseRoad2Url}/valhalla/isochrone?json={"locations":[{"lat":-1.0825252603106321,"lon":35.47230915879868}],"costing":"pedestrian","contours":[{"time":10,"color":"0000ff"}],"polygons":true}

Tasks

• Create an openAPI JSON for OSRM (documentation/apis)
  • Route URL
  • GetCap URL to make the link between resources and profile/optimization
• Create an openAPI JSON for Valhalla (documentation/apis)
  • Iso URL
  • GetCap URL to make the link between resources and profile/optimization
• Modify the configuration to add the new APIs (docker/config/service.json)
• Create the directories src/js/apis/osrm/1.0.0 and src/js/apis/valhalla/1.0.0
• For each directory, code the init.js, update.js and index.js
  • For the init.js : create the getcap
  • For the update.js : update the getcap
  • For the index.js : create the router which handle requests.
• Add new properties inside the routeRequest class to handle osrm options
• Add new properties inside the isoRequest class to handle valhalla options
• Add an optional option object inside the computeRequest function to indicate that we want the raw response of the motor or create two new classes for each motor responses
• Add unit and integration tests for any new classes or properties of existing ones
• Add request tests (test/functional/request/cucumber/features)
• Modify the default configuration and run configuration functional tests
• Update documentation
• Update changelog

Create GET /services/:service like it is specified inside https://github.com/IGNF/road2/blob/develop/documentation/apis/administration/1.0.0/api.yaml.

• Code it (see next paragrah)
• Update doc (ex. functionalities, changelog)
• Update tests (new classes if there are; rtest for this route)
• Run tests inside docker (rtest, ctest, utest, itest)

Code

• Add the route inside the router of admin/1.0.0 (no need of an interface)
• Create a function getServiceConfiguration() inside administrator.js to handle the request and to get the information. Everything is in this._configuration of the administrator instance (this._configuration is the content of road2.json) :
  • Id is in the config of the admin (aka road2.json)
  • the rest is the content of the config file of the service (aka service.json)
• Refactor function getServicesConfigurations() to use the new getServiceConfiguration() to get configuration for each service

Need

We need to know available projections, and to manage them on a specific service.

Tasks

• Specify all the IO inside the API yaml
• GET /services/{service}/projections
• POST /services/{service}/projections
• GET /services/{service}/projections/{projection}. See #63 and #64
• DELETE /services/{service}/projections/{projection}

For a specific API (ex. simple/1.0.0), you have differents routes (ex. /route, /nearest, etc...).

But sometime, you may want to expose thoses routes with a different label (ex. /route -> /navigation). It is easy to hide this technical aspect behind a front but the getCap will not know it.

Create a new .github/workflows/tests.yml to add tests to CI.

Tests to add:

• unit tests npm run utest
• integration tests npm run itest

load tests will be added in a different issue.

smartrouting sources were used to have a backup service for isochrones to avoid using slow procedure with pgrouting since isochrones was not available with osrm.

This is not needed anymore with the support of valhalla.

We need to remove all smartrouting related code / test / documentation.

Is your feature request related to a problem? Please describe.
When I deploy a Road2 service I would like to be able to deploy also a swagger-ui to document available requests to user.

Describe the solution you'd like
We could use swagger-ui-express to add route for api documentation

Describe alternatives you've considered
Pointing user to the documentation for swagger-ui display with docker. But it's not very user-friendly.

The administration API exists in the project but it must be updated and fully implemented. I see different actions :

• Update api.yaml. See #34
• Code /heath route. See #35
• Code /configuration routes. See #37
• Code /services routes. See #66
• Code /sources routes. See #55
• Code /resources routes. See #56
• Code /projections routes. See #65