A bicycle path mapper

• status: IN PROGRESS
• this issue is linked to [C] BACKEND SERVER on #4 User sequence diagram

Environments

• Python 3
• Flask docker image
• OpenCV 3^
• Kakao Map API

Drafts

• get_image(coordinates, perspective)
  • description
    • get a map image from Kakao Map API
  • output
    • image in numpy matrix object
• get_graph(Image, road_level)
  • description
    • get interpreted graph
  • output
    • class Graph_map

Project Design

• written in 2020. 10. 16. (Fri)

Objectives

• "map bicycle lanes & suggest the fastest bicycle or electric kickboard path"
• In order to achieve it, these modules should be developed (see #1 Modules) :
  • a ROS package for traffic sign recognition
  • a bicycle path mapper
  • traffic sign estimation system
  • shortest path suggestion

Definitions

Devices to use

An edge camera

• While riding a bicycle or an electric kickboard, assume that every vehicle has front-view camera (or a smartphone) which can detect pedestrian traffic lights. Let's call it an edge camera.
• An android smartphone would work as an edge device at the beginning.
• The camera is turned off in general.
• The camera starts to record the view when the vehicle is near intersection based on GPS coordinate.
• Send a frame, the orientation, the compass data, time, and the GPS coordinate per a second until an edge server tells it to stop capturing.

An edge server

• An edge server is a type of edge device that provides an entry point into a network.
• A Nvidia Jetson nano would work as an edge device at the beginning.
• The edge server receives every data from edge cameras.
• It processes the images to a boolean value if the pedestrian traffic light is green or not.
• When the boolean value changed from red to green, it means the signal edge arises. Send a stop signal to an edge camera in every signal rising edge.

A backend server

• A backend server is a device to understand the data.
• It does machine learning, and predicts when the traffic light changes.
• Based on the machine learning model, suggest the shortest path.
• The server also maps bicycle lanes using Kakao Map API.

Modules

a ROS package for traffic sign recognition

• ROS is applied to make a stable data publisher/subscriber design, and it also provisions multi-platform device services.
• An edge camera sends images without interpreting them. An edge server interprets them with image processing. This design will help edge cameras to save their battery & will help to process the images faster using CUDA.
• A backend server shouldn't know of this package.
• Used frameworks & libraries:
  • An edge camera
    • ROS1 (Melodic) as a publisher
  • An edge server
    • ROS1 (Melodic) as a subscriber (roscore)
    • OpenCV for image processing

a mobile application for traffic sign recognition

• an alternative way to get images using smartphones, which also sends images without interpreting them.
• Used frameworks & libraries:
  • An edge camera
    • Nativescript-Vue
    • Android API level ^21
  • An edge server
    • RESTful web service
    • OpenCV for image processing

a bicycle path mapper

• A bicycle path mapper maps intersections & roads to a graph using Kakao Map API.
• A backend server does image processing because the API doesn't provide conceptual mapping data or a graph.
• Used frameworks & libraries:
  • A backend server
    • Kakao Map API
    • OpenCV for image processing
• When an edge camera sends its GPS coordinates via an edge server, a bicycle path mapper returns intersections & roads neighborhood.

traffic sign estimation system

• Every single data of an edge camera sends images to a backend server via an edge server. This will help machine learning model more accurate.
• Hope to guess the accurate timing of traffic lights only within hundreds of images. It is possible because a single ride of a bicycle captures not only a single status of a traffic light, but also the changing from red to green.
• Used frameworks & libraries:
  • A backend server
    • PyTorch
    • numpy
    • pandas
    • keras
    • postgreSQL

shortest path suggestion

• A backend server suggests an edge camera where to go by based on the estimated timing of traffic lights & current position of the bicycle path map.
• Assume that the delay of traffic lights of intersection is much larger than traffic jam. It's very down to Earth assumption because we can hardly see a traffic jam on bicycle lane!
• Selecting the best shortest path algorithm will be the issue.

User sequence diagram

Version logs

• Ver 1.0.0. (30, Oct, 2020): Initial user sequence diagram

A mobile application for traffic sign recognition

• status: IN PROGRESS
• this issue is linked to [A] EDGE CAMERA APP on #4 User sequence diagram

Environments

• on developing workstation:
  • OpenCV
  • NativeScript-Vue
  • Android
• on android smartphones (as an edge camera):
  • Nativescript-preview / Nativescript-playground
• on an edge server:
  • Python3
  • Flask

Submodules

on an edge camera

• android app for image sender using Vue & NativeScript
• roslaunch package in the android app to act like a ROS node. (planning)

on an edge server

• Flask app which supports RESTful API
• ROS melodic to work as ROS subscriber

Requirements

Android app

• required hardware seonsors:
  • GPS (with nativescript-geolocation)
  • Accelerometer (with native android api)
  • Camera (with native android api)

Reference

• https://github.com/NickIliev/NativeScript-cameraApp-poc/blob/master/app/main-page.js

Due to unsupporting static DISTRICT-overlay map API on Kakao map API, should ask permission to use them.

정적 이미지 맵이 카카오 API에서 제공되지 않으나 본 소스코드에 분석용으로 사용하고 있습니다. 저작권 이슈 해결이 필요합니다.

• such as: