Git Product home page Git Product logo

rvi_core's Introduction

Copyright (C) 2014-2017 Jaguar Land Rover

This document is licensed under Creative Commons Attribution-ShareAlike 4.0 International.

Build Status

REMOTE VEHICLE INTERACTION (RVI)

This document gives a brief introduction to the codebase of the RVI project and explains the reasoning behind some of the technical choices.

ADDITIONAL DOCUMENTATION AND RESOURCES#

For a high level description, with an exhaustive master use-case walkthrough, please see the High Level Design document here (NOTE: HLD not updated to reflect RVI Core 0.5.1)

Packages are available for some distributions. See installation instructions for Ubuntu, Debian, and Raspbian.

For build instructions, please check the build instructions: Markdown | PDF

For configuration and launch instructions, please check the configuration documentation: Markdown | PDF

For instructions on how to create RVI Core certificates, keys and credentials, please check the certificates documentation: Markdown | PDF

For instructions on using the Services API, please check the services documentation: Markdown | PDF

For a detailed description of the RVI Core Peer-to-peer protocol, please check the rvi_protocol documentation: Markdown | PDF

Technical RVI disussions are held at the GENIVI project mailing list: GENIVI

GENIVI RIV Expert Group (RVI-EG) discussions are held at the members only list: GENIVI

PROJECT MISSION STATEMENT

The Remote Vehicle Interaction project will specify, design, plan and build a reference implementation of the infrastructure that drives next generation's connected vehicle services.

  • Specify
    Requirement specifications, test suites, integration tests.

  • Design
    High Level Description. Detailed Description. Use Cases.

  • Plan
    Roadmap. Milestones. Deliverables. Budgeting. Resource planning.

  • Build
    Implement. Document. Test. Demonstrate. Deploy for download.

  • Reference Implementation
    Provides a baseline and starting point for organizations' production-grade connected vehicle projects.

TECHNICAL SCOPE

RVI provides P2P based provisioning, authentication, authorization, discovery and invocation between services running inside and outside a vehicle.

  • P2P
    Internet connection not required for two peers to exchange services.

  • Provisioning
    Add, delete, and modify services and network nodes.

  • Authentication & Authorization
    Proves that a service is who it claims to be, and has the right to invoke another service.

  • Discovery
    RVI is considered a safe bet that can be used without unforeseen consequences.

  • Invocation
    RVI shall be able to function over transient and unreliable data channels, but also over a reliable in-vehicle LAN.

TECHNOLOGY CHOICES

The following chapters describe the technology choices made for the reference implementation. An overall goal was to avoid technology lock-in while easing adoption by allowing organizations to replace individual components with in-house developed variants using their own technology.

INTERCHANGEABILITY

The only fixed parts of the entire RVI project are the JSON-RPC protocol specifications used between the components themselves and their connected services.

GENIVI provides an RVI implementation as a starting point and a reference. The adopting organization is free to use, rewrite, or replace each component as they see fit. A typical example would be the Service Discovery component, which must often be extended to interact with organization-specific provisioning databases.

COMMUNICATION AGNOSTICISM

In a similar manner, RVI places no requirements or expectations on the communication protocol between two nodes, such as between a vehicle and a backend server, in an RVI network. An organization is free to integrate with any existing protocols, or develop new ones, as necessary.

The reference implementation provides an example of how to handle a classic client-server model. The RVI design, however, can easily handle such cases such as wakeup-SMS (used to get a vehicle to call in to a server), packet-based data, peer-to-peer networks without any dedicated servers, etc. The adopting organization can implement their own Data Link and Protocol components to handle communication link management and data encoding / decoding over any media (IP or non-IP).

ERLANG

Erlang was chosen to implementation the core components of the RVI system. Each component (see the HLD for details) run as an Erlang application inside a single Erlang node.

Several reasons exist for this somewhat unorthodox choice of implementation language:

  • Robustness
    Erlang has the ability to gracefully handle component crashes / restarts without availability degradation. This makes a deployment resilient against the occasional bug and malfunction. In a similar manner, redundant sites can be setup to handle catastrophic failures and geographically distributed deployments.

  • Tool availability
    There are a multitude of open source Erlang components available to handle SMS, GPIO, CAN buses, GPS, PPP links, and almost any protocol out there. Since Erlang was designed to handle the mobile communication requirements that is at the center of the connected vehicle, integrating with existing systems and protocols is often a straight-forward process.

  • Scalability
    The concurrent nature of an Erlang system sets the stage for horizontal scalaility by simply adding hosts to a deployment, allowing an organization to expand from a pilot fleet to a full fledged international deployment.

  • Carrier grade availability
    The robustness and scalability, in conjunction with the built-in Erlang feature of runtime code upgrades, is a part of Erlang's five-nines uptime design that is rapidly becoming a core requirement of the automotive industry.

  • Proven embedded system solution
    Erlang has been adapted to operate well in embedded environments with unreliable power, limited resources, and the need to integrate with a wide variety of hardware.

PYTHON

Python is used to implement all demonstrations, beginning with the HVAC demo available in the hvac_demo subdirectory.

By using Python for the demos, which is better known than Erlang, examples are given on how to write applications and services interfacing with the RVI system.

PERFORMANCE

Performance is not a goal of the RVI reference implementation. Instead, code readability and component interchangeability takes priority in order to ease design understanding and adoption.

One example is the use of JSON-RPC (over HTTP) to handle internal communication between components in a single Erlang node.

Using a traditional Erlang solution such as genserver, the overhead for internal transactions could be cut to a few percent in comparison with the current JSON-RPC implementation. That route, however, would force all components of the RVI system to be implemented in Erlang, thus severely limiting an organization's abilities to replace individual components with their own versions.

CODE STRATEGY

All code in the RVI reference implementation and its demonstrations are written with a minimum of complexity and "magic". Readability is paramount, even if it severely impacts performance and memory usage.

All components in the RVI are kept small and distinct, with a well-defined JSON-RPC external interface and simple call flows.

Only three external modules (lager, bert and exo) are used by the code, with two more (setup and edown) used for release and documentation management.

The reason for minimizing external module usage is to make the code comprehensible and minimize the time a developer has to traverse through obscure libraries trying to understand what a specific call flow actually does.

The entire reference implementation (as of the first alpha release) is 2800 lines of code, broken down into six standalone modules and one library of shared primitive functions.

JSON-RPC

JSON-RPC is used for all communication between components in an RVI system, and also to communicate with services connected to it. The ubiquity of JSON-RPC, and its close relationship with Java/JavaScript, provides maximum of freedom of technology choices when new components, services, and applications are developed.

rvi_core's People

Contributors

afan1 avatar amcgee7 avatar jochenschneider avatar magnusfeuer avatar mfeuer avatar mjone308 avatar pragma1ce avatar tjamison avatar uwiger avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

rvi_core's Issues

rvi_common:get_static_node(Service) should return best-matching result

When a serivce is matched against static nodes in rvi_common:get_static_node(Service),
the service should be prefix matched against all available static nodes, returning the static node with the longest prefix match against the provided service.

This allows for a static node to be configured for jlr.com/backend/sota that only handles software over the air, while another node can be configured for jlr.com/backend to handle all other types of requests.

RVI crashes on service invocation with malformed name

On master:

python rvi_call.py will_crash

Will trigger a crash in service_edge_rpc.erl:handle_call({ rvi, handle_local_message,...) when authorize_rpc:authorize_local_message() returns not_found instead of [ ok, Signature].

I think Ulf fixed this, but it may have been in another branch than master.

Invalid/wrong device cert in credential throws warning, connection proceeds

Connecting to a backend server via TLS, client displays this info in log:

2016-07-06 14:27:12.748 [warning] <0.139.0>@authorize_keys:process_cred_struct_:555 Wrong device_cert in cred
<<... (certificate) ...>>
<<... (certificate) ...>>
2016-07-06 14:27:12.749 [info] <0.116.0>@rvi_log:store_event:350 RVI_LOG: <<"conn:19-8fEt">>/1/<<"authorize">>/<<"cred stored 15eca337 Conn={<<"192.168.18.76">>,<<"9007">>}">>
2016-07-06 14:27:12.749 [info] <0.203.0>@dlink_tls_rpc:connection_authorized:739 dlink_tls:authorize(): Announcing local services: [] to remote "192.168.18.76":9007

Snipped certificate is indeed not the certificate presented for the handshake. However, the connection proceeds as if the credentials checked out OK.

Expected behavior is to terminate connection upon receiving invalid credentials.

Service directory functions in service_edge

A locally connected service should be able to request a list of local and remote services that are available for invocation.

Only services that are allowed to be invoked by the calling service should be returned.

The calling services should be able to provide a regexp pattern to filter the result against.

BT connection manager does not split up multi-command packages.

components/dlink_bt/src/bt_connection.erl:handle_info(..., {data, Data}) does not split up multiple JSON-RPC commands bundled into a single RFCOMM transmit and invokes dlink_bt_rpc:handle_socket() separately for each command.

This problem has been fixed in components/dlink_tcp/connection.erl, where a receive buffer is kept in the process state and complete commands are extracted and forwarded to dlink_tcp_rpc:handle_socket(), with residual data being returned to state to be prepended to the next received network packet.

The TCP solution should be copied to the BT connection code.

This bug fix must be back ported to release-0.4.0, creating release-0.4.1

Implement timeout notifications

A timeout is specified as a unix time stamp after which the given transaction should be cancelled.

When a transaction cannot forward the transaction toward its target service before the given time stamp, the transaction should be dropped and a timeout notification is sent back to the transaction-originating service.

This means that a service-generated transaction_id must be provided by the originating service.
The given service will have its registered timeout function invoked when a timeout occurs.

Deadlock.

Under load, the RVI deadlocks in several instances when Component A calls Component B, while Component B calls Component A.

An example is service_edge_rpc's handle_remote_message(), called by protocol_rpc, which can be blocked if, at the same time the service_edge_rpc is currently processing a handle_local_message gen_server call which is indirectly calling protocol. The to call chains will, in this event, be blocked.

The solution is to replace synchronous calls (gen_server:call()), with asynchronous notifications (gen_server:cast()) that do not wait for a return value before continuing operations.

WIll be fixed in the gen_server_fix feature branch and 0.3.2

Allow environment variables in node_service_prefix config.

The config file's node_service_prefix is used to specify the string to prefix all locally connected services with to make them globally unique.

Today this is a static string.

It should support integrated Linux environment variables, allowing the prefix to be dynamically set depending on what the environment variable is set to.

Now we have a statically configured value.
jlr.com/vin/saw813433993cv4/

Instead we should have:
jlr.com/vin/$env{VIN}/

Where $env{VIN} is replaced by the value of the VIN environment variable.

We should also support file content:
jlr.com/vin/$file{/etc/vin}
Where $file{/etc/vin} is replaced by the single line stored in /etc/vin

Deadlock in service_discovery

A deadlock occurs in service_discovery:unregister_remote_services_by_name() when it calls service_edge:unregister_remote_services while, at the same time, service_edge:handle_local_message() calls service_edge:resolve_local_service().

The solution isa to move the service_edge:unregister_remote_services() call from service_discovery to the invoking data_link_bert_rpc:handle_socket(...,unavailable,).

The same should be done for all service_discovery:unregister_single_remote_service_by_name_() and service_discovery:register_remote_services()

Don't unregister services for a disconnected remote node that have already been re-registered.

  1. RVI(client) connects to RVI(server)
  2. RVI(client) registers jlr.com/x/y
  3. RVI(client) power fails, and does not shut down connection to RVI(server)
  4. RVI(client) comes back up
  5. RVI(client) connects to RVI(server)
  6. RVI(client) registers jlr.com/x/y
  7. The old TCP/IP connection from Step 1 finally times out in RVI(server)
  8. RVI(server) deletes jlr.com/x/y from its tables since it was associated with the old connection.

Solution is to have service_discovery:unregister_remote_services_by_address() verify that each service to be unregistered has not been registered from another peer address. Those that have been registered should not be dropped.

rvi_0.4.0-1ubuntu1_amd64.deb missing runtime dependencies

dpkg -i rvi_0.4.0-1ubuntu1_amd64.deb

Selecting previously unselected package rvi.
(Reading database ... 11517 files and directories currently installed.)
Preparing to unpack rvi_0.4.0-1ubuntu1_amd64.deb ...
Unpacking rvi (0.4.0-1ubuntu1) ...
dpkg: dependency problems prevent configuration of rvi:
rvi depends on libbluetooth3 (>= 4.91); however:
Package libbluetooth3 is not installed.
rvi depends on bluez; however:
Package bluez is not installed.
rvi depends on erlang-base (= 1:16.b.3-dfsg-1ubuntu2.1); however:
Package erlang-base is not installed.
rvi depends on python-jsonrpclib (>= 0.1.3-1build1); however:
Package python-jsonrpclib is not installed.
rvi depends on python (>= 2.7.5-5ubuntu3); however:
Package python is not installed.

dpkg: error processing package rvi (--install):
dependency problems - leaving unconfigured
Processing triggers for ureadahead (0.100.0-16) ...
Errors were encountered while processing:
rvi

My assumption here is that when you tested the package you already had all the needed dependencies on your system. I tested the package in a minimal Ubuntu 14.04 system which has missing dependencies as shown above. We can add these to the debian/control file and they will get pulled in automatically when the package is installed.

Javascript bridge needs to be cleaned up.

Today the javascript bridge, ej.js, wse.js and rvi.js is a hack and slash of a much more comprehensive erlang-javascript integration package provided by Tony Rogvall. This needs to be stripped down so that we only have remote function invocation and nothing else.

escript is likely a build dependency

root@sylph:~/rvi_core# make ubuntu_package
./rebar clean
/usr/bin/env: escript: No such file or directory
Makefile:49: recipe for target 'clean' failed
make: *** [clean] Error 127

Cannot open device key protected by pass phrase

Attempting to connect a client (whose device key has a pass phrase) to a backend (whose device key has no pass phrase) results in a crash on the client side.

Client side console output:

(rvi@tjamison-Precision-T5610)1> 2016-07-06 16:05:02.517 [error] <0.194.0>@dlink_tls_conn:handle_upgrade:407 Cannot upgrade to TLS: {error,{options,{keyfile,"/usr/lib/rvi_core/priv/keys/device_key.pem",{error,enoent}}}}

Backend side console output:

(rvi@tjamison-VTC1010)1> 16:05:02.833<dlink_tls_listener/101>new_connection(): Peer IP: "0.0.0.0" (ignored)
16:05:02.834<dlink_tls_listener/102>new_connection(): Peer Port: 9007 (ignored)
16:05:02.834<dlink_tls_listener/103>new_connection(): Sock: #Port<0.3501>
16:05:02.898<dlink_tls_conn/67>setup() IP = undefined; Port = 0; Mod = dlink_tls_rpc; Fun = handle_socket
16:05:02.903<dlink_tls_conn/68>CompSpec = {component_spec,[{service_edge_rpc,gen_server,[{json_rpc_address,{"38.129.64.31",9001}},{msgpack_rpc,[{ip,"38.129.64.31"},{port,9021}]},{websocket,[{port,9008}]}]}],[{schedule_rpc,gen_server,[]}],[{service_discovery_rpc,gen_server,[]}],[{authorize_rpc,gen_server,[]}],[{dlink_tls_rpc,gen_server,[{server_opts,[{port,9007},{ping_interval,5000}]}]}],[{proto_msgpack_rpc,gen_server,[]}],[{rvi_log,gen_server,[{json_rpc_address,9009}]}],[{tls_opts,[{port,9007},{ping_interval,5000}]}]}
16:05:02.905<dlink_tls_conn/153>connection:init(): self(): <0.221.0>
16:05:02.906<dlink_tls_conn/154>connection:init(): IP: undefined
16:05:02.906<dlink_tls_conn/155>connection:init(): Port: 0
16:05:02.906<dlink_tls_conn/156>connection:init(): Sock: #Port<0.3501>
16:05:02.906<dlink_tls_conn/157>connection:init(): Module: dlink_tls_rpc
16:05:02.906<dlink_tls_conn/158>connection:init(): Function: handle_socket
16:05:02.907<rvi_common/647>get_module_config(): Missing component spec: data_link:dlink_tls_rpc:packet_mod{...}: [{server_opts,[{port,9007},{ping_interval,5000}]}]
16:05:02.907<dlink_tls_conn/160>packet_mod = dlink_data_msgpack
16:05:02.938<rvi_common/647>get_module_config(): Missing component spec: data_link:dlink_tls_rpc:frag_opts{...}: [{server_opts,[{port,9007},{ping_interval,5000}]}]
16:05:02.940<rvi_common/654>get_module_config(): data_link:dlink_tls_rpc:server_opts -> [{port,9007},{ping_interval,5000}]:

Removing the pass phrase from the client's key results in successful connection.

Fix out of order execution on remote nodes

If two different services are invoked on a remote node that is currently not available, the services may be invoked out of order when the node does become connected. Scheduluer should queue messages on a per-node basis, not per-service.

Backend configuration

See PDXostc/rvi_backend#37.

After digging in the Erlang code, I came to the conclusion, that the crash is caused by the dlink_bt being present at the backend side, however { proto_json_rpc, dlink_bt_rpc} not configured in 'backend.config'. Thus the service discovery registration message crashes the service discovery server, which therefore looses its ets-table and is re-initialized by the supervisior with an empty ets-table afterwards.
This change in 'backend.config', even if BT is not used seems to solve it as workaround:

    { "", 
      [
       { proto_json_rpc, dlink_tcp_rpc},
       { proto_json_rpc, dlink_bt_rpc}]
    }

rebar compile error

Hi,

I'm trying to build rvi_core from source. I get this error;

./rebar compile
==> lager (compile)
==> src (compile)
==> exo (compile)
==> edown (compile)
/home/jeremiah/rvi-core-0.4.0/deps/edown/src/edown_doclet.erl:116: field packages undefined in record doclet_gen
/home/jeremiah/rvi-core-0.4.0/deps/edown/src/edown_doclet.erl:118: field filemap undefined in record doclet_gen
ERROR: compile failed while processing /home/jeremiah/rvi-core-0.4.0/deps/edown: rebar_abort
Makefile:43: recipe for target 'compile' failed
make: *** [compile] Error 1

Automatic reconnect to static nodes not implemented.

If a connection to a remote note, specified through the static_nodes config, no automatic reconnect is attempted.

A reconnect should be done at a regular interval to all currently unconnected remote nodes, just like what happens during startup.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.