Git Product home page Git Product logo

retropilot-server's Introduction

retropilot-server

Uptime Robot status Uptime Robot ratio (7 days) CI workflow

Replacement for comma.ai backend and useradmin dashboard. Bundled with a modified version of comma's cabana to allow viewing & analyzing drives.

If you don't want to host your own instance, check out https://api.retropilot.org/useradmin for a hosted version of the backend, useradmin and cabana.

[Server] Summary

The server consists of 2 node scripts.

  • src/server is using expressjs and runs the backend (file upload / communication with openpilot) and the useradmin dashboard to manage / view / download drives & logs.
  • src/worker is a background worker that is processing drives (analyzing video files & logs) to prepare drives for playback in cabana and to gather statistics. It automatically terminates itself after 60 minutes to make sure the video/log libraries do not cause memory leaks.

Attention: Minimum required node version is node 10.

[Server] Installation

TODO: write instructions for PostgreSQL database

npm install
cp config.sample.js config.js
cp database.empty.sqlite database.sqlite
> EDIT config.js

[Server] Running

node -r esm src/server

node -r esm src/worker

[Server] CABANA Support

A compiled version of a custom cabana fork (https://github.com/RetroPilot/cabana) is directly bundled in the cabana/ subdirectory and will be served by the express app. After starting index.js, cabana is ready to use.

[Device] Preparation / Enable Custom Server

On the device or in your fork's code, replace all API endpoints with your own server endpoint. This could be executed directly on the device in the shell to use https://api.retropilot.org as backend:

find /data/openpilot -type f -exec sed -i 's/https:\/\/api.commadotai.com/https:\/\/api.retropilot.org/g' {} +

TODO: add git patch instructions

[Device] Swapping Servers (Back)

To switch a device between different servers, you have to remove the old DongleId and reboot:

rm /data/params/d/DongleID
reboot

There is no need to backup the DongleId, as the new server will identify your device based on its imei, serial and public key.

[Device] Raw Drives Not Uploading (fcamera & rlog)

  1. Raw data is only uploaded if the device is sufficiently charged, not connected to an active panda (offroad) and there are no immediate files (boot, crash, qcamera, qlog) remaining.
  2. Your branch might have raw uploads disabled, check Device Settings > Upload Raw Logs.

If that doesn't help or the option is not available, try:

echo "1" > /data/params/d/IsUploadRawEnabled
echo "1" > /data/params/d/UploadRaw
reboot

Current Limitations

OpenPilot before 0.8.3 will not display any statistics or pairing status in the dashboard. The reason is that pre 0.8.3, the offroad.apk with react and comma-api would require recompilation to accept the new endpoints.

The athena websockets interface is not implemented yet, so the comma app and athena specific remote control commands (including "upload on demand") are not functional as of now.

Screenshots

image

image

image

image

Deployment

For deployment instructions and configuration, take a look at the environment directory.

Code of Conduct

https://github.com/RetroPilot/.github/blob/main/profile/CODE_OF_CONDUCT

retropilot-server's People

Contributors

adamsblack avatar florianbrede-ayet avatar incognitojam avatar jor3l avatar sconway-datto avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

stlnick2 moretore tomkosm chunpengwu watchhub florianbrede-ayet paperboypaddy sheaduncan

retropilot-server's Issues

User registration potential for duplicates

There's nothing directly preventing a user to register with the same email other than the "unique" attribute in the database.

    at Query.run (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\sequelize\lib\dialects\postgres\query.js:50:25)
    at C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\sequelize\lib\sequelize.js:313:28
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
    at async PostgresQueryInterface.insert (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\sequelize\lib\dialects\abstract\query-interface.js:297:21)
    at async model.save (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\sequelize\lib\model.js:2417:35)
    at async Function.create (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\sequelize\lib\model.js:1329:12)
    at async C:\Users\adam\Documents\Retropilot\public\retropilot-server\dist\server\router\useradmin.js:174:16 {
  name: 'SequelizeUniqueConstraintError',
  errors: [
    ValidationErrorItem {
      message: 'email must be unique',
      type: 'unique violation',
      path: 'email',
      value: '[email protected]',
      origin: 'DB',
      instance: [accounts],
      validatorKey: 'not_unique',
      validatorName: null,
      validatorArgs: []
    }
  ],
  parent: error: duplicate key value violates unique constraint "accounts_un"
      at Parser.parseErrorMessage (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\pg-protocol\dist\parser.js:287:98)
      at Parser.handlePacket (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\pg-protocol\dist\parser.js:126:29)
      at Parser.parse (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\pg-protocol\dist\parser.js:39:38)
      at Socket.<anonymous> (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\pg-protocol\dist\index.js:11:42)
      at Socket.emit (events.js:400:28)
      at addChunk (internal/streams/readable.js:293:12)
      at readableAddChunk (internal/streams/readable.js:267:9)
      at Socket.Readable.push (internal/streams/readable.js:206:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
    length: 204,
    severity: 'ERROR',
    code: '23505',
    detail: 'Key (email)=([email protected]) already exists.',
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: undefined,
    schema: 'public',
    table: 'accounts',
    column: undefined,
    dataType: undefined,
    constraint: 'accounts_un',
    file: 'nbtinsert.c',
    line: '670',
    routine: '_bt_check_unique',
    sql: 'INSERT INTO "accounts" ("id","email","password","created","admin") VALUES (DEFAULT,$1,$2,$3,$4) RETURNING "id","email","password","created","last_ping","2fa_token","admin","email_verify_token","g_oauth_sub","two_factor_enabled";',
    parameters: [
      '[email protected]',
      'password_hash***',
      1651328540828,
      false
    ]
  },
  original: error: duplicate key value violates unique constraint "accounts_un"
      at Parser.parseErrorMessage (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\pg-protocol\dist\parser.js:287:98)
      at Parser.handlePacket (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\pg-protocol\dist\parser.js:126:29)
      at Parser.parse (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\pg-protocol\dist\parser.js:39:38)
      at Socket.<anonymous> (C:\Users\adam\Documents\Retropilot\public\retropilot-server\node_modules\pg-protocol\dist\index.js:11:42)
      at Socket.emit (events.js:400:28)
      at addChunk (internal/streams/readable.js:293:12)
      at readableAddChunk (internal/streams/readable.js:267:9)
      at Socket.Readable.push (internal/streams/readable.js:206:10)
      at TCP.onStreamRead (internal/stream_base_commons.js:188:23) {
    length: 204,
    severity: 'ERROR',
    code: '23505',
    detail: 'Key (email)=([email protected]) already exists.',
    hint: undefined,
    position: undefined,
    internalPosition: undefined,
    internalQuery: undefined,
    where: undefined,
    schema: 'public',
    table: 'accounts',
    column: undefined,
    dataType: undefined,
    constraint: 'accounts_un',
    file: 'nbtinsert.c',
    line: '670',
    routine: '_bt_check_unique',
    sql: 'INSERT INTO "accounts" ("id","email","password","created","admin") VALUES (DEFAULT,$1,$2,$3,$4) RETURNING "id","email","password","created","last_ping","2fa_token","admin","email_verify_token","g_oauth_sub","two_factor_enabled";',
    parameters: [
      '[email protected]',
      'password_hash*****',
      1651328540828,
      false
    ]
  },
  fields: { email: '[email protected]' },
  sql: 'INSERT INTO "accounts" ("id","email","password","created","admin") VALUES (DEFAULT,$1,$2,$3,$4) RETURNING "id","email","password","created","last_ping","2fa_token","admin","email_verify_token","g_oauth_sub","two_factor_enabled";'
}
TypeError: Cannot read property 'dataValues' of undefined
    at C:\Users\adam\Documents\Retropilot\public\retropilot-server\dist\server\router\useradmin.js:181:16
    at processTicksAndRejections (internal/process/task_queues.js:95:5)```

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.