janejeon / express-objection-starter Goto Github PK

This would involve mapping the production logs index somehow.

https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html

• list all current sessions for a user
• track location/IP/device/etc for each session
• allow revoking sessions manually
• update the updatedAt time every time the session is used, not created
• Somehow parse IP into geolocation (country/city would be enough), either server-side or client-side

https://github.com/heroku-examples/node-workers-example/blob/master/worker.js
https://github.com/OptimalBits/bull#separate-processes

• https://github.com/kibertoad/objection-swagger
• https://github.com/Zooz/express-ajv-swagger-validation
• https://github.com/kibertoad/express-swagger-oauth-scopes
• https://github.com/Surnet/swagger-jsdoc
• https://github.com/bcherny/json-schema-to-typescript

• Admins shouldn't be able to affect other admins' status
• Allow admins to "step down" on their own

Update tests! Should I have an "owner" role?

I'm only using ajv-errors for is the username verification. However, that should be done in the frontend... does that mean I can take off this dependency?

Either through config or documentation, protect API from being exposed directly. Something like Cloudflare's free tier would do great, because if it's laid bare to the internet, it's going to get DDOS'd to hell and back

• https://blog.cloudflare.com/ddos-prevention-protecting-the-origin/
• https://help.heroku.com/3EUIOIVB/why-can-t-i-access-herokudns-com-cname-endpoints-in-my-browser

Related to #22

Assuming #23 is set up, we can use rate limiting on the API side as a means of limiting actions (not DDOS, because if we hit Redis for every request during DDOS... yikes)

The question remains how we would protect per IP vs. per account, and how to protect endpoints differently

Some possible libraries:

• https://github.com/animir/node-rate-limiter-flexible
• https://github.com/AdamPflug/express-brute
• https://github.com/nfriedly/express-rate-limit

DATABASE_URL needs to be mapped to database:connection (IF it exists), and the same can go for BONSAI_URL and whatever config elasticsearch takes in... can we have a central place to manage this?

Either make them compatible, or drop support for one of them.

Should be able to:

• CRUD
• Bulk "force" update

https://www.algolia.com/doc/
https://www.elastic.co/guide/en/elasticsearch/client/javascript-api

eg. any demotions by admins

https://learnxinyminutes.com/docs/yaml/

especially the numbers (1e+x) and anchors

createValidator as a one-time hook and setting this.foo = bar might not work/persist. Check that the jsonSchema, relationMappings, and whatever else is injected in #21 actually works (better yet, write a test!)

Really stretching it, but having the counter would serve as an example for online systems using Redis

If we want FTS via external document stores, then we need to pipe changes from the "truth store" to the search index.

While there are native ways to stream changes (such as zombodb for Postgres), 1. you often can't install plugins in DBaaS providers, 2. it doesn't work for different db's, 3. how would you even filter objects (e.g. private) and properties?

Using the bulk update option in #34, it might be possible to do this by running a repeating job that checks for last "updated_at" (inclusive, since we don’t care if we have to do it more than once; plus, this way we don’t lock the table), drags all objects and push them to the search index.

And we can avoid locking by keeping a "bookmark" thru job data (eliminating the need for transactions) instead.

https://gocardless.com/blog/syncing-postgres-to-elasticsearch-lessons-learned/

• API integration tests

Should come after #49

• Setup email
• Route for re-requesting verification email
• Route for account recovery/password reset

While we want the whole "setup" thing to run automatically (especially after pulling, since people always forget to do that), it's annoying that it runs after yarn add!

I only want it to run on these cases:

• after a git pull (except in CI)
• before a yarn test
• after a production push

I've tried experimenting with this but I have NO idea how to make it work!

Ideally, the workflow would be like:

1. write to a template production YAML file (that is based on production.example.yml) to make changes
2. run something that'll produce the encrypted production.yml
3. read that encrypted production.yml that can be committed

How the fsck do I do step 2?

Either make them compatible, or drop support for one of them.

Should be able to:

• CRUD
• Image size/thumbnail support?
• Client-side uploading? (e.g. Presigned URL image uploading: https://devcenter.heroku.com/articles/s3-upload-node
  https://github.com/krisgholson/serverless-thumbnail)
• Limit image size to 2000x2000

Default to gravatar

Should come after #2

• npm publish (when tag is present)
• automated GitHub release?

See: https://github.com/JaneJeon/bazaar-web-API/blob/master/config/error.js

Specifically, we'd need something like https://github.com/epoberezkin/ajv-errors to make pretty error messages?

Also we'd need flash messages for validation errors!

Copy and paste the default CLI output from vue/angular/react/etc

tutorial: https://www.twilio.com/blog/2018/01/protect-your-node-js-app-from-cross-site-request-forgery.html

Might be related to #35, but to avoid storage buildup, we either need all images to "expire" at some point, or have some kind of mechanism to delete images when their parent resources are deleted and/or changed (e.g. avatar change)

Should come before #4.

In addition, on every error, we should redirect to the page it was at and display the flash

If doing SSR, then you might care about this stuff. For APIs, this doesn't matter.

• remove trailing slash: https://www.npmjs.com/package/connect-slashes
• "canonical" lowercase form

Use RBAC to control access to resources.

Possibly add GitHub/Twitter/Google sign-in?
See here: https://github.com/sahat/hackathon-starter/blob/master/config/passport.js

But I can't figure out how to render the non-callback part since we're using a pure REST API

Or OAuth2 integrations: https://github.com/simov/grant

Things like pageSize and hidden you have to add manually to model classes every time.

Or do I leave it alone?

Cloudflare IP geolocation using CF-IPCountry HTTP header (format: https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2)

Related: #17

https://app.leanboard.io/board/1c4ff2a9-f84a-49c1-a7d3-0d6c711b619d

Intercept reCAPTCHA tokens at the base of the router and verify them?

Hell, maybe even integrate Akismet?

Should this even be handled server-side?

Write an example CircleCI config to push directly to Heroku, perhaps using the Orb?

Goes hand-in-hand with #23.

See #34, #46

Ideally, redirecting HTTP => HTTPS should be handled by the reverse proxy. But if you're directly exposing Heroku API, then you kinda need it.

This would also be a good time to look at the whole "reverse proxy" section of config

We have to make a call to Redis for active session support, and another call to Redis for rate limiting, on every request. Can we cut this down into 1?

A lot of things need to happen automatically:

• typescript generation from json schema
• json generation from yaml files
• prettier/eslint/whatever

The goal is to make them run automatically, but not unnecessarily:

• linting should happen only before a commit, and shouldn't trigger a restart (i.e. nodemon)
• json generation should happen every time yaml file changes during development, and should be done before running any tests
• typescript generation would be a "nice to have" feature during development, but could be considered part of linting as well.

Ideally, these should be handled by hooks.

Look at https://github.com/chrisvfritz/vue-enterprise-boilerplate for "inspiration"

Docs: https://github.com/expressjs/multer

The way it's implemented in #10, SCAN takes O(n) time in relation to the total number of keys on Redis, which is obviously not performant.

I could also write it the way GitLab does, but managing the extra set and somehow expiring them (sets don't expire, so over time there will be a buildup of sessions from people who don't use the service anymore) seems like a huge PITA.

Finally, this would be a good opportunity to rethink the way to store and query sessions entirely, as per #13.

Objection.js and db-errors both support sqlite3, mysql, postgres, and mssql. That means this repo needs to be tested against all of those.

https://circleci.com/docs/2.0/workflows/

• Set up build matrix on CircleCI
• Rewrite "magic queries" on the base class to make it work on all databases (.returning('*) tricks)

Should be optional.

https://docs.newrelic.com/docs/accounts/install-new-relic/partner-based-installation/heroku-install-new-relic-add
https://devcenter.heroku.com/articles/newrelic
https://docs.newrelic.com/docs/agents/nodejs-agent/hosting-services/nodejs-agent-heroku
https://beachape.com/blog/2013/02/04/nodejs-on-heroku-with-newrelic/
https://github.com/newrelic/node-newrelic