Git Product home page Git Product logo

nodejs-reactive-moviebase's Introduction

Reactive Nodejs-MongoDB Movies

Build Status codebeat badge code style: prettier

Introduction

A simple Movie database with ratings and comments with limited social functionality, registration options and private lists control.

Design and architecture

For this project, although it has very limited functional requirements, there will be developed a variety of services with a main layer strongly influenced by DDD and microservices distributed architectures. Microservices pretend to determine an exact context boundaries to the DDD model of all project, accordingly divided by local models, aggregates and pass messages through Redis to delegate tasks and information flow.

Technologies

  • Node.js LTS
  • React
  • MongoDB
  • Redis
  • Typescript
  • Gulp
  • Webpack
  • Variety of other tools

Infrastructure

Whole application is dockerized and managed through docker-compose, node process manager is pm2

Application structure

  • It is divided by two subdirectories, client and server.
  • @types is for the project typings, to be able reuse them across front/back
  • configs are jest, babel configs
  • docker docker configuration

Server

  • launch contains server launching fabric for different microservices what are going to be executed.
  • services contain external / independent services
  • handlers contain all controllers to which the routes will claim for
  • routes files with routing for express
  • pkg it is the main directory for the application services/repositories
  • pkg/storage different repositories for each different DB motor we implement
  • pkg/{verbose} self-explainable services

Installation

Requirements

  • docker
  • Node LTS

Launch

  • yarn install
  • cd docker && docker-compose build
  • cd docker && docker-compose up
  • if mongo did't start in replicaSet -> docker-compose exec -T mongo mongo rs.initiate()
  • yarn serve:dev:client

nodejs-reactive-moviebase's People

Contributors

dependabot[bot] avatar greenkeeper[bot] avatar sckv avatar

Stargazers

 avatar

Watchers

 avatar

nodejs-reactive-moviebase's Issues

[spec] - Users - Register / Authorization / Forgotten password service

Current Scenarios

The users will have an option to access full features of the service using a registration form and authorization. The authorization has to be secure, to have expiry date, to be using a sessions token stored in the database. Cache mechanism for sessions easing fast access with obligated cache invalidation.

Proposed Solution

Registration

With an email, username (nick) and a password we can access to the registration procedure, a generated unique token with an expiry date for access by an URL, confirming user registration intention. The emails with the URL will be sent to registered email (using Email service)

Authorization

  • a) Generated custom sessions token.
    The token will be stored with the necessary user data for internal usage and scope. redis database for the data with a simple KV storage. Is needed to implement a cache invalidation mechanism.
  • b) OAuth service e.g. hydra or custom-made JWT basic implementation with the data still stored in redis (need to discuss exact mechanism)

The data will be stored at client side by secured httpOnly cookies.

Forgotten password

If the user can't remember hes password, there will be a dedicated form and field for that purpose, where with hes email or username submitting will be triggered a mechanism for password resetting. The unique URL will be "generated" and sent to hes registered email. (using Email service)

User Stories

Actor Story
User I want to register in the application and obtain an account to access, have to fill a valid email, username and double password. After that procedure I will receive an email and follow provided URL where I will be notified about activation of my account.
User I want to access to my account through login form introducing email or username with a correct password. If the information is correct I will gain the access to the service.
User I want to restore my forgotten password and will enter my email or username and if it's correct, I will receive an email with the similar URL as register stage. That link will lead me to the new password screen, where I will fill the field two times and confirm. After successful change I will be authorized directly to the service.

Requirements

  • Authorization endpoint implementation with bcrypt and redis.
  • Authorization session invalidation (ban, logout)
  • Registration endpoint
  • Forgotten password endpoints

[spec] - Emails - Emails queued service

Current Scenarios

The users, once on the website, maybe would like to register to share their tastes or/and have private lists with some notes and/or ratings for the movies they've already seen. The interaction for the user will be provided through an account, which will have to be activated, and for user there will be an option to recover his password if he loses one.

Proposed Solution

Use an email service e.g. SendGrid. The integration with Nodejs is quite easy and extended with the official library @sendgrid/mail.
The mails has to be queued to the mail service for the most performant solution.

User Stories

Actor Story
User As user I can register in the web-page and activate my account by the hyperlink included in email. This link is unique and have an expiration date, after which my account data self deletes.
User As user I can request a link for resetting my old password for a new one, the link has to have expiration date also

Requirements

  • SendGrid account + API Key
  • Email templates with an email template library e.g. email-templates
  • Emails service + internal api + queue e.g. bull
  • ? Sending feedback

[spec] - Users - Interaction between accounts

Current Scenarios

Users will have an option to find another user by search endpoint, to look into hes profile and hes movie lists, if they are not private. He will be able to follow users he want to and have a direct access area with all users he follows with their latest added movies.

Proposed Solution

An internal search with the data. Retrieving an ID for the necessary profile information.

User Stories

Actor Story
User I want to find an another user in the system and see hes profile, hes lists and follow him to get a comfortable area where I can recall to hes profile in a handy way

Requirements

  • Username search API
  • Full profile information API

Not the Goal

We do not want users being found by a name or email

[spec] - Structure - API files structure and scope

Current Scenarios

It's needed a clear structure with determined scopes for an easy understanding for any developer possibly involved in the project. Not speaking about ubiquitous language but with and intention for easy management and expansion.

Proposed Solution

Modified and inspired in DDD structure with repositories, services and execution layer, without DI, composition instead.

Requirements

  • File structure
  • Initial repository methods
  • Readme explanation

[spec] - Infrastucture - Caching

Current Scenarios

GET calls caching. A reverse proxy or server side caching mechanism is needed for the first meaningful information being retrieved the fastest way. The cached information can vary in different situations, e.g. private list information for a user. There has to be a solution to cache-per-user maybe.

Proposed Solution

Usage of the runtime cache (memo) - Redis. Redis is much more scalable option. Cache invalidation can be provided with SHA256 hashing of the contents + access URL + (maybe) user session information.

Addition. Each Node.js connection while keepalived can behave as a cache information submitter, we can store a list of Mongo ChangeStream (CS) subscribers as atomic entries in Redis, and submit the data to Pub/Sub Redis queue for the Caching service consumption. If the data in Redis is available BUT the CS subscription list does not contain an active CS subscriber regarding that concrete URL/UserData, we open a new connection and sign the keepalive to the list (if the Endpoint precises that)

Addition:2. Each client what ONLY consumes the data from Redis cache (no Cache publisher, no Mongo CS subscriber) will subscribe OR initiate a singleton local preloading from Redis process cache:<url>:<hash> with setInterval which, with the average latency request time in Spain (20-60ms) will be 40-45ms. Each request which is NOT Cache publisher, NOR Mongo CS subscriber will stop the interval for the Redis GET calls, to prevent unnecesary Redis DB overhead with unnecesary data
edit: There is a subscription to namespace events which leaves obsolete a constant polling solition, the data can be accessed upon change notification, saving huge overhead over the database

Addition:3. Internal Cache polling can provide an Observable with RxJS to the controllers (clients) to consume (subscribe to) with the updated data of polled Redis cache data slice. The observable is the best way for now I consider to get streamable sets of date directly communicated to the clients.

Addition:4. RxJS use is not needed with a native EventEmitter. Each controller will subscribe to concrete event and remove the subscription under closing of the connection.

cacheProposal

User Stories

Actor Story
User I want to access the services as fast as it possible, with only necessary infromation retrieved

Requirements

  • Caching / Cache-invalidation mechanism/service

[spec] - Frontend BL - Fetch stream to API

Current Scenarios

When the web-page makes GET calls to the API service, we would like to keep open a socket connection for the long-polling updates overtime from the server. We need a web fetch client capable of taking chunks of data, packing them and stream them to the frontend state BL.

Proposed Solution

Fetch API. We can leverage our needs for open streams with the native Fetch API wrapper implementation for (possibly) stream the data to RxJS subscription client.

User Stories

Actor Story
User As a user I want to have data I visualize about the movies dynamically updated overtime without my interaction.
User As a user I want to see different users movie lists updated in real-time with the data another user modifies/adds

Requirements

  • Streamable FetchAPI wrapper implementation

[spec] - Testing - Backend services coverage

Current Scenarios

Tests coverage for the services/repositories for assurance of the working state of the application.

Proposed Solution

Create fixtures, initial scripts, split the neccesary services for ease of use. We can use TravisCI testing mongodb or the local docker/system testing MongoDB.

Requirements

  • Authorizing
  • Users control
  • Listing
  • Movies control

[spec] - Movies - Initial page movies list

Current Scenarios

Initial page of the service will be a list populated with movies that has been added by the users to the database, choosing that movies during their lists composition. The information will be dynamic (real-time) from the database streaming directly to the frontend view.

Proposed Solution

Stream the collection to the request connection using long-polling or WebSocket with the necessary transformations and possible server-side caching/memoization if no changes been observed. Use of Change Stream for collection or database. The response has to be paginated or front-virtualized.

User Stories

Actor Story
User I can see the list of the movies currently in the service database added by users in their different lists. They will be ordered by date, {desc}
User I can switch between latest and the most popular movies in that home list

Requirements

  • Movies service
  • Query filter for newest/most popular movies

[spec] - Movies - External movies API storage

Current Scenarios

Users will need a database of Movies to add them to their lists, this movies will be retrieved from an internal database source or if needed, will be added there. The usage of an external API for movies and translation is needed. (Spanish, English)

Proposed Solution

OMDB and Google Translation API. Would be a good candidates for the solution. User language preferences have to prevail for the movies synopsis and another information retrieved by the user. Once the movie is added to the local database, it will be retrieved from there. Collateral search data (during user requests) will be stored in the local database too, to avoid external API calls at all cost.

Addition:2 For fast searches we can use a public IMDB search API, and then develop the selected option with the OMDBAPI for further information.

Addition:3 Another option is to parse public movies data files provided by IMDB and replicate them to the MongoDB

User Stories

Actor Story
User I want to add a movie to my list. I don't have to know where and how this works, but I want to have the movie data and description on language of my preference (spanish/english)

Requirements

  • API Keys and documentation for OMDBAPI and Google Translation API
  • External services integration into the Movies service

Not the Goal

We do not want maintain updated and synced with external API all the movie details.

[spec] - Movies - Create movie lists and Add / Rate / Comment movies

Current Scenarios

Users would like to add movies to their personal lists, for that they will have to leave a comment (optional) and to rate the movie (non optional). There has to be a mechanism to rename, edit and delete any own list.

Proposed Solution

A series of endpoints for managing this type of service, in main Movies service. Basically this are CRUD operations regarding the data of which user is in possession.

User Stories

Actor Story
User As a user I want to create, rename or delete a movie list. Mark it private or public at any time.
User As a user I add/delete new/old movies to/from the list, can add a comment to my choice and will have to put a permanent rating (which I won't be able to change never again).

Requirements

  • List CRUD endpoint
  • Movies to List CD endpoint

[unit] - Errors Handler - Human readable errors handler

Problem/Implementation question

Human readable errors feedback. Besides status codes, we have to have a human readable explanation for the error / status received by the client.

What is going to be done

Express Error handling middleware. The middleware will match URL/StatusCode/Throw from the underlying controllers and add a json body with clear information to the response.

[spec] - Users - Edit personal information

Current Scenarios

Change personal users information. The user has to have an option and resources to change it's username and password.

Proposed Solution

RUD.

User Stories

Actor Story
User As user I want to change my username and/or password. After the password change I won't be logged out

Requirements

  • An endpoint for that purpose in Users service

[spec] - Infrastructure - Services cluster

Current Scenarios

We need an automatized services cluster, initially with docker-compose configs, to be able to scale horizontally with ease. Initially, in will be a MongoDB, Envoy, Nginx, a pair of Node instances

User Stories

Actor Story
User As a user I want to have a reliable service with minimum downtimes possible.

Requirements

  • docker-compose.yml tested configuration

[spec] - API - Endpoints list

Current Scenarios

The API will have accesible a list of endpoints for users consumption.

Proposed Solution

Common RESTful application.

List

Authorization - prefix: /auth

  • POST - /login
  • POST - /logout
  • POST - /forgot
  • POST - /check-recovery
  • POST - /reset-password

Movies - prefix: /movies

  • GET - / - ?{s - sort, c -criteria, p - page, ps - pageSize, l - language}
  • GET - /{id}
  • GET - /{ttId}/by-ttid

Lists - prefix: /lists

  • GET - /{userId}/get - ?{u - userId}
  • GET - /{id}
  • POST - /
  • PATCH - /{id}
  • DELETE - /{id}
  • PATCH - /{id}/add-to
  • PATCH - /{id}/remove-from

Users - prefix: /users

  • POST - /activate
  • GET - / - ?{un - username}
  • GET - /{id} - ?{pd - personalData, md - moviesData, ld - listsData, follows, followers}
  • PATCH - /{id}
  • POST - /follow
  • POST - /unfollow
  • DELETE - /self
  • POST - /register

An in-range update of eslint-plugin-jest is breaking the build 🚨

The devDependency eslint-plugin-jest was updated from 22.5.1 to 22.6.0.

🚨 View failing branch.

This version is covered by your current version range and after updating it in your project the build failed.

eslint-plugin-jest is a devDependency of this project. It might not break your production code or affect downstream projects, but probably breaks your build or test tools, which may prevent deploying or publishing.

Status Details
  • ❌ continuous-integration/travis-ci/push: The Travis CI build could not complete due to an error (Details).

Release Notes for v22.6.0

22.6.0 (2019-05-22)

Features

Commits

The new version differs by 9 commits.

  • 14d83ef feat(rules): add no-commented-out rule (#262)
  • 83ff198 chore: migrate no-jest-import to typescript (#259)
  • 718c08c chore: upgrade @typescript-eslint
  • ca2aa27 chore: port lowercase-name to TypeScript (#258)
  • 3df0058 chore(ci): run danger with lts version of node
  • 48e3a59 chore: precompile with babel (#257)
  • 8670804 chore: bump deps
  • 05eb11a chore: fix lint error
  • dff6446 docs: link to eslint-plugin-jest-formatting

See the full diff

FAQ and help

There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.


Your Greenkeeper Bot 🌴

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.