Rapid Development Node.js Hackathon template

This template based on KeystoneJS. KeystoneJS is just a glue between Express, ApolloGraphql, Mongoose/Knex (as ORM).

Todo

VERSION: 0.0.1.ALPHA

Learn

You should have a basic knowledge about HTML/CSS
You should know Modern JavaScript
You should know GraphQL

Server side

You know something about Node.js
You should follow the KeystoneJS tutorial
Something about mongo and mongoose

Client side

You should know React
You should follow the Next.js tutorial

yarn workspaces: https://classic.yarnpkg.com/en/docs/workspaces/
CSS animations: https://css-animations.io/ (base level), https://tympanus.net/codrops/ (pro examples)
JS Egghead.io courses: https://egghead.io/
Docker compose: https://docs.docker.com/compose/

Philosophy

BACKEND: You write domain models and services on backend side (it's better if each service have only one action)
TESTS: TDD is matter! Tests should work parallel and independently! You should write tests in mind of extensibility (you don't need to rewrite the test if you add extra fields). You can move test to another git repository. You can run the tests on a remote production API by URL! We should test not only the our Unit.
STYLE: You should follow the common style or change the whole style everywhere (not only in your files)
THINK AND DESCRIBE BEFORE CODE! TODO(pahaz): write diagrams like c4 before write a code!

Init new Project

You should have docker-compose, git and node commands.

# Clone and create STARTUP dir
git clone https://github.com/8iq/nodejs-hackathon-boilerplate-starter-kit my-new-startup
cd my-new-startup

# Change git origin to your STARTUP repo
git remote set-url origin [email protected]:USERNAME/REPOSITORY.git
# Add template origin for some core tempate updates
git remote add template https://github.com/8iq/nodejs-hackathon-boilerplate-starter-kit

# Create .env file!
cp .env.example .env

# Run databases by docker
docker-compose up -d mongodb postgresdb

# Install dependencies and link workspaces
yarn

# 1. Write BACKEND domain models: `schema`
# 2. Create FRONTEND app: `apps`

# Run BACKEND
yarn dev

Keystone Admin UI is reachable via http://127.0.0.1:3000/admin.

Write Backend

Create new Keystone.js APP

# Create Keystone.js new backend
mkdir apps/_back01keystone

# 1. create package.json, index.js and initial-data.js (check exampe)
# 2. define your schema by following https://www.keystonejs.com/tutorials/add-lists
# and use GQLListSchema wrappers for style checking and easy test writing

# Link new package and Install dependencies
yarn

# create .env file (development mode)
cat > .env << ENDOFFILE
DATABASE_URL=mongodb://mongo:[email protected]/main?authSource=admin
DISABLE_LOGGING=true
NODE_ENV=development
ENDOFFILE

# Start dev server on http://localhost:3000
yarn workspace @app/_back01keystone dev

# Follow the Keystone.js tutorial: https://www.keystonejs.com/tutorials/add-lists
# You can run `yarn <command>` inside new app workspace by
# `yarn workspace @app/_back01keystone <command>`

Write Frontend APP

Create new Next.js APP

# Create Next.js new React APP folder
mkdir apps/_example01app
cat > apps/_example01app/package.json << ENDOFFILE
{
  "name": "@app/_example01app",
  "version": "1.0.0",
  "scripts": {
    "dev": "next",
    "build": "next build",
    "start": "next start"
  },
  "dependencies": {
    "isomorphic-unfetch": "^3.0.0",
    "next": "^9.3.5",
    "react": "^16.13.1",
    "react-dom": "^16.13.1"
  }
}
ENDOFFILE
mkdir apps/_example01app/pages
cat > apps/_example01app/pages/index.jsx << ENDOFFILE
function HomePage() {
  return <div>Welcome to Next.js!</div>
}

export default HomePage
ENDOFFILE

# Link new package and Install dependencies
yarn

# Start Next.js dev server on http://localhost:3000
yarn workspace @app/_example01app dev

# Follow the Next.js tutorial: https://nextjs.org/docs/getting-started#related
# You can run `yarn <command>` inside new app workspace by
# `yarn workspace @app/_example01app <command>`

# You can also check `apps/_example02app` and others examples

Write mobile app

Create Expo APP

cd apps
expo init _mobile01
cd ..
# based on https://stackoverflow.com/questions/59920012/monorepo-expo-with-yarn-workspace-and-using-expo-install
cat > apps/_mobile01/package.json << ENDOFFILE
{
  "name": "@app/_mobile01",
  "version": "1.0.0",
  "main": "__generated__/AppEntry.js",
  "scripts": {
    "start": "expo start",
    "android": "expo start --android",
    "ios": "expo start --ios",
    "web": "expo start --web",
    "eject": "expo eject",
    "postinstall": "expo-yarn-workspaces postinstall"
  },
  "dependencies": {
    "expo": "~38.0.8",
    "expo-status-bar": "^1.0.2",
    "react": "^16.13.1",
    "react-dom": "^16.13.1",
    "react-native": "https://github.com/expo/react-native/archive/sdk-38.0.2.tar.gz",
    "react-native-web": "~0.11.7"
  },
  "devDependencies": {
    "@babel/core": "^7.8.6",
    "babel-preset-expo": "~8.1.0",
    "expo-yarn-workspaces": "^1.2.1"
  }
}
ENDOFFILE
cat > apps/_mobile01/metro.config.js << ENDOFFILE
const { createMetroConfiguration } = require("expo-yarn-workspaces");

module.exports = createMetroConfiguration(__dirname);
ENDOFFILE
yarn
yarn workspace @app/_mobile01 postinstall

yarn workspace @app/_mobile01 start --clear

Deploy

We use docker-compose to deploy your application. You can check the docker-compose.yml and important variables in .env.example file.

There are two important variables for building production image:

DOCKER_FILE_INSTALL_COMMAND -- install extra packages and system requirements (ex: python3 -m pip install 'psycopg2-binary>=2.8.5' && python3 -m pip install 'Django>=3.0.6' or apt-get -y install nano)
DOCKER_FILE_BUILD_COMMAND -- run build static or prepare some project files for production (ex: yarn build or yarn workspace @app/_example05app build)

You can build a production docker container by docker-compose build command.

There are important variables for runtime:

DOCKER_COMPOSE_START_APP_COMMAND -- docker start command (ex: yarn workspace @app/_back02keystone start)
DOCKER_COMPOSE_COOKIE_SECRET -- the keystone important variable for sessions store (ex: AWJfbsbaf! or some secret random string)
DOCKER_COMPOSE_SERVER_URL -- the next.js important variable for API calls from frontend to backend (ex: https://example.dok.8iq.dev)

That's it!

You can build a production image localy and copy it to the production server by command: docker save apps:prod | bzip2 | pv | ssh [email protected] 'bunzip2 | docker load' (required brew install pv or just rm it from command)

Final script should looks like:

# 0. warm docker cache (it's speed up your rebuild)
bash ./bin/warm-docker-cache
# 1. You should build a prod image.
docker-compose build
# 2.1 Copy to prod server `docker image`
docker save apps:prod | bzip2 | pv | ssh [email protected] 'bunzip2 | docker load'
# 2.2 Copy to prod server `docker-compose.yml` and required `.env`
scp ./docker-compose.yml [email protected]:~
scp ./.env [email protected]:~
# 3. Run redeploy command
ssh [email protected] 'docker-compose down && docker-compose up -d'

NOTE: If you need some extra containers or you want to customize existing containers you can create docker-compose.override.yml file.

DOKKU Deploy

# Prepare dokku
dokku plugin:install https://github.com/dokku/dokku-letsencrypt.git
dokku plugin:install https://github.com/dokku/dokku-postgres.git postgres
dokku config:set --global [email protected]

# BUILD CONTAINER LOCALY AND SEND IT TO DOKKU SERVER
export DOCKER_COMPOSE_APP_IMAGE_TAG=coddi
docker-compose build
docker save apps:${DOCKER_COMPOSE_APP_IMAGE_TAG} | bzip2 | pv | ssh [email protected] 'bunzip2 | docker load'

# CREATE DOKKU APPLICATION ON DOKKU SERVER SIDE
export APP=node5
export APP_VERSION=v5
export DOCKER_IMAGE=apps:coddi
export START_COMMAND='yarn workspace @app/CODDI start'

dokku apps:create ${APP}
dokku postgres:create ${APP}
dokku postgres:link ${APP} ${APP}

dokku config:set --no-restart ${APP} NODE_ENV=production
dokku config:set --no-restart ${APP} SERVER_URL=https://${APP}.dok.8iq.dev
dokku config:set --no-restart ${APP} DOKKU_DOCKERFILE_START_CMD="${START_COMMAND}"
dokku config:set --no-restart ${APP} COOKIE_SECRET=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1)
dokku checks:disable ${APP}
dokku proxy:ports-set ${APP} http:80:5000
dokku nginx:set ${APP} hsts false
dokku nginx:set ${APP} hsts-include-subdomains false

docker tag ${DOCKER_IMAGE} dokku/${APP}:${APP_VERSION}
dokku tags:deploy ${APP} ${APP_VERSION}
docker exec -it -u root ${APP}.web.1 yarn workspace @app/CODDI migrate

dokku letsencrypt ${APP}

Others

Add package to existing APP

# Add new package to APP
yarn workspace @app/_example01app add antd

How-to use with postgres

You need to set DATABASE_URL like postgresql://postgres:[email protected]/main.

# 1 create tables
yarn workspace @app/_back02keystone create-tables

NOTE: And there is no solutions to migrations yet... You can try knex migrations and write about it here.

Upgrade packages versions

# just run the command, and select packages for update (this command will fix the package.json files) 
yarn upgrade-interactive --latest

TESTS_FAKE_CLIENT_MODE

@core/keystone/test.utils can work in two modes:

Real client mode. Allow sending HTTP/1.1 requests to a remote server
Fake client mode. Allow using fake express requests by supertest lib

The Real mode is better for end-to-end testing. You can also use it to test your production.

The Fake mode is better for debugging. Because it uses one process for the whole request/response process. It's better for TDD like development process and to easy debug end-to-end request/response process.

You can use TESTS_FAKE_CLIENT_MODE to change test mode. This option setup the express app and the fake client in one process. This allows you to put debugger breakpoints in any part of the request/response process. And use IDE integrations for easy debugging.

Postgres migrations

https://github.com/keystonejs/keystone/discussions/3067

# download
curl -o kmigrator https://raw.githubusercontent.com/8iq/nodejs-hackathon-boilerplate-starter-kit/master/apps/_back02keystone/kmigrator.py
chmod +x kmigrator
# install dependencies
python3 -m pip install django
python3 -m pip install psycopg2-binary

# create new migrations based on the changes you have made
./kmigrator makemigrations

# applying database migrations
./kmigrator migrate

Add to package.json:

  ...
  "scripts": {
    "makemigrations": "./kmigrator.py makemigrations",
    "migrate": "./kmigrator.py migrate",
    ...
  }

Run by yarn makemigrations and yarn migrate

DEBUG QUERIES

For postgres you can set env DEBUG=knex:query,knex:tx. Example:

DEBUG=knex:query,knex:tx yarn dev @app/_back02keystone @app/_example05app @app/_realtime01app

For mongo you can set env DEBUG_MONGOOSE=1. Example:

DEBUG_MONGOOSE=1 yarn dev @app/_back02keystone @app/_example05app @app/_realtime01app

Run multiple apps at the same time

You can use bin/run-multiple-apps to run more then one app.

Example:

node ./bin/run-multiple-apps @app/_back02keystone @app/_example05app @app/_realtime01app

Every app should have multi-app-support.js file. Check _back02keystone, _example05app and _realtime01app examples.

Tips

yarn add

yarn add <package> -W -- add package for all apps (yarn add react react-dom -W)
yarn workspace @app/<name> add <package> -- add package for special app (yarn workspace @app/web add react react-dom)

yarn

yarn <command> -- run command (yarn dev)
yarn workspace @app/<name> <command> -- run command inside workspace (yarn workspace @app/web dev)
yarn --cwd <app-path-name> <command> -- run command inside app (yarn --cwd apps/web dev)

luis-herrera-aburto / parvulo Goto Github PK

parvulo's Introduction