expatiating / hi_score Goto Github PK

A modern full-life-cycle starter project for SPAs

This is an SPA starter project that installs best-in-class assets and tools to save time and guide best practice. Install hi_score today and start writing Test-driven native JS client code immediately. The project comes with recommended libraries, but feel free to swap them out as needed. That's the point.

• Code coverage at 98.9% per coveralls.io using Istanbul
• Fractal MVC architecture and proven file structure per the diagram below
• Vendor asset management (npm install && npm run setup) including support for executables, stylesheets, fonts, images, and libraries and automated patching.
• Libraries written to exacting standards to ensure readability and modularity
• Full code standard and quick-reference guides included
• Automatic namespacing and run-time control of CSS using PowerCSS
• Automatic commit-hook linting using JSLint, whitespace check, and strict checks (git commit...)
• Automatic in-line browsable HTML documentation using markdown and pandoc (git commit...)
• TDD and regression tests using Nodeunit + JSDOM (npm test)
• Type safety with type-cast libraries
• Build compression including object properties (npm run make)
• Creation of distribution-ready build/dist from build process (npm run make)
• Two simple demo apps that show compression and namespacing (npm run make)

  mkdir -p ~/GitHub
  cd ~/GitHub
  git clone [email protected]:mmikowski/hi_score.git
  cd hi_score
  npm install && npm run setup

We use the code style presented in Single Page Web Applications - JavaScript end-to-end (see reviews on Amazon). The quick reference and the full code standard are available online and are included in the docs directory.

The xhi libraries are structured to facilitate loose coupling but strict call precidence. For example the xhi/00.js library must be loaded to the browser before any other xhi code, and it may not call any library with a higher precidence number. The 08.app.js, in comparison, must be loaded after all the 00-07 libraries, but it may call any library of the same or lower precidence:

  /|                                                       //////
 +  ======================== API CALLS =========================
  \|                                                       \\\\\\

  +---------+    +----------+                       +----------+
  | 02.data |    | 03.model |<--+-------------------|  Shell   |
  |  Data   |<---|  Model   | ..... events .....))) | 07.shell |
  |  Fetch  |    +----------+            :          +----------+
  +---------+      |                     :                    |
       |           |                     :      +---------+   |
       |           |                     :      | 06.lb   |   |
       |           |                     :..))) | litebox |<--+
       |           |  +----------+       :      | feature |   |
       v           |  | Browser  |<--+----------+---------+   |
  +---------+      |  |  Utils   |   |   :                    |
  | 01.util |      |  | 04.utilb |   |   :      +---------+   |
  |  Utils  |<-----+--+----------+   |   :      | 06.*    |   |
  +---------+                        |   ...))) | feature |<--+
        |                            |          | modules |   |
        v                            +----------+---------+   |
  +-----------+                      |                        |
  |   00.js   |                      |      +-------------+   |
  | namespace |                      |      | 05.css_*    |   |
  +-----------+                      |      | 06.css      |<--+
                                     |      | feature css |
                                     +------+-------------+
 \\\\\\                                                       |\
  ========================== DATA FLOW =======================| +
 //////                                                       |/

We use model events to broadcast changes to the Shell and Feature modules and we keep our feature modules isolated from each other. This enhances portability and quality.

Our baseline compatibility is IE9+. Those supporting IE 8 have our sympathy.

Everything should just work on recent Ubuntu and derivative distributions like Mint or Kubuntu. Here are the steps to install prequisites on Ubuntu 17.04:

  # Install development libs
  sudo apt-get install build-essential openssh-server git pandoc \
    libfile-slurp-perl liblist-moreutils-perl libgetopt-mixed-perl

  # Install nodejs
  curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
  sudo apt-get install -y nodejs

  # Install mongodb
  sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 \
    --recv 0C49F3730359A14518585931BC711F9BA15703C6
  echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" \
    | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list
  sudo apt-get update && sudo apt-get install -y mongodb-org

Other modern Linux distributions should generally work as long as the same tools can be installed as as above. It works fine on CentOS with development libraries installed:

 yum install gcc gcc-c++ make openssl-devel

See this guide for NodeJS package installation on other Linux distros. Here is a more generic guide for Kubuntu and Ubuntu.

Use AWS or a Virtual Box image using Ubuntu 16.04 Server using the the same steps above.

This is probably the best way to get familiar with the project. Any work here will pay off during deployment as hi_score is designed to run on industry-standard Ubuntu servers, cloud instances like Amazon EC2, and containers.

We recommend using a virtual machine if possible. However one should be able to develop natively on Mac. At the very least one will need Bash 4+, GNU Core utilities, NodeJS, Git, PanDoc, and SSH.

We recommend using a virtual machine as detailed above. Installation might work with the new Linux subsystem on Windows 10 but we don't have any experience with it.

Use the Quick start guide to install hi_score and prepare it for development. One can also use npm install hi_score but the git method is preferred. No errors should be reported.

Use npm test to run the regression tests. You may expand tests by adding to the test/xhi_level_0 file. Tested xhi modules include the root namespace (00.js), the utilities (01.util.js), the browser utilities (04.utilb.js), and the litebox (06.lb.js).

Use npm run coverage to calculate code coverage. We can open the coverage/lcov-report/index.html file with a browser to inspect coverage. We use the excellent Istanbul code coverage tool along with the JSDOM package. Current coverage is reported at the top of this document.

Use npm run publish-coverage to publish a report to the coveralls.io website. The process to set up coveralls is described in hi_score/README.coveralls.md.

Use npm run make to create a distribution for deployment. The client deployment files are found in build/dist. We can inspect our sample applications as follows:

  cd ~/GitHub/hi_score
  npm run make
  cd build/dist
  google-chrome ex01.html ex02.html

Yes, we know the examples are lame. We are working on that.

One may update all the npm libraries, assets and the package.json file as follows:

PATH=$PATH:node_modules/.bin

# Check module versions
ncu

# Update package and manifest
ncu -u --packageFile package.json

# Reinstall vendor assets
npm run setup

When we open our example apps (google-chrome ex01.html ex02.html) we see they provide near-identical features and share a great deal of code and assets. However, they use separate namespaces to avoid data collisions. Namespaces enable us to provide a suite of web apps that share a great deal of code but have instances and data cleanly isolated. With namespacing, one can trace behaviors to the controlling code faster and with greater accuracy.

When we view the Example 1 app we can open the browser development tools (press <shift>-<ctrl>-i or <shift>-<cmd>-i on a Mac) and enter ex01 into the JavaScript console to inspect that value. We can see that ex01 is the single variable that contains all our app code. When we enter ex02 we see that it is undefined. When we visit the Example 2 we can see that ex01 is undefined and ex02 contains all our app code.

We namespace our CSS classes as well. When we inspect the HTML of the Example 1 app we can see that nearly all classes start with an ex01- prefix. When we inspect Example 2 tab we find the prefix is ex02-. As with the JavaScript namespacing, the prefixes are hierarchical. For example, the ex02-_lb_ class was generated for use by the ex02-_lb_ module.

Use npm run setup to deploy vendor assets. The configration for vendor assets are in package.json. The devDependencies map listing the assets, xhiVendorAssetGroupTable lists the deployment groups and files.

Assets are copied to their destination directory with their version number appended to their names. The .gitignore file instructs git to ignore all these files as their development management is external to our project. Everytime npm setup is run the vendor directories are deleted and recreated.

Vendor executables are copied to the bin/vendor directory.

Vendor font files are copied to the font/vendor directory. These fonts are currently installed:

• Font Awesome: Icon fonts
• Open Sans: OSS Font face

Vendor images are be copied to the img/vendor directory.

Client libraries are copied to the js/vendor directory. This makes them available to the web server. The following libraries are installed:

• jQuery: DOM manipulation
• PowerCSS: JS-powered CSS
• jQuery Plugin: event.dragscroll: Inertia scroll
• jQuery Plugin: event.gevent: Global events
• jQuery Plugin: event.ue: Touch and desktop gestures
• jQuery Plugin: scrolli: Scroll indicators
• jQuery Plugin: urianchor: SPA routing
• SprintF: Sprintf library
• TaffyDB: Client data management

NodeJS libraries are not copied to a vendor directory. We may changes this if we decide to create a server distribution. The following libraries are installed:

• clusterjs: Server multiplexer
• express: Minimalist Sinatra HTTP server
• mongodb: Official mongodb node client
• mysql2: Faster mysql interface
• websocket: Websockets interface

Developent libraries are used for testing a building code. They are not copied to a vendor directory and probably never will be as they are for development, not deployment. The following libraries are installed:

• coveralls: Code coverage reporting
• istanbul: Code coverage
• jsdom: DOM mock for testing
• jslint: Linting for commit hook
• nodeunit: Unit testing
• node-inspector: Debugging
• uglifycss: CSS minification
• uglifyjs: JS minitifcation
• buildify: Build script

Vendor CSS libraries are copied to the css/vendor directory. We copy the Font Awesome CSS files to this directory:

• Font Awesome: Icon fonts

Use npm run setup to apply patches. The configuration for patches are in package.json in the xhiPatchMatrix map. The patches are stored in the patch directory.

The patches mechanism allows us to use great tools tweaked for our needs while maintaining the upstream source. For example, we patch uglify-js to support object property name compression and shuffling by superpack.

Use npm run make to build a distribution. The build script concatenates, compresses, and obsufucates JavaScript and CSS. It copies only the required assets into the the distribution directory (build/dist). The result can load up to 10x faster and typically consumes only 5% of the disk space of the development code. We can inspect the files and disk usage as follows:

  cd ~/GitHub/hi_score

  # Make sure we have built the distribution
  npm install && npm run setup && npm test && npm run make

  # Get disk usage of all development files
  du -sh .

  # Inspect distribution directory
  cd build/dist
  tree
  du -sh .

The buildify build script uses superpack to analyze variable names and object properties and replaces them with shuffled keys. The shortest keys are used for the most-used symbols. It reports the symbol-to-key mapping and the frequency of use which makes further optimizations by pruning code easier. Projects with many object properities can be compressed an additional 50% using superpack.

Buildify reduces the dozens of HTTP calls to just a few. This can reduce load time significantly as illustrated below.

The load time measurements were made using a local HTTP server which is almost certainly a best-case scenario. We hope to add results for a remote server soon.

Any improvements or suggestions are welcome through the issues tracker. Pull requests are especially appreciated.

2016, 2017 Michael S. Mikowski (mike[dot]mikowski[at]gmail[dotcom])

MIT

• (x) Initial preparation

• (x) Library updates

• (x) Regression and integration testing
• (x) Rudimentary sample app

• (x) Add code coverage
• (x) Replace getDeepMapVal and setDeepMapVal with more powerful and tested getStructData and setStructData
• (x) Updates to xhi/01.util.js

• (x) Replace jscoverage with much more complete and recent istanbul
• (x) Added cast routines and detail their use
• (x) Consolidate utilities to increase coverage
• (x) Update lite-box using cast methods

• (x) Add jsdom to expand testing to modules that use jQuery
• (x) Continue regression test expansion
• (x) Rationalize libraries
• (x) Add lite-box regression tests

• (x) Remove vendor code from repo and auto-copy on install
• (x) Add native utils makeThrottleFn and makeDebounceFn
• (x) Add links to updated code style guides
• (x) Replace install script with prep-libs (v0.6.17+)

• (x) Move to consturctor approach to easily create multiple concurrent namespaced apps using the common xhi core
• (x) Update index page to illustrate
• (x) Make example app less trivial
• (x) Number code library level

• (x) Work on build system
• (x) Unify shell scripts nomenclature
• (x) Add constructor where only selected components are added
• (x) Add dependency levels for xhi libs

• (x) Add distribution build system npm run buildify
• (x) Add utilities and tests

• (x) Initial feature complete
• (x) Add utils and tests

• (x) Rename npm run prep-libs to npm run setup
• (x) Rename npm run cover to npm run coverage
• (x) Rename npm run covera to npm run publish-coverage
• (x) Rename npm run buildify to npm run make
• (x) Syntax refinements
• (x) Update libs, add express
• (x) Add utils and tests

• (x) Convert bin/setup in JavaScript
• (x) Configure setup completely in package.json

• (o) Test load time using remote server
• (o) Provide unique build number like build/0001/
• (o) Move coverage reports to build/xxxx/ directories
• (o) Convert buildify from Bash to JavaScript
• (o) Convert superpack from Perl to JavaScript
• (o) Increase complexity of example apps

absurd.js, responsive.js