Goal

What happens in a Vulcan app should be transparent to the developers and devops team.

ideas

We should use a debug logger as soon as possible, to have relevant debug logs.
I currently use debug, which is a bit low level

• Setup debug client side programmatically based on DEBUG environment variable
• Demo Sentry
• Setup fetch-HAR to track server requests. See #69

Ressources about Sentry

• https://www.npmjs.com/package/@sentry/nextjs
• vercel/next.js#8592
• Official with sentry example, works well

Goal

Vulcan is not about coding, it's about creating application. It should be easy to add persistence solutions and a backend to your application.

Vulcan is also about choice, because no application is the same. It should also be easy NOT to add a backend, in case you want to create a classical Single Page Application.

Ideas

• Easy spawn of a Mongo database
• Easy install of Prisma?
• Include an API route with Apollo GraphQL
• Ability to use Vulcan's schema to generate the API (Work In Progress)

Goal

Contributor should have a great experience out of the box

Ideas

• List of VS Code packages to install
• Linting with Prettier, eslint
• VS Code debugging for server-side render, of client-side, of API routes
• VS Code debugging for Jest tests (done in Vulcan-NPM, we need to paste the vs code config here)
• Add recommended packages autopatically

There is a way to add recommended packages to the workspace config, so that VS Code recommend them directly:
https://medium.com/better-programming/how-to-configure-vs-code-like-a-pro-782d2d718586

Recommended packages (use Ctrl+P in VS code to run the command):

ext install vscode-styled-jsx

Goal

Keep Vulcan as fun as possible!

Having a powerful professional setup is cool. But Open Source is a place of freedom where you can break bounds, moonshot and try new things.

Vulcan should include features that 20 years ahead will make you say, "Ha, remember in Vulcan, we add this cool thing, what a wonderful era!"

Random ideas:

• Define what a love feature would be for Vulcan community. Something that helps you become a better developer? An innovative UI/UX?
• Include a Storybook design system, based on this course: https://frontendmasters.com/courses/design-systems/)

git checkout bugfix/res-locals
yarn && yarn run test:e2e
One test will fail.

For unknown reason, it does not fail in dev (using yarn dev and yarn cypress:open).

Concerned files

• Failing test is in cypress/integration/vns/i18n.spec.ts (with it.only so it's the only one to run)
• src/pages/_app.tsx contains a commented quickfix in "getInitialProps"

apollographql/react-apollo#3678 (comment)
vercel/next.js#9542
apollographql/react-apollo#3251

Next with Apollo allows to activate/deactivate SSR with Apollo queries

Important note

Testing SSR based on "error" state can be problematic, as getDataFromTree might just fail in error case (eg gql server non available). See a loading state in this case can be normal.

Trying this package, want to learn, want to help. But after git clone and yarn install and then yarn dev with Vulcan running in another project, I get this error: Cannot find module '@vulcan/multi-env-demo'. I can't find that package here or in the main Vulcan repo.

We need to manually add command typings to cypress/support/index.d.ts, command per command.
This is of course inelegant.

Ideally, Cypress commands should export their own typing, and export it in a single interface.

It could look like this:

// cypress/support/myCustomCommands.ts
// ... my commands here

export interface MyCustomCommands {
     someCommand(param: string): Chainable<Element>
}

// cypress/support/index.d.ts
import { MyCustomCommands } from "./myCustomCommands"
declare namespace Cypress {
  interface Chainable extends MyCustomCommands {
  }
}

The problem is that I've never managed to import the Chainable type in my custom command file. I am not able to extend the "Cypress" namespace directly from the interface either.

Relevant doc: https://docs.cypress.io/guides/tooling/typescript-support.html#Types-for-custom-commands

Steps to solve

• Check example with Cypress recently merged in Next examples folder (a new example has been added in oct 2020)
• Improve Vulcan Next accordingly

There are great addons in the ecosystem.

We should include in this starter a handful addons that make sense for user that want to design accessible applications.

Goal

Progressively transition from Meteor to Next app.

• First being able to connect any type of JS frontend to a Vulcan backend
• Then reuse Vulcan code in Next, both backend and frontend
• Then use Next as our main framework

Some code will be specific to Next, however this transition will also allow to reuse Vulcan code in other type of React frameworks (eg Gatsby), at least for the frontend.

Things to get sorted out

Multi-environment NPM packages

• How to create npm packages that works differently on the client and the server?
  The idea is to provide fullstack package as we are used to do in Vulcan.

For Storybook I've created a specific Webpack handler, but it would be better to have some out-of-the-box pattern.

Usually NPM expect you to define a unique main field that points to your package built code (eg dist/index.js). Instead, we may want to point a directory (related SO question), maybe with `exports: { ".": "./dist"}. We could also put built file directly at the project root.

We probably will need to define Webpack loaders/config at least for client side imports, that can differentiate environments.

Mongo

• How to handle the Mongo database? Probably a simple Docker command in development

Others

• How to handle startup logic. On the client it's ok (see Storybook code in Vulcan) but on the server we don't really have a startup system. We might probably explicitely call a function in our Next app.
• Equivalent to Meteor boilerplate (setting up the barebone HTML page) => Custom document
• What kind of example app people want to see in 2020?
• Easy deployment, including deploying a mongo database
• Figure out npm link with Lerna, to allow a pattern similar to METEOR_PACKAGE_DIRS under development => calling lerna bootstrap or lerna link should work
• ... others

Things to do

Steps to go from Meteor packages to NPM packages

• Copy the package in the packages folder of vulcan-next-starter
• Make it work in the Next application, demeteorize
• When the package is ok, move it to the Lerna monorepo vulcan-npm (WIP: we need to add a Webpack config in this repo to handle multi-env package)
• Update the Meteor packages to reuse the NPM packages as much as possible => from meteor/vulcan-core to @vulcan/core

The goal is to both allow to use Vulcan outside Meteor, in Next, and keep maintaining the Meteor version with a minimal amount of code.
To be done for most existing package, however note that packages may exist in Next already. Eg handling styled components. In this case no additional work is needed, it's user responsibility to install relevant packages.

Specific demeteorization tasks

• Move to Mongoose for collections VulcanJS/Vulcan#2479
• Graphql Accounts issue PR
• From Jest to Mocha (should not be too hard)

Documentation of equivalent patterns

Coming from Meteor, we need to get a grasp of equivalent patterns in Next.

• npm link vs a 2-repo install with METEOR_PACKAGE_DIRS => lerna bootstrap should install and link. However npm i [email protected] may sometime fails. Adding manually the package in package.json seems to work.
• Handling environment specific settings , with next.config
• Testing packages altogether
• Create a new package in the monorepo: cd packages && mkdir vulcan-whatever && cd $_ && npm init, define index.server, index.common and index.client (or just index for only one environment). Or lerna create
• ...

Tooling

• Setup Jest in the monorepo
• Jest in the Next app
• TypeScript in the monorepo
• Export to JS when publishing on NPM
• Eslint/prettier in Lerna
• A webpack config in the monorepo able to handle NPM package with index.client and index.server to have 2 different build. For example @vulcan/core index.server.ts reexports @vulcan/users => it needs Webpack + our custom helper to know which index file to load from @vulcan.users automatically.

Feature-wise goals

In order to make the process easier, here are some intermediate goals. Each may need some of tasks listed above to be achieved beforehand.

• Use @vulcan/users useCurrentUser hook in vulcan-next-starter: need to be able to connect to Vulcan OK, need a custom Webpack config in the Next app to load packages with environment-linked index files, need to be able to build our TypeScript code in the monorepo
• Same but in Vulcan Meteor meteor/vulcan:core package, in order to start reusing the same code. Not sure how to import our multi-env NPM packages in Meteor though, we may need explicit @vulcan/core/index.[environment] imports.

Use case idea

Working on a simple business use case would allow us to create example more easily. We could for example develop a Job board, or an app similar to "Awesome Vulcan" to publish vulcan ressources.

[This document is a Work in Progress. It will be updated with additional questions, answers, and process to move from Meteor to Next]

Ressources

• https://github.com/comus/react-vulcan-proposal
• Lerna with TypeScript

Contribute

Post a comment on this issue if you want to help or take care of one of the tasks.

Problem

Currently, Vulcan rely massively on the application startup process to dynamically generate features:

• the graphQL schema
• the Components registry, with replacement support
• the routes
• the graphQL fragments
• the Mongo collections

This approach was a great innovation when Meteor was just a young framework.

But nowadays, web development framework are moving away from dynamic generation, and prefer richer build step.

• Server side, this is necessary to reduce the startup time of lambdas in serverless architecture
• Client side, this is necessary to dump the startup step altogether, in order to fasten rendering
• It's a game changer for 3rd party tools, that can import built code easily (GraphQL schema static analysis, Storybook...).

Vision

For Vulcan, that means replacing all registerComponent, addRoute, registerFragment and the associated startup calls to populate various objects (the router, Components, Collections...) by build-step patterns whenever possible.

We suppose here that the code is structured in packages. The package syntax will probably be specific to Vulcan, but should rely on Next incoming plugin system.

An export standard

See #9

Relevant patterns to combine static build but allow modifications at the developer level

To be completed, multiple patterns can coexist.

Copy paste (appliable to Mongoose schema, Graphql schema)

Allow user to copy autogenerated files into his code for modification (eg for the graphql schema)

Examples

Copy .vulcan/graphqlSchema.graphql to src/lib/graphqlSchema
Generate routes directly in src/pages

Limit

Then how do you update the auto-generated part? How do you detect that users applied modification to the pages?

Mix (for replaceable Components)

Mixing autogenerated components and user override in a single file at build time?
The issue is with nested components.

See VulcanJS/Vulcan#2549

Example

In Vulcan, we define default components:

// in vulcan code

// some non replaceable component
const Footer = ({children}) => <footer>{children}</footer>

// replaceable components (exported)
export const DatatableFooter = ({children}) => (<Footer>{children}</Footer>)
export const DatatableContent = ({children}) => (<main>{children}<DatatableFooter /></main>)
export const Datatable = ({children}) => (<DatatableFooter>{children}</Footer>

In users code, we define some overrided versions, but only for some components. We keep the default for other components:

// in user override
import { DatatableFooter } from `vulcan/ui`

// some non exported code
const StyledDiv = (props) => <div {...props} style={{backgroundColor: "red"}} />
export const DatatableContent = ({children}) => (<StyledDiv>MY CUSTOM CONTENT {children}<DatatableFooter /></StyledDiv>)

It will be mixed at build time into:

// FINAL BUILT FILE
// We first have code necessary to components
// copy pasted as-is from Vulcan
const Footer = ({children}) => <footer>{children}</footer>
// copy pasted as-is from user override
const StyledDiv = (props) => <div {...props} style={{backgroundColor: "red"}} />

// Then components
// from vulcan
export const DatatableFooter = ({children}) => (<Footer>{children}</Footer>)

// nested component that has been overriden
export const DatatableContent = ({children}) => (<div>MY CUSTOM CONTENT {children}<DatatableFooter /></div>)

// from vulcan again
export const Datatable = ({children}) => (<DatatableFooter>{children}</Footer>

Limit

How do you even code that?
How to handle imports in the user code?
How to correctly type props?
How do you handle all additional code, for example 3rd party package used to implement some component? Eg package imported at the top level for each component? The risk of name clashes? Will we need to define only one file per component in Vulcan, so that we explicit the dependency to some additional code on a per-component basis?

Others?

Per feature

Routes

Build step

This means copying package "pages" into the "pages" folder of the Next application.

User overrides

??

Updates in case of user overrides

??

Collections

Build step

This means translating Vulcan schemas into Mongoose schemas.

User overrides

??

Updates in case of user overrides

??

GraphQL schema

Build step

This means creating a .graphql file.

User overrides

???

User extension

We can probably stitch schemas. Use should be able to enjoy text editor features, so writing directly into .graphql files or gql tags.

Relevant tools:

• https://github.com/Urigo/merge-graphql-schemas

Query and mutation resolvers

??

Fragments

Build step

This means creating multiple .graphql file.

User overrides

We can expect user to create its own fragment, independant from Vulcan default fragments. So no need for overrides.

Mutations and resolvers

We need both the GraphQL part and the JS implementation. For the GraphQL part the process is the same as for any part of a grapqhl schema.

For the function we can simply export and merge into the global schema.

Components

Here the tricky part is replacement. See this issue

i18n files

They are currently a pain point because they may bloat exports. Relying directly on an i18n JSON file could work.

Open questions:

• Where to we put the generated code? In a .vulcan folder similar to .next and .meteor?
• How user can extend this code ?
• How user can override certain parts of the code ? Eg modifying a generated page, a generated graphql schema ? A component ?
• How do we update the auto-generated part when the user has applied overrides ? For example in the graphql schema?
• When static analysis is needed (eg to mix Component overrides), where to apply it? To TypeScript code? To built babel code? It probably needs some AST feature?
• How to handle build step in the dev process? Rebuild on file change?
• Easy creation of Storybook addon (eg to add i18n to preview for instance)

Goal

Vulcan is a big framework. We want to include a lot of features for you to get started.
But sometimes, you need something simpler. Maybe you don't need MDX to write your doc, or you only want to keep the server. You may prefer a CSS solution to Material UI.

What's more annoying in a starter than needing to remove all the default samples?

We need to include a few code examples in Vulcan Next Starter, to demo features but also to be able to test that everything is ok.

Ideas

• Isolate code samples, demos, and framework related stuff in folders named vn. The packages folder also isolates features.
• Write a script that eject from Vulcan Next, to get a clean start.
• Write scripts to remove specific features. UI libs like Material UI are a first good example
• Put configs into .vn: even if you don't remove them, they are isolated and easier to locate and update

Getting rid of Material UI

Material UI is an opinionated choice, but we should at least be able to document where it impacts the app even if you want to use only some components

• Material UI setups a default, global theme => remove the CSSBaseline component will remove styling of body

Goal

This helps setting up API keys for instance, that you may want to be sent in all your graphql queries.
Eg to setup API keys #54

Steps

• Add a configuration item in next.config.js where we can add default headers (check the progress of #62 before doing this)
• Load the default headers based on this configuration in apollo client
• Advanced: demo a custom header computation with Apollo client setContext and middleware system

Goal

The goal is to make Vulcan Next frontend compatible with Vulcan backend, be it a preexisting Meteor application or a newer version built on top of vulcan-npm packages.
This should allow you to easily spawn a full-stack application, using Vulcan schema-system, while keeping it very easy to use any other kind of backend technology in respect with the JAMStack philosophy.

Ideas

• Easy authentication to a Vulcan backend with hooks
• Data fetching and mutation hooks
• Pre-made components
• Easy authentication to legacy Meteor Vulcan Backend

Specific to Vulcan Meteor

• id format of older Meteor Mongo databases doesn't seem to be recknognized by Mongoose v5? To be confirmed. Can be checked like this: const mongoose = require("mongoose"); mongoose.Types.ObjectId.isValid("5ff8709323fa1715ff457866")
• preserver user accounts

=> best solution seems to create a "vulcan-next-with-meteor-backend" clone of Vulcan Next with a demo connexion to a Vulcan backend, so we keep both versions separated.

Next does some magic that makes the iconic import React from "react" optional.
But... that barely works with 3rd party tools, that will cringe when they encounter JSX with React not present as a global (it make sense, as built code will massively rely on the React global under the hood).

Repro branch: "bugfix/jest-optional-react-import"

Steps to solve

• Decide whether we keep React standard behaviour (import React mandatory in JSX/TSX) or Next standard behaviour (import React optional)
• if we prefer Next manage to make this import optional in Jest, Storybook, Cypress
• if we prefer React show a warning when React is not imported in Next

Goal

Most file uploading/downloading tutoriels on the web limits to public files.

The main goal here is to handle both public and private files, with the advanced permission system of Vulcan:

• ability to share a link to the file, that will only be usable by authorized people only

Which translates from the app point of view:

• ability to check ownership of a file, based on a document in the database representing file permissions, before we deliver it

Basically it should be easy to create a datatable of files, exactly like we do with other types of documents on Vulcan. The link to the file acts exactly as the unique _id of a document or as a slug, except that we can return the raw data instead of JSON or a graphql response.

Ideas

Possible solution:

• Connections to cloud based service?
• Using on-premise 3rd party tools?

Idea

Calling registerComponent or addRoute anywhere in the code is magic but we won't be able to track those down during a build step.

Instead, we could expect developer to explicitly export routes and components at the root of every package.

This approach is now recommended in storybook, that moved from the dynamic pattern :

storiesOf("whatever").add("some story", () => (<MyComponent />)

to Component Story Format

export default {
  title: 'Path/To/MyComponent',
  component: MyComponent,
  decorators: [ ... ],
  parameters: { ... }
}
export const someStory = () => (<MyComponent />)

The big difference is that now stories can be understood by a build tool, since they are explicitly exported. Advantages are basically infinite, mostly regarding 3rd party tools interaction.

Apollo Universal Starter Kit is close to this pattern, but still rely on a dynamic approach by exporting a ServerModule object:

import Counter from './sql';

import schema from './schema.graphql';
import createResolvers from './resolvers';
import ServerModule from '@gqlapp/module-server-ts';

export default new ServerModule({
  schema: [schema],
  createResolversFunc: [createResolvers],
  createContextFunc: [() => ({ Counter: new Counter() })]
});

I'd like to go even further and expect the user to simply export an object. Vulcan would take care of generating relevant code.

Next

Eventually, this format should be somehow compatible with Next plugin format: vercel/next.js#9133.

It could be done through a specific field of the export or a specific file in our Vulcan packages, like next.plugin.ts.

Examples

• Material UI is a good example for packages that should affect the frontend/SSR at a low level.
• Vulcan automated backoffice is a good example of page generation
• Vulcan auth is a good example of full-stack feature (auth resolvers, mutations, and UI
• ...to be completed

List of needs

• Wrap the custom "_app" component with new items, eg Material UI Theme
• Add hooks to the custom "_app" component, eg Material UI JSS removal
• Export custom values specific to the package, eg Material UI themes
• Extends the document "getInitialProps" method with Material UI (not easy, because we need to enhance the ctx object, then compute the raw "Document.getInitialProps", then compute Material UI own props, so there is an imbrication)
• Extend the Storybook Webpack configuration
• Extend the app Webpack configuration
• Extend the Cypress Webpack configuration
• Declare the possible settings
• Export graphql fragment, schemas
• ...to be completed progressively

Next does some magic with tsconfig, so it's safer to use it as much as possible.
But some questions stays open.
Specifically, how do we exclude stories and tests from the build, without breaking VS code autocompletion + specific builds (storybook, tests)
vercel/next.js#12076

Steps to solve

• Wait for a fix from Next

Hi,

I suggest for improvement in Getting Started as following:

In another terminal, run the Next development server:

GRAPHQL_URL="your-vulcan-server-url" npm run dev
#or
GRAPHQL_URL="your-vulcan-server-url" yarn dev

Should change to =>

In another terminal, run the Next development server:
git clone https://github.com/VulcanJS/vulcan-next-starter.git
cd vulcan-next-starter
npm install next react react-dom
yarn add @apollo/react-hooks apollo-cache-inmemory apollo-client apollo-link-http apollo-link-ws graphql graphql-tag subscriptions-transport-ws isomorphic-unfetch next-with-apollo
yarn add typescript -D

GRAPHQL_URL="your-vulcan-server-url" npm run dev
#or
GRAPHQL_URL="your-vulcan-server-url" yarn dev

Thank You
Adalidda

Happens when running test:vns, which are special internal tests that validates coverage.

Probably a clash between browse tests with JSDOM and server-side testing of the Node server.

http://mongoosejs.com/docs/jest.html

Goal

Next is all about scalable and performant deployment, let's demo that.

Ideas

Support various host

• Easy Vercel (ex Zeit's now) deployment.
• Add a "deploy now" button that clone and deploy to Vercel https://vercel.com/new/git/external?repository-url=https%3A%2F%2Fgithub.com%2FVulcanJS%2Fvulcan-next
• Check that static hosts are working ok with dynamic pages like [id].tsx
• Easy Cloudfare deployment (static)
• Easy Heroku deployment
• Easy AWS, Azure and GCP deployment
• Easy Github Pages deployment (static). Recently add an issue, that requires to create an empty .nojekyll in the out folder after a next export: https://github.blog/2009-12-29-bypassing-jekyll-on-github-pages/
• Easy Netlify deployment
  Netlify is slightly trickier to support because it may slightly lag behind Next.js features that are strongly tight to Vercel, like next.config.js rewrites or _middleware. Appropriate equivalent feature from Netlify must be used in this case (Edge Handlers etc.). Hopefully Netlify aims at providing a great support for Next.js.
  At the time of writing 11/2021 we wait for this ticket to be merged: netlify/next-runtime#706

Provide dockerfiles

• For Cypress e2e testing
• For the application
• For Jest

Control docker image size

Image size tend to explode, due to node_modules (prod but also dev) => can reach multiple GOs!.
We should be able to run the app correctly after pruning non production modules, using npm prune --production.
Can't work yet with Yarn sadly yarnpkg/yarn#6373
We should also find a compromise between caching and image size.

CI/CD

• Easy setup of a CI/CD server for Vulcan Next
• Check bundle size using Github actions https://github.com/hashicorp/nextjs-bundle-analysis

After exploring various styling approaches with Next, I came to the 2 following possible patterns

1. The CSS-in-JS approach: Material UI+ Styled Components Material UI v5 (emotion)

Why not material UI alone

makeStyles may induce performance issues, especially with TypeScript. Styled Components is a powerful CSS-in-JS replacement solution, officialy proposed as a replacement into the incoming v5 of Material UI.

Note: Emotion is probably also a valid candidate, but Styled Components is a bit simpler.

UPDATE MATERIAL UI V5: We have updated Material UI to V5, so using Emotion.
This is still a beta version, we need to figure a few thing:

• Theme switching sample is not working, we may need to follow https://stackoverflow.com/questions/68382679/toggling-between-dark-light-mode-material-ui/68385247#68385247
• styled API is still a bit confusing (https://next.material-ui.com/system/styled/)
• There are conflict between using vscode-styled-components plugin (needed to highlight styled.XXX template stringsà and vscode-styled-jsx (needed for Next), see styled-components/vscode-styled-components#310 (comment). It probably affects emotion as well.
  It means the JS syntax should probably preferred when using this CSS-in-JS approach.
• Find learning resources, eg https://www.joshwcomeau.com/css/styled-components/
• Example have been updated recently, they should help using the lib as it's intended to be used

Limitations

You have to learn additional lib, and they add more complexity to your Next app. Some people strongly dislike CSS-in-JS

2. The purist approach: Styled-jsx and PostCSS, (+ bootstrap or tailwind or whatever CSS framework you like)

See official doc of built-in CSS support in Next

Why PostCSS

Styled-jsx is a very "raw" solution. It introduces CSS in React the right way, but without much additional feature.
Nesting is a powerful feature, not included into styled-jsx. So in VNS we also bundle PostCSS.

~~### Why NOT CSS modules

They don't behave correctly with Storybook. However they are still included as a default in Next so if you don't bother about Storybook you can already use them in VNS.
Edit: we now are able to use CSS modules in Storybook, it simply needs the right plugin.~~

Limitations

Styling child component leads to a poor syntax and requires wrapping with div or span. Basically, you can't restyle Material UI components easily.

Also, you can't setup VS Code with PostCSS + styled-jsx.

To be done: easy swap

Currently, we have both solutions preinstalled in VNS. In the future you should be able to enable only one of them. In particular, the Material UI + Styled components approach should be optional. We currently wait for plugins to be added to Next before investigating more.

vercel/next.js#9133

Edit: even better, we could mix Emotion and Tailwind: vercel/next.js#23830

To be done: explore CSS modules

CSS modules didn't work in Storybook, until we figured the right package to support them. Now they work as expected so we may want to investigate them a bit more (ability to restyle existing components, syntax, should it be prefered to styled-jsx etc.).

Check doc here: /docs/from-vulcan-v1.md

See vercel/next.js#14308

Runtime configs are not advised, because they can't be deployed in a serverless environment. However, environment variable are clearly not sufficient to setup the app: how to parse arrays, how to dynamically set a value? If only because they are string based, and not very intuitive to consume.

See src/lib/api/cors for instance:

• we can't easily tell that cors can be set using APOLLO_SERVER_CORS_WHITELIST
• we need to parse it, since its not an array but a string
• no nesting, so except dirty names with long prefixes

This cypress/support file is imported client side => we have no way to pass it process.env.COVERAGE (or at least, I found none).

It will thus add annoying warning when running tests in Cypress:open, saying that code is not instrumented, even when we actually don't run e2e testing.

Solution would be to check how to pass an custom boolean to "support.ts" or if there is a relevant option in Cypress.

See relevant issue in Storybook: storybookjs/storybook#9610 (comment)

Tracking branch:
https://github.com/VulcanJS/vulcan-next-starter/tree/bugfix/css-modules-storybook

Main Next+TS+storybook issue:
storybookjs/storybook#9610

CSS modules imports are not correctly loaded in stories, they get an empty object.
However, surprisingly, styled-jsx worked out-of-the box with new Storybook zero-config TS support.

Steps to solve

• Wait for a fix from Storybook or Next

Issue can be seen in Storybook.

Goal

Have a clear vision of the ecological impact of Vulcan application.

It means limiting consumption during development (optimal build), during deployment (static build), and during consumption (caching data etc.).

Ideas

• Check what's known about Next performances and power consumption, in dev and when it runs
• Find tools to analyze energetic consumption either at runtime or during the development process
• Find possible improvements (smarter build etc.)
• Advertise green data centers to host Vulcan applications

Resources

• From the following article it seems that a key element, in addition to limit the computation volumes, useless heavy images etc., is picking a data center fueld by renewal energy
• Vercel green energy policy
• [Netlify green energy policy](https://community.netlify.com/t/is-netlify-powered-by-renewable-energy/3951=
• https://www.netlify.com/sustainability/
• https://erjjiostudios.com/#about
• The green web foundation

In French 🇫🇷

• https://richardhanna.dev/blog/2021/open-source-et-numerique-soutenable.html

Update based on Next improvements

Future version of Next (9.6 most probably) will include a smarter setup for i18n, that we should include into Vulcan when it's more mature.

We already support basic i18n features.

Something to dig for the key extraction workflow: https://formatjs.io/

Update 2021 10

A big players:

• https://github.com/vinissimus/next-translate
• https://github.com/isaachinman/next-i18next

For both, we will probably lose next export, though that's acceptable as there is no real use case for it as soon as you use dynamic routes (Next.js cannot handle dynamic routes in exported app automatically, you need a gateway to handle them)

They however sound quite over-engineered...

Playing around with rewrites and an URL parameters might help simplifying this setup?

Old issue

i18n is now working ok, using next-i18next, but we need to test in more contexts to confirm:

• i18n with SSR in dynamic mode => OK, and tests with Cypress
• Route based i18n ("vulcan/fr/foobar")
• i18n in static mode
• i18n when deploying to Vercel #36
• i18n in API routes, GraphQL queries like in Vulcan

Other features:

• Powerful language detection like in Vulcan
• Package based expansion of locales (using namespaces?)

Older issue (for reference)

Next-i18next: currently does not support Serverless mode, as it needs a custom server.
But maybe that's overkill?

Next Right Now seems to have achieved i18n without custom server, by simply using react-i18n. To my best knowledge, Vulcan is also able to do that without actually using any kind of specific Express middleware.
In NRN, see src/middlewares/localeMiddleware.ts

• e2e test of i18n https://glebbahmutov.com/blog/cypress-tips-and-tricks/#control-navigatorlanguage, https://stackoverflow.com/questions/54671266/setting-the-browser-language-in-cypress
• Route based i18n ("vulcan/fr/foobar")
• i18n basic client side
• i18n SSR https://react.i18next.com/latest/ssr
• i18n in static mode
• i18n in dynamic mode

Step 1: clarify why "next-i18n" chose to rely on a custom server

We need to fully know the technical limitations of a serverless approach, at least to document it.

i18next/next-i18next#274

Answer to this probably lies into i18next-http-middleware. It used to be bound to Express, but the "http" version is agnostic: it's probably usable directly in Next _app?
It seems to handle language detection in requests for instance.

Step 2: figure out how i18n work in Vulcan

Provider and language detection is done in:
packages/vulcan-core/lib/modules/components/App.jsx

Intl Provider is defined here:
packages/vulcan-i18n/lib/modules/provider.js

Step 2: implement a serverless i18n version

We can tolerate to have limitations in the first implementation, as they will probably mostly affect advanced i18n patterns for static application, eg adding the locale in the URL.

i18next/react-i18next#715
https://gist.github.com/odensc/216288159aaa2cb41fc924774fce5859
https://github.com/borispoehland/next-i18next-boilerplate/blob/master

git checkout feature/29-i18n

cypress-io/cypress#4771
Update our custom ssr Cypress command when its resolved by Cypress.

@see vercel/next.js#11106 (comment)

• Add recommended packages useful for Next
  Recommended packages (use Ctrl+P in VS code to run the command):

ext install vscode-styled-jsx
# ext install vscode-styled-jsx-languageserver # Do not seem to work correctly

There is a way to add recommended packages to the workspace config, so that VS Code recommend them directly:

https://medium.com/better-programming/how-to-configure-vs-code-like-a-pro-782d2d718586

Goal

🇪🇺

Ideas

• Get more knowledgeable about GDPR and the scenarios where it applies
• Help user setting up relevant components/checks when GDPR is needed
• See https://github.com/InteractiveAdvertisingBureau/GDPR-Transparency-and-Consent-Framework
• Ability to embed Google Fonts used by Material UI: https://thehackernews.com/2022/01/german-court-rules-websites-embedding.html and https://leerob.io/blog/fonts#self-hosting (we might need to play around with vercel.json for specific HTTP cache optimizations)

See cypress/support/commands/ssr.ts. This command is based on @bahmutov recipe.

However it seems to work only with one test. cy.state is probably doing some weird asynchronous stuffs, and is not yet documented.

Issue for cy.state documentation: cypress-io/cypress-documentation#108

To repro:

• git checkout bugfix/40-cypress-ssr-visit
• yarn run dev
• yarn run cypress:open
• Run i18n tests. The fr test will pass, but the document is not correctly rewritten for the en test.
  Code is in cypress/integration/vns/i18n.spec.ts and cypress/support/commands/ssr.ts

I've tried to clone document, no success.

Goals

Make your experience as a contributor a great moment.
Get an higher proportion of PR merged
Shorten the time for issues to be solve
Keep the framework easy to maintain
Involve the user community

Ideas

We should seek examples and good practices from other successful open source projects, and from articles in the wild.

• Listing contributors / people / companies automatically in the Readme (see open collective)
• List things made with Vulcan in the Readme
• Create issue template
• Create PR template
• Guide to write a good PR that have huge chances of being merged
• Onboard beginners
• Find an efficient branching model for managing new features, bugfixes, and bugfixes on older -version
• Document release process
• Document the perfect VS code setup, especially for Prettier/Eslint linting #50
• Run linting on commits/merges (eg with Husky)
• Style guide
• Make bug reproduction as easy as possible => should document all information to create a new clone and send it to us? Or even run Vulcan Next in some kind of online playground? See https://www.pixiplayground.com/#/edit for example, but this is a lib so it's easier to do this.
• e2e and unit tests (see tests/vns) to avoid breaking change
• CI/CD

Goal

Add depcheck to the release process

Seems to work ok, we could add it to the release process (I've had issue with very big node_modules bundles).

https://www.npmjs.com/package/depcheck

Steps

•  Test depcheck on the current codebase
• If it works ok, document in the release doc, add to the release process

Similar to this closed issue istanbuljs/babel-plugin-istanbul#62

Steps to solve

• Check the open issue linked above
• Fix accordingly by moving nyc config out of package.json into an independant config file
• Check that coverage still work ofr e2e and unit tests

Since Next is supposed to be an efficient static build tool, I think it would be relevant to put the docs within the starter.

• Install MDX in the starter. You can now edit src/pages/docs folder and get the documentation automatically generated when you run the app. It's available on http://localhost:3000/docs locally.
• Automatically generate documentation based on TS typings? From @Apollinaire: the Apollo documentation is an example of how to generate the doc from TypeScript. They seem to combine Gatsby, Typedoc, and MDX to render the documentation while being able to automate many tasks and keep the API in sync.
• Host the TS documentation on Vercel instead of Netlify
• Docusaurus seems more apprioriate thant he Next.js embedded documentation: https://docusaurus.io/
  We should keep the Next.js .mdx setup as a demo, but use Docusaurus for Vulcan documentation, both for NPM and Next.js versions
• Publish Vulcan NPM Storybook on either Chromatic or Vercel as a build site
• Write the best doc ever
• Find patterns for interactive docs, eg Vulcan's getting started or Next dynamic quizz => this the /learn page
• Organize Q&A, write articles, do videos...

• The storybook MDX setup should belong to a package
• Wait for a better support of ".md" files in Storybook (currently conflicts with default behaviour, note that this issue does not apply to ".mdx", only ".md": storybookjs/storybook#7644)
• Easier opt out
• Currently have to explicitely import src/types/mdx.d.ts, it should be loaded from tsconfig instead

Currently, there is a small issue that blocks deployment of internationalized app on Vercel:
vercel/next.js#13624
i18next/next-i18next#274

Goal

Slow build is the number 1 productivity killer. Building a Vulcan Next app should be as fast as possible, even when your app grows big.

Ideas / leads

First, we need information, so we should facilitate bundle analysis

• Stop server-side code leak / verify bundle size. Seems that for instance @vulcan/schema is exposing resolver, in particular client side. We might need to create 2 endpoints for those packages.
• Find tools to diagnostic TypeScript/Next build performance
• Add webpack bundle analysis to Next
• Also find tools to diagnostic other tools build performance (Storybook, Cypress, Jest)
• Keep automated tests fast
• Automate Lighthouse audit

Then, if we notice specific issues, we should solve them one at a time.

In Cypress, if you run vulcan-next-starter/basic.spec.ts, everything is fine.
But if you run vulcan-next-starter/basic.spec.js, (a JS file) then basic.spec.ts build will be triggered, and will fail with errors regarding mocha globals (describe etc.).

To reproduce:

• git checkout bugfix/11-webpack-cypress-js-import && yarn && yarn run dev
• open Cypress yarn run cypress:open
• run spec basic.spec.ts > ok
• run spec basic.spec.js > not ok! It acts as if TypeScript types were unknown.

• If you disable typechecking in cypress/webpack.config.js, it will work fine. But it hides the underlying issue.

Currently we are forced to use transpileOnly option of ts-loader in Cypress to bypass this.

Steps to solve

• Decide if we should keep it possible to use vanilla JS or not in Vulcan.
• Define on which criteria we should take this decision (speed of build, complexity of Vulcan, beginner-friendliness for instance)
• If we keep .js, have a better support for JS files within Typescript

Will probably be solved soon but to be noted.

Relevant issue:
storybookjs/storybook#11132

Hi,

The doc of vulcan say:

Connecting Remotely
You might sometimes need to connect to your GraphQL endpoint (typically accessible at http://yourapp.com/graphql) from outside your app.

API Key
You can make authorized requests to your GraphQL endpoint by defining an arbitrary vulcan.apiKey setting in your settings.json file (make sure to keep it private by keeping it out of the public block), and then including an apikey header whose value matches that API key:

{
"vulcan": {
"apiKey": "123foo"
}
}

How do we use vulcan api in vulcan-next-starter ?

Thank You
Adalidda

kulshekhar/ts-jest#1678
https://github.com/microsoft/TypeScript/wiki/Node-Target-Mapping

Currently the target is ESNext, with Node version 10, but ts-jest seems unhappy with this.

We should also define the expected Node version for VNS.

Goal

App created with VNS should be usable with eyes closed, literally.

Next 11 introduced a conformance system with eslint => we could add eslint rules for a11y

Ideas

• Automated LightHouse audit? => cypress-audit is the perfect package https://github.com/mfrachet/cypress-audit
• Check for a11y best practices
• Make it easy to test the app with a screen readern or as a colorblind user?
• Add a11y addon to Storybook
• Investigate "axe" https://www.deque.com/axe/
• This chrome extension helps testing in Testing Library a lot: https://chrome.google.com/webstore/detail/testing-playground/hejbmebodbijjdhflfknehhcgaklhano it helps having the right selectors for elements of the page
• Adding a code of conduct

Types are clashing with Jest cypress-io/cypress#1091 cypress-io/cypress#1087 cypress-io/cypress#1087

Steps to solve

• Check if the issue is still reproducible
• Check links above for a solution

We need PostCSS, and in particular nesting, for correct override of nested components styles in Next, see relevant discussion.

However at this point vscode-styled-jsx doesn't reckognize PostCSS. There are alternatives for various styling solutions like Stylus or SCSS with corresponding language server, but none for PostCSS as far as I know.

Next makes it difficult to cleanly handle isomorphic patterns with different implementation server and client.
We need to check for SSR to be enabled with a similar pattern as Meteor.isServer, which we actually tended to avoid in Vulcan, considering it an anti pattern.

For instance, we use the same function to create the Apollo client during SSR and in the app. However, it would be more efficient to rely on SchemaLink server side than having the server querying itself.

It could be a relevant use case for our isomorphic build system.

In Vulcan, server side: https://github.com/VulcanJS/Vulcan/blob/devel/packages/vulcan-lib/lib/server/apollo-ssr/apolloClient.js

Goal

Account system out of the box is one of the main love feature of Meteor.

Ideas, leads

Looking for a replacement to Meteor Accounts.

Pure soft implementation, like Passport, for a basic password auth?
Support for 3rd party systems like FusionAuth?

Connect to Vulcan via oauth?
And Vulcan as an oauth server?

Step 1: 2021

• Demo private pages
• Implement password change and password reset
• Implement email verification
• Easy oauth with usual providers => refer to Passport, which allow to add strategies easily
• Auth server => with passport
• Stop using localStorage for auth, including in Vulcan, use Set-Cookie instead
• Create a function to check crendentials => necessary for the update password workflow
• Create a token system for password reset, with a token sent via email => we need to create the token (see Blitz helper for instance) + store it in the user data. Then we can compare the provided token to the one in the db. We might also add additionnal securities like an expiration date. Ignore current code that uses the session token, it's simply the wrong kind of tokens.
  In the meantime cy.request does the job in test.
• Add e2e tests for login, logout, signup, password change
• Add e2e tests for email verification, password reset => ok for password reset, need to be done for user verfication
• Add an SMTP server for email sending, or use console.log if none is set during development => started to work on it, not yet configurable (localhost only + setup in Cypress)

Step 2

• Add tests for API endpoints=> we wait for vercel/next.js#13389 and vercel/next.js#28173 to progress, so that we can test API routes more cleanly. Those discussions should bring ways to easily start and stop the Next server during testing, which is more efficient that trying to spawn a fake Express server. Tests are located in our .vn folder.
• Add API test for error scenarios: trying to reuse the same token twice
• Provide auth components like in Vulcan => ongoing work, we still lack role based + more example (currently, the useUser hook has an option to trigger a redirect if there is/there is no user, but that's simplistic and a tad confusing)
• Refactor by wrapping reusable methods into services (both client side and server side) => on progress, using an "account" lib + the User model. Responsibility of each is not completely clear yet however.
• E2E tests are working but sometimes flaky, in particular the token is sometimes wrongly considered as expired already. Maybe the page is loaded twice by Next during dev? To be investigated.

State of the art

•  Blitz.js has a cool demo and secure setup
• Auth js => we could reuse the token based methods from this library
• Vulcan implementation => we can take inspiration from the logic there, but we need to reimplement the lower level primitives out of Meteor
• Frame project
• Email verification question on SO
• Hackathon starter
• Next auth
• Apollo example with auth
• Our own issue about private pages
• React Roles

Edit: current progress:

• An HOC for pages is available here: withPrivateAccess. You can customize the server side auth check and the client side auth check. It works in all scenarios, from SSR to client-only, and is compatible with next export 🎉.
• I lack a pattern to handle hook based auth (eg useCurrentUser like we do in Vulcan, based on Apollo's useQuery)
• I find the client part clumsy/inelegant, but maybe there is no alternatives?
• I lack a way to handle purely synchronous check client-side, like just reading localStorage
• Improve based on Vulcan's AccessControl
• Check newer examples and technologies: Next-auth, and Next examples related to auth

Goal

Easy, intuitive, way of creating private pages visible only by logged in users.
Should work with and without SSR.
Should also work in client-only context, like any other "JAMstack"-friendly framework.

Previous art

• Vulcan Access Component

• Discussion I've just opened on Next

• Question "How to redirect in getStaticProps" in Next repo. This is the most insightful ressource on the question, as it raises the broader question of redirect in a static page. The page being private is a special case of this, where you not only want the user to be redirected if not logged in, but you also want the static version of the page to never exist in the first place.

• Stack overflow question

• Example that works in dynamic mode, but does not talk at all about static render

• My unanswered question in spectrum regarding redirection Dead link...

• RFC about custom route, don't seem to adresse the issue

• Hosting on github pages => leads to a dead wiki link

• Issue with admin routes

• What about Gatsby? The key difference between Gatsby and Next is the Next's ability to do dynamic server side rendering, which is necessary to create real "private" pages with SSR. In Gatsby, private pages are client-side only. That defeats the purpose of dynamic SSR.

What we need

• During dynamic server render, use an HTTP redirection.
• During client side rendering, use the Router
• During static rendering, disable the route altogether. Private pages should become "client-only" pages, like in a normal SPA. Spoiler alert: this is the use case that bothers me the most here

Private pages only make sense either client-side, or during dynamic server-render, because they depend on whether the user is logged in or not.
Static render happens on your dev machine, once, like the application build itself, so it does not know the end user. Private page of a statically rendered Next app should be rendered client-side only.

First attempt, with getInitialProps in custom _app

Intuitively, if you are used to react-router or alike, you'll want to setup a global redirection at the top level, so basically adding auth checking and redirection in _app getInitialProps

TL;DR: this is wrong. This is intuitively what you'll do when used to React router or alike. You'll define an array of public routes, and let the user pass when the route match. That's wrong. You'll redirect to the login page if the user is trying to open a private route while not being connected. Wrong wrong wrong.

In Next, the router is the page. Everything must happen at the page level. Defining the layout, defining redirections, checking auth, etc., etc.
I've never met yet a page with only private pages, if only the login page.

My takeaways:

• getInitialProps is called client-side too when you click a link. Beware of this scenario, in this case, you can call Router.push to redirect.
• appContext.ctx.res is available both during STATIC server-render and DYNAMIC server-render.You need to check for res.writeHead to be available if you want to redirect during server render.
• Do not change _app render based on props status computed during getInitialProps. I've had a weird behaviour breaking apollo SSR of private pages. Basically the app props were not directly enhanced with currentUser, computed during this call. So the Apollo queries were not collected as expected during getDataFromTree, resulting in an app in loading state.

Code looks like this and you should not do that because this is wrong:

// Various helpers
export const redirectServer = (ctx: NextPageContext) => (pathname: string) => {
  ctx.res.writeHead(302, { Location: pathname });
  ctx.res.end();
};

const isServerSideRenderCtx = (ctx?: NextPageContext) => {
  return !!(ctx && ctx.res && ctx.res.writeHead);
};
const isStaticRenderCtx = (ctx?: NextPageContext) => {
  return !!(ctx && ctx.res && !ctx.res.writeHead);
};
const isClientRender = () => {
  return typeof window !== undefined;
};

// The page
...
// getInitialProps
PrivatePage.getInitialProps = async (ctx?: NextPageContext) => {
  // We simulate private connexion
  const isAllowed = !!ctx.query.allowed; // just for the demo, replace with a check that the user is logged in
  const pageProps = {  isAllowed };
  if (isAllowed) {
    return { ...pageProps, isAllowed };
  }
  if (isStaticRenderCtx(ctx)) {
    debugNext("Detected static render, not doing anything");
    // TODO: this is bad...
  } else if (isServerSideRenderCtx(ctx)) {
    // Scenario 2: we are in a server-side render
    debugNext("Detected dynamic server-side rendering");
    redirectServer(ctx)("/vns/debug/public"); // redirect server
  } else if (isClientRender()) {
    // Scenario 3: we are client-side
    debugNext("Detected client render");
    debugNext("Redirecting (client-side)");

    Router.push("/vns/debug/public");
  }
  return pageProps;
};

Second attempt, with getServerSideProps and getStaticProps, in the page

• the function is not called client side, but is reruns server side

// So beautiful
export const getServerSideProps = async (ctx: NextPageContext) => {
  const isAllowed = !!ctx.query.allowed; // demo
  if (!isAllowed) {
    debugNext("Redirecting (dynamic server render)", ctx.req.url);
    redirectServer(ctx)("/vns/debug/public");
  }
  return { props: { isAllowed } };
};

Issues I struggle to solve

• I can't tell from the doc and all articles and answer what's the best course of actions to create a private page in Next.

• As far as I understand, next export is the only way to have an application that do not rely on the server. Something I could host on GitHub pages.
  From the doc:

Next.js automatically determines that a page is static (can be prerendered) if it has no blocking data requirements. This determination is made by the absence of getServerSideProps and getInitialProps in the page.

Agree with that. However, it's not because a page cannot be prerendered, that you don't want the whole app not to be statically exported.

next build WON'T create such an app, as far as I understand. Or at least, I found no doc about this.

If you define a page with getServerSideProps, which you need to in order to have server-side redirect, Next considers that you opt out from static export. This feels wrong, I could still want to have a simple client-side redirect in scenarios where

• More broadly, it seems that Next that your app could both be deployed as a static app, and using SSR. However I often encounter the scenario where the app as a canonical SaaS version, but could also be deployed on-premise. In this case, having a client-only version is nice, even when it includes private pages.