mapbox / batfish Goto Github PK
View Code? Open in Web Editor NEWA static-site generator for React and Markdown
License: ISC License
A static-site generator for React and Markdown
License: ISC License
The dotcom repo includes the MagicImage system.
Two parts to this, both of which rely on a single config:
I think this system has worked well for dotcom, and we should carry it over. This could be a standalone open source package — so we could put it in its own repo, or we could make this repo a Lerna MegaRepo.
Maybe we should call the image system ChunkLite, instead of MagicImage ...
@samanpwbb @mayagao @tristen: Have you run into any problems with MagicImage, or do you have ideas for how it should be improved before we generalize it?
I notice that create-react-app does this. Seems like a good way to avoid having to add forgotten extensions to the list every time you think of some other file type you'd like to use.
I forgot to document that you should use react-helmet to modify the document <head>
. It is in there as a peer dependency — just forgot to document.
If hijack-links.js works well, it is useful for any user of a client-side router. For example, you could avoid the need to use ReactRouter's Link
component — something I've always wanted.
We need to thoroughly test this across browsers and devices, though, before we make presumptions.
This would be the best way for us to provide helpful clear error messages if, for example, your wrapperPath
is incorrect and does not actually point to a file.
Gotta establish what happens if you hit a route that doesn't exist.
An unknown property probably means a typo, or a missed guess. It makes debugging that mistake easier if there's an error thrown about the unknown property.
Webpack v3 came out. Let's see if everything here is compatible with it, and if so upgrade.
We should upgrade the demo directory to be a viable (and not super ugly) example. We might want to add a few examples. We should make sure examples illustrate all of the functionality. Here's a checklist (all of these need to work for both dev and static builds):
this.props.frontMatter
(@jfurrow self-assigned this)batfishConfig.pagesDirectory
batfishConfig.outputDirectory
batfishConfig.siteBasePath
batfishConfig.siteOrigin
(@jfurrow self-assigned this)batfishConfig.wrapperPath
batfishConfig.notFoundPath
batfishConfig.temporaryDirectory
batfishConfig.dataSelectors
, selectively injected into pagesbatfishConfig.vendorModules
batfishConfig.webpackLoaders
, including asset-optimization loadersbatfishConfig.webpackPlugins
batfishConfig.babelPlugins
batfishConfig.babelPresets
batfishConfig.babelExclude
, maybe using a promise-fun module, which requires compilationbatfishConfig.externalStylesheets
batfishConfig.fileLoaderExtensions
batfishConfig.port
batfish/prefix-url
batfish/route-to
batfish/with-location
There are two kinds of Markdown support to consider: Markdown embedded in JS, and Markdown files used as pages.
Looking something like this:
import md from 'some-module-name';
const text = md(`Some **markdown**`).
This would provide the flexibility we'd want to be able to use Markdown anywhere that we have extended prose.
However, we don't want to bundle a Markdown parser and send it to the browser. I think the way to do this is to build a Babel plugin that uses remark-react at compile time to transform all uses of md()
.
Here's an existing Babel plugin to learn from: https://github.com/threepointone/markdown-in-js
This is the class Jekyll format. Both Phenomic and Gatsby support it, so it should be easy to learn from their open-source code. We'd use front matter to pass props to a specified layout component.
There's a big limitation with this format: Although (I think) you could embed HTML directly into the Markdown in a document like this, you could not embed JSX and use React components. If you want to embed JSX, it makes more sense to avoid this additional level of abstraction and write the page as a (relatively simple) JS file. I think that's ok. If people need to do a minimal amount of custom styling interspersed in their prose, they can do that with HTML. If they need more, they can create a page component.
Because pages are found and front matter is parsed once when the Webpack config is created, it will not update dynamically while the watching & dev server are running.
Definitely should be documented. Maybe can be improved.
@davidtheclark wrote:
Let's figure out a good cut-the-mustard tests for IE<11, Android<4, Safari<7, and any other browsers like those, and show a banner to those users.
Here's some additional context about "cutting the mustard" from BBC News engineers.
Let's identify which features might be problematic in the supported browser ranges and create simple tests for them. If a test fails, we should display some warning to the user, indicating that they may experience some issues, and provide a link to help them upgrade (maybe http://outdatedbrowser.com).
Here are some implementation ideas:
localStorage
Next steps:
We should validate the config upfront, instead of allowing it to produce mysterious, less-easily-debugged errors.
https://github.com/mapbox/batfish/blob/master/docs/configuration.md#jsxtrememarkdownoptions includes a list of configuration options. The list includes both https://github.com/mapbox/jsxtreme-markdown#tojsx options as well as https://github.com/mapbox/jsxtreme-markdown#tocomponentmodule – some clarification in batfish about the differences between these two types of options and how they should be used, even just an extra sentence, would be helpful for folks new to batfish.
We need to build a sitemap.
I would really like for this to be possible. I have no idea if it is. This will take some experimentation.
To be clear about the intention: It would be great if a user could nest React Router inside a page for their own ends.
It's not a huge deal, but it seems that changes to front matter aren't reflected in the props that get passed to their respective React components. The dev server reloads the app when the file changes, but the values in props remain the same.
Draft Pages in README could have a little bit more information on
Draft pages are built during development but are not included in production builds.
and should point to /404 in production.
I'd like to carry over from dotcom the convention that a pages/
directory determines your routes. This mimics the systems that work well in Jekyll, Next.js, and other things.
One improvement I'm thinking about, over what we have in dotcom: Let's allow people to put asset files that are not part of any processing pipeline into pages/
directories. We'd make this work the same way Jekyll does it: anything that is not processed is copied directly into the built directories, at the same location.
Forgot this one.
Batfish is kind of nice. But we could consider other names, before we get too far. Please populate this list with your best ideas. Or else affirm your appreciation for the name "batfish".
It would be great to have a table of contents for the configuration docs, and it would also be nice to move them out of the README so it doesn't overwhelm the thing. These goals go hand-in-hand: once it's in its own doc, we can use https://github.com/jonschlinkert/markdown-toc (or something similar) to generate a table of contents.
In dotcom we have a little system that parses static HTML with Cheerio, then uses PostCSS to and document.querySelector
to create a CSS file that only contains selectors that match something on the page.
Because it's pretty slow, we split it across available processes.
I ran across an npm module with the same Cheerio-based system: https://github.com/tscanlin/css-razor. We could try using and contributing to that public module, instead of keeping our own. The code isn't super complicated, so I think it's perfectly fine to keep our own. But do we want to contribute to that project? @tristen what do you think?
Just add published: false
to the YAML front matter:
batfish/demo/src/pages/posts/three.js
Line 4 in f6b4811
Jekyll's collections include some derived from the build, such as pages
and posts
, that can be handy — mostly when creating lists.
Ideally all data is available at compilation time, but somehow limited in the client bundle.
(This is what Gatsby v1 is using GraphQL for. I don't want to introduce another hurdle with GraphQL. So let's figure out another way.)
scroll-restoration.js
has generic utility for client-side-routed sites, not just Batfish ones. So we should make it into its own package.
We want client-side routing because it feels fast, and the feeling of speed is super important for websites that are built to impress.
I want to avoid using ReactRouter if possible, because it is a very big dependency (much bigger, I think, than is justified by the problems we need it to solve) and it does not actually provide several features that we'd like to have. As I've thought about this more, I've realized that client-side routing is a collection of problems that could (probably should) be broken into different pieces and put together according to the needs and constraints of particular situations. I think we can build something pretty small that fits our needs and constraints pretty well.
Requirements:
pages/
directory. (I suppose it may not be much trouble at all to include the ability to dynamically add and remove routes.)routeTo
function that can be used to programmatically change routes.Link
components. There's a bunch of prior art trying to do this, so hopefully we can make it work well.scrollRestoration
API.Key question: Do we care about using the Express-style route definition syntax (e.g. people/:id
)? Or can we just use regular expressions, instead, to avoid the extra dependency (e.g. something like people\/([^\/]*)$
? Or should we create a more simple little domain-specific-language for our limited needs that would be easy to convert to regexps (e.g. people/{id}
)?
Here's some prior discussion of ReactRouter, motivations and ideas for a more minimalistic router:
Here are some open source libraries we can use as sources of ideas:
Gatsby released v1 and I'm sure a question on the minds of anybody who comes across this repo will be "Why not Gatsby?" So I should write up a document that describes the differences of the approaches.
The Symbol polyfill from core-js creeps into the bundles. I believe this is because babel-plugin-transform-runtime
adds a polyfill automatically when Symbol is mentioned; and React, at least, mentions Symbol — though it does so to check for its existence and not use it if it's not available.
Alternatives I can think of:
I don't think there's a good way right now to inject some JS that will end up being inlined in the HTML. We'll want this.
This will allow you to do things like:
It might be worth adding a step in here to complain about invalid HTML. The reason in my mind is that interpolation within Markdown can make this especially easy to do — e.g. nest a <div>
inside a <p>
.
All CSS is run through Autoprefixer. Forgot to document that.
Besides inlining critical CSS in the static build (#4), there are other considerations:
<link>
elements to add to the head.This configuration property is used in two places currently:
prefixUrl.absolute()
, if you use that.Sitemaps won't work at all without it. Should it be required?
The dotcom repo has a module that converts raw SVGs to React components.
We can port this over.
@tristen @samanpwbb @mayagao: Do you think there are any problems with the dotcom SVG system that we should definitely address before generalizing it?
I forgot to create a way to add CSS that is only going to be used on one page, so should not be added to the full site bundle.
Writing out some initial thoughts about the API. Adding to this list as more things come to mind.
start
and build
commands.build
needs to be able to distinguish between production and non-production builds.require('batfish/context')
or something like that.batfish
variable. This will have a routeTo
function for programmatic routing. It could also contain other stuff.I forgot routeTo.prefixed
, and also forgot to add examples.
Once we have a fully-functional prototype, we need some thorough cross-browser testing.
The use of ES2015 module syntax is established in Studio, and currently in dotcom and assembly-components.
With this tool we need to make prescriptions about the way users export pages. Either Node.js module syntax or ES2015 module syntax. And I'm a little bit inclined to go with Node.js module syntax, for these reasons:
require
to make the system work. If we're going to use require
in a bunch of places, why not everywhere?@mapbox/studio: What do you think? Any defenders of ES2015 module syntax?
We'll need this option.
The file-loader is used to hash and copy images, videos, and fonts. We should make sure that Batfish is configurable such that a user could throw in an image-optimization loader of their choice, e.g. https://www.npmjs.com/package/image-webpack-loader.
@samanpwbb and @tristen, I'd love it if you could try this out in its current state and give me feedback about the API.
I wrote up some documentation in the README. I think I captured the core features and configuration options.
One obviously missing feature right now is Markdown documents, as we discussed this morning. I'll add that soon.
There is a demo/
directory where I've been experimenting. You can try that out to start, to see how the CLI works and look at some simple examples. Then I think it'd be great if you tried creating a new directory for a fresh start on your own experimental demo.
This would save you the trouble of going to the URL when it's not yet ready.
The following images are based on my example in the markdown-world branch.
Scenario 1
On development, paths that don't exist and aren't draft pages get a 404:
On production, these paths get the following:
Scenario 2
On development, draft pages that use published: false
are visible by feature default:
On production, draft pages also get the following:
TODO
Ideally for production, in both the above scenarios these paths would go to the 404 page
"The React-powered static-site generator you didn't know you wanted."
There are good reasons for batfish! The project readme should lead with a clearer statement of purpose / value
This needs to be able to build sites that aren't at the root of the domain. Dotcom was able to cut some corners because it did not need to account for this. Making this issue so I don't forget about this vital feature.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.