Add an API to JSX Loader about dataformsjs HOT 17 CLOSED

Hello Bart!

Thanks so much for your kind words and checking out my project!

I'm actually in the middle of publishing the next major release so it's a good time for adding this. Next release 5.0.0 (current version is 4.8.0) will likely be published this month. Most updates are related to Web Components however I'm adding in a variety of changes as well so any ideas you have on how to improve or update the jsxLoader is welcome. An if you want to contribute options that would help react-styleguidist feel free.

I gave you credit for helping out with in the changelog 😄
https://github.com/dataformsjs/dataformsjs/blob/master/CHANGELOG.md

This is something I thought about before but didn't have a use case for it myself as I wrote it to run from a browser and unit testing occurs from within the browser as I want to test with actual browsers and not a controlled environment. For example in doing this I discovered a bug specifically to UC Browser that I fixed and it helps compare babel result with IE vs the jsxLoader on modern browsers.

Anyways for the initial update I added this code block and had to make just a few tweaks to get it too work.
https://github.com/dataformsjs/dataformsjs/blob/master/js/react/jsxLoader.js#L1540

I created an initial node script here based on your code:
https://github.com/dataformsjs/dataformsjs/blob/master/scripts/jsx-loader-node-demo.js

Because I'm actively making a large number of changes for the next release I'm doing this on the main/master branch. Perhaps the change will be removed (and added to a separate repository) or changed further but that can be decided on before during development as this doesn't break anything with the existing code.

Related to this I actually have Vue 3 version using this JSX Loader. I created when Vue 3 was still in development and was able to help make minor updates on Vue based findings from the page. The Vue 3 version uses code hint for the h function:
https://www.dataformsjs.com/examples/hello-world/en/vue3-with-jsx.htm

I'd would definitely be interested in helping you use it to make react-styleguidist faster so here are my answers to your questions:

• Do you think it could be made into a separate npm package?
  • Probably, I'll have to think about this more and consider different options.
  • When I look at the code if I keep just the functions needed for the compiler I can remove about 500 lines of code from the full file.
  • However minimized it probably saves only about 5 KB and gzipped it maybe 1 kb.
  • If I were to keep everything in the initial file I would probably want to add some sort of option so it doesn't load and run on the page against all Babel scripts. Currently it looks for all scripts with <script type="text/babel"> and automatically processes them and checks the browser for needed polyfills, etc.
  • As I write this it seems to make sense to include in a separate repository as it could be optimized for node and webpack.
  • I want to keep the original version a zero-dependency one script file so the needed code could likely just duplicated then synced when there are changes.
• Do you think one could ask for a specific pragma to package the jsx?
  • For now {pragma: 'h'} gets converted to a code hint but I like the idea of adding it as a proper option without the code hint.
• Most importantly, do you think that this transformation could be done without running the code?
  • Yes, the update I made does that (assuming it runs from node). Not sure about if the file is included in a page with webpack though. As I mentioned above the script currently does a lot when it's added to the browser but I could add an option to turn it off before the page is loaded. Probably wouldn't be friendly for webpack though but it can easily be done.

from dataformsjs.

Thank you @ConradSollitt

It's a rare pleasure to have such a complete and committed answer on GitHub.
I may probably use the JSXLoader as it is for refactoring vue-styleguidist.

Since I have to add some secret sauce on vue examples to make them work live in the browser:

• mocking the require/import
• importing (and registering) by default the proper components

1. Build an AST (I need it for import/require management)
2. Using the AST, fix the require/imports
3. Using the AST, if there is JSX, transform the JSX into its pragmas
4. If the browser does not support modern code (IE) transform it
5. Run the code and get its return into a variable
6. Register this variable as our mini vue app

Step 3 and 4 could be accomplished using JSXLoader.
I have also thought about skipping step 4 since Internet Explorer end of life is around the corner.

The example you gave on vue usage will be of great help.

from dataformsjs.

Your Welcome @elevatebart

Since I don't have many issues opened here it allows for proper time compared to a project with a large number of opened issues. And for me I like learning and getting new ideas so stuff like this is great and I'm happy to help.

I'll have to try out vue-styleguidist but likely not today. Probably later this week or next week.

I assume you've seen the JSX Loader Docs but if not here is the part on AST:
https://github.com/dataformsjs/dataformsjs/blob/master/docs/jsx-loader.md#advanced-usage-and-internals-microscope

Basically JSX Loader understands very little JavaScript so the AST is built around JSX Elements and designed around returning the transformed JavaScript string. Common traverser() and transformer() compiler steps are combined with codeGenerator() which helps keep the code small (and because it's working well and fast in current form I don't feel the need to refactor it).

I still need to test the change I did yesterday with webpack to see if it works which can help with the items from steps 3 and 4. However in the meantime I created an additional Vue 3 Demo below to see if something like this could be done in plain HTML using the existing API and the basic demo is working so far.

Online Demo Page:
Currently I'm not syncing changes back to the main dataformsjs server until the next release is ready. I use a bash script that syncs the full server directly from the main branch and right now most examples are not set to point to a CDN build so I'm waiting till the next release (later this month) before I sync.
https://dataformsjs.s3-us-west-1.amazonaws.com/vue-3-testing/vue3-dynamic-jsx.htm

Source Code
All related app code (HTML, CSS, JS, JSX) is in this one file (excluding linked scripts).
https://github.com/dataformsjs/dataformsjs/blob/master/examples/vue3-dynamic-jsx.htm

4 demos so far (all simple). Later I would like to test components by mocking require/import or including them somehow.

Each one uses it's own <App> object.

For the demo in the current form, not much code is needed. Basically I use the JSX Loader to compile and then I use new Function() to evaluate the code. I tested eval() as well and it works as well. new Function() is used in Vue 3 Source and many other templating languages such as Underscore source use this when evaluating code generated from templates.

Even though each demo defines it's own <App> object none overwrite the original one that is defined at page load.

With the above demo I created the page does not need to know AST or how it works internally. And in this case (what I'll try to test later) is for importing I plan on passing impoart/require components to the new function.

I actually started doing this recently with several DataFormsJS Web Components to allow for minimal templating using JavaScript template literals (template strings).

https://www.dataformsjs.com/examples/image-gallery-web.htm

<data-list
    data-bind="images"
    template-selector="#image"
    root-element="div"
    root-class="image-gallery">
</data-list>

<template id="image">
    <image-gallery image="${image}" image-avif="${image_avif}" image-webp="${image_webp}">
        <img src="${thumbnail}" alt="${title}" tabindex="${index+5}" />
    </image-gallery>
</template>

https://github.com/dataformsjs/dataformsjs/blob/master/js/web-components/data-list.js#L162

// Template Compiled with `new Function()`
// [index, render, escapeHtml, and format] are made available to the new function.
if (listItemName) {
    tmpl = new Function(listItemName, 'index', 'render', 'escapeHtml', 'format', 'return ' + renderFn + '`' + tmplHtml + '`');
} else {
    tmpl = new Function('item', 'index', 'render', 'escapeHtml', 'format', 'with(item){return ' + renderFn + '`' + tmplHtml + '`}');
}

// Later the `tmpl` function is used and passed several parameters
html.push(tmpl(item, index, render, escapeHtml, format));

Related to IE - Vue 3 currently doesn't work with IE 11 yet.
https://github.com/vuejs/vue-next#changes-from-vue-2

from dataformsjs.

Just did some testing to see how safe using new Function() over and over while compiling from jsxLoader and rendering on Vue each time.

Memory (on my tested machine - Win 7 / Chrome Latest) starts out small at 2 to 3 MB.

Single clicks don't show any real change and after manually testing an automated 100 clicks at a time memory went up and down.

After several tests of clicking over 100,000 times it appears there may a memory leak somewhere as it went up to around 50 MB of memory. However when running 1000 clicks at a time over and over memory went up and down at more reasonable levels (10 - 20 MB).

Although one test when I snapped the screenshot it came back with little memory (5.6 MB) after 100,000 clicks.

I'm going to look into the leak some more to see if it is quick to solve but overall this seems mostly safe unless someone tries hard to break it like I'm doing.

If the issue can be solved and turns out to be in Vue it can be possible to use a custom build. When I first created the Vue 3 JSX Demo (link below) I modified the Vue.h as shown below and then after discussion with Vue authors using the demo they put a fix in so it works out of the box.

dd09383

from dataformsjs.

Great job,

I believe there is a small memory leak indeed in Vue 3, I heard Evan talk about it recently.

I was wondering why you did not rely on acorn for parsing.
It would have made your life easier.
Using rollup you could have helped you bundle it in your code.

Is it something you considered?

from dataformsjs.

Thanks,

Yeah, regarding the memory leak the number of Vue.h elements and Vue.render calls is so large 100,000+ I wouldn't expect this to cause an issue on 99% of site at the moment.

Rather than using acorn or other items for parsing one of the main goals was to make the JSX Loader as small and fast as possible when I wrote it and to have it be compatible with Babel Standalone. When I had the original idea for it I tested many options but simply ended up writing the full JSX Loader because I could then keep it as small and fast as possible. It did take a while (month or two) to write and get everything working properly but so if I could have used something else that would have definitely made my life easier. I'm happy the the result in doing it this way because the current download size is only (5.4 kB - min and gzip) and it's very fast and all code is in a single file.

When I started writing one It was more of a proof of concept to take simple Hello World JSX and transform it to JS for modern browsers or download and use polyfills and babel for older browsers. Often developers think "IE" as the older browser but this also includes older iPhones (small usage now as of 2020 but still used) and older chromium based browser that are widely used. For example UC Browser (heavy use in Asia) and many Samsung devices that use pre-installed Samsung Internet. After the proof of concept work I kept developing the compiler in a single file and simply used browser DevTools for debugging and stepping through the code and that development workflow worked well for it.

As for code splitting and other common tasks that webpack or a bundler would normally handle I wrote a separate <LazyLoad> component that uses the JSX Loader API to handle lazy loading that would normally be done with React: https://github.com/dataformsjs/dataformsjs/blob/master/docs/jsx-loader.md#code-splitting-scissors

I still plan on update the example I created to see how it can create and import separate Vue Components. Hopefully later today or tomorrow but I'm not 100% sure I'll have the time yet.

from dataformsjs.

Hi @elevatebart

Good news, I've updated the demo to show how you can combine multiple Vue Components using JSX with the example. This demo doesn't let you edit the code but you can see how it would be possible if using a <textarea> instead of <script> elements.

The updated page is posted here:
https://dataformsjs.s3-us-west-1.amazonaws.com/vue-3-testing/vue3-dynamic-jsx.htm

You can also run it locally if using the most recent update of the main DataFormsJS repository.

# Download this repository
npm start
# 1) open main page in browser and filter "jsx"
#     Note - filter is currently broken in Safari / likely fixed later today
# 2) or directly open:
#     http://127.0.0.1:8080/vue3-dynamic-jsx

Custom Component Section:

Browser support is very good based on initial testing:

• Chrome
• Firefox
• Safari
• UC Browser (desktop) - works with error warning because the browser runs both [module/nomodule] scripts. Safari 10.1 has this issue as well and in this specific example the user can simply close the alert and continue using the page (screenshot below).
• IE users will see only the error message because Vue 3 does not currently work with IE.

I'll follow up later today or later this week with more info on the jsxLoader as I've been testing with webpack and am able to get it working without side effects. I'll have more details to provide once I'm done with development related to webpack for this.

Items of interest on the latest demo:

• By default the jsxLoader checks environment for needed polyfills such as Promise.prototype.finally and downloads the if needed. Additionaly when it runs automatically it also checks for very old browsers (IE 11, older mobile Safari) and then downloads Babel Standalone. This version of the demo blocks the check, it has one very minor side effect (in code comments) but that will be fixed in next release later this week or next.
• Since jsxLoader no longer runs automatically I was able to replace <script type="text/x-template> with <script type="text/babel"> so code editors (specifically VS Code) will highlight the JSX.
• Each Vue Component lives in it's own <script> and regex is used to determine what should be imported.

from dataformsjs.

That is super cool !!
Great work!

Given the size and speed of your jsxLoader, you have a real goldmine in your hands.
For my tooling, I only need to add the transform of import statements into require() so that I can pre-package external modules for use at runtime. But that seems like a breeze, I even feel like I could do that with a good old replace.

Can't wait to try it this week-end.

from dataformsjs.

Thanks @elevatebart I appreciate your nice words about my project!

Yeah after creating the jsxLoader it allowed me to include React on sites or projects that I might not have done otherwise simply because I can just include the file and start writing the page. Sometimes it feels overkill when a single HTML page for a small app needs so much tooling. Here are some more demos of the jsxLoader in case you are interested:
https://awesome-web-react.js.org/

Related to the Vue jsxLoader page I have not posted it on the main site as I'm syncing server with the repository again. I've added a simple markdown editor demo using the replace method.

https://www.dataformsjs.com/examples/vue3-dynamic-jsx.htm

I did however find one issue that might prevent the jsxLoader from bing used with vue-styleguidist depending on the needs.

Basically I'm doing this:

import md from 'markdown-it'

const Markdown = {
    props: {
        content: {
            type: String,
            required: true,
        },
    },
    methods: {
        renderMarkdown(e) {
            this.$refs.markdown.innerHTML = md.render(e.target.value);
        }
    },
    render() {
        return <div className="markdown-editor">
                <textarea rows="4" cols="40" value={this.content} onInput={this.renderMarkdown} />
                <div className="markdown-content" ref="markdown" innerHTML={md.render(this.content)}></div>
            </div>
    }
}

function App() {
    return <>
            <Markdown content="# **Hello World from <Markdown>**" />
            <Markdown content="## _Hello World from <Markdown>_" />
        </>
}

But I would rather use Vue directives - for example vModel and bind the data without handling onInput. I'm not sure if this would work if using Babel but it would be nice if it did.

const Markdown = {
    props: {
        content: {
            type: String,
            required: true,
        },
    },
    render() {
        return <div className="markdown-editor">
                <textarea rows="4" cols="40" vModel={this.content} />
                <div className="markdown-content" innerHTML={md.render(this.content)}></div>
            </div>
    }
}

It turns out that Vue directives are handled by a Babel plugin and the transformed JS code for Vue from JSX is very different just using standard Babel. The main repo links to a playground where you can try it against Babel:
https://github.com/vuejs/jsx-next @vue/babel-plugin-jsx
https://babeljs.io/repl/

Also I have completed testing with webpack so here is how you can import and use the jsxLoader without any side effects:

import 'dataformsjs/js/react/jsxLoader'

// Update to use Vue
jsxLoader.compiler.pragma = 'Vue.h';
jsxLoader.compiler.pragmaFrag = 'Vue.Fragment';

// Update jsxLoader settings so that doesn't download polyfill or check browser.
// A `document.addEventListener('DOMContentLoaded'` event will still run, however
// it will perform no actions with these settings.
jsxLoader.isSupportedBrowser = true;
jsxLoader.needsPolyfill = false;
jsxLoader.setup = function() { };

// Optional setting to consider
jsxLoader.compiler.isMinimized = function() { return false; };

// App code:
const jsx = '<Hello>World</Hello>';
const js = jsxLoader.compiler.compile(jsx);
console.log(js);

A I found when including all of jsxLoader in the current form vs removing un-needed code only saved about 5 KiB of space on the build and when gzipped it only saved about 1.2 KiB. Since the space saved is tiny I'll be keeping jsxLoader in the main repository rather than moving it out and maintaining a separate copy. The file is intended for browser over webpack and node so I want to keep it that way for now as the API is easy enough to use to make it work with webpack I fell.

Hope all this helps. Let me know if you have more questions and I'll be happy to help.

from dataformsjs.

One final update (for now)!

Since vModel can't be used with jsxLoader (at least not easily) I updated the demo to use a combination of props, data, onInput, and computed to make it more Vue like while being friendly for React developers.

const Markdown = {
    props: {
        content: {
            type: String,
            required: true,
        },
    },
    data() {
        return {
            userContent: this.content
        }
    },
    methods: {
        handleInput(e) {
            this.userContent = e.target.value;
        }
    },
    computed: {
        renderedMarkdown() {
            return md.render(this.userContent);
        }
    },
    render() {
        return <div className="markdown-editor">
                <textarea rows="4" cols="40" value={this.userContent} onInput={this.handleInput} />
                <div className="markdown-content" innerHTML={this.renderedMarkdown}></div>
            </div>
    }
}

from dataformsjs.

@elevatebart
Did you end up working on vue-styleguidist and using jsxLoader? Any questions or do you need additional help?

from dataformsjs.

Not yet, I did not get to work on v5 as much as I would have liked.
Maybe this weekend if all goes well.

I would see 3 reasons that would make me hesitate:

• The "ambient" configuration makes me uneasy for some reason.
• Having to install all dataformsjs to only use the loader
• The API being loader.compiler.compile() seems ... weird?

and 2 reasons why I need to do it

• The babel integration "on the fly" seems magical
• The initial weight & perf of what you have done is unbeatable

To fix the hesitation, I was thinking of copying the part of your code that I need and publish it as a separate package with a simpler API, but it is entirely unfair to your ownership.
I will get to try it out this weekend to see if it works with your API in a branch. We can plan for extra packages and different APIs then.

from dataformsjs.

Sounds like a good plan.

All the breaking changes for v5 were related to Web Components while jsxLoader mostly had some minor updates.

Feel free to copy it to a new separate package if that seems the best option for you. All the code is MIT license so it's entirely fair to copy 😄 A link back to DataFormsJS would be appreciated of course.

I understand the issue with the "ambient" configuration and the jsxLoader.compiler.compile. Basically you found a new use case for it that I never intended or needed myself so in order to do this undocumented lower-level objects and functions are used. Normally it would just be included on the page and in some cases much higher level API called:

<!--
    Standard JSX Loader Usage
    Include file on page, no additional JavaScript to call
-->
<script src="node_modules_or_cdn/dataformsjs/js/react/jsxLoader.min.js"></script>

<script>
// ------------------------------------------------
// Typical Advanced Usage
//
// Show Compiler details (used for some development and can be used for learning)
jsxLoader.logCompileTime = true;
jsxLoader.logCompileDetails = true;
//
// Use Preact instead of React
//
jsxLoader.usePreact();

// ------------------------------------------------
// Advanced Usage - Load a JSX file dynamically
//
// The code below was copied from the DataFormsJS <LazyLoad> React Component
// which would be the easier method (by design) for dynamically loading JSX scripts.
var script = document.createElement('script');
script.type = 'text/babel';
script.setAttribute('src', url);
document.head.appendChild(script);
jsxLoader.loadScript(script).then(function() {
    resolve();
});

// ------------------------------------------------
// Advanced Usage - Compiling JSX
//
// The `compiler` object keeps the compiler code organized separately
// in an object model from `loadScript` and high level features. So far
// other than this issue it's undocumented as it was made available for
// Unit Testing but there were no plans to use it in an actual app.
var js = jsxLoader.compiler.compile(jsx);
</script>

I just did a quick test of a basic webpack and Babel install on an empty project and the size was almost 43 MB while create-react-app is now over 250 MB when I tried it last month. npm i dataformsjs adds under a 1.5 MB; granted its a lot to include if you only need one small file but relative to many other npm packages it's small overall.

Another thing to consider is the target market of your site is developers (from a desktop on a learning/playground) so waiting an extra few seconds for Babel on the initial page load or action to compile is acceptable in my opinion. If you use that over JSX Loader then have full JSX and JS Syntax Support. JSX Loader should work in 99%+ percent of the time but since it doesn't understand JS it can't handle all code, see known issues here:
https://github.com/dataformsjs/dataformsjs/blob/master/docs/jsx-loader.md#known-issues-%EF%B8%8F

So far every realistic issue I've come across I've been able to support it but I'm sure additional minor syntax items will be added in the future as they are found. During development I made sure to push the limit of JSX in terms of what it can support. For example the unit testing file (highlights correctly in VS Code but not on Github - link below). It includes a lot of deeply nested JS and JSX that would not commonly be used in most React apps but needs to be supported for real world production use.

https://github.com/dataformsjs/dataformsjs/blob/master/test/js/unit-testing.jsx

On another computer that I don't have on me right now I have a webpack version were I tested separating the code. I can later provide it later today or this week as starting point in case you want to create a new repository.

from dataformsjs.

Hi @elevatebart

Here is a working copy of the latest jsxLoader (5.4.1) renamed jsx-transformer.js and optimized for webpack and node because I've removed un-needed code that runs when the browser loads and I removed the jsxLoader.compiler.isMinimized() function.

This is using your original API idea transform(jsx, options). This file can simply be copied to a webpack/node project and used as-is. If the original JSX Loader compiler portion were to have added features it could be merged fairly quickly using a diff program.

https://gist.github.com/ConradSollitt/f88166e079da3d00cc7a3e40847392a8

Tested with webpack:

import { transform } from './jsx-transformer'

document.querySelector('button').onclick = function() {
  const options = { pragma:'Vue.h', pragmaFrag:'Vue.Fragment' }
  const jsx = '<Hello>World</Hello>'
  const js = transform(jsx, options)
  console.log(js)
}

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>JSX Testing</title>
  </head>
  <body>
    <button>JSX Testing</button>
    <script src="bundle.js"></script>
  </body>
</html>

from dataformsjs.

Hello @elevatebart!

Hope you are having a good 2021. Wanted to follow up and see if you needed any help otherwise I'll close out this issue.

Looks like you already solved it using Buble https://github.com/bublejs/buble based on https://github.com/vue-styleguidist/vue-styleguidist/blob/dev/packages/vue-inbrowser-compiler/package.json

I think Buble is a very good choice for your needs. I actually tested it a few months ago in the jsxLoader for the fallback engine and found it was about 10x smaller and must faster than Babel for tested demos that I created. Granted for jsxLoader as of 2021 this would affect mostly IE or very old Safari on iOS (small single 0.1% of global traffic - example iPhone 6 model that is not upgraded) so after thinking about it I decided to keep Babel by default for fallback on jsxLoader since it's so widely used and supported. Buble is very appealing though for specific sites due to it's size and speed; I also like how it transpiled for (const x of x) compared to Babel.

from dataformsjs.

Hello @ConradSollitt

I actually had a baby over Christmas and did not get any time to try it yet.

My new gig at cypress will surely bring me around to use it.

from dataformsjs.

Congratulations @elevatebart

from dataformsjs.