Add UMD about scribe HOT 69 OPEN

+1

from scribe.

+1

from scribe.

Also, I would really appreciate support for component.

from scribe.

👍

If not, then exposing it as a global like Flight have done would be useful.

from scribe.

👍

@OliverJAsh I'd like to take this work on, but I need a little guidance. In your opinion, at which step of the build process should the conversion to UMD be done?

Given the fact that AMD is baked into all of the modules, one route might be to let RequireJS resolve all the dependencies and create the AMD module, and then bundle a minimal define implementation (perhaps almond). Then, wrap the whole thing in UMD's returnExports.

Another (and more future-proof) route would be to convert all existing AMD modules to ES6 modules. Then, use Square's ES6 module transpiler to output to AMD, CommonJS, or browser globals as needed. It's not UMD, but it avoids the need for a define shim and is just as flexible in the long run.

EDIT: This would also likely require a plumber-es6-module-transpiler plugin.

Of course, a 3rd option would be to try and find something that knows how to translate AMD to UMD, but I'm not aware of any such tools that are well-maintained and in wide use.

from scribe.

@mjijackson @OliverJAsh There's a library called amdclean which I find very useful for stuff like this. It takes the output of the RequireJS optimiser and removes all the AMD boilerplate (sample config code), exposing each module as a uniquely-named variable. Then all you need to do is wrap the whole thing up:

(function (global) {
  'use strict';

  /* amdclean'd r.js output goes here... */

  if (typeof define === 'function' && define.amd) {
    define( function () { return Scribe; });
  } else if (typeof module !== 'undefined' && module.exports) {
    module.exports = Scribe;
  } else {
    global.Scribe = Scribe;
  }
}(window)); 

This removes a fair amount of unnecessary code and means you don't need to include almond.

from scribe.

For what it's worth, I'd love to use scribe in a node-webkit application, which uses node's native module system, like this:

"use strict";
var Module = (function(){
  var dependency = require("./some/deeply/nested/dependency");
  function blah() {}
  exports.blah = blah; 
})();

I started trying to do this myself, but I got somewhat tangled up in some of the deeply-nested dependencies where scribe plugins return functions that return functions (that also return functions?), and I wasn't entirely sure how to transform the code into a node-like module system without screwing things up. But if others are interested, I can try taking another crack at it.

from scribe.

@Rich-Harris Thanks for your input!

I tried using amdclean as you suggested, and it looks good for the most part. The only problem is that it doesn't quite do the trick for modules that are already defined using UMD, like the EventEmitter module that scribe depends on.

For example, the EventEmitter module definition looks something like this:

(function () {

  function EventEmitter() {}

  // ...

  // Expose the class either via AMD, CommonJS or the global object
  if (typeof define === 'function' && define.amd) {
    define('event-emitter',[],function () {
      return EventEmitter;
    });
  }
  else if (typeof module === 'object' && module.exports){
    module.exports = EventEmitter;
  }
  else {
    this.EventEmitter = EventEmitter;
  }

}.call(this));

But amdclean transforms it into this:

(function () {

  function EventEmitter() {}

  // ...

  // Expose the class either via AMD, CommonJS or the global object
  if (true) {
    var event_emitter = function () {
      return EventEmitter;
    }();
  }
  else if (typeof module === 'object' && module.exports){
    module.exports = EventEmitter;
  }
  else {
    this.EventEmitter = EventEmitter;
  }

}.call(this));

which effectively conceals the event_emitter variable inside the closure.

@benjismith Is this similar to the problem that you ran into?

@gfranko Since amdclean is specifically meant for AMD modules, I don't suppose it should be responsible for anticipating scenarios where it's already wrapped as UMD, right?

from scribe.

As a side note, I'm starting to believe that using AMD modules isn't a good idea for code that you intend to ship in multiple different module formats, unless you absolutely need to be able to asynchronously load modules at runtime, which I don't believe scribe does.

@OliverJAsh @theefer Please correct me if I'm wrong.

from scribe.

@mjijackson You just need to set the transformAMDChecks option to false and it will not transform the UMD logic. By default, it expects to be used for web apps (which is why the transformAMDChecks option is set to true).

If terms of scoping problems for variables, you need to set the globalObject option to true. In the next major release of AMDClean, this will no longer be necessary since variables will all be hoisted correctly.

from scribe.

@gfranko I see. Thanks for the help!

Unfortunately that option doesn't quite solve my issue in this case. Now, the output looks like this:

(function () {

  function EventEmitter() {}

  // ...

  if (typeof define === 'function' && define.amd) {
    var event_emitter = function () {
      return EventEmitter;
    }();
  } else if (typeof module === 'object' && module.exports) {
    module.exports = EventEmitter;
  } else {
    this.EventEmitter = EventEmitter;
  }

}.call(this));

which still conceals the event_emitter variable inside the UMD wrapper. It's almost like we need a umdclean (which may be difficult to create given the variance in UMD wrapper styles) for the EventEmitter module, and then we can use amdclean on everything else.

from scribe.

@mjijackson If you don't want the event emitter module to be "cleaned", then you can include it in the ignoreModules option. Like this: ['event-emitter'].

Then none of your UMD logic would be touched.

from scribe.

@gfranko Thanks for the input. It still doesn't resolve this particular issue, but it's good to know for using amdclean in the future.

from scribe.

I think using ES6 modules would be the preferred option, though we'd need to see if we can still use the es6-module-transpiler to transpile to AMD or CJS in spite of depending on non-ES6 code (e.g. EventEmitter).

from scribe.

@theefer Agreed. I doubt es6-module-transpiler would mangle non-ES6 code, but can't say for sure.

from scribe.

So what is the current situation? What is the best way to use it as CommonJS module?

from scribe.

@mjijackson No, my issues were much more mundane: I'm primarily a Java programmer, so I'm accustomed to seeing a more straightforward import mechanism, one class per file, multiple class files organized into hierarchical packages, etc. I've also done a lot of node.js programming over the past year, and its module import mechanism is conceptually pretty similar to the stuff I'm already familiar with.

But things are a bit less predictable with client-side AMD modules. Some modules are implemented as functions that get immediately called to create a closure, while other modules define a function that's not called until later, creating a closure upon call. The lack of predictable structure provides a lot of implementation flexibility, but it looks a bit foreign to somebody coming from a different dev background.

I'd like to use scribe in a node-webkit project, so I need to use node's require implementation, but it's not always clear to me how to take scribe's closure and expose them as node module exports.

from scribe.

I want us to move to using ES6 modules internally, however the Plumber operation for Traceur does not currently output source maps, so I can’t use it in production just yet. Nonetheless, there’s no reason we can’t do the work to move to ES6 modules and keep that on a separate branch (without source maps). Hopefully it won’t belong before I give that a go, but if anyone else wants to pick it up, don’t hesitate!

If we use ES6 modules internal, my hope is that we can compile that source code to AMD/CommonJS/UMD-like source code.

Right now I’m wondering if it’s possible – when using ES6 modules – to import a module that uses AMD/CommonJS/UMD, as that will be the case for html-janitor and EventEmitter.

from scribe.

@benjismith I think you are speaking about the plugins. They are functions that return functions, yes:

var plugin = function () {
  return function (scribe) {
    …
  }
};

To enable a plugin, scribe.use simply expects one parameter that is a function, which will be invoked with the Scribe instance as its first and only argument. (We should have API docs to explain this; sorry. The code is very straightforward.)

So to use my example plugin plugin, you just do scribe.use(plugin()). In CommonJS that could look like:

var plugin = require('plugin');
scribe.use(plugin());

Although we don’t yet support CommonJS, unless you’re doing something funky.

Some plugins need to be configured, in which case we need to use higher-order functions so that you can pass in any options to the plugin function call. We made all the plugins higher-order functions as a normalisation step.

I hope that helps to clear things up for you!

from scribe.

Thanks Oliver! That's a very clear explanation. I'll take a closer look and see if I can make it work in my setup :)

from scribe.

Is there a usable process available at the moment for converting ES6 modules into UMD? I could only find this year-old fork of Squares transpiler, which supposedly isn't always reliable itself.

Having spent some time with Scribes' sources, I am quite eager to migrate my own projects over to it from HalloJS. The only thing holding me back is the pain of dealing with require/AMD/CJS, which does not integrate very well with either the Rails Asset Pipeline, nor the rest of my projects.

So I would be grateful for any solution in the near future that does not involve me having to wrangle in UMD support manually and playing catchup with upstream from then on :)

from scribe.

Do you really need UMD if you have ES6 modules that you can transpile to AMD, CommonJS and globals, and distribute separate dist files for each case?

from scribe.

You could do ES6 modules => AMD => UMD, it’s just another step. Or what @theefer said.

from scribe.

Could also be interesting to see how Lodash generates global, AMD, and CommonJS versions. There’s a lot going on in here: https://github.com/lodash/lodash-cli/blob/master/bin/lodash

from scribe.

@theefer you're right, I am only really concerned with using Scribe via globals, and using UMD would just be a means to that end - whatever works, works. It simply seemed to be the simplest, least involved approach.

from scribe.

@theefer and @janfoeh I agree with both of you. Browser globals will also work from within node-webkit, which is fine by me.

from scribe.

Btw I just fixed sourcemaps in the plumber-traceur operation. Still quite experimental, but at least it's not blocking us to try using ES6 anymore!

from scribe.

This is currently being held by up #111.

from scribe.

I'm looking forward to using scribe in our project (post.fm) and wanted to chime in on this issue.

In my humble opinion, keeping it simple is the way to go. I’d take the same approach as Backbone: a single js file (with a UMD definition at the top) and underscore as a required dependency. Reasons:

• Scribe isn’t a huge library, number of lines is the same as Backbone’s if not less (1500 or so)
• Many people already include Underscore.js in their projects (or use lodash as an alternative) and Scribe can include any custom lodash methods missing from underscore API.
• No fancy packaging needed, no dependency on RequireJS/AMD/ES6/you name it
• Plugins can attach themselves to the ‘Scribe’ global (as jQuery plugins do) thus avoiding global pollution #146
• Works as an NPM & CommonJS module for browserify users #77

I’d have a folder with optional plugins in the same repo, with a standalone js file for each plugin (adding a plugin is then as easy as adding a script element).

PS I'm using browserify (CommonJS) and backbone in my project currently. I’d be happy to do this for you, however syncing with upstream would be hard so should be done in one go really.

from scribe.

We now have CommonJS support via Browserify. See #175. For an example, check out https://github.com/guardian/scribe/blob/master/examples/cjs.html.

Unless someone can provide a reason to implement support for globals, I’m happy to consider this issue closed.

from scribe.

My projects are quite small - low five digit KLOC, hundred-ish files and no more than one to two dozen dependencies. While I have tried JS dependency managers, I have found that they do not provide me with tangible benefits over plain IIFEs and a manually managed namespace, while introducing quite a lot of conceptual overhead and toolchain complexity.

While they certainly are a boon for some workflows and projects, for others like mine they are just process for processes' sake. While I cannot provide hard data, I have a hard time believing that the latter does not describe a significant percentage of use cases out there right now, if not the majority.

Using UMD would make Scribe accessible to them as well. While I respect opinionated projects, I would appreciate it if you would reconsider.

from scribe.

@OliverJAsh @janfoeh I know that you guys are fond of Pluming.js for your dist build, however browserify has an option that does exactly this (that is, does UMD export style): the --standalone option. Perhaps that would make everybody happy.

See this blog post for a more detailed explanation: http://www.forbeslindesay.co.uk/post/46324645400/standalone-browserify-builds

from scribe.

So can you get Scribe via globals by using browserify and --standalone (since #175 added browserify support)?

from scribe.

Try something like this from the root of the scribe repo:

$ npm install deamdify
$ browserify --global-transform deamdify --standalone Scribe . > build.js

Now build.js is a standalone copy of Scribe with "UMD" syntax, so it works in CommonJS, AMD, and in plain' ol <script> tag imports (will be global as window.Scribe in that case).

I think that final use-case is the one that @janfoeh was desiring.

from scribe.

So, theoretically, we could create the bundle using browserify instead of
RequireJS, and that would leave us with a UMD compatible module? I think
this would still be done in the form of a Plumber task (this doesn't really
matter).

If it’s that easy, we should do it! :-)

from scribe.

@TooTallNate, @OliverJAsh thank you! It looks like this would indeed solve my problems. I'll give it a whirl as soon as possible.

from scribe.

Also needing a global version. Using Angular without CommonJS/AMD (quite common I expect, thanks to their take on modules) so the standalone Browserify options sounds great.

from scribe.

Let’s do this then: #83 (comment)

Anyone want to give it a shot?

from scribe.

@OliverJAsh I will give it a try 📦

from scribe.

Well, I finally got back to my Scribe-related project and managed to spend a couple hours on this. It was a bit of a pain, but I now have a converted UMD builds of Scribe and all official plugins i could find.

@TooTallNate, @OliverJAsh would you mind if I put up a scribe-umd repo as a quick fix in the interim? Judging by this thread this might be of use to some.

from scribe.

@janfoeh How are they built?

from scribe.

I've been fiddling a little bit to see how Scribe might work with ES6 modules. Here's a untested first pass at converting the module syntax:

https://github.com/callum/scribe/commit/2357d5b696a2b365b364b328dc63dc68c49c914d

I wanted to experiment with various aspects of ES6 modules on a smaller scale, so I've scrappily implemented es6-module-transpiler on my own fork of scribe-common:

https://github.com/callum/scribe-common

I'm coming at this from a CommonJS angle, but in its current state, if you installed the module with npm, you could:

var element = require("scribe-common/dist/cjs/element");

element.isBlockElement();

As @OliverJAsh points out above, I'm not sure how dependency management works; whether you're able to use npm for libraries such as Lo-Dash or other. (https://github.com/lodash/lodash-es6).

If scribe-common is written in ES6 modules, how does scribe then consume that module? Via npm, Bower, jspm? I don't know.

Hope this is somewhat helpful.

from scribe.

I believe you can use jspm to consume a variety of module formats. We
should give it a go!
On 13 Sep 2014 19:52, "callum" [email protected] wrote:

I've been fiddling a little bit to see how Scribe might work with ES6
modules. Here's a untested first pass at converting the module syntax:

callum@2357d5b
https://github.com/callum/scribe/commit/2357d5b696a2b365b364b328dc63dc68c49c914d

I wanted to experiment with various aspects of ES6 modules on a smaller
scale, so I've scrappily implemented es6-module-transpiler on my own fork
of scribe-common:

https://github.com/callum/scribe-common

I'm coming at this from a CommonJS angle, but in its current state, if you
installed the module with NPM, you could:

var element = require("scribe-common/dist/cjs/element");
element.isBlockElement();

As @OliverJAsh https://github.com/OliverJAsh points out above, I'm not
sure how dependency management works; whether you're able to use NPM for
libraries such as Lo-Dash or other. (https://github.com/lodash/lodash-es6
).

If scribe-common is written in ES6 modules, how does scribe then consume
that module? Via NPM, Bower, JSPM? I don't know.

Hope this is somewhat helpful.

—
Reply to this email directly or view it on GitHub
#83 (comment).

from scribe.

Trying to get my head around a stack that uses jspm and is able to build to AMD, CJS and globals. @OliverJAsh what ideas do you have around this?

from scribe.

I've got a working example of a jspm/SystemJS implementation with ES6 modules working in this commit https://github.com/callum/scribe/commit/978f8eeab157ebf43c0a8215bec1075ec27def69. It's just the regular example sans toolbar for now, but it works brilliantly.

from scribe.

External dependencies are the pain point at the moment. I tried es6-module-transpiler on the above and it chokes at things like this:

import flatten from 'lodash-node/modern/arrays/flatten';

Only jspm/SystemJS cleverness knows how to resolve that because of its internal configuration. (https://github.com/callum/scribe/blob/es6-modules/examples/jspm/config.js#L11-L12). I can't think of a way to normalise imports across AMD, CJS and globals. Let alone how you would manage versioning between them.

There is an NPM resolver for es6-module-transpiler, but it requires that libraries are authored in ES6 module syntax. (https://github.com/caridy/es6-module-transpiler-npm-resolver).

One option would be to factor out external dependencies, but that's probably a narrow minded solution. https://github.com/jakearchibald/es6-promise doesn't have any, for example.

There's also the option of bundling external dependencies, either using jspm or Browserify bundling. Not ideal, but it would work.

@guybedford you mention tooling for plugins at the end of your JSConf talk, is that relevant to this problem? Would appreciate your input.

from scribe.

One idea I've been trying to push for a while is the idea of a module bundle. That is, you build your internal modules into a bundle, but still create a UMD file that has external dependencies.

I've been suggesting this for both the Traceur and ES6 Module Transpiler projects, but haven't been able to convince anyone yet that it is worth doing. Relevant posts are at - esnext/es6-module-transpiler#140 (comment), google/traceur-compiler#844 (comment).

Let me know if you think something like that sounds like a possible direction here. Very keen to see work done along these lines myself.

from scribe.

Have you any thoughts on how those external dependencies are referenced? I've done something similar with Browserify in the past, using https://github.com/pluma/literalify. The idea being that you bundle your internal modules as you say, and swap your external require calls with window globals.

The ideal solution is something that transpiles to different formats, referencing external dependencies in a manor that each format understands with regard to package managers (npm for Browserify, Bower for RequireJS or other).

I don't know how well bundling would suit that requirement.

As a side note, I've just seen this:
https://github.com/polyfills/es6-module-crosspiler

from scribe.

My ideal workflow for this would be the following:

1. Write in ES6:

   main.js

   import { f } from './dep';
   import _ from 'lodash-es6';
   f(_);
   export function apiFunc() {}

   dep.js

   export function f() {...}

2. Compile into a "module bundle" that is still ES6:

   bundle.js

   import _ from 'lodash-es6';
   $$dep$f(_);
   function $$dep$f() { ... }
   export function apiFunc() {}

   note that we've inlined the private modules (./), while leaving in the public third-party modules as dependencies. This is what I mean by module bundle. The square ES6 module transpiler already does the private concat process.

3. Now, I have one ES6 module, that I can compile for the environment I want:

   • For npm & Browserify I can compile this ES6 module into CommonJS, rewriting lodash to a suitable name for npm perhaps (simple mapping).
   • For AMD & Globals / a UMD pattern I can compile this ES6 module into the good old UMD monster (https://github.com/umdjs/umd/blob/master/amdWebGlobal.js)

etc.

Let me know if that makes sense? These workflows are where we should be working towards, but not where we are today, so it is great to be discussing this stuff.

from scribe.

I like the idea. Your third point is the tricky part to my understanding, because as an author, you want to ensure that consumers are using consistent versions for dependencies. Sort of like "we don't know if you're using CommonJS or AMD, but you should be using Lo-Dash 2.4.1". I'm not sure how you propose the rewriting/mapping would work, but I'm interested to know more about that.

from scribe.

As a package author your priority is typically ensuring that it can be consumed as widely as possible. So having a flexible process for authoring to a wide number of formats / destinations is important.

A UMD pattern will always require manual config of third-party dependencies, but it will be useful to some users.

The only way to automate third-party dependencies is by using a registry system. npm kind of wins by ubiquity right now so you could compile to CommonJS and publish to npm. The dependency parameters are handled during the publish configuration process.

One can skip the middle steps and just author directly to npm, but it's the flexibility I'm trying to advocate.

from scribe.

Might be worth exploring that npm route. I foresee the publish configuration being particularly tricky.

Thanks @guybedford.

from scribe.

I believe self-executing bundles through jspm was also mentioned as a possible solution to this issue. Just to update that this feature is live in jspm 0.7 now - https://github.com/jspm/jspm-cli#4-creating-a-self-executing-bundle.

from scribe.

Lots of great thoughts in here, I'm hoping to get the time to pitch in soon as it'd be great to move towards an ES6 and UMD-compatible world!

from scribe.

@janfoeh Would you publish your umd build somewhere? I'm 3/4 of the way to having my own build, but getting stuck with the scribe-plugin-intelligent-unlink-command.js with Browserify's --standalone option.

My attempt attaches all the plugins to a Scribe.plugins global.

from scribe.

@raztus yes, I've had issues as well, although with the sanitizer plugin. Here is my intelligent unlink build. Three caveats, though: it's from September, I believe I haven't used it yet, and it exports to window.scribePluginIntelligentUnlinkCommand.

from scribe.

+1 for more module formats or global somehow. Wiring up Angular and Require JS isn't the prettiest of things.

from scribe.

Is there a way to use this without any fancy dependency management wrappers via just normal script tags. I also get a 'define is not defined' error when loading and I really don't want to have to include RequireJS to get this working.

from scribe.

+1 any news on this?

from scribe.

Would you accept a PR converting scribe to CommonJS, adding in browserify and outputting a UMD build?

from scribe.

@WickyNilliams - we'd certainly look at it!

from scribe.

👍

from scribe.

UMD is on the way out, really. Would be more progressive to port Scribe to ES6 modules, and optionally build CJS/AMD targets.

from scribe.

Even better :)

from scribe.

@theefer I've already ported Scribe to ES6 modules, and it works great internally - mind if I try pushing it upstream to this repo?

from scribe.

I can confirm. We needed to have Scribe in ES6 modules format, and it has been relatively painless to rewrite the codebase itself. Would it be a good idea to have a separate Scribe branch that just has all its source code remapped to modules, even if tests, and build tools, and other parts that now depend on AMD need to be redone? I can see the value of just allowing the source of Scribe to be imported in a project, and let the project team decide how to bundle it and its dependencies.

from scribe.

I have gone through everything I've been able to find on this subject, and still haven't really understood what the status is other than that apparently using Scribe via npm is not possible right now, unless you manually build the npm build as is done in the examples/cjs.html example.

Even a global binding as an option would be nicer. I'll much rather include a script-tag to the page than fiddle with some random build process for one dependency (or include require.js).

As others have suggested, it would be simple to port Scribe to use ES6 modules, and after that it would be just one browserify command in the build process to compile it to all module systems (CommonJS, AMD, global).

I'm willing to take part in the process of porting to ES6 modules (the repetitive work). I don't have extensive experience on publishing such a package via npm, but I'm sure someone would be able to help with that (@WickyNilliams for example proposed it already).

Question to maintainers: would this be a pull request you'd merge, assuming it would simply be a port of the current version in ES6 and with a build process outputting a fits-everything module?

Questions to others: anyone interested in helping with the PR, if maintainers confirm it'd be merged? As said, I can do the porting to ES6 modules, but would appreciate help ensuring the build process works great for npm and others.

from scribe.

I can chip in with whatever needs to be done on this.

from scribe.

@danburzo why not?

from scribe.