Git Product home page Git Product logo

Comments (21)

sokra avatar sokra commented on March 28, 2024

can you please give little more detail on this?

from webpack.

jhnns avatar jhnns commented on March 28, 2024

I've written a loader that turns every .server.js-module into a .client.js-module. It looks like this:

function myLoader() {
    var filename = this.request.split("!")[1],
        clientFilename = filename.replace(/\.server\.js$/i, ".client.js"),
        amphibiousFilename;

    if (fs.existsSync(clientFilename)) {
        return fs.readFileSync(clientFilename, "utf8");
    }

    amphibiousFilename = filename.replace(/\.server\.js$/i, ".js");
    if (fs.existsSync(amphibiousFilename)) {
        return fs.readFileSync(amphibiousFilename, "utf8");
    }

    throw new Error("You're trying to include the server-only module '" +
         filename + "' within '" + this.context + "'");
}

myLoader.loader = __filename;
myLoader.test = /\.server\.js$/i;

But now the .client.js appears as .server.js in the browser's developer tools which is right because the server-module was initially requested. I'm thinking of something like

this.filename = clientFilename;

which updates the @sourceURL for the debug view.

Ok, another solution would be to generate a module that requires the client module but this seems like too much overhead since in this case "thousands" of forwarding ghost-server-modules would appear in the browser.

from webpack.

sokra avatar sokra commented on March 28, 2024

Use the new resolve postprocessor feature:

resolve: {
  postprocess: {
    normal: [
      function(filename, cb) {
        var file = filename.split("!").pop();
        var clientFile = file.replace(/\.server\.js$/, ".client.js");
        fs.exists(clientFile, function(client) {
          if(client) return callback(null, filename.replace(/\.server\.js/, ".client.js");
          var amphibiousFile = file.replace(/\.server\.js$/, ".js");
          fs.exists(amphibiousFile, function(amphibious) {
            if(amphibious) return callback(null, filename.replace(/\.server\.js/, ".js");
            return callback(new Error("You're trying to include the server-only module '" + filename + "'"))
          });
        });
      }
    ]
  }
}

This do a lot of things better than a loader.

  • loader can be overwritten require("!./file.server.js") to ignore the security
  • correct filename in debug
  • better caching
  • no duplicate modules included

from webpack.

jhnns avatar jhnns commented on March 28, 2024

Why are they called post processor? I'm just curious, but shouldn't they be called preprocessor because they're applied before the actual module is included?

from webpack.

jhnns avatar jhnns commented on March 28, 2024

Anyhow, I like the new feature! :)

from webpack.

sokra avatar sokra commented on March 28, 2024

It's named post processor because it's executed after the resolving. resolve.postprocess

I added pre and post loaders, which are executed before and after the normal loaders.

from webpack.

jhnns avatar jhnns commented on March 28, 2024

I'm a bit confused. Could you explain the purpose of pre- and postloaders? Can't loaders be piped? And why are pre and postloaders not part of the resolve-object in the options?

Additionally to that there are normal and context post-processors which are executed before the loaders.

I'm concerned that the API will be way too complicated. I've expected to configure different loaders more like middlewares. My current stack for the framework for instance looks now something like:

    preLoaders: [
        // compile .class files 
        { test: /\.class\.js/i, loader: "compile" }
    ],
    resolve: {
        loaders: [
            // load all html files with the raw loader by default
            // (yes, someday we will minify the html here)
            { test: /\.html$/i, loader: "raw" },
            // require all modules within the page folder as lazy bundle
            { test: /([\/\\])pages\1(\w+\1)*(\w+)\1\3Page\.class\.js$/i, loader: "bundle/lazy"  }
        ],
        postprocess: {
            normal: [
                // swap server modules to client modules if present
                { test: /\.server(\.class)?\.js$/i, loader: "lookForClientModule" }
            ]
        }
    }

And I'd expect to do it like this:

loaders: [
    { test: /\.server(\.class)?\.js$/i, loader: "lookForClientModule" },
    { test: /\.class\.js/i, loader: "compile" },
    { test: /\.html$/i, loader: "raw" },
    { test: /([\/\\])pages\1(\w+\1)*(\w+)\1\3Page\.class\.js$/i, loader: "bundle/lazy"  } 
]

Notice how some loaders would be called accordingly their position in the loaders-array? For the file pages/MyPage.class.js first the compile-loader would compile the class and then bundle as lazy bundle. For the file models/MyModel.server.class.js first the lookForClientModule-loader would change the module to a client module (if present) and than compile it.

I really appreciate your willingness to help me to adjust webpack to my needs, but I'm afraid the API is getting too complicated.

from webpack.

jhnns avatar jhnns commented on March 28, 2024

Maybe we should split it into

PreProcessors
Are called with the filename. They can filter special files or swap server-files to client-files
Loaders
Do the actual loading. I like your specification of loaders. It seems very elaborate.
PostProcessors
Are called with the content. Useful to add additional code like compilation stuff

from webpack.

sokra avatar sokra commented on March 28, 2024

Maybe I should add a flow chart...

This are the steps for adding a require call:

+ required string
| [option] resolve.loaders is prepended if no loaders specified
| resolve process
| [option] resolve.postprocess
+ resolved filename
| check cache (if in watch mode)
|  reading file
|  [option] preLoaders
|  loaders
|  [option] postLoaders
|  parse file
|  save to cache (if in watch mode)
+ parsed source
| fork require calls
+ parsed source (requires are resolved)
| replace requires with ids
+ source fragment

There are more stuff running for the chunking and some specific features, but this is not included here.

from webpack.

sokra avatar sokra commented on March 28, 2024

That new API are totally optional and are intended for extensions to the compile system. They acts as middleware at 3 mayor points of the process.

Loaders operate on the "content" of the module.
resolve postprocessors operate on the filename of the module.

resolve.loaders is the point where you mainly bind filetypes to loaders.
resolve.postprocess is a point where you may alter the filename of a module, load a different one.
preLoader is a point where you can give a difference view on the files.
postLoaders is a point where you can enhance the generated javascript code.

I. e. if you want to add the functionality of exclusion of files to webpack. preLoaders is the right point. Excluded files are replaced with some mock code.

// config:
preLoaders: [{test: /\.excluded\.js$/, loader: "excluded"}]
// loader:
module.exports = function(source) {
  this.cacheable(); // flag for caching
  this.clearDependencies(); // result do not depend on "source"
  return "module.exports = undefined";
}

I. e. if you want to inject a testing framework into the compiled code, which replaces the require method. postLoaders is the right point.

// config:
postLoaders: [{test: /./, loader: "testingframework"}]
// loader:
module.exports = function(source) {
  this.cacheable(); // flag for caching
  return "require = require('testingframework')(require.valueOf());\n" + source;
}

from webpack.

jhnns avatar jhnns commented on March 28, 2024

What do you mean by different view on files?

from webpack.

sokra avatar sokra commented on March 28, 2024

The preLoaders can alter the file before the normal stuff sees it. So it offers a different view on the file.

Examples for preLoaders:

  • Minimizing Image Files
  • Linting Code Files
  • Hidding sensitive content

preLoaders (and postLoaders) are some kind of extensions.

from webpack.

jhnns avatar jhnns commented on March 28, 2024

Ok, I guessed something like that. But it seems quite strange that pre- and postLoaders are not part of the resolve options.

I also don't get it why I can't configure loaders like middlewares.

In your flowchart preLoaders are executed right before the loaders and postLoaders right after the loaders. Why don't remove pre- and postLoaders and say: "Ok, we have this stack of loaders that is executed in the order they are defined according as the regexp matches or not"?

from webpack.

jhnns avatar jhnns commented on March 28, 2024

I think that configuring middlewares is very familiar to most node developers. And since webpack loaders can be piped they are actual middlewares.

On the other hand configure objects that are too complex can be very confusing.

from webpack.

sokra avatar sokra commented on March 28, 2024

pre and postLoaders are not part of the resolve options, because they are not part of the resolve process (see flowchart).

The normal developer should not even touch any of the loader options in configuration. He should use loaders through the require call. i. e. require("bundle/lazy!./pages/HomePage.class.js") (See Lazy Pages in wiki). Loaders in config are for extending the compiler or compile process.

I've a bigger project using wbepack and this is my config:

var config = {
    input: path.join(__dirname, "lib", "client.js"),
    output: path.join(__dirname, "public_compiled", prefix + "asserts", "[hash].js"),
    publicPrefix: prefix + "asserts/",
    includeFilenames: !live,
    watch: !live,
    watchDelay: 100,
    debug: !live,
    minimize: live,
    filenames: !live,
    resolve: {
        alias: {
            "nls": "xxxxxx/nls"
        },
        paths: [path.join(__dirname, "node_modules")]
    },

    // options for specific loaders
    css: {
        requireUrl: "url/auto!"
    },
    url: {
        dataUrlLimit: 100000
    },
    i18n: {
        locales: ["de"]
    }
};

No need to put any loader into the config. I use some custom loaders and use code splitting often (maybe too often^^). It's still in work and I expect it to grow 3-4 times. Included libaries: bootstrap, jquery, three.js, CodeMirror, some jquery-plugins, and a few smaller libaries.

Hash: f6270e85cf828dd275463d4acd8ea434
Compile Time: 7419ms
Chunks: 24
Modules: 185
Modules including duplicates: 293
Modules per chunk: 12.2
Modules first chunk: 36
   f6270e85cf828dd275463d4acd8ea434.js:   560555 characters
 1.f6270e85cf828dd275463d4acd8ea434.js:     1006 characters
 2.f6270e85cf828dd275463d4acd8ea434.js:      552 characters
 3.f6270e85cf828dd275463d4acd8ea434.js:      991 characters
 4.f6270e85cf828dd275463d4acd8ea434.js:      721 characters
 5.f6270e85cf828dd275463d4acd8ea434.js:     1524 characters
 6.f6270e85cf828dd275463d4acd8ea434.js:      921 characters
 7.f6270e85cf828dd275463d4acd8ea434.js:      622 characters
 8.f6270e85cf828dd275463d4acd8ea434.js:     1003 characters
 9.f6270e85cf828dd275463d4acd8ea434.js:      758 characters
10.f6270e85cf828dd275463d4acd8ea434.js:     3279 characters
11.f6270e85cf828dd275463d4acd8ea434.js:     7120 characters
12.f6270e85cf828dd275463d4acd8ea434.js:     7119 characters
13.f6270e85cf828dd275463d4acd8ea434.js:     7023 characters
14.f6270e85cf828dd275463d4acd8ea434.js:    43749 characters
15.f6270e85cf828dd275463d4acd8ea434.js:    51866 characters
16.f6270e85cf828dd275463d4acd8ea434.js:    30083 characters
17.f6270e85cf828dd275463d4acd8ea434.js:    76852 characters
18.f6270e85cf828dd275463d4acd8ea434.js:   202312 characters
19.f6270e85cf828dd275463d4acd8ea434.js:    82299 characters
20.f6270e85cf828dd275463d4acd8ea434.js:    67141 characters
21.f6270e85cf828dd275463d4acd8ea434.js:    80581 characters
22.f6270e85cf828dd275463d4acd8ea434.js:   256183 characters
23.f6270e85cf828dd275463d4acd8ea434.js:     6642 characters
  b68045c65bc5d59e7578d34daecd8fd8.jpg:    13538 characters
  955b61e79835a426b0db6bdc13ea0830.png:   117014 characters
  fd9f6ae247ab82fa107f07d86db965b5.png:    86799 characters
  f12c547dbdb845aae39504315f7f7782.png:    74395 characters

Takes 15s to compile produtive version. In watch mode 1,6s after each change.

from webpack.

jhnns avatar jhnns commented on March 28, 2024

But in a context of a client-server-framework - that is build on webpack - I want to hide the webpack loaders for developers that are not familiar with bundling techniques. I want to provide a good default, that uses different loaders based on their file ending and their folder. For instance, every Page-Class is loaded via bundle/lazy because it makes totally sense to split an application according to its pages.

But if requested the developer may customize the loading behavior via the require("myLoader!myModule") syntax.

That's my goal. Trying to provide a good default that works for 95% of all cases but with the possibility to customize it. That's why I'm always fiddling around with the config instead of using the exclamation mark syntax. Because than I have to explain it to the developer that wants to use my framework.

from webpack.

sokra avatar sokra commented on March 28, 2024

ok that are good reasons... should be possible with the existing api :)

from webpack.

jhnns avatar jhnns commented on March 28, 2024

Yes, actually it is working. But I'm not 100% sure why ... that's why I'm kind of nervous :).

I'll explain shortly my current setup: Basically I have a class compiler which evaluates the module code and adds some lines to the end of the module that makes it possible to write private, protected, public, static or even abstract properties. This compiler is applied if the file ends on .class.js. Additionally all classes within the pages-folder are bundled as lazy bundles. Therefore all pages are touched by the compiler and the lazy bundler.

So I would've expected the class compiler to be a postLoader since it enhances the existing JavaScript. But in this case the output results in one big chunk. So what I actually have to do is to add it as a preLoader, which works perfectly fine.

But maybe that's already too implementation specific.

from webpack.

sokra avatar sokra commented on March 28, 2024

It should work with both loader types. You loader may corrupt the module generated by the bundle loader (format). As nodeclass is valid javascript, enhanced by the compiler, I would think it fit postLoaders best. You should not filter the postLoader by regexp, but enhance only valid source. So it would be possible to generate nodeclass source by loader.

from webpack.

sokra avatar sokra commented on March 28, 2024

I. e. require("Cup.class.coffee") should work. So it must be a postLoader.

If we follow the sheme that file format conversion should happen in (normal) loaders:

preLoader: fileFormatA -> fileFormatA
loader: fileFormatA -> ... -> javascriptFileFormat
postLoader: javascriptFileFormat -> javascriptFileFormat

fileFormatA may be coffeescript, javascript, text, png, css, less, etc.

from webpack.

ackvf avatar ackvf commented on March 28, 2024

So, how can I change the imported filename in 2019 not using an alias? 😅

from webpack.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.