Git Product home page Git Product logo

japr's Introduction

JAPR (Jekyll Asset Pipeline Reborn)

Gem Version Build Status Coveralls Status

Update On December 20, 2017 JAPR has been merged back into Jekyll Asset Pipeline. As my time permits I'll try to maintain both repositories.

JAPR is a powerful asset pipeline that automatically collects, converts and compresses / minifies your site's JavaScript and CSS assets when you compile your Jekyll site.

Table of Contents

Features

  • Declarative dependency management via asset manifests
  • Asset preprocessing/conversion (supports CoffeeScript, Sass / Scss, Less, Erb, etc.)
  • Asset compression (supports YUI Compressor, Closure Compiler, etc.)
  • Fingerprints bundled asset filenames with MD5 hashes for better browser caching
  • Automatic generation of HTML link and script tags that point to bundled assets
  • Integrates seamlessly into Jekyll's workflow, including auto site regeneration

How It Works

JAPR's workflow can be summarized as follows:

  1. Reviews site markup for instances of the css_asset_tag and javascript_asset_tag Liquid tags. Each occurrence of either of these tags identifies when a new bundle needs to be created and outlines (via a manifest) which assets to include in the bundle.
  2. Collects raw assets based on the manifest and runs them through converters / preprocessors (if necessary) to convert them into valid CSS or JavaScript.
  3. Combines the processed assets into a single bundle, compresses the bundled assets (if desired) and saves the compressed bundle to the _site output folder.
  4. Replaces css_asset_tag and javascript_asset_tag Liquid tags with HTML link and script tags, respectively, that link to the finished bundles.

Getting Started

JAPR is extremely easy to add to your Jekyll project and has no incremental dependencies beyond those required by Jekyll. Once you have a basic Jekyll site up and running, follow the steps below to install and configure JAPR.

  1. Install the japr gem via Rubygems.
$ gem install japr

If you are using Bundler to manage your project's gems, you can just add japr to your Gemfile and run bundle install.

  1. Add a _plugins folder to your project if you do not already have one. Within the _plugins folder, add a file named japr.rb with the following require statement as its contents.
require 'japr'
  1. Move your assets into a Jekyll ignored folder (i.e. a folder that begins with an underscore _) so that Jekyll won't include these raw assets in the site output. It is recommended to use an _assets folder to hold your site's assets.

  2. Add the following Liquid blocks to your site's HTML head section. These blocks will be converted into HTML link and script tags that point to bundled assets. Within each block is a manifest of assets to include in the bundle. Assets are included in the same order that they are listed in the manifest. Replace the foo and bar assets with your site's assets. At this point we are just using plain old javascript and css files (hence the .js and .css extensions). See the Asset Preprocessing section to learn how to include files that must be preprocessed (e.g. CoffeeScript, Sass, Less, Erb, etc.). Name the bundle by including a string after the opening tag. We've named our bundles "global" in the below example.

{% css_asset_tag global %}
- /_assets/foo.css
- /_assets/bar.css
{% endcss_asset_tag %}

{% javascript_asset_tag global %}
- /_assets/foo.js
- /_assets/bar.js
{% endjavascript_asset_tag %}

Asset manifests must be formatted as YAML arrays and include full paths to each asset from the root of the project. YAML does not allow tabbed markup, so you must use spaces when indenting your YAML manifest or you will get an error when you compile your site. If you are using assets that must be preprocessed, you should append the appropriate extension (e.g. '.js.coffee', '.css.less') as discussed in the Asset Preprocessing section.

  1. Run the jekyll build command to compile your site. You should see an output that includes the following JAPR status messages.
$ jekyll build
Generating...
Asset Pipeline: Processing 'css_asset_tag' manifest 'global'
Asset Pipeline: Saved 'global-md5hash.css' to 'yoursitepath/assets'
Asset Pipeline: Processing 'javascript_asset_tag' manifest 'global'
Asset Pipeline: Saved 'global-md5hash.js' to 'yoursitepath/assets'

If you do not see these messages, check that you have not set Jekyll's safe option to true in your site's _config.yml. If the safe option is set to true, Jekyll will not run plugins.

That is it! You should now have bundled assets. Look in the _site folder of your project for an assets folder that contains the bundled assets. HTML tags that point to these assets have been placed in the HTML output where you included the Liquid blocks. You may notice that your assets have not been converted or compressed-- we will add that functionality next.

Asset Preprocessing

Asset preprocessing (i.e. conversion) allows us to write our assets in languages such as CoffeeScript, Sass, Less, Erb or any other language. One of JAPR's key strengths is that it works with any preprocessing library that has a ruby wrapper. Adding a preprocessor is straightforward, but requires a small amount of additional code.

In the following example, we will add a preprocessor that converts CoffeeScript into JavaScript.

CoffeeScript

  1. In the japr.rb file that we created in the Getting Started section, add the following code to the end of the file (i.e. after the require statement).
module JAPR
  class CoffeeScriptConverter < JAPR::Converter
    require 'coffee-script'

    def self.filetype
      '.coffee'
    end

    def convert
      return CoffeeScript.compile(@content)
    end
  end
end

The above code adds a CoffeeScript converter. You can name a converter anything as long as it inherits from JAPR::Converter. The self.filetype method defines the type of asset a converter will process (e.g. .coffee for CoffeeScript) based on the extension of the raw asset file. A @content instance variable that contains the raw content of our asset is made available within the converter. The converter should process this content and return the processed content (as a string) via a convert method.

  1. If you haven't already, you should now install any dependancies that are required by your converter. In our case, we need to install the coffee-script gem.
$ gem install coffee-script

If you are using Bundler to manage your project's gems, you can just add coffee-script to your Gemfile and run bundle install.

  1. Append a .coffee extension to the filename of any asset that should be converted with the CoffeeScriptConverter. For example, foo.js would become foo.js.coffee.

  2. Run the jekyll build command to compile your site.

That is it! Your asset pipeline has converted any CoffeeScript assets into JavaScript before adding them to a bundle.

SASS / SCSS

You probably get the gist of how converters work, but here's an example of a SASS converter for quick reference.

module JAPR
  class SassConverter < JAPR::Converter
    require 'sass'

    def self.filetype
      '.scss'
    end

    def convert
      return Sass::Engine.new(@content, syntax: :scss).render
    end
  end
end

Don't forget to install the sass gem or add it to your Gemfile and run bundle install before you run the jekyll build command since the above SASS converter requires the sass library as a dependency.

If you're using @import statements in your SASS files, you'll probably need to specify a base load path to the SASS engine in your convert method. You can use the @dirname instance variable for this, which contains the path to the current asset's directory:

...
def convert
  return Sass::Engine.new(@content, syntax: :scss, load_paths: [@dirname]).render
end
...

LESS

module JAPR
  class LessConverter < JAPR::Converter
    require 'less'

    def self.filetype
      '.less'
    end

    def convert
      return Less::Parser.new.parse(@content).to_css
    end
  end
end

Don't forget to install the less gem or add it to your Gemfile and run bundle install before you run the jekyll build command since the above LESS converter requires the less library as a dependency.

As with the SASS convertor, you'll probably need to specify a base load path and pass that to the LESS Parser:

...
def convert
  return Less::Parser.new(paths: [@dirname]).parse(@content).to_css
end
...

Successive Preprocessing

If you would like to run an asset through multiple preprocessors successively, you can do so by naming your assets with nested file extensions. Nest the extensions in the order (right to left) that the asset should be processed. For example, .css.scss.erb would first be processed by an erb preprocessor then by a scss preprocessor before being rendered. This convention is very similar to the convention used by the Ruby on Rails asset pipeline.

Don't forget to define preprocessors for the extensions you use in your filenames, otherwise JAPR will not process your asset.

Asset Compression

Asset compression allows us to decrease the size of our assets and increase the speed of our site. One of JAPR's key strengths is that it works with any compression library that has a ruby wrapper. Adding asset compression is straightforward, but requires a small amount of additional code.

In the following example, we will add a compressor that uses Yahoo's YUI Compressor to compress our CSS and JavaScript assets.

Yahoo's YUI Compressor

  1. In the japr.rb file that we created in the Getting Started section, add the following code to the end of the file (i.e. after the require statement).
module JAPR
  class CssCompressor < JAPR::Compressor
    require 'yui/compressor'

    def self.filetype
      '.css'
    end

    def compress
      return YUI::CssCompressor.new.compress(@content)
    end
  end

  class JavaScriptCompressor < JAPR::Compressor
    require 'yui/compressor'

    def self.filetype
      '.js'
    end

    def compress
      return YUI::JavaScriptCompressor.new(munge: true).compress(@content)
    end
  end
end

The above code adds a CSS and a JavaScript compressor. You can name a compressor anything as long as it inherits from JAPR::Compressor. The self.filetype method defines the type of asset a compressor will process (either '.js' or '.css'). The compress method is where the magic happens. A @content instance variable that contains the raw content of our bundle is made available within the compressor. The compressor should process this content and return the processed content (as a string) via a compress method.

  1. If you haven't already, you should now install any dependencies that are required by your compressor. In our case, we need to install the yui-compressor gem.
$ gem install yui-compressor

If you are using Bundler to manage your project's gems, you can just add yui-compressor to your Gemfile and run bundle install.

  1. Run the jekyll build command to compile your site.

That is it! Your asset pipeline has compressed your CSS and JavaScript assets. You can verify that this is the case by looking at the contents of the bundles generated in the _site/assets folder of your project.

Google's Closure Compiler

You probably get the gist of how compressors work, but here's an example of a Google Closure Compiler compressor for quick reference.

class JavaScriptCompressor < JAPR::Compressor
  require 'closure-compiler'

  def self.filetype
    '.js'
  end

  def compress
    return Closure::Compiler.new.compile(@content)
  end
end

Don't forget to install the closure-compiler gem before you run the jekyll build command since the above compressor requires the closure-compiler library as a dependency.

Templates

When JAPR creates a bundle, it returns an HTML tag that points to the bundle. This tag is either a link tag for CSS or a script tag for JavaScript. Under most circumstances the default tags will suffice, but you may want to customize this output for special cases (e.g. if you want to add a CSS media attribute).

In the following example, we will override the default CSS link tag by adding a custom template that produces a link tag with a media attribute.

  1. In the japr.rb file that we created in the Getting Started section, add the following code.
module JAPR
  class CssTagTemplate < JAPR::Template
    def self.filetype
      '.css'
    end

    def html
      "<link href='#{output_path}/#{@filename}' rel='stylesheet' " \
        "type='text/css' media='screen' />\n"
    end
  end
end

If you already added a compressor and/or a converter, you can include your template class alongside your compressor and/or converter within the same JAPR module.

The “self.filetype” method defines the type of bundle a template will target (either .js or .css). The “html” method is where the magic happens. output_path is a helper method and @filename is an instance variable which are available within the class and contain the path and filename of the generated bundle, respectively. The template should return a string that contains an HTML tag pointing to the generated bundle via an html method.

  1. Run the jekyll command to compile your site.

That is it! Your asset pipeline used your template to generate an HTML link tag that includes a media attribute with the value screen. You can verify that this is the case by viewing the generated source within your project's _site folder.

Configuration

JAPR provides the following configuration options that can be controlled by adding them to your project's _config.yml file. If you don't have a _config.yml file, consider reading the configuration section of the Jekyll documentation.

asset_pipeline:
  bundle: true
  compress: true
  output_path: assets
  display_path: nil
  gzip: false
Setting Default Description
bundle true controls whether JAPR bundles the assets defined in each manifest. If set to false, each asset will be saved individually and individual html tags pointing to each unbundled asset will be produced when you compile your site. It is useful to set this to false while you are debugging your site.
compress true tells JAPR whether or not to compress the bundled assets. It is useful to set this setting to false while you are debugging your site.
output_path assets defines where generated bundles should be saved within the _site folder of your project.
display_path nil overrides the path to assets in generated html tags. This is useful if you are hosting your site at a path other than the root of your domain (e.g. http://example.com/blog/).
gzip false controls whether JAPR saves gzipped versions of your assets alongside un-gzipped versions.

Octopress

Octopress is a popular framework for Jekyll that can help you get a blog up and running quickly. JAPR can be added to an Octopress site using the Getting Started steps above with the following modifications:

  1. Octopress uses Bundler to manage your site's dependencies. You should add gem japr to your Gemfile and then run bundle install to install.

  2. Instead of adding a _plugins folder, you should put japr.rb in the plugins folder included by default in the root of your Octopress site.

  3. You should still store your assets in an Jekyll ignored folder (i.e. a folder that begins with an underscore _), but note that this folder should be located within the source folder of your Octopress site (e.g. source/_assets).

  4. No change to this step.

  5. Instead of running the jekyll command to compile your site, you should use Octopress' rake commands (e.g. rake generate) as outlined here.

If you have any difficulties using JAPR with Octopress, please open an issue.

Contribute

You can contribute to the JAPR by submitting a pull request via GitHub. There are a few areas that need improvement:

  • Tests, tests, tests. This project is now fully tested.
  • Successive preprocessing. Currently you can only preprocess a file once. It would be better if you could run an asset through multiple preprocessors before it gets compressed and bundled. As of v0.1.0, JAPR now supports successive preprocessing.
  • Handle remote assets. Right now, JAPR does not provide any way to include remote assets in bundles unless you save them locally before generating your site. Moshen's Jekyll Asset Bundler allows you to include remote assets, which is pretty interesting. That said, it is generally better to keep remote assets separate so that they load asynchronously.

If you have any ideas or you would like to see anything else improved please use the issues section.

Changelog

See the changelog.

Community

Credits

License

JAPR is released under the MIT License.

japr's People

Contributors

brandonmwest avatar caseyscarborough avatar dependabot-preview[bot] avatar gavinballard avatar geniou avatar janosrusiczki avatar matthodan avatar szemek avatar tbjers avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

japr's Issues

Support html & image compression

HTML & image compression may be a rather useful feature to have in jekyll-asset-pipeline. Since jekyll-asset-pipeline preprocesses & compresses css & js, why not have it compress html and images as well?

There're a couple of utilities that can do this. For example, image-optimizer (haven't tried this yet), htmlcompressor.

The test suite fails randomly

  1) Error:
JAPR::Template::with custom template::class methods::::priority#test_0001_anonymous:
NameError: uninitialized constant TestTemplate
Did you mean?  TestTemplate
    /home/dev/japr/spec/template_spec.rb:102:in `block (5 levels) in <top (required)>'

  2) Error:
JAPR::Template::with custom template::instance methods::html#test_0001_anonymous:
NameError: uninitialized constant TestTemplate
Did you mean?  TestTemplate
    /home/dev/japr/spec/template_spec.rb:107:in `block (4 levels) in <top (required)>'

  3) Error:
JAPR::Template::with custom template::instance methods::#new(path, filename)#test_0001_anonymous:
NameError: uninitialized constant TestTemplate
Did you mean?  TestTemplate
    /home/dev/japr/spec/template_spec.rb:107:in `block (4 levels) in <top (required)>'

  4) Error:
JAPR::Template::with custom template::instance methods::#new(path, filename)#test_0002_anonymous:
NameError: uninitialized constant TestTemplate
Did you mean?  TestTemplate
    /home/dev/japr/spec/template_spec.rb:107:in `block (4 levels) in <top (required)>'

process YAML header first

I'd like to be able to make use of the YAML header / liquid template processing in out-of-the-box Jekyll for .less files. i.e.:

---

---

.{{ someTargetNameFromConfig }} {
  /* ... */
} 

so I can only have to specify those classes (e.g.) in one place to use them across the pages / css / js.

As it stands, the assets aren't initially checked for any YAML / liquid processing before they are then run through the (e.g.) Less compiler.

Gemspec file updates

  • Update rubygems version
  • Remove trailing ' from the description
  • Include CHANGELOG.md in the files array

filename hash changes too often

Using jekyll build out-of-the-box appears to update the mod times on js / css and other precursor files regardless of actual changes to the file. This in turn means changes to the file name md5 component, which (for a github pages site using jekyll w/ japr) means lots of not-real changes as script and style head elements change.

Not a performance killer, but it definitely adds up on site with lots of pages and many small changes.

Request either usage guidance with jekyll to avoid this issue (there may be some config option I'm missing) or modification of japr to md5 on, say, the file contents instead.

Support for gem theme assets

Hi @janosrusiczki,

Thank you for the awesome asset pipeline! Any idea on what work is needed to fetch assets from a Jekyll theme? I'll happily submit a PR if you can point me in the right direction?

I thought Jekyll pulled assets over from the theme (see - https://jekyllrb.com/docs/themes/#understanding-gem-based-themes). Maybe the plugin is running before these have been synced over? Jekyll doesn't add the assets to the site directory but they must be referenced in Ruby array?

Kind regards,

Kyle

Jekyll enters infinite regenerate loop when using JAPR

I added JAPR to my project and it works fine generating a single JS and CSS file.

However, if I'm in Jekyll watch mode, the first change I do to any page, layout, include, etc. triggers an infinite loop regenerating everything. Does that work for anyone currently?

I'm using jekyll 2.5.2.

Soft hash

Would it be possible to add an option for hard and soft hashes?

Soft hash: assets/global.js?v=md5hash
Hard hash: assets/global-md5hash.js

JAPR neither concatenating stylesheets nor producing error

I followed along with the instructions, downloaded the gem and included a jekyll_asset_pipeline.rb file in my plugins directory that requires the gem. I also tried using the _config.yml gem option to include the plugin.

I have two CSS files that aught to be concatenated, but the resultant html file in _site contains the same liquid tags as the input with no error:

{% css_asset_tag global %}
- /_assets/test.css
- /_assets/testing.css
{% endcss_asset_tag %}

I have another plugin running so I know that I'm not in safe mode. I'm using Jekyll version 2.4.0.

Fix random coverage jumps

For some reason LiquidBlockExtension::ClassMethods are covered in one run and not covered in another.

Create a changelog

Mainly for ease of tracking of what's been included in a specific version.

Getting Dependency error while running jekyll build

I followed all the steps in Getting Started Section from step 1 to 4 but as soon as I run the "bundle exec jekyll build" command from step 5 I get the below Dependency error.

Dependency Error: Yikes! It looks like you don't have C:/Kommunicate.io/_plugins/japr.rb or one of its dependencies installed. In order to use Jekyll as currently configured, you'll need to install this gem. The full error message from Ruby is: 'cannot load such file -- japr' If you run into trouble, you can find helpful resources at https://jekyllrb.com/help/!

If i run "jekyll build" alone I am getting this error:

WARN: Unresolved specs during Gem::Specification.reset:
      rb-fsevent (>= 0.9.4, ~> 0.9)
      ffi (< 2, >= 0.5.0)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/bundler-1.16.0/lib/bundler/runtime.rb:313:in `check_for_activated_spec!': You have already activated liquid 4.0.0, but
your Gemfile requires liquid 3.0.6. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
        from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/bundler-1.16.0/lib/bundler/runtime.rb:31:in `block in setup'
        from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/bundler-1.16.0/lib/bundler/runtime.rb:26:in `map'
        from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/bundler-1.16.0/lib/bundler/runtime.rb:26:in `setup'
        from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/bundler-1.16.0/lib/bundler.rb:107:in `setup'
        from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/jekyll-3.6.2/lib/jekyll/plugin_manager.rb:50:in `require_from_bundler'
        from C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/jekyll-3.6.2/exe/jekyll:11:in `<top (required)>'
        from C:/Ruby23-x64/bin/jekyll:22:in `load'
        from C:/Ruby23-x64/bin/jekyll:22:in `<main>'

Any solution for this?

Support Jekyll 3.x

Jekyll 3.0.0 was released on 2015-10-26 and 3.1.0 has been released two days ago at the time of this writing (making it 2016-01-23) but japr locks itself to Jekyll 2.x at this time.

It would be nice if it could be updated and a new release made to support the latest version of Jekyll. Looking at the changes it does not look like there would be significant API changes to adapt to.

Unable to activate

I get this error
ise_if_conflicts': Unable to activate japr-0.3.2, because liquid-3.0.6 conflicts with liquid (~> 2.4), jekyll-3.0.0 conflicts with jekyll (~> 2.0) (Gem::LoadErr

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.