rspivak / slimit Goto Github PK

View Code? Open in Web Editor NEW

551.0 551.0 94.0 545 KB

SlimIt - a JavaScript minifier/parser in Python

License: MIT License

Python 100.00%

slimit's People

Contributors

Stargazers

Watchers

Forkers

firemyst duilio agriffis gfmartins rskumar lelit shobhit nuklea weaver-viii liftoff harig inkymail autumnjolitz joegermuska allbuttonspressed niwinz cherry-wb zer0- ngs2410 pombredanne leonardoo acatton rojer sgallagher moses-palmer slideclick luiseduardohdbackup sexst cainiaocome armenarmen dalduba fangbei ghosthamlet stevenvanacker longlongtech prideauxx alanwooo adamchainz ztane avishkar58 snehesht redyyu zlzw comawill pretix tobby2002 fflexo bstrdsmkr xtqxk wediors mrj0n3s kaishuibaicai ajinkyakulkarni alvinvinvinvin ale-rt brandonwillard lostwater ovwane phillipdwright takluyver sowle khongten001 saviodcunha matthew-carpenter ykankaya enfold afcarl dtuan chubbymaggie scopatz hmyit kepy909 shalevy1 ap-wtioit n3x15 mvondracek ra2003 atempcode1 yousafe007 592767809 tzunieh jasiken collindever dong-cg assertive-yield prd-hop-duong

slimit's Issues

Add [Block | Line]Comment Node support

Currently there is no way to keep

licence
jsdoc

in syntax tree and as a result in to_ecma output which is essential in some cases.

Accessing .default in object crashes Slimit.

Slimit crashes when trying to minify anything that tries to get a reference to .default. As a workaround, you can access it with object['default'].

>>> slimit.minify("var a={b:'b', c:'c', default:'def'}; console.log(a.default)")

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python27\lib\site-packages\slimit\minifier.py", line 38, in minify
    tree = parser.parse(text)
  File "C:\Python27\lib\site-packages\slimit\parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "C:\Python27\lib\site-packages\ply\yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "C:\Python27\lib\site-packages\ply\yacc.py", line 1047, in parseopt_notra
ck
    tok = self.errorfunc(errtoken)
  File "C:\Python27\lib\site-packages\slimit\parser.py", line 116, in p_error
    self._raise_syntax_error(token)
  File "C:\Python27\lib\site-packages\slimit\parser.py", line 89, in _raise_synt
ax_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (DEFAULT, 'default') at 1:21 between LexToken(COMM
A,',',1,19) and LexToken(COLON,':',1,28)

Miscompilation of for statement

The following code gets miscompiled by current slimit

var i=0;
for(i; i<10; i++) {}

as it emits something like

var i=0;
for(i i<10; i++) {}

I found such a (silly) construct in the ExtJS sources.

Line number tracking

slimit removes parentheses from ternary expression, causes syntax error in jQuery

There is a bug in the Slimit JSParser / serializer, which we hit when trying to convert some files including jQuery. It removes some parentheses it shouldn't remove, causing a syntax error in jquery.js.

e.isPlainObject(d) ? (a = [c.createElement(j[1])], e.fn.attr.call(a, d, !0))

becomes

e.isPlainObject(d) ? a = [c.createElement(j[1])], e.fn.attr.call(a, d, !0)

which breaks the ternary expression.

Probably another variant of #21 ?

The full line of code is:

d=d instanceof e?d[0]:d,k=d?d.ownerDocument||d:c,j=m.exec(a),j?e.isPlainObject(d)?(a=[c.createElement(j[1])],e.fn.attr.call(a,d,!0)):a=[k.createElement(j[1])]:(j=e.buildFragment([g[1]],[k]),a=(j.cacheable?e.clone(j.fragment):j.fragment).childNodes);

slimit.py src.js: list index out of range

minifier.py:57,58

if len(args) == 1:
text = open(args[1]).read()

should be args[0]...

for loop minification

Assigning the iterating variable before the loop causes issue in minification

var i = 0;
for (i; i < 5; i++) {
    // Do something
}
// Outputs: var i=0;for(ii<5;i++){} (broken)

var i;
for (i = 0; i < 5; i++) {
    // Do something
}
// Outputs: var i;for(i=0;i<5;i++){} (correct)

Using $ for mangled function names conflicts with jQuery

I have a "class" function with many subfunctions inside of it. When compressing it with Slimit, the function name mangling uses $ and _ for function names which results in errors because I am using jQuery and expect $ to remain reserved for jQuery.

See the testcase at https://gist.github.com/3806569
Names of functions 52 and 53 are mangled to $ and _ which breaks jQuery usage.

$ and _ are both used by popular JS libraries (jQuery and Underscore.js) so I'd suggest not to use them for function names.

Removing those 2 chars from ID_CHARS in scope.py fixes the problem for me, but I don't know if it might have any side effects.

Generate source maps

It would be very useful for debugging if slimit could generate source maps.

Problem with "+", "++" and whitespace

Needed whitespace is being lost in this example.

>>> minify('''"begin"+ ++a+"end"''')
'"begin"+++a+"end";'
>>>

The result is interpreted as a ++ postfix operator on "begin".

Add IF statement optimization N1

Add the following optimization:

if (foo) bar(); else baz(); ==> foo?bar():baz();

SlimIt breaks on unicode byteorder marks

Example: "jquery.blockUI.js" and "jquery.caret.min.js"

Both have the unicode byteorder mark "EF BB BF" before or after the first line of code. Result is:

Illegal character '\xbb' at 1:1 after LexToken(ID,'\xef',1,0)
Illegal character '\xbf' at 1:2 after LexToken(ID,'\xef',1,0)

and errors in the browser with compressed versions of those:
ERROR: ILLEGAL TOKEN

No tar.gz download

I'm about to package slimit for Debian and it would be nice if it was also available as a tar.gz download.

Breaks on object attributes that use certain special characters

I just ran slimit -m jquery-1.9.1.js > jquery.min.js and this resulted in errors in the browser:

Uncaught SyntaxError: Unexpected identifier

...so I did some digging and it turns out this is the line that caused the problem:

jQuery.expr[":"] = jQuery.expr.pseudos;

The problem is that slimit converted that expr[":"] into expr.: which is not valid JavaScript. It seems there's a number of special and "illegal" characters where slimit has this problem.

I've put together a simple test case to demonstrate the problem:

var testObj = {};
testObj[":"] = undefined; // Breaks
testObj["::"] = undefined; // Breaks
testObj["a:"] = undefined; // Breaks
testObj["."] = undefined; // OK
testObj["{"] = undefined; // OK
testObj["}"] = undefined; // OK
testObj["["] = undefined; // Breaks
testObj["]"] = undefined; // Breaks
testObj["("] = undefined; // OK
testObj[")"] = undefined; // OK
testObj["="] = undefined; // Breaks
testObj["-"] = undefined; // OK
testObj["+"] = undefined; // OK
testObj["*"] = undefined; // OK
testObj["/"] = undefined; // OK
testObj["\\"] = undefined; // Breaks
testObj["%"] = undefined; // OK
testObj["<"] = undefined; // Breaks
testObj[">"] = undefined; // Breaks
testObj["!"] = undefined; // OK
testObj["?"] = undefined; // Breaks
testObj[","] = undefined; // OK
testObj["@"] = undefined; // Breaks
testObj["#"] = undefined; // OK
testObj["&"] = undefined; // OK
testObj["|"] = undefined; // OK
testObj["~"] = undefined; // OK
testObj["`"] = undefined; // Breaks
testObj["."] = undefined; // OK

This is the result (newlines added by me):

var testObj={};
testObj.:=undefined;
testObj.::=undefined;
testObj.a:=undefined;
testObj["."]=undefined;
testObj["{"]=undefined;
testObj["}"]=undefined;
testObj.[=undefined;
testObj.]=undefined;
testObj["("]=undefined;
testObj[")"]=undefined;
testObj.==undefined;
testObj["-"]=undefined;
testObj["+"]=undefined;
testObj["*"]=undefined;
testObj["/"]=undefined;
testObj.\\=undefined;
testObj["%"]=undefined;
testObj.<=undefined;
testObj.>=undefined;
testObj["!"]=undefined;
testObj.?=undefined;
testObj[","]=undefined;
testObj.@=undefined;
testObj["#"]=undefined;
testObj["&"]=undefined;
testObj["|"]=undefined;
testObj["~"]=undefined;
testObj.`=undefined;
testObj["."]=undefined;

Please take a look when you get a chance. I hope it is as simple as adding a few more characters to a list of specials somewhere ▶️

mangler ignores catch() var

When I try to mangle the following code:

function a() {
  var $exc = null;
  try {
    lala();
  } catch($exc) {
    if ($exc.__name__ == 'hi') {
      return 'bam';
    }
  }
  return 'bum';
}

I get the following result:

function a() {
  var a = null;
  try {
    lala();
  } catch ($exc) {
    if (a.__name__ == 'hi') {
      return 'bam';
    }
  }
  return 'bum';
}

As you can see, the $exc variable in the catch() clause was just ignored.

Slimit/PLY explodes on unicode input

Slimit/PLY throws:

Traceback (most recent call last):
  File "vardetect.py", line 35, in collect_var_dec_globals
    tree = parser.parse(source)
  File "/Library/Python/2.7/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "/Library/Python/2.7/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "vardetect.py", line 242, in     monkey_patch_parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Library/Python/2.7/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)
  File "/Library/Python/2.7/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (DIV, u'/') at 1:12757 between     LexToken(RPAREN,u')',1,12755) and LexToken(BXOR,u'^',1,12758)

On Javascript from URL:
http://www.510.com/style1_3/510.js

Environment:
Python 2.7,
pip install git+https://github.com/rspivak/slimit.git#egg=slimit

monkey_patch_parseopt_notrack is a line by line copy of parseopt_notrack from yacc.py with try/except around the del errok statements. Assume it is same as yacc.py.

Bad LexToken(STRING) or invalid javascript?

$ echo "var SohoPriceSlideData = { min_price : '\3,000' };" | slimit

Illegal character "'" at 1:39 after LexToken(COLON,':',1,37)
Illegal character '\\' at 1:40 after LexToken(COLON,':',1,37)
Illegal character "'" at 1:46 after LexToken(NUMBER,'000',1,43)

Traceback (most recent call last):
  File "/home/nuklea/.virtualenvs/project/bin/slimit", line 9, in <module>
    load_entry_point('slimit==0.7.4', 'console_scripts', 'slimit')()
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/minifier.py", line 69, in main
    text, mangle=options.mangle, mangle_toplevel=options.mangle_toplevel)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/minifier.py", line 38, in minify
    tree = parser.parse(text)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (SEMI, ';') at 1:48 between LexToken(NUMBER,'000',1,43) and LexToken(RBRACE,'}',1,48)

Maybe a bug?

when i try to slim this:

var tagRegExp = new RegExp('<(/*)(FooBar)', 'gi');

I get:
SyntaxError: Unexpected token (LT, '<') at 1:30 between LexToken(LPAREN,'(',1,28) and LexToken(LPAREN,'(',1,31)

This same error occurs across several different libraries i use. One of them being monernizr.

Removal of necessary parentheses

c||(c=393); // is processed correctly as:
c||(c=393); // parentheses kept.

However, in this situation:

c||(c=393,a=323,b=2321);

result is this:

c||c=393,a=323,b=2321;

causing a syntax error.

Version: slimit-0.5.5

SlimIt fails to parse jQuery fullcalendar

SlimIt returns the following error when minifying jQuery fullcalendar 1.5.4:

SyntaxError: Unexpected token (SEMI, ';') at 1:93792 between LexToken(LINE_TERMINATOR,'\n',1,93780) and LexToken(REGEX,"/ opt('slotMinutes')\n\t\t\t\t\t\t\t\t\t\t)\n\t\t\t\t\t\t\t\t\t);\n\t\t\t\t\t\t\t\t\teventElement.draggable('option', 'grid', [colWidth, 1]);\n\t\t\t\t\t\t\t\t\tallDay = false;\n\t\t\t\t\t\t\t\t}\n\t\t\t\t\t\t\t}else{\n\t\t\t\t\t\t\t\trevert = true;\n\t\t\t\t\t\t\t}\n\t\t\t\t\t\t}\n\t\t\t\t\t\trevert = revert || (allDay && !dayDelta);\n\t\t\t\t\t}else{\n\t\t\t\t\t\tresetElement();\n\t\t\t\t\t\t/",1,93792)

Add optimization: foo["bar"] ==> foo.bar

Unable to Parse valid statement...

In Short: Like it perfectly parses a=function(){}(); slimit should be able to parse a||function(){}(); too.

In Detail:
slimit parses

(function() { return "slimit!"; } )();

and

a=function () { return "slimit!" ; }();

perfectly.

It throws an exception for
function() { return "slimit!"; } ();

as
SyntaxError: Unexpected token (LPAREN, '(') at 1:8 between LexToken(FUNCTION,'function',1,0) and LexToken(RPAREN,')',1,9)

which is again perfect since function declaration needs a name here.

It throws the similar (unexpected token is FUNCTION this time, instead of LPAREN) exception here:

window.done_already || function () { return "slimit!" ; }();

which is syntactically correct like the second example above.

Problem with multiline strings

First of all, thanks for slimit! Nice tool.

Ran into a bug with multiline strings. If you provide this input:

var a = " \
";

You get this traceback:

Illegal character '"'
Illegal character '\\'
Illegal character '"'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/databakery/deployments/deployment-development/virtualenv/lib/python2.6/site-packages/slimit/minifier.py", line 38, in minify
    tree = parser.parse(text)
  File "/home/databakery/deployments/deployment-development/virtualenv/lib/python2.6/site-packages/slimit/parser.py", line 64, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "/home/databakery/deployments/deployment-development/virtualenv/lib/python2.6/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/home/databakery/deployments/deployment-development/virtualenv/lib/python2.6/site-packages/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/home/databakery/deployments/deployment-development/virtualenv/lib/python2.6/site-packages/slimit/parser.py", line 84, in p_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (SEMI, ';') at 1:13 between LexToken(LINE_TERMINATOR,'\n',1,11) and None

Mangling and top level symbols

Some of the javascript on my page is loaded from elsewhere and the rest I'm running though slimit with mangle=True. Unfortunately, this doesn't work for me because slimit is mangling global symbols which I need to remain unmangled (i.e. module identifiers).

I came up with the following patch which seemed to do the trick for me:

diff --git a/src/slimit/visitors/scopevisitor.py b/src/slimit/visitors/scopevisitor.py
index 56585dc..7e0a2fd 100644
--- a/src/slimit/visitors/scopevisitor.py
+++ b/src/slimit/visitors/scopevisitor.py
@@ -145,12 +145,13 @@ def mangle_scope_tree(root):
             scope.mangled[name] = mangled_name
             scope.rev_mangled[mangled_name] = name

-    def visit(node):
-        mangle(node)
+    def visit(node, mangle_this_node=True):
+        if mangle_this_node:
+            mangle(node)
         for child in node.children:
             visit(child)

-    visit(root)
+    visit(root, mangle_this_node=False)


 def fill_scope_references(tree):

I have no idea if that is the appropriate way to handle this but it worked for me.

Invalid semicolon causes infinite loop

slimit version = 0.6

Try the following code:

from slimit import minify

src = """var a = 1;
, b = 2;"""
minify(src)

On my machine (os x 10.7.3, python 2.7, slimit 0.6) the function doesn't return and it hogs all the cpu. I know it's invalid JS, but no error is reported by minify, it just doesn't return.

If you remove the "," before "b = 2" then it returns with the result.

In fact, you can get it to get stuck on simpler code:

src = """var a;
, b;"""

minify(src)

The above code shows the same problem as the first code.

If you put both on the same line:

src = """var a; , b;"""

It returns with a Syntax Error: "Unexpected token (COMMA, ',') blah blah"

Illegal character '\xbb' when slimit head.js

Reproducing:

$ wget https://raw.github.com/headjs/headjs/master/dist/head.js
$ slimit head.js
Illegal character '\xbb' at 1:1 after LexToken(ID,'\xef',1,0)
Illegal character '\xbf' at 1:2 after LexToken(ID,'\xef',1,0)
�;

Allow certain comments in minified output

It's nice to be able to designate certain comments to remain in minified files (e.g. license info).

In source file, for example:

/* omit this comment from minified output */
/*! include this comment in minified output */

By the way, thank you very much for making this project available. Nice work!

Compressed files crash in IE8 because of .class

Compressed files use .class which freaks IE8 out and makes it crash. To replicate just compress jQuery 1.7.2 and try to run it in IE8.

Missing semicolon after the first part in for-loop

echo "var x,y,z=true; for ( z&&(x = 0, y = 0); x < 10 && y < 10; ++x, ++y) { console.log(x, y) }" | slimit

Get

var x,y,z=true;for(z&&(x=0,y=0)x<10&&y<10;++x,++y)console.log(x,y);
                               ^

Changing PyPI hosted mode -- speedy installation

See: http://pypi-externals.caremad.io/help/what/

Please consider switching the hosting mode to the first option (Do NOT extract URL)
(And also remove unwanted links)

Currently the pypi index lists too much irrelevant links : https://pypi.python.org/simple/slimit/
It makes pip crawl too much and eventually hit dead servers

Illegal character "'" at 1:35518 after LexToken(COMMA,',',1,35516)

slimit seems to be having trouble minifying bootstrap-datepicker.js

$ wget https://raw.github.com/eternicode/bootstrap-datepicker/511c1b0241eb9804892df6f9388e0afd00107253/js/bootstrap-datepicker.js
$ slimit bootstrap-datepicker.js

Results in:

Illegal character "'" at 1:35518 after LexToken(COMMA,',',1,35516)
Illegal character '\\' at 1:35519 after LexToken(COMMA,',',1,35516)
Illegal character '\\' at 1:35531 after LexToken(STRING,"').split('",1,35521)
Traceback (most recent call last):
  File "/home/fletch/my-venv/bin/slimit", line 9, in <module>
    load_entry_point('slimit==0.8.1', 'console_scripts', 'slimit')()
  File "/home/fletch/my-venv/local/lib/python2.7/site-packages/slimit/minifier.py", line 69, in main
    text, mangle=options.mangle, mangle_toplevel=options.mangle_toplevel)
  File "/home/fletch/my-venv/local/lib/python2.7/site-packages/slimit/minifier.py", line 38, in minify
    tree = parser.parse(text)
  File "/home/fletch/my-venv/local/lib/python2.7/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "/home/fletch/my-venv/local/lib/python2.7/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/home/fletch/my-venv/local/lib/python2.7/site-packages/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/home/fletch/my-venv/local/lib/python2.7/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)
  File "/home/fletch/my-venv/local/lib/python2.7/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (STRING, "').split('") at 1:35521 between LexToken(NUMBER,'0',1,35520) and LexToken(NUMBER,'0',1,35532)

Add IF statement optimization N2

Add the following optimization:

if (!foo) bar(); else baz(); ==> foo?baz():bar();

Unexpected token (DIV, '/') at 1:23 between LexToken(RPAREN,')',1,21) and LexToken(BXOR,'^',1,24)

Try this valid javascript:
for (index in [1,2,3]) /^salign$/i.test('salign')

Try to parse it:

>>> from slimit.parser import Parser
>>> parser = Parser()
>>> tree = parser.parse('for (index in [1,2,3]) /^salign$/i.test(\'salign\')')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Python/2.7/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "/Library/Python/2.7/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/Library/Python/2.7/site-packages/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Library/Python/2.7/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)
  File "/Library/Python/2.7/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (DIV, '/') at 1:23 between LexToken(RPAREN,')',1,21) and LexToken(BXOR,'^',1,24)
>>> ```

slimit.org down

because I don't know what other way to contact you; http://slimit.org/ is down.

allow for multiple files on the command line

Hey, I'm just getting started with slimit, but I was wondering if it would be possible to pass more than one file at a time to the script? Or is the intended use to concat all JS files together first, and then apply slimit?
Thanks

Add IF statement optimization N3

Add the following optimization:

if (foo) bar(); ==> foo&&bar();

Slimit crashes on Flot

While using slimit on flot (https://github.com/flot/flot), it crashes:

$ cat jquery.flot.js | slimit

Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/bin/slimit", line 9, in 
    load_entry_point('slimit==0.6dev', 'console_scripts', 'slimit')()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/slimit-0.6dev-py2.6.egg/slimit/minifier.py", line 62, in main
    minified = minify(text, mangle=options.mangle)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/slimit-0.6dev-py2.6.egg/slimit/minifier.py", line 38, in minify
    tree = parser.parse(text)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/slimit-0.6dev-py2.6.egg/slimit/parser.py", line 64, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/slimit-0.6dev-py2.6.egg/slimit/parser.py", line 87, in p_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (NULL, 'null') at 1:25224 between LexToken(VAR,'var',1,25220) and LexToken(ID,'ify',1,25228)

Quite possibly on line 558:

var nullify = p == null;

Optimize consecutive declarations

var a = 10; var b = 20;  ==>  var a=10,b=20;

Add IF statement optimization N5

Add the following optimization:

if (foo) return bar(); else return baz(); ==> return foo?bar():baz();

Add Python 3.x support

Invalid removal of parentheses of immediately called function

Consider this JS snippet:

(function($) {
    $.hello = 'world';
}(jQuery));

The result of running Slimit on it is:

function($){$.hello='world';}(jQuery);

which is invalid JS. The outer parentheses are required here, so Slimit shouldn't remove them.

The real-world issue that I stumbled upon was with jQuery Timeago plugin: https://github.com/rmm5t/jquery-timeago
It's using the above-mentioned construct and slimit is unable to pack it :-(

Unable to compress pdf.js code

With slimit 0.7.1:

Get http://mozilla.github.com/pdf.js/build/pdf.js
Execute slimit on the file:

% slimit pdf.js > pdf2.js
Traceback (most recent call last):
  File "/usr/bin/slimit", line 9, in <module>
    load_entry_point('slimit==0.7.1', 'console_scripts', 'slimit')()
  File "/usr/lib/python2.7/site-packages/slimit/minifier.py", line 69, in main
    text, mangle=options.mangle, mangle_toplevel=options.mangle_toplevel)
  File "/usr/lib/python2.7/site-packages/slimit/minifier.py", line 38, in minify
    tree = parser.parse(text)
  File "/usr/lib/python2.7/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "/usr/lib/python2.7/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/usr/lib/python2.7/site-packages/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/usr/lib/python2.7/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)
  File "/usr/lib/python2.7/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (ID, 'content') at 1:3095 between LexToken(ID,'get',1,3091) and LexToken(LPAREN,'(',1,3102)

division with "this" fails

I'm trying to minify code generated by pyjs/pyjamas. Slimit fails on this snippet:

Number.prototype.rfloordiv = function (y) {
if (!y.number || isNaN(y = y.valueOf())) return @{{NotImplemented}};
if (this == 0) throw @{{ZeroDivisionError}}('float divmod');
return Math.floor(y / this);
};

Traceback (most recent call last):
...
File "...\slimit\minifier.py", line 38, in minify
tree = parser.parse(text)
File "...\slimit\parser.py", line 64, in parse
return self.parser.parse(text, lexer=self.lexer, debug=debug)
File "C:\Python26\lib\site-packages\ply\yacc.py", line 265, in parse
return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
File "C:\Python26\lib\site-packages\ply\yacc.py", line 1047, in parseopt_notrack
tok = self.errorfunc(errtoken)
File "...\slimit\parser.py", line 84, in p_error
self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (REGEX, "/ y);\n};\n\nNumber.prototype.rfloordiv = function (y) {\n if (!y.number || isNaN(y = y.valueOf())) return $p['NotImplemented'];\n if (this == 0) throw $m['ZeroDivisionError']('float divmod');\n return Math.floor(y /") at 1:601787 between LexToken(THIS,'this',1,601782) and LexToken(THIS,'this',1,602015)

I tried browsing the code, but it would take me much too long to understand the parser (I've only worked with OMeta/PyMeta2). Could you please help?

Problem with for loop with ? operator in initializer position

The following javascript is not minimized correctly:

>>> minify("for(a?b:c;d;)e=1;")
'for(a?b:cd;)e=1;'
>>>

It doesn't properly emit a needed semicolon.

Conflict with underscore

I recently used this in a case where the minifier called a local function "_". Since I'm using underscore later, it breaks because it's going to the local function, where the original code ran fine. Is there any way to set a no-set-list for cases like this? Thanks :)

Add IF statement optimization N4

Add the following optimization:

if (!foo) bar(); ==> foo||bar();

Slimit does not support the use of keywords as method names

If you are to use indexeddb, one of the instance method is continue.

This would cause a syntax error with slimit: SyntaxError: Unexpected token (CONTINUE, 'continue') at 1:6306 between LexToken(PERIOD,'.',1,6305) and LexToken(LPAREN,'(',1,6314)

Here are some sample codes:

>>> import slimit
>>> js = """
... var result = getCursor();
... result.continue();
... """
>>> slimit.minify(js) # This errors.

Disappearing quotes

foo['bar"']=42 is minified into foo.bar=42, losing the double quote.

Invalid output when for loop's initialization expression is constant

Slimit produces invalid output with this example:

function test(callback) {
    var index = 3, length = 5;
    for(index; index < length; index++) {
    }
  }

The output in this case is:

function test(callback){var index=3,length=5;for(indexindex<length;index++){}}

As you can see, the semicolon after init expr is missing.
In real world, infinity.js uses such syntax, see https://github.com/airbnb/infinity/blob/master/infinity.js

Parser fails to parse a code block with a new line at the end of a file

Parser fails to parse a code block that doesn't have a semicolon at the end of the block and the next token is a new line at the end of a file:

$ cat test.js 
function add(x, y) {
 return x + y;
}\n

rspivak / slimit Goto Github PK

slimit's People

Contributors

Stargazers

Watchers

Forkers

slimit's Issues

Recommend Projects

Recommend Topics

Recommend Org