koalaman / shellcheck Goto Github PK
View Code? Open in Web Editor NEWShellCheck, a static analysis tool for shell scripts
Home Page: https://www.shellcheck.net
License: GNU General Public License v3.0
ShellCheck, a static analysis tool for shell scripts
Home Page: https://www.shellcheck.net
License: GNU General Public License v3.0
I prefer this construct a lot over having an obscure redirect after "done" to feed the while loop:
cat "$pkglist" | while read pkg; do
^-- Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
If we add shellcheck to Hackage, users can simply cabal install shellcheck
.
I like to run shellcheck *.sh
to scan a whole directory at once. But the helpful messages are often hidden inside long lists of No comments for <file>
. Could we turn this message off by default, so that shellcheck
only shows warnings, like how CheckStyle works?
1 test=grep "\"" test
^––
SC1009 The mentioned parser error was in this simple command.
^––
SC1073 Couldn't parse this double quoted string.
^––
SC1072 Unexpected eof. Fix any mentioned problems and try again
This is questionable, since it ignores the very real cornercase of "$0" being a plain word ("dirname foo" is ".", not "foo").
scriptdir="$(cd "$(dirname "$0")" && pwd)"
^-- Use parameter expansion instead, such as ${var%/*}.
It would be very helpful to turn of warnings in specific cases. Pylint warnings can be silenced with a comment:
Something similar would be very useful. Another option would be a config file where you could turn off specific warnings globally.
...
and ?
are valid function names in bash, however shellcheck chokes on them and stops parsing
On my machine, cabal install
puts the Shellcheck binaries in a weird place:
$ cabal install
...
Installing executable(s) in
/Users/apennebaker/Library/Haskell/ghc-7.6.3/lib/ShellCheck-0.2.0/bin
Not sure if this is an error in Shellcheck's Cabal configuration, or an error in Haskell Platform's Cabal configuration.
In any case, I was able to get around this by manually adding export PATH="$PATH:~/Library/Haskell/ghc-7.6.3/lib/ShellCheck-0.2.0/bin"
to my ~/.profile
.
System:
$ specs haskell os
Specs:
specs 0.7
https://github.com/mcandre/specs#readme
cabal --version
cabal-install version 1.16.0.2
using version 1.16.0 of the Cabal library
ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.6.3
ghc-pkg field haskell-platform version
version: 2013.2.0.0
system_profiler SPSoftwareDataType | grep 'System Version'
System Version: OS X 10.9 (13A603)
There are some legitimate use cases for variables in printf strings, like
printf "%${pad_len}s\n" "pad me"
We should reduce false positives, e.g. by looking for %s or valid format codes in the printf string, indicating that the user is familiar with them
This example script converts first letter to capital of first argument passed to it.
Shellcheck reports array "small" is not being used despite it is getting processed inside printf expression.
#!/bin/bash
small=( a b c d e f g h i j k l m n o p q r s t u v w x y z )
capital=( A B C D E F G H I J K L M N O P Q R S T U V W X Y Z )
pos() {
declare -a my_array=("${!2}")
for (( i = 0; i < ${#my_array[@]}; i++ )); do
if [ "${my_array[$i]}" = "${1}" ]; then
echo $i;
fi
done
}
printf "%s%s" "${capital[$(pos "${1::1}" "small[@]")]}" "${1#?}"
exit 0
Consider:
foo=$(something)
rm -r "/usr/$foo"
The assignment could fail due to program errors or ulimit, and rm would then delete /usr/
Is there a way to warn about these things that's useful and not full of false positives?
To wit:
case $var in
a )
foo
b )
bar
esac
On OS X, the shellcheck
compiled output collides with the ShellCheck
directory due to case insensitivity. I don't know the convention in Haskell, but perhaps you could output the binary in a build
directory (ghc --make shellcheck -o build/shellcheck
). Or maybe use cabal.
$ make
: Conditionally compiling shellcheck
ghc -O9 --make shellcheck
[1 of 5] Compiling ShellCheck.AST ( ShellCheck/AST.hs, ShellCheck/AST.o )
[2 of 5] Compiling ShellCheck.Parser ( ShellCheck/Parser.hs, ShellCheck/Parser.o )
[3 of 5] Compiling ShellCheck.Analytics ( ShellCheck/Analytics.hs, ShellCheck/Analytics.o )
[4 of 5] Compiling ShellCheck.Simple ( ShellCheck/Simple.hs, ShellCheck/Simple.o )
[5 of 5] Compiling Main ( shellcheck.hs, shellcheck.o )
Linking shellcheck ...
ld: can't open output file for writing: shellcheck, errno=21 for architecture x86_64
collect2: error: ld returned 1 exit status
make: *** [shellcheck] Error 1
Shellcheck says to replace expr ...
, as expr
is deprecated. For mathematical expressions, I can use $((...))
instead. But what should I use for string expressions?
FizzBuzz example:
Add a option to create shell and html output compareable to http://www.shellcheck.net/
and create checkstyle output.
Example:
$ shellcheck --recursive --outputdir=target/shellcheck --checkstyle-file target/checkstyle.xml target/ shellscriptfolder
This example would:
This might be very useful for automatic build environments.
Jenkins/Hudson provide plugins for publishing html outout or for interpreting checkstyle reports (provide a awesome presentation and statistics from the checkstyle files to the build metrics of the project)
There should be a way to disable false positives in comments, e.g.
# shellcheck disable-msg SC2086
echo $1
These should be scoped for structure for which they appear, or the entire file if at the top.
I plan to run Shellcheck over some zsh, ksh scripts, etc. etc. Would Shellcheck print warnings specific to these shell languages? Would Shellcheck even recognize the syntax?
Shellcheck is awesome, by the way!
Interesting project.
I am wondering if it would be possible to use the AST to generate correct shell scripts from Haskell. The use case I have in mind is what Ansible is doing: managing remote hosts with SSH and shell scripts.
I'm not entirely sure if this is a bug, so please forgive me if it's a problem with my script instead
shellcheck suggests that i replace "dirname
however, when i run the script with "bash scriptname" or "sh scriptname", the parameter expansion does not result in the same string:
dirname: "."
parameter expansion: "scriptname"
hope that helps
#!/bin/bash
var=ab
echo "${"$(echo "${var//a/a1}")"//b/b1}"
exit 0
this erronous script reports bad substitution when run in bash shell but reports OK with shellcheck.
The shorthand for for arg in "$@"; do
, for arg do
(note: without the requirement of a semicolon), is reported as a violation of SC107{3,2}, but it is explicitly permitted by POSIX, and in fact is more portable than for arg; do
(with semicolon).
A script that triggers this:
for arg do
echo "$arg"
done
This fails, when it should not:
time { true; }
if (($#!=2)) || [[ ! -f "$2" ]] || [[ ! "$1" =~ ogg|flac ]]; then
^-- The mentioned parser error was in this if expression.
^-- Couldn't parse this test expression.
^-- Unexpected keyword/token. Fix any mentioned problems and try again.
Thanks to Norbert Varzariu for reporting.
12 echo $FIELDMATCH | sed -r "s/#FIELD#/$1/"
^–– Unquoted variable may contain spaces/globs, and will word split.
VERSION=$(expr "$VERSION" : '.*"\(1.[0-9\.]*\)["_]')
^-- expr is antiquated. Consider rewriting this using $((..)), ${} or [[ ]].
In this case the regular expression cannot be handled by the suggested mechanisms.
expr "$VERSION" \< 1.2 >/dev/null && continue
^-- expr is antiquated. Consider rewriting this using $((..)), ${} or [[ ]].
expr can compare floats
Hi, would it be possible that you tag a stable release of the software?
That would help getting the package accepted in Homebrew: Homebrew/legacy-homebrew#21231
Also, being a person who's experienced with Haskell, could you please see the discussion at the aforementioned ticket and add your thoughts about the Cabal issues we're having? Namely preventing it from writing into the user's HOME directory (~/.cabal
)
Hello
It looks like this expression is not parsed correctly.
[ -"${option_type:0:1}" "$value" ]
^-- Couldn't parse this test expression.
^-- Unexpected """. Fix any mentioned problems and try again.
For the record, the line context is available at https://github.com/Anvil/bash-argsparse/blob/master/argsparse.sh#L614
Thank you.
Hi,
ShellCheck is awesome, but it fails to parse this code:
for i do
echo $i
done
It works only when the do
statement is clearly separated:
for i; do
echo $i
done
# or
for i
do
echo $i
done
All shells (except [t]csh) I have tried support all three syntaxes. POSIX [1] doesn't clearly state how the do
reserved word is supposed to delimit the loop's body.
Best Regards,
Dridi
[1] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_09_04_03
in
cat << EOF > /dev/null
foo
EOF
the here document is parsed as starting with > /dev/null rather than foo
If we add shellcheck to Homebrew, users can simply brew install shellcheck
.
This parse error doesn't seem right:
In test.sh line 3:
for (( i=0 ; i<10 ; i++ )) ; do
^-- Couldn't parse this for loop.
^-- Expected 'do'.
^-- Unexpected ";". Fix any mentioned problems and try again.
Apparently, the parser is not expecting the semicolon before 'do', but man bash says it is a correct syntax:
for (( expr1 ; expr2 ; expr3 )) ; do list ; done
First, the arithmetic expression expr1 is evaluated according to the rules described below under ARITHMETIC EVALUATION. The arithmetic expression expr2 is then evaluated repeatedly until it evaluates to
zero. Each time expr2 evaluates to a non-zero value, list is executed and the arithmetic expression expr3 is evaluated. If any expression is omitted, it behaves as if it evaluates to 1. The return value
is the exit status of the last command in list that is executed, or false if any of the expressions is invalid.
Compare:
foo() { typeset lul; lul=42; }; foo; echo $lul
function foo { typeset lul; lul=42; }; foo; echo $lul
This warning is incorrect:
ls -1N | cat
^-- Don't parse ls output; it mangles filenames.
When used with -N and a pipe, ls does not mangle file names, but passes them on raw. What you have after the pipe might mangle them, of course, but that's a different issue.
This warning is invalid:
find trunk -type f -name '*.gcda' -print0 |\
^-- Don't use find | xargs cmd. find -exec cmd {} + handles whitespace.
find ... -print0 | xargs -0 ... handles whitespace.
And piping to xargs can be orders of magnitude(!) faster than using -exec {}, which is single-threaded, blocking, and not file system cache friendly:
$ rm -f /tmp/foo; time find /usr/lib64 -type f 2>/dev/null -print0 | xargs -0 md5sum >>/tmp/foo 2>/dev/null
real 0m3.304s
user 0m2.971s
sys 0m0.373s
$ rm -f /tmp/foo; time find /usr/lib64 -type f 2>/dev/null -exec md5sum >>/tmp/foo {} ;
real 0m14.842s
user 0m4.928s
sys 0m5.174s
If anything, the warning should be the other way - never use exec {} if there's an opportunity to use -print0 | xargs -0
Hi,
I have packaged ShellCheck for Fedora, it should be available for Fedora 19+ in less than a week :)
Now this is my duty to get a man page in the package, but even if it weren't for Fedora I would've asked. Of course I can help with that, if you're not familiar with it. I'd recommend a markup language such as rst (with rst2man, my favorite one) or markdown (with ronn), but man pages are written in (not so) plain text.
There may also be an existing tool in the Haskell ecosystem that would fit better, I don't know.
The following test case was created from /etc/cron.daily/prelink
on RHEL6:
#!/bin/sh [ "`find /var/lib/prelink/quick -mtime -${PRELINK_NONRPM_CHECK_INTERVAL:-7} 2>/dev/null`" \ -a -f /var/lib/rpm/Packages \ -a /var/lib/rpm/Packages -ot /var/lib/prelink/quick ] && exit 0
# shellcheck testcase.sh In testcase.sh line 2: [ "`find /var/lib/prelink/quick -mtime -${PRELINK_NONRPM_CHECK_INTERVAL:-7} 2>/dev/null`" \ ^-- Couldn't parse this test expression. In testcase.sh line 3: -a -f /var/lib/rpm/Packages \ ^-- Unexpected "/". Fix any mentioned problems and try again.
Here's a slightly modified version with an explicit "-n" which doesn't show the parsing problem:
#!/bin/sh [ -n "`find /var/lib/prelink/quick -mtime -${PRELINK_NONRPM_CHECK_INTERVAL:-7} 2>/dev/null`" \ -a -f /var/lib/rpm/Packages \ -a /var/lib/rpm/Packages -ot /var/lib/prelink/quick ] && exit 0
# shellcheck testcase2.sh In testcase2.sh line 2: [ -n "`find /var/lib/prelink/quick -mtime -${PRELINK_NONRPM_CHECK_INTERVAL:-7} 2>/dev/null`" \ ^-- Use $(..) instead of deprecated `..`
I.e. there is a parsing problem when the explicit "-n" is missing. According to the man page test(1) both variants should be equivalent:
-n STRING the length of STRING is nonzero STRING equivalent to -n STRING
yum install cabal-install ghc ghc-parsec-devel ghc-QuickCheck-devel ghc-json-devel ghc-regex-compat-devel
Thanks to Norbert Varzariu
Hi,
Running Ubuntu 12.04.2 LTS with the packages given in the README:
ghc6 libghc6-parsec3-dev libghc6-quickcheck2-dev libghc6-json-dev libghc-regex-compat-dev
and also cabal
. Current shellcheck HEAD cloned:
~/github/shellcheck$ git log -1 --oneline
de1fa61 Warn about client side expansion in ssh strings/heredocs.
configure
seems okay but build
fails:
~/github/shellcheck$ cabal configure
Resolving dependencies...
Configuring ShellCheck-0.1.0...
~/github/shellcheck$ cabal build
Building ShellCheck-0.1.0...
Preprocessing executable 'shellcheck' for ShellCheck-0.1.0...
ShellCheck/Simple.hs:23:8:
Could not find module `Text.Parsec.Pos'
It is a member of the hidden package `parsec-3.1.2'.
Perhaps you need to add `parsec' to the build-depends in your .cabal file.
Use -v to see a list of the files searched for.
~/github/shellcheck$
With -v
flag as suggested:
~/github/shellcheck$ cabal build -v
creating dist/build
creating dist/build/autogen
Building ShellCheck-0.1.0...
Preprocessing executable 'shellcheck' for ShellCheck-0.1.0...
Building executable shellcheck...
creating dist/build/shellcheck
creating dist/build/shellcheck/shellcheck-tmp
/usr/bin/ghc --make -o dist/build/shellcheck/shellcheck -hide-all-packages -fbuilding-cabal-package -package-conf dist/package.conf.inplace -i -idist/build/shellcheck/shellcheck-tmp -i. -idist/build/autogen -Idist/build/autogen -Idist/build/shellcheck/shellcheck-tmp -optP-include -optPdist/build/autogen/cabal_macros.h -odir dist/build/shellcheck/shellcheck-tmp -hidir dist/build/shellcheck/shellcheck-tmp -stubdir dist/build/shellcheck/shellcheck-tmp -package-id base-4.5.0.0-40b99d05fae6a4eea95ea69e6e0c9702 -package-id containers-0.4.2.1-cfc6420ecc2194c9ed977b06bdfd9e69 -package-id directory-1.1.0.2-ebacad9b5233212b1abbebce9b7e6524 -package-id json-0.5-b3efb968dbdfc514365c5250445af3ff -package-id mtl-2.0.1.0-db19dd8a7700e3d3adda8aa8fe5bf53d -package-id parsec-2.1.0.1-defe69eb7a92d23008966c94e32574a7 -package-id regex-compat-0.95.1-851005df9f3cd69b337623025f7c092b -O -XHaskell98 ./shellcheck.hs
ShellCheck/Simple.hs:23:8:
Could not find module `Text.Parsec.Pos'
It is a member of the hidden package `parsec-3.1.2'.
Perhaps you need to add `parsec' to the build-depends in your .cabal file.
Use -v to see a list of the files searched for.
~/github/shellcheck$
I have three parsec-dev packages installed:
un libghc-parsec-dev <none> (no description available)
un libghc-parsec-dev-2.1.0.1-defe6 <none> (no description available)
un libghc-parsec-dev-3.1.2-a6715 <none> (no description available)
ii libghc-parsec2-dev 2.1.0.1-6 Haskell monadic parser combinator library for GHC
ii libghc-parsec3-dev 3.1.2-1 Haskell monadic parser combinator library for GHC
ii libghc6-parsec3-dev 1:6 transitional dummy package
cabal info parsec
says:
Versions available: 2.0, 2.1.0.0, 2.1.0.1, (3.0.0), (3.0.1), (3.1.0)
Versions installed: 2.1.0.1, (3.1.2)
(and Text.ParserCombinators.Parsec.Pos
is included in the modules listing).
An explicit dependency parsec == 2.1.0.1
does not change the build error
(as I expected). However setting parsec == 3.1.2
reveals:
~/github/shellcheck$ cabal configure
Resolving dependencies...
Configuring ShellCheck-0.1.0...
Warning: This package indirectly depends on multiple versions of the same
package. This is highly likely to cause a compile failure.
package json-0.5 requires parsec-2.1.0.1
package ShellCheck-0.1.0 requires parsec-3.1.2
~/github/shellcheck$
No more recent version of json
appears to be available in Ubuntu (although there is a slightly older version of 0.5).
Because I'm a WILD AND CRAZY guy I tried cabal build
in any case and
that seems to produce a working executable. But obviously something a
little fishy here...
Like with variable modification, break/continue are ineffectual in subshells:
while true; do { foo || break; } | bar; done
cat <<-!SOME_MARKER | xargs echo
test
test2
!SOME_MARKER
1 cat <<-!SOME_MARKER | xargs echo
^––
SC1009 The mentioned parser error was in this simple command.
^––
SC1073 Couldn't parse this here document.
^––
SC1072 Unexpected keyword/token. Fix any mentioned problems and try again.
2 test
3 test2
4 !SOME_MARKER
5
Perl tends to incorrectly trigger the variable-in-single-quotes warning, as in
perl -F: -lane 'print $F[0]' /etc/passwd
It should be added to the accepted list along with awk and trap.
I cloned the tree and did a cabal install and it failed. Here's the relevant error:
[6 of 6] Compiling Main ( shellcheck.hs, dist/build/shellcheck/shellcheck-tmp/Main.o )
shellcheck.hs:159:39:
Not in scope: catch' Perhaps you meant
catch#' (imported from GHC.Exts)
shellcheck.hs:220:5:
Not in scope: catch' Perhaps you meant
catch#' (imported from GHC.Exts)
Failed to install ShellCheck-0.2.0
cabal: Error: some packages failed to install:
ShellCheck-0.2.0 failed during the building phase. The exception was:
ExitFailure 1
Have I screwed up somewhere?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.