This module improves Python's usefulness as a tool for writing shell one-liners. A brief usage example:
# show line numbers
python -m oneliner -ne '"%5d: %s" % (NR, _)' < input.txt
# convert unix timestamp to something more human readable
date +%s | pyl -j ' => ' -line '_, datetime.datetime.fromtimestamp(int(_))'
# 1355256353 => 2012-12-11 22:05:53
# triple space a file
pyl -ne 'line + "\n"*2' < input.txt
# double space only non-blank lines:
pyl -ne '_ if re.match("^$", _) else _+"\n"' input.txt
Python is a wonderful general purpose scripting language with a great breadth of excellent libraries. While the simplicity of its syntax is often cited as a major strong point, I believe that the negative attitude towards any kind of syntax magic has prevented Python from becoming a useful tool for writing shell one-liners. When stacked against the likes of Ruby and Perl, Python is ill-equipped to be a part of a shell pipeline. Writing one-liners is possible, but their verbosity makes them impractical. This may be attributed to the following factors:
-
Lack of a read-print-loop command line option. Ruby and Perl provide the
-n
and-p
switches that makes them assume the following loop around code:for line in sys.stdin: <code> if '-p' in options: sys.stdout.write(line)
-
No syntax magic and no special variables. Ruby and Perl provide a multitude of cryptic, yet useful variables such as:
$_ last input line $. current input line number $~ the match object returned by the last successful pattern match $& the string matched by last successful pattern match $3 the string matched by the 3rd group of the last successful pattern match $* sys.argv $> sys.stdout (default output file) $< sys.stdint (default input file) $; input field separator (default value to str.split()) $/ record separator (os.linesep) $$ os.getpid()
Please refer to English.rb and English.pm for more information.
-
Lack of a flexible command line import mechanism. For example, Perl has the
-M
options:perl -MDigest::MD5=md5_hex => from Digest::MD5 import md5_hex perl -MDigest::MD5 => import Digest::MD5
While the CPython interpreter has the
-m
switch, it is not suitable for the task at hand. For example, the following one-liner will run random's test suite, instead of printing a random number:python -m random -c "print(random.randint(10))"
All these points add up to the verbosity of Python one-liners. The following example demonstrates this:
# convert a date to a unix timestamp (using strptime)
ruby -rdate -pe '$_=Date.strptime($_, "%Y-%m-%d").strftime("%s")'
perl -MTime::Piece -pe '$_=Time::Piece->strptime($_, "%Y-%m-%d\n")->epoch()'
python -c 'import sys,datetime; [sys.stdout.write(datetime.datetime.strptime("%Y-%m-%d", i).strftime("%s") for i in sys.stdin]'
But why would anyone want to write one-liners in Python, given these shortcomings and the available alternatives? I believe that when doing interactive work on the shell, the first solution that comes to mind is usually good enough. If that solution is a Python one, why not use it?
Python comes with all the building blocks for implementing a practical method of writing one-liners. This module tries to address the issues outlined above. The command line interface is kept as close as that of Ruby and Perl as reasonable.
-
To help with the processing of input and output, oneliner provides the the
-n
,-p
and-l
command line switches.-
-n
: assume the following loop around expressions or statements (the distinction will be clarified later):for line in sys.stdin: ...
-
-p
: like-n
, but write the value ofline
to stdout at the end of each iteration:for line in sys.stdin: ... sys.stdout.write(line)
-
-l
: automatic line-ending processing. Roughly equivalent to:for line in sys.stdin: line = line.strip(os.linesep) ... sys.stdout.write(line) sys.stdout.write(os.linesep)
-
-
Makes the following variables available in the local namespace of each one-liner:
-
line
,L
,_
: The current input line. Unless the-l
switch is given, the line separatator will be a part of this string. -
words
,W
: Corresponds to the value ofre.split(delimiter, line)
where delimiter is the value of the-d
option. Defaults to\s+
.The
words
list will return an empty string instead of throwing anIndexError
when a non-existent item is referenced. This behavior is similar to that of arrays in Ruby and field variables in Awk. -
NR
: Current input line number. -
FN
: Current input file name. If oneliner is processing input from stdinFN
will be equal to<stdin>
, otherwise it corresponds to the current input file given on the command line. For example:echo example | python -m oneliner -ne '"%s:%s\t %s" % (FN, NR, L)' => <stdin>:1 example python -m oneliner -ne '"%s:%s\t %s" % (FN, NR, L)' example.txt => example1.txt:1 line 1
-
-
Provide the
-m
and-M
options and a mini-language for specifying imports. This is best illustrated by an example:-m os,sys,re,pickle => import os, sys, re, pickle -m os -m sys -m re => import os, sys, re -m os sys re pickle => import os, sys, re, pickle -m os.path.[*] => from os.path import * -m os.path.[join,exists] => from os.path import join, exists -m subprocess=sub => import subprocess as sub -m datetime.[datetime=dt] => from datetime import datetime as dt -M os.path => from os.path import *
The latest stable version of python-oneliner is available on pypi and may be installed with pip.
$ python -m pip install oneliner
Alternatively, you may simply put the oneline.py file anywhere in [sys.path]{.title-ref}.
- Support one-liners that don't deal with input/output only. If
-n
or-p
are not given, python-oneliner should behave mostly likepython -c
does. - Persistent variables in statement one-liners.
- The result of an expression one-liner is always written to stdout
(even if
-n
). - Define the behaviour of multiple expression/statements specified on the command line.
- Some means of emulating
BEGIN
andEND
(perhaps a-b
and-d
flag?) - Add more examples.
Python-oneliner is released under the terms of the Revised BSD License.