Git Product home page Git Product logo

pyjq's Introduction

pyjq: Binding for jq JSON Processor

Build Status

pyjq is a Python bindings for jq (http://stedolan.github.io/jq/).

jq is like sed for JSON data โ€“ you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text.

http://stedolan.github.io/jq/

You can seamlessly call jq script (like regular expression) and process plain python data structure.

For your information, https://pypi.python.org/pypi/jq is a also jq bindings but different and incompatible with pyjq.

Example

>>> data = dict(
...     parameters= [
...         dict(name="PKG_TAG_NAME", value="trunk"),
...         dict(name="GIT_COMMIT", value="master"),
...         dict(name="TRIGGERED_JOB", value="trunk-buildall")
...     ],
...     id="2013-12-27_00-09-37",
...     changeSet=dict(items=[], kind="git"),
... )
>>> import pyjq
>>> pyjq.first('.parameters[] | {"param_name": .name, "param_type":.type}', data)
{'param_type': None, 'param_name': 'PKG_TAG_NAME'}

Install

It requires build tools such as make, automake, libtool, etc...

You can install from PyPI by usual way.

pip install pyjq

API

For jq script, see its manual.

Only four APIs are provided:

  • all
  • first
  • one
  • compile

all transforms a value by JSON script and returns all results as a list.

>>> value = {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
>>> pyjq.all('{user, title: .titles[]}', value)
[{'user': 'stedolan', 'title': 'JQ Primer'}, {'user': 'stedolan', 'title': 'More JQ'}]

all takes an optional argument vars. vars is a dictonary of predefined variables for script. The values in vars are avaiable in the script as a $key. That is, vars works like --arg option and --argjson option of jq command.

>>> pyjq.all('{user, title: .titles[]} | select(.title == $title)', value, vars={"title": "More JQ"})
[{'user': 'stedolan', 'title': 'More JQ'}]

all takes an optional argument url. If url is given, the subject of transformation is got from the url.

>> pyjq.all(".[] | .login", url="https://api.github.com/repos/stedolan/jq/contributors") # get all contributors of jq
['nicowilliams', 'stedolan', 'dtolnay', ...

Additionally, all takes an optional argument opener. The default opener will simply download contents by urllib.request.urlopen and decode by json.decode. However, you can customize this behavior using custom opener.

first is almost some to all but it first returns the first result of transformation.

>>> value = {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
>>> pyjq.all('{user, title: .titles[]}', value)
[{'user': 'stedolan', 'title': 'JQ Primer'}, {'user': 'stedolan', 'title': 'More JQ'}]

first returns default when there are no results.

>>> value = {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
>>> pyjq.first('.titles[] | select(test("e"))', value) # The first title which is contains "e"
'JQ Primer'

first returns the first result of transformation. It returns default when there are no results.

>>> value = {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
>>> pyjq.first('.titles[] | select(test("T"))', value, "Third JS") # The first title which is contains "T"
'Third JS'

one do also returns the first result of transformation but raise Exception if there are no results.

>>> value = {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
>>> pyjq.one('.titles[] | select(test("T"))', value)
IndexError: Result of jq is empty

Limitation

jq is a JSON Processor. Therefore pyjq is able to process only "JSON compatible" data (object made only from str, int, float, list, dict).

Q&A

How can I process json string got from API by pyjq?

You should apply json.loads in the standard library before pass to pyjq.

Author

OMOTO Kenji

License

Released under the MIT license. See LICENSE for details.

Development

We DO commit _pyjq.c

When you edit _pyjq.pyx, you need to run cython _pyjq.pyx before to run python setup.py develop. It is because setup.py in this project does not compile .pyx to .c.

Of course, we can use Cython.Build.cythonize in setup.py to automatically compile .pyx to .c . But, it cause bootstrap problem in pip install.

So, we DO commit both of _pyjq.pyx and _pyjq.c.

Changes

2.1.0

  • API's translate JS object not to dict but to collections.OrderedDict.

2.0.0

  • Semantic versioning.
  • Bundle source codes of jq and oniguruma.
  • Supported Python 3.5.
  • Dropped support for Python 3.2.
  • Aeded all method.

1.0

  • First release.

pyjq's People

Contributors

doloopwhile avatar jellesmet-tomtom avatar mfeit-internet2 avatar megahall avatar

Watchers

James Cloos avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.