Git Product home page Git Product logo

jmespath.spec's Introduction

This project is a community initiative that aims to discuss frequently requested features for JMESPath. It tracks updates to the original specification and brings improvements that may not be necessarily included in the original spec. This project supports an ecosystem of high quality, fast paced implementations in popular programming languages.

JMESPath Specification

Any changes to the JMESPath specification must have a JEP (JMESPath Enhancement Proposal) which is tracked in this repo. While there are implementations of JMESPath in over 9 different languages, we currently track implementations of 4 implementations of the JMESPath Community specification. We want to make sure that any modifications to the spec make sense for all JMESPath and JMESPath Community libraries.

A JEP helps to work through the design process for new additions and ensures that the JMESPath community has a chance to give feedback before it's officially part of the specification.

Things that need a JEP

Any functionality change that would require an update to the specification requires a JEP.

This includes, but is not limited to:

  • New syntax
  • New functions
  • New semantics/functionality

Things that do not need a JEP

Anything that is specific to a JMESPath library does not need a JEP. You should defer to the specific library's contributing guide. This can include additional language specific APIs, extension points (e.g. adding custom functions), configuration options, etc.

Guidelines for proposing new features

First, make sure that the feature has not been previously proposed. If it has, make sure to reference prior proposals and explain why this new proposal should be considered despite similar proposals not being accepted.

Writing a JEP can be a lot of work, so it can help to get initial guidance before getting too far. You can chat on the JMESPath gitter channel (https://gitter.im/jmespath/chat) to get an initial pulse of a new feature.

Then open a discussion to discuss the feature, its merit, and any possible alternatives. Once the discussion has reached a mature level of feedback, it will be assigned a JEP number N. At that point it is safe to begin composing JEP-N along with all the required changes and documentation.

In rare circumstances, JEPs will be superceded by newer versions of a JEP. Those updated JEPs will have an amended letter sequence added as a suffix to the original JEP#. The sequence follows the 26 US-ASCII alphabetical order 'a', 'b', to 'z'. It then follows with 'aa', 'ab', to 'az' and so on.

For instance, JEP-12 introduced raw-string literals but the grammar defined there did not match that which was eventually specified. An update has been published as JEP-12a.

Tenets of JMESPath

When proposing new features, keep these tenets in mind. Adhering to these tenets gives your proposal a higher likelihood of being accepted:

  • JMESPath is not specific to a particular programming language. Avoid constructs that are difficult to implement in another language.
  • JMESPath strives to have one way to do something.
  • Features are driven from real world use cases.
  • Syntax is generally used to express data structure manipulation while functions are used to express datum manipulation

jmespath.spec's People

Contributors

springcomp avatar innovate-invent avatar gibson042 avatar jamesls avatar sebastien-rosset avatar

Stargazers

 avatar  avatar AriESQ avatar Pilou avatar NaveenKumar Namachivayam ⚡ avatar Chip Zoller avatar

Watchers

 avatar  avatar  avatar  avatar

jmespath.spec's Issues

Consider reviewing type-check consistency of function arguments

Some functions are defined as taking arguments of a specific type, but are still allowed to return null when given invalid types.

For instance the ceil() function returns null on invalid input in the original implementations.
This seems to contradict the blanket statement from the specification that:

“Function arguments have types. If an argument for a function has the wrong type, an invalid-type error will occur.”

JMESPath Community implementations enforce strict type checking as much as possible, unless a compliance test exists.
Please, consider reviewing type-checking and make it more consistent.

Maybe this should be documented in the compatibility section of the compliance test suite?

[styling] Consider enabling different semantic classes for the HTML header tags.

This is part of a series or minor issues that aim to enable support for flexible styling and theming of the JMESPath site.

Multiple semantically distinct captions all share the same HTML h1 tag.

image

Please, consider adding a CSS class to differentiate those semantic usages of the captions.

For instance, each website section – like the Tutorial section shown in the screenshot above – has both an h1 tag with the section heading (i.e 'Tutorial') and a h1 tag with chapter headings (i.e 'Basic Expressions').

I think we may need to distinguish between:

  • The website welcome header
  • Each website main section (Tutorial, Examples, Libraries, Syntax and Functions)
  • Each website section’s HTML tags like h2, h3 and so on might also need to be mapped to the same CSS class.

Linter/Autoformatter for JEPs

I am finding the existing markdown files have line terminators that break markdowns autowrapping.

PRs should trigger a autoformatter and linter to ensure correct document structure.

Clarification needed for `let()` function behaviour and sub-expressions

In light of this comment, is appears clarification is needed.

In fact, @jamesls was aware of potential confusions. and @mtdowling talked him out of moving the JEP forward.

[@jamesls] In this scenario we're evaluating foo.qux. foo evaluates to {"bar": "baz"}. So far so good. Next we evaluate the RHS of the sub expression, qux. We see the qux is not defined in the current object {"bar": "baz"} so we look in the scope object and see it's defined as "qux", so we use that value. I can see how some people might find that confusing. Maybe not.

[@mtdowling] We've spoken about this in person, but I wanted to leave feedback here to hopefully help drive this proposal forward. I stated this earlier, but I think that the current behavior of this JEP would be confusing.

I still think this feature brings more benefits as is as it brings confusions but I agree clarification is needed and changes may be made if necessary.

Sub Expressions

It appears the most common source for confusion is trying to use the scope when evaluating the RHS of a sub-expression.
The current implementations of the specification behave is unconditionnally simple:

  • When evaluating an identifier, first lookup the current context.
  • If not found in the context, lookup the call chain.

The confusion arises when trying such expressions:

  • search( let({bar: 'bar'}, &foo.bar ) , {"foo": "foo"} ) -> "bar"

Whereas intuition would dictate that expression to evaluate to null instead, as bar is not a property of the "foo" string.

Pipe Expressions

It’s worth mentioning that JEP-19 standardized a small difference between evaluating a sub-expression vs a pipe-expression.

Thoughts must be applied when clarifying this JEP-11 in this context.
For the record, assuming we closed the confusion with sub-expression, I would still be comfortable with the following evaluations:

  • search( let({bar: 'bar'}, &foo.bar ) , {"foo": "foo"} ) -> null
  • search( let({bar: 'bar'}, &foo | bar ) , {"foo": "foo"} ) -> "bar"

Link to https://jmespath.site/specification.html broken (?)

The README.md file has a link to https://jmespath.site/specification.html, and currently this link is broken. I'm not sure if it's a temporary issue or not.
That links returns "We are sorry, but that page could not be found."

Any changes to the JMESPath specification (https://jmespath.site/specification.html) must have a JEP (JMESPath

It would help to include a link to the source for the spec, not just the generated html page. Or maybe it's already there in the HTML page, but since the link is broken, I cannot see it right now.

Reconcile JEP-12 and raw-string grammar rule

While JEP-12 Raw String Literals has been accepted, it has subsequently fallen behind and no longer matches the official grammar.

For instance, this test/literal.json test is not compliant with JEP-12 that does not allow backslash characters explicitely. On the other hand, it is not compliant with the official grammar either because \\ is expected to be a valid escape sequence for a single \ character.

https://github.com/jmespath-community/jmespath.test/blob/aa6fb5f942e852af1781d1f208c4069a15794a80/tests/literal.json#L194-L196

Please, consider updating JEP-12 to align with the grammar and the updated tests.

'foo'[::-1].length(@)

In most implementations, the rhs of a sub-expression is implemented differently when the lhs is a projection.
Due to the shape of the AST, the lhs is actually evaluated on the lhs projection.

However, when slicing a string, the rhs should be treated as a regular sub-expression.

As a result, in most implementations, the following expression does not evaluate correctly:

  • 'foo'[::-1].length(@) should evaluate to 3.

In some implementations, the result is ``"oof"ornull`.

Please, consider adding a compliance test.

Document pre-JEP-12 legacy syntax for literals

With JEP-12, raw-string literals where introduced and json-value was tighten to more strictly adhere to the JSON specification.

As a result, syntax like `foo` was being deprecated in favor of `"foo`".

As JMESPath Community intends to fully deprecate legacy syntax, a few changes will be made.

Please, consider documenting legacy syntax, clarifying its deprecation policy.

Please, consider adding notes about supporting legacy syntax and its relationship with regards to standards compliance.

Implementations were free to support legacy syntax after JEP-12 was introduced. This may mean that expressions like `foo` were still successfully parsed automatically. The deprecation process SHOULD discourage automatic parsing and qualify that as non-compliant behaviour. Additionally, the deprecation process SHOULD promote explicit opt-in to the legacy syntax if at all desirable for an implementation.

Fix examples in wiki

Numbers are incorrectly wrapped with [].

diff --git a/Examples.md b/Examples.md
index 423ff84..ce19262 100644
--- a/Examples.md
+++ b/Examples.md
@@ -8,7 +8,7 @@ value is greater than 20, we're creating a sub list of the name and age
 values.

 ```jmespath
-people[?age > [20]].[name, age]
+people[?age > `20`].[name, age]
 {
   "people": [
     {
@@ -39,7 +39,7 @@ greater than `20`. If instead we want to create the same hash structure
 but only include the `age` and `name` key, we can instead say:

 ```jmespath
-people[?age > [20]].{name: name, age: age}
+people[?age > `20`].{name: name, age: age}
 {
   "people": [
     {

Consider implementing simple arithmetic operations.

I think the spec should support simple arithmetic operations in the way of the +, -, * and / operators.

A few things to keep in mind:

  • What happens in case of illegal or overflowing operations. Does that return null or does that raise an error ?
  • Parentheses would need to be involved to override precedence.
  • The geek in me would love to support proper symbols for arithmetic operators in addition to their ASCII approximations:
    • (U+2212 MINUS SIGN)
    • ÷ (U+00F7 DIVISION SIGN)
    • × (U+00D7 MULTIPLY SIGN)

ABNF: `escaped-char` rule included potentially unwanted `/`(SOLIDUS) character

From the grammar:

literal           = "`" json-value "`"

; The ``json-value`` is any valid JSON value with the one exception that the
; ``%x60`` character must be escaped.  While it's encouraged that implementations
; use any existing JSON parser for this grammar rule (after handling the escaped
; literal characters), the grammar rule is shown below for completeness::

json-value =/ json-quoted-string
json-quoted-string = %x22 1*(unescaped-literal / escaped-literal) %x22

escaped-literal   = escaped-char / (escape %x60)
escaped-char      = escape (
                        %x22 /          ; "    quotation mark  U+0022
                        %x5C /          ; \    reverse solidus U+005C
                        %x2F /          ; /    solidus         U+002F
                        %x62 /          ; b    backspace       U+0008
                        %x66 /          ; f    form feed       U+000C
                        %x6E /          ; n    line feed       U+000A
                        %x72 /          ; r    carriage return U+000D
                        %x74 /          ; t    tab             U+0009
                        %x75 4HEXDIG )  ; uXXXX                U+XXXX

The / (U+002F SOLIDUS) character is included in the escaped-char rule but I cannot understand why.

To me the / character does not need escaping and thus should be removed from this list.

Consider supporting regular expressions

There seems to be some demand for supporting regular expressions.

This presents some challenges as there are many different dialects of regexp, although there is an effort around
specifying an interoperable regex format.

Also, regular expressions uses cases fall into three main categories:

  • Matching patterns, which could be useful in filter-expressions. It either matches, or it does not.
  • Replacing substrings.
  • Capturing and extracting substrings in named (or numbered) groups.

So a JEP would need to come up with intuitive syntax for those use cases that would be relevant.

The interoperable regex format proposal referred to above only supports the first scenario. So maybe that’s the only scenario that requires extending the JMESPath syntax.

The last two scenarios may be supported using dedicated functions.

JMESPath vNext

This issue tracks the progress towards the next iteration of the JMESPath specification.
These are the planned current issues and proposals included in the next version.

Structured JEPs

I recommend we establish a structured format for JEPs that allows the markdown to be baked into html documentation to be included in the site. Only some elements would be included from the JEP and additionally some elements would only exist for the purpose of inclusion in the documentation and would not directly contribute to the language spec.

This will require a actions workflow to automatically bake the JEPs and commit it to the jmespath.site repo.

All existing specification should also be loaded into a JEP-0 to include in this pipeline

[styling] Consider keeping the branding of the JMESPath logo

This is part of a series or minor issues that aim to enable support for flexible styling and theming of the JMESPath site.

I feel the JMESPath logo is part of the original branding and should not be changed,
Here is the original logo:

image

Here is the new logo:

image

In particular the "Path" portion of the logo used to be in all uppercase. Whereas the "JME" portion of the logo used to be formatted with a very thin font.

Empty quoted string should be allowed

The quoted-string rule currently requires at least one character between the quotation marks ( quoted-string = quote 1*(unescaped-char / escaped-char) quote), but it should not because JSON object members can be named like "". For example, an expression like `{"": "foo"}` | "" should be valid and result in "foo" (as is in fact already the case in the live demo at the top of https://jmespath.site/ ).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.