jg-rp / liquid Goto Github PK
View Code? Open in Web Editor NEWA Python engine for the Liquid template language.
Home Page: https://jg-rp.github.io/liquid/
License: MIT License
A Python engine for the Liquid template language.
Home Page: https://jg-rp.github.io/liquid/
License: MIT License
The reference implementation allows range literals to be assigned to variables for later use.
Template
{% assign nums = (1..5) -%}
{% for x in nums limit: 2 %}{{ x }}{% endfor -%}
{% for x in nums %}{{ x }}{% endfor -%}
Output
1212345
Python Liquid currently raises a LiquidSyntaxError: unexpected '(', on line 1
The built-in truncate
filter should have a default length of 50. Length is currently a required argument in Python Liquid.
The built-in truncatewords
filter should have a default word count of 15. The word count is currently a required argument in Python Liquid.
Provide an example loader and include
(or render
) tag subclass that uses a context variable to give the loader a prefix or scope to search. Much like the "section" tag found in Shopify's Liquid.
Ruby Liquid's implementation of the strip_html
filter has some specific rules for handling HTML comments, <script>
blocks and <style>
blocks. Python Liquid's built-in strip_html
filter does not implement these rules.
For example, {{ "<style type='text/css'>foo bar</style>" | strip_html }}
will render an empty string with Ruby Liquid, but foo bar
with Python Liquid.
The date
filter should accept 'today'
as well as 'now'
as its left value.
Currently Python Liquid raises an exception if 'today'
is used.
Error: filter 'date': unexpected error: Unknown string format: today, on line 1
Add filters for encoding and decoding strings to and from Base64. As per the reference implementation.
New filters are:
Add HTML auto-escape functionality to Python Liquid.
In keeping with the reference implementation, auto-escape would be disabled by default. We don't want to assume that everyone is using Liquid for generating HTML or XML, or that all Liquid generated HTML contains user edited content, but common use cases like Django and Flask web apps really do need an auto-escaping option.
If markupsafe is installed and the autoescape
argument to Environment
or Template
is True
, context variables will be escaped before output, unless explicitly marked as "safe".
Provisionally, it looks like we'll need:
liquid.builtin.statement.StatementNode
liquid.expression.StringLiteral
liquid.builtin.tags.capture_tag.CaptureNode
Unknowns:
safe
filter or block tag required?When given an additional argument, the built-in uniq
filter will use the value of that argument as a key into each item in the sequence to dedupe. Python Liquid currently raises an exception if any of a sequence's items don't have that property/key. The reference implementation does not.
data:
{
"things": [
{"title": "foo", "name": "a"},
{"title": "foo", "name": "b"},
{"title": "bar", "name": "c"},
{"heading": "bar", "name": "c"},
{"heading": "baz", "name": "d"},
]
},
template:
{% assign uniq_things = things | uniq: 'title' %}
{% for obj in uniq_things %}
{% for x in obj %}
x[0]: x[1]
{% endfor %}
{% endfor %}
expected output
title: foo
name: a
title: bar
name: c
heading: bar
name: c
Liquid will suppress blocks that only contain whitespace characters. Python Liquid currently checks for whitespace only blocks after rendering the block. Whereas the reference implementation will output whitespace only string literals if they are in an output statement or echo tag.
Template
{% assign array = '1,2,3' | split: ',' -%}
{% for value in array -%}
{{ value -}}
{% if forloop.first %} {{ ' ' }} {% endif -%}
{% endfor %}
Expected output
1 23
Actual output
123
When using offset: continue
and looping over the same sequence three or more times, a for
loop's limit
can be calculated incorrectly.
Example template
{% for item in (1..6) limit: 2 %}a{{ item }} {% endfor -%}
{% for item in (1..6) limit: 2 offset: continue %}b{{ item }} {% endfor -%}
{% for item in (1..6) offset: continue %}c{{ item }} {% endfor %}
Expected output
a1 a2 b3 b4 c5 c6
Actual output
a1 a2 b3 b4 c3 c4 c5 c6
The built-in first
and last
properties should follow the semantics of the first
and last
filters. That is they should not work on strings.
This should render as an empty string or raise an exception in strict mode. Python Liquid currently renders f
.
{% assign x = "foo" %}{{ x.first }}
When using Liquid's special size
property on a collection, you get the number of items in that collection, equivalent to calling len()
on the object in Python.
data:
{
"some_list": [1,2,3],
"some_dict": {"a": 1, "b": 2},
}
template:
{{ some_list.size }}
{{ some_dict.size }}
output
3
2
If an object already has a size
property or key, the reference implementation will return its value rather than the object's length. Python Liquid always returns the object's length.
data:
{
"some_list": [1,2,3],
"some_dict": {"a": 1, "b": 2, "size": 42},
}
template:
{{ some_list.size }}
{{ some_dict.size }}
expected output
3
42
The same goes for .first
and .last
, although .last
should not work on a dictionary (or Mapping).
Hey there,
I am trying to use your package to parse some liquid template strings without a "real" environment. When creating a new Environemnt like
from liquid import Environment
env = Environment()
I get the error message from the title:
ModuleNotFoundError: No module named 'liquid.utils'
I installed the library using pipenv: pipenv install python-liquid
. When checking the virtualenv created by pipenv, the utils
module indeed is not installed:
site-pacakges $ ls liquid
ast.py context.py environment.py exceptions.py expression.py filter.py __init__.py lex.py loaders.py mode.py parse.py __pycache__ stream.py tag.py template.py token.py
I assume the setup.py
does not explicitly include the submodules needed in its packages (i.e. it does not include liquid.utils
).
Is my assumption correct or did I install the package wrong?
In Ruby Liquid, forloop.parentloop
is available inside nested {% for .. %}
tags, Giving access to the parent forloop
object.
Python Liquid's forloop
object does not currently have a parentloop
property.
Extend the outward facing API to allow for instantiating a Template
directly, without the need for an Environment
.
A suitable Environment
would be created automatically and used for subsequent templates created without an explicit environment.
When comparing nil
, null
, blank
and empty
, some of the corner cases are not quite what I had expected.
In the reference implementation, where foo
is the empty string, the following expressions all evaluate to true
.
"" == ""
"" == foo
"" != nil
"" != null
"" == empty
"" == blank
blank == ""
blank == foo
blank == nil
blank == null
blank != empty
blank != blank
empty == ""
empty == foo
empty != nil
empty != null
empty != empty
empty != blank
nil != ""
nil != foo
nil == nil
nil == null
nil != empty
nil == blank
null != ""
null != foo
null == nil
null == null
null != empty
null == blank
foo == ""
foo == foo
foo != nil
foo != null
foo == empty
foo == blank
Note, for example, that blank != blank
but blank == nil
.
Python Liquid does not stay true to all these "rules".
Python Liquid follows the reference implementation's default behaviour of returning nil
for any variables that are undefined. No warnings are given and no exceptions are raised.
Template
Hello, {{ noshuchthing }}.
Output
Hello, .
Whereas the reference implementation offers a strict_variables
mode and the render!
method, Python Liquid does not currently offer an alternative to the default mode.
Note that, in the reference implementation, strict_variables
is not affected by the error_mode
.
I'm inclined to follow Jinja2's example, by offering a choice of Undefined
type.
The page on creating custom filters at https://liquid.readthedocs.io/en/latest/user/filters.html is out of date.
It shows examples of inheriting from the depreciated Filter
class instead of using the preferable filter decorators.
When the built-in slice
filter is given a negative start index and a length, Python liquid can calculate the wrong stop index, potentially returning an empty sequence when it shouldn't.
{{ "Liquid" | slice: -3, 3 }}
Expected output
uid
Actual output is an empty string.
Remove the @abstractmethod
decorator from liquid.loaders.BaseLoader.get_source
so that custom loaders can implement get_source_async
without having to implement get_source
too.
I'm thinking of a database loader, where most async database drivers/packages don't expose a synchronous API. Meaning we'd need to either raise a NotImplementedError
inside an otherwise empty get_source
, or use multiple database drivers/packages.
Many filters built in to Liquid will automatically convert a string representation of a number to an integer or float as needed.
When converting integers, Ruby Liquid uses Ruby's String.to_i method, which will disregard trailing non-digit characters. In the following example, '7,42'
is converted to 7
template:
{{ 3.14 | plus: '7,42' }}
{{ '123abcdef45' | plus: '1,,,,..!@qwerty' }}
output
10.14
124
Python Liquid currently falls back to 0
for any string that can't be converted to an integer in its entirety. As is the case in Ruby Liquid for strings without leading digits.
This does not apply to parsing of integer literals, only converting strings to integers (not floats) inside filters.
The reference implementation has added support for "drops" that can resolve to primitive values when used in some Liquid expressions.
For example, if a class defines a to_liquid_value
method, like this
class SomeDrop:
def __init__(self, val):
self.val = val
def to_liquid_value():
return self.val
It could be used to access a Liquid array like this
from liquid import Template
source = "{{ greetings[foo] }}, World."
template = Template(source)
print(template.render(foo=SomeDrop(1), greetings=["Hello", "Goodbye"]))
# Goodbye, World.
There might be some other subtleties that require investigation. Like having a drop that resolves to an integer work well with maths filters.
The template
object built-in to Python Liquid is not part of "core" Liquid. It is Shopify specific.
It's tempting to disable the template
object by default, and allow it to be enabled via an argument to the Environment
constructor. This, however, risks setting a bad precedent.
Maybe some pre/post render hooks/signals are in order. Then we can move this kind of functionality to liquid-extra, along side non standard tags and filters.
The following tasks need to be completed before releasing Python Liquid version 1.0.
Both template and expression lexing functions are currently defined in liquid.lex
, and parsers for template tags and tag expressions are bundled into liquid.parse
. Moreover, all tag expressions are parsed through liquid.parse.ExpressionParser.parse_expression()
, which handles liquid identifiers, loops and boolean expressions.
For reasons of easier maintenance and potential improvements in performance, I intend to move and refactor each of the expression lexers into their own package, along with a specialised parser and independent TokenStream
(independent from the top-level token stream).
Built-in tags will transition to use these new parsers now, via liquid.Environment.parse_*_expression_value
functions. Existing tokenize*
functions and the ExpressionParser
will be maintained until at least Python Liquid version 2.0, which is quite some time away, for those who use them in custom tags.
Some possible optimisations that can be realised include:
NamedTuple
s. Benchmarks show the former to be faster.Python Liquid currently raises a NoSuchFilterFunc
exception upon rendering an output statement that uses an unknown filter.
By default, the reference implementation seems to silently ignore unknown filters, passing over them if there are more filters later in the chain. Ruby Liquid also offers a strict_filters
mode, which captures filter errors for the caller to explicitly check. In strict_filters
mode, a single undefined filter causes the whole output statement to return nil
.
Python Liquid should probably follow suit. Although without the need to explicitly check for errors. Let the exceptions raise.
When exposing templating to end users, which is one of the primary use cases for Liquid, it is vital for the stability of existing deployments that minor and patch updates to the template engine don't change template rendering behaviour.
At the same time we want to offer bug fixes, including behavioural changes, to new deployments or to developers that don't rely on built-in tags and filters.
This dilemma should be familiar to anyone who's glanced at Shopify's Liquid issue tracker. Here's an example of one bug that the Shopify developers can't fix because it could break existing templates maintained by their vast user base.
To solve this problem I propose we add tag and filter version pinning to Python Liquid. This means:
Environment
creation time.Come Python Liquid version 2, we'll reverse this, making new Environment
s default to the latest version of each tag and filter. Pinning older version will be done at the developer's discretion.
The only unknown at this stage is how we might go about testing multiple tag and filter implementations, and maintain a test suite for each.
We need an async version of liquid.template.BoundTemplate.is_up_to_date
to use when checking the template cache from liquid.Environment.get_template_async
.
An async database loader will probably want to make an async query as part of is_up_to_date
.
I'm proposing adding asynchronous support to Python Liquid (using async
/await
syntax) by way of the following additional methods.
Template.render_async
Template.render_with_context_async
Environment.get_template_async
Drops (classes that mimic Liquid primitive values) can implement __getitem_async__
, which is assumed to be a coroutine function. When defined, Liquid will use (and await) __getitem_async__
instead of __getitem__
when doing Liquid attribute and array access, and hash lookups. This will allow drops to perform asynchronous lazy loading of objects.
Initially async support will be targeted at only some Python Liquid use cases. Specifically those that ..
I am wondering if this library has the same security guarantees as the Ruby version. If I run an untrusted template from a client, is it dangerous? Is security a design goal in addition to the API compatibility?
The built-in captialize
filter currently uses Python's str.captialize
internally. Which makes "the first character have upper case and the rest lower case".
The reference implementation's capitalize
filter is roughly equivalent to this..
def captitalize(val):
if val:
return val.replace(val[0], val[0].upper(), 1)
return val
Where characters after the first are not forced to lower case.
I want to extend Python Liquid's template loader API to optionally include template meta data. Said meta data will be added to a template's globals
mapping, making it available to each render context.
This gives custom loaders the chance to emulate "front matter" style templates, similar to that found in Jekyll, or include extra data from a database of templates, for example.
By default, meta data loaded with a template will take priority over variables from existing Environment
or Template
globals, but not keyword arguments passed to .render()
. Foreseeing situations where that order of priority does not make sense, we'll implement this in BoundTemplate.make_globals
, making it simple enough to subclass BoundTemplate
to change this behaviour.
Python Liquid currently maintains an isolated namespace for named counters. This namespace is not accessible from output statements ({{ some_name }}
), and {% increment %}
and {% decrement %}
do not touch context global or local variables.
template:
{% increment foo %} {% increment foo %} {% increment foo %} {{ foo }}!
current output:
0 1 2 !
With the same template, the reference implementation outputs..
0 1 2 3!
Not only is foo
in scope outside of increment
and decrement
, but it increments/decrements the value after rendering it.
If foo
is assigned with {% assign %}
or {% capture %}
, foo
will exist in the local scope independently of any named counter.
template:
{% assign foo = 5 %}{% increment foo %} {% increment foo %} {% increment foo %} {{ foo }}!
output
0 1 2 5!
If foo
is defined as a global variable, increment
and decrement
use it. Where foo
is equal to 5
..
template:
{% increment foo %} {% increment foo %} {% increment foo %} {{ foo }}!
output:
5 6 7 8!
In addition to to_liquid_value
(allows drops to behave like a Liquid primitive value inside some expressions), Ruby Liquid uses to_number
to allow drops to work with filters that expect numbers as arguments.
The same effect can be achieved in Python Liquid using an __int__
and/or __float__
method.
Python Liquid's documentation for drop objects needs to be expanded to cover this use case.
Offer an Undefined
type that returns debug information when rendered.
The Django Liquid template backend will use this to provide contextual line information.
The special blank
keyword has not been implemented.
Currently, when blank
appears in an expression, it will be treated as any other identifier. If that identifier is not in scope, it will default to nil
.
The intended behaviour is that blank
be equivalent to an empty string. Making it possible to write conditions like this.
{% unless settings.heading == blank %}
<h1>{{ settings.heading }}</h1>
{% endunless %}
The default template cache created with every environment has a "least recently used" policy with a capacity of 300 templates.
Add a cache_size
argument to the liquid.Environment
constructor for controlling the default template cache capacity.
Python liquid will successfully parse a chained identifier containing a bracketed index or identifier, followed immediately by another identifier, with no separating dot.
{{ products[0]title }}
Ruby Liquid will raise a Liquid::SyntaxError
when parsing the above template, instead expecting {{ products[0].title }}
. Python Liquid will accept either without error.
Similarly, Python Liquid will handle {{ products.[0].title }}
, while Ruby Liquid will again raise a Liquid::SyntaxError
.
Liquid supports accessing array items from the end using negative indexes. For example.
{%- assign some_numbers = "1,2,3,4,5" | split: "," -%}
{{ some_numbers[-1] -}}
Which is expected to render 5
. Python Liquid currently raises a LiquidSyntaxError: invalid identifier, found negative
.
when
expressions should accept one or more literals or identifiers, separated by commas.
Template
{% assign a = "foo" %}
{% case a %}
{% when "" %}
no a
{% when "foo", "bar" %}
a is foo or bar
{% endcase %}
Expected output
a is foo or bar
Python Liquid currently raises a LiquidSyntaxError
as it expects exactly one literal or identifier.
Also, if multiple when
expressions match the case, including multiple matches from a comma separated list, the when
block will be rendered multiple times. One for each match.
Template
{% assign a = "foo" %}
{% assign b = "foo" %}
{% case a %}
{% when b %}
when b
{% when "foo", b %}
when "foo" or b
{% endcase %}
Expected output
when b
when "foo" or b
when "foo" or b
Python Liquid currently breaks after the first match.
When the cycle
tag is given a name, Python Liquid will use that name and all other arguments to distinguish one cycle from another. Ruby Liquid will disregard all other arguments when given a name. For example.
{% cycle a: 1, 2, 3 %}
{% cycle a: "x", "y", "z" %}
{% cycle a: 1, 2, 3 %}
Ruby Liquid Output:
1
y
3
Python Liquid Output:
1
x
2
The reference implementation allows for else
and elsif
blocks to appear in unless
blocks. Like this.
{% unless true %}
foo
{% else %}
bar
{% endunless %}
Where the expected output is bar
. Python Liquid currently raises a LiquidSyntaxError: unexpected tag 'else', on line 3
From the recently updated reference documentation..
"To start a loop from where the last loop using the same iterator left off, pass the special word continue."
Template
<!-- if array = [1,2,3,4,5,6] -->
{% for item in array limit: 3 %}
{{ item }}
{% endfor %}
{% for item in array limit: 3 offset: continue %}
{{ item }}
{% endfor %}
Output
1 2 3
4 5 6
Python Liquid does not currently support the special continue
keyword as a for loop offset.
The reference implementation includes a tag called ifchanged
that has not been implemented in Python Liquid.
Usage appears to be something like this.
Template
{% assign list = "1,3,2,1,3,1,2" | split: "," | sort %}
{% for item in list -%}
{%- ifchanged %} {{ item }}{% endifchanged -%}
{%- endfor %}
Output
1 2 3
Write some documentation.
TODO:
The sort
filter (and probably sort_natural
too) is inconsistent with the reference implementation.
Trying to sort an array of hashes without providing a key/property should raise an error. Python Liquid doesn't have an issue comparing hashes.
puts Liquid::Template.parse("{{ a | sort }}").render(
{ "a" => [{"title": "foo"}, {"title": "bar"}, {"heading": "apple"}] })
# Liquid error: comparison of Hash with Hash failed
Also, when a sort key/property is given and items in the target array support subscripts but don't have the key property, the sort
comparison block will always return 1. This has the effect of reversing the target array if all items don't contain the key property. Python Liquid currently raises a liquid.exceptions.FilterArgumentError
.
puts Liquid::Template.parse("{{ a | sort: 'title' | join: ',' }}").render(
{ "a" => ["Z","b", "a", "B", "C", "A"] })
# A,C,B,a,b,Z
Unlike liquid.context.extend
, liquid.context.copy
does not guard against recursive use.
extend
, as used by the built-in include
tag, will raise a ContextDepthError
when MAX_CONTEXT_DEPTH
is reached. copy
, as used by the built-in render
tag, should do the same.
Example include
:
>>> from liquid import Environment
>>> from liquid.loaders import DictLoader
>>> loader = DictLoader({"some": r"{% include 'some' %}"})
>>> env = Environment(loader=loader)
>>> template = env.get_template("some")
>>> template.render()
.
.
ContextDepthError: maximum context depth reached, possible recursive include, on line 1
If we swap include
for render
in the example above, we get a RecursionError: maximum recursion depth exceeded while calling a Python object
.
Update the built-in FileSystemLoader
to accept either a single search path string or a sequence of paths to search for templates.
Use markupsafe.soft_str instead of, for instance ..
if not isinstance(arg, str):
arg = str(arg)
Note that soft_str
was renamed from soft_unicode
at some point and speed ups were added at the same time.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.