Git Product home page Git Product logo

atoml's Introduction

UNMAINTAINED: This project has been merged back to tomlkit and is no longer maintained. Please switch to tomlkit>=0.8.0.

PyPI Version Python Versions License Github Actions codecov

ATOML - Yet another style-preserving TOML library for Python

ATOML is a 1.0.0rc1-compliant TOML library.

It includes a parser that preserves all comments, indentations, whitespace and internal element ordering, and makes them accessible and editable via an intuitive API.

You can also create new TOML documents from scratch using the provided helpers.

The name comes from the famous Japanese cartoon character 鉄腕アトム(Atom).

Implementation Change: Start from 1.0, ATOML is a fork of tomlkit v0.7.0 with less bugs and inconsistency.

Usage

Parsing

ATOML comes with a fast and style-preserving parser to help you access the content of TOML files and strings.

>>> from atoml import dumps
>>> from atoml import parse  # you can also use loads

>>> content = """[table]
... foo = "bar"  # String
... """
>>> doc = parse(content)

# doc is a TOMLDocument instance that holds all the information
# about the TOML string.
# It behaves like a standard dictionary.

>>> assert doc["table"]["foo"] == "bar"

# The string generated from the document is exactly the same
# as the original string
>>> assert dumps(doc) == content

Modifying

ATOML provides an intuitive API to modify TOML documents.

>>> from atoml import dumps
>>> from atoml import parse
>>> from atoml import table

>>> doc = parse("""[table]
... foo = "bar"  # String
... """)

>>> doc["table"]["baz"] = 13

>>> dumps(doc)
"""[table]
foo = "bar"  # String
baz = 13
"""

# Add a new table
>>> tab = table()
>>> tab.add("array", [1, 2, 3])

>>> doc["table2"] = tab

>>> dumps(doc)
"""[table]
foo = "bar"  # String
baz = 13

[table2]
array = [1, 2, 3]
"""

# Remove the newly added table
>>> doc.remove("table2")
# del doc["table2] is also possible

Writing

You can also write a new TOML document from scratch.

Let's say we want to create this following document:

# This is a TOML document.

title = "TOML Example"

[owner]
name = "Tom Preston-Werner"
organization = "GitHub"
bio = "GitHub Cofounder & CEO\nLikes tater tots and beer."
dob = 1979-05-27T07:32:00Z # First class dates? Why not?

[database]
server = "192.168.1.1"
ports = [ 8001, 8001, 8002 ]
connection_max = 5000
enabled = true

It can be created with the following code:

>>> from atoml import comment
>>> from atoml import document
>>> from atoml import nl
>>> from atoml import table

>>> doc = document()
>>> doc.add(comment("This is a TOML document."))
>>> doc.add(nl())
>>> doc.add("title", "TOML Example")
# Using doc["title"] = "TOML Example" is also possible

>>> owner = table()
>>> owner.add("name", "Tom Preston-Werner")
>>> owner.add("organization", "GitHub")
>>> owner.add("bio", "GitHub Cofounder & CEO\nLikes tater tots and beer.")
>>> owner.add("dob", datetime(1979, 5, 27, 7, 32, tzinfo=utc))
>>> owner["dob"].comment("First class dates? Why not?")

# Adding the table to the document
>>> doc.add("owner", owner)

>>> database = table()
>>> database["server"] = "192.168.1.1"
>>> database["ports"] = [8001, 8001, 8002]
>>> database["connection_max"] = 5000
>>> database["enabled"] = True

>>> doc["database"] = database

Installation

If you are using PDM, add atoml to your pyproject.toml file by using:

pdm add atoml

If not, you can use pip:

pip install atoml

Migrate from TOMLKit

ATOML comes with full compatible API with TOMLKit, you can easily do a Replace All of tomlkit to atoml or:

import atoml as tomlkit

ATOML differs from TOMLkit in the following ways:

  • Python 3.6+ support only
  • Tables and arrays are subclasses of MutableMapping and MutableSequence respectively, to reduce some inconsistency between the container behaviors
  • load and dump methods added
  • Less bugs

atoml's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

abravalheri

atoml's Issues

New method for array to control the format

As discussed in #38, the array element is missing a method to manually add comments and whitespace.

Proposed Solution

  • Add a method Array.raw_append(item_or_whitespace_or_comment)
  • To keep it simple, it only supports appending, i.e. is not possible to insert in the middle.
  • No auto-format is enabled in this method
  • Perform whitespace merging?

About commas and whitespace, do you want to auto add them when adding an actual value, or leave it to users for control? @abravalheri

Public API seems to rely on a private type (type hints)

Hi @frostming, I recently submitted an issue and associated fixing PR to tomlkit. Would you like me to submit it to atoml too?

python-poetry/tomlkit#130
python-poetry/tomlkit#131

The gist of the problem is the following:

Unfortunately, it seems to me that both dumps and loads type signatures depend on what seems to be a private class... This is specially suggested by:

from .toml_document import TOMLDocument as _TOMLDocument

Here the class is imported as _TOMLDocument which seems to imply that it is an implementation detail, and private.

The problem arises when trying to use those functions in a type-safe manner, since users are not supposed to import private terms and TOMLDocument is not exposed in __init__.py...

Moreover the implementation of dumps suggests that the type hint is not accurate, I suppose the correct would be (Union[TOMLDocument, dict], bool) -> str?

Error when dumping `list` that contains a `dict` and other objects

Hello, it seems that when atoml tries to convert a list whose first element is a dict (but the other aren't), it assumes the list is necessarily an AoT.

This leads to an unexpected error, that can be reproduced with the following steps:

>>> import atoml
>>> example = {"x": {"y": [{"a": 3}, "b", 42]}}
>>> print(atoml.dumps(example))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".venv/lib/python3.8/site-packages/atoml/api.py", line 39, in dumps
    data = item(dict(data), _sort_keys=sort_keys)
  File ".venv/lib/python3.8/site-packages/atoml/items.py", line 46, in item
    val[k] = item(v, _parent=val, _sort_keys=_sort_keys)
  File ".venv/lib/python3.8/site-packages/atoml/items.py", line 46, in item
    val[k] = item(v, _parent=val, _sort_keys=_sort_keys)
  File ".venv/lib/python3.8/site-packages/atoml/items.py", line 71, in item
    a.append(v)
  File "/usr/lib/python3.8/_collections_abc.py", line 962, in append
    self.insert(len(self), value)
  File ".venv/lib/python3.8/site-packages/atoml/items.py", line 1350, in insert
    raise ValueError(f"Unsupported insert value type: {type(value)}")
ValueError: Unsupported insert value type: <class 'str'>

Unsupported insert value type: <class 'str'>

I am using Python 3.8.0 and installing atoml e5d8e4b via:

$ pip install -I 'git+https://github.com/frostming/atoml@main#egg=atoml'

Future of Atoml

Hi @frostming, now that the changes from atoml were merged back into toolkit, what is the future of atoml? Will it continue to exist as an independent package?

`atoml` generates error for input that `toml` and `tomli` think is valid.

Hi, @frostming. I was trying to switch from pipenv to pdm and I came across the following error when pdm tried to parse my pyproject.toml with atoml==1.1.0:

>>> f = open( 'pyproject-debug.toml', 'rb' )
>>> import atoml
>>> d = atoml.load( f )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/api.py", line 54, in load
    return parse(fp.read())
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/api.py", line 68, in parse
    return Parser(string).parse()
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/parser.py", line 158, in parse
    body.append(key, value)
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/container.py", line 100, in append
    if key is not None and key in self:
  File "/usr/lib/python3.8/_collections_abc.py", line 666, in __contains__
    self[key]
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/container.py", line 513, in __getitem__
    return OutOfOrderTableProxy(self, idx)
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/container.py", line 693, in __init__
    self._internal_container.append(k, v)
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/container.py", line 153, in append
    current.append(k, v)
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/items.py", line 1061, in append
    self._value.append(key, _item)
  File "/home/me/python-venvs/atoml-bug/lib/python3.8/site-packages/atoml/container.py", line 159, in append
    raise KeyAlreadyPresent(key)
atoml.exceptions.KeyAlreadyPresent: Key "BASIC" already exists.

I narrowed down the input to the following reproducer:

[tool.pylint.TYPECHECK]
ignore-mixin-members = false
[toll.pylint.SPELLING]
spelling-dict = 'en_US'
spelling-ignore-comment-directives = [ 'mypy:', 'pylint:' ]
[tool.pylint.BASIC]
good-names = [ "_", "__" ]
include-naming-hint = true
const-rgx = '([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$'
const-name-hint = '([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$'
[tool.pylint.FORMAT]
max-line-length = 79
ignore-long-lines = '^(\s*(#\s*)?<?https?://\S+>?|.*\s+#\s*.*)$'

The problem goes away if I properly spell toll.pylint.SPELLING as tool.pylint.SPELLING. (I know, it's ironic that I misspelled the configuration for the spelling checker. 🤦‍♂️ But, that's beside the point.) The problem also goes away if I remove either the tool.pylint.TYPECHECK table or the tool.pylint.FORMAT table.

Neither toml nor tomli have a problem with this input:

>>> f = open( 'pyproject-debug.toml', 'rb' )
>>> import tomli
>>> d = tomli.load( f )
>>> d[ 'tool' ][ 'pylint' ][ 'BASIC' ]
{'good-names': ['_', '__'], 'include-naming-hint': True, 'const-rgx': '([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$', 'const-name-hint': '([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$'}
>>> import toml
>>> d = toml.load( 'pyproject-debug.toml' )
>>> d[ 'tool' ][ 'pylint' ][ 'BASIC' ]
{'good-names': ['_', '__'], 'include-naming-hint': True, 'const-rgx': '([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$', 'const-name-hint': '([A-Za-z_][A-Za-z0-9_]{2,30}|(__.*__))$'}

Inconsistent `dumps` when replacing existing item with nested table.

Hello, I am using atoml to try editing an existing document while preserving the original comments and formatting as much as possible.
I noticed that when I try to replace an existing item with a more complex entry (e.g. table), the produced TOML document (via dumps) does not correspond to the underlying data structure.

The follow is a minimal example to try to reproduce this error:

>>> import atoml
>>> example = """\
... [a]
... x = 1 # comment
... y = 2
... z = 3
... """
>>> doc = atoml.loads(example)
>>> doc
{'a': {'x': 1, 'y': 2, 'z': 3}}
>>> doc['a']['y'] = {'nested': 1}
>>> text = atoml.dumps(doc)
>>> doc  # this is the data-structure what we expect to obtain if parsing the dumped text
{'a': {'x': 1, 'y': {'nested': 1}, 'z': 3}} 
>>> atoml.loads(text)  # here 'z' is wrongly under 'nested'. The correct would be 'z' under 'a'
{'a': {'x': 1, 'y': {'nested': 1, 'z': 3}}}
>>> print(text)
[a]
x = 1 # comment
[a.y]
nested = 1

z = 3

Please notice in this example we expect the key z to still be under the a table, but the dump-ed document ends up putting z inside the nested sub-table, which is inconsistent with the underlying data-structure.

I believe the expected behaviour of dumps, after the modification would be producing the following:

[a]
x = 1 # comment
z = 3
[a.y]
nested = 1

or at least

[a]
x = 1 # comment
y = { nested = 1 }
z = 3

I am using Python 3.8.0 and atoml 1.0.3 on Ubuntu 18.04.5 LTS.

Please notice the same happens even when using a Table instead of a plain dict:

>>> t = atoml.table()
>>> doc['a']['y'] = t
>>> t['nested'] = 1
>>> print(atoml.dumps(doc))
[a]
x = 1 # comment
[a.y]
nested = 1

z = 3

>>> atoml.loads(atoml.dumps(doc))
{'a': {'x': 1, 'y': {'nested': 1, 'z': 3}}}

Invalid `dumps` output when appending to a multiline array

Hello @frostming, sorry for reaching out again 😝.
I noticed an unexpected output (and invalid TOML syntax) for dumps, when trying to append a dict object to an existing multiline array. Maybe this should also be considered a bug?

Please find bellow the steps to reproduce the error:

>>> import atoml
>>> example = """\
... x = [
... ]
... """
>>> doc = atoml.loads(example)
>>> doc["x"].append({"name": "John Doe", "email": "[email protected]"})
>>> doc
{'x': [{'name': 'John Doe', 'email': '[email protected]'}]}
>>> print(atoml.dumps(doc))
x = [
, name = "John Doe"
email = "[email protected]"
]

Here we can see that the value dumped for x does not conform with the TOML syntax. I would expect the output of dumps to be similar to the following:

x = [
   {name = "John Doe", email = "[email protected]"}
]

Please notice that even when using explicitly inline_table, there still will be errors:

>>> doc = atoml.loads(example)
>>> t = atoml.inline_table()
>>> t.append("name", "John Doe")
{'name': 'John Doe'}

>>> t.append("email", "[email protected]")
{'name': 'John Doe', 'email': '[email protected]'}

>>> doc['x'].append(t)
>>> doc
{'x': [{'name': 'John Doe', 'email': '[email protected]'}]}
>>> print(atoml.dumps(doc))
x = [
, {name = "John Doe", email = "[email protected]"}]

Here the extra initial , will cause problems when parsing the produced string.

I am using Python 3.8.0 and atoml 1.0.3 on Ubuntu 18.04.5 LTS.

Incorrect string returned by `dumps` when moving/renaming table

Hello @frostming, thank you for providing this fork and the effort in fixing the bugs and improving the inconsistencies found in the upstream project.

I recently notice something that seems to be another bug/inconsistency with dumps...
It happens when trying to move/rename an existing table via __setitem__ and pop. When the result is printed, the moved table appear with the old name. Please see the steps bellow to reproduce the problem:

>>> import atoml
>>> example = """\
... [x]
... a = 3 # comment
... """
>>> doc = atoml.parse(example)
>>> doc["y"] = doc.pop("x")
>>> doc
{'y': {'a': 3}}

>>> print(atoml.dumps(doc))

[x]
a = 3 # comment

>>>

I would expect as a result:

[y]
a = 3 # comment

Please notice the same inconsistent behaviour happens when using __delitem__ instead of pop:

>>> doc = atoml.parse(example)
>>> doc["y"] = doc["x"]
>>> del doc["x"]
>>> doc
{'y': {'a': 3}}

>>> print(atoml.dumps(doc))

[x]
a = 3 # comment

>>> doc["y"]
{'a': 3}

>>> doc.get("x", "No table found")
'No table found'

>>>

I am using Python 3.8.0 and atoml 1.0.3 on Ubuntu 18.04.5 LTS.

`dumps` producing invalid output

Hello, I have been recently working with TOML and I notice dumps produce an invalid output (not according to the v1.0.0 standard). loads is also not able to parse the produced output.

The steps to reproduce this issue are:

>>> import atoml
>>> atoml.dumps({"": "src"})
' = "src"\n'

>>> atoml.loads(atoml.dumps({"": "src"}))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".venv/lib/python3.8/site-packages/atoml/api.py", line 31, in loads
    return parse(string)
  File ".venv/lib/python3.8/site-packages/atoml/api.py", line 68, in parse
    return Parser(string).parse()
  File ".venv/lib/python3.8/site-packages/atoml/parser.py", line 138, in parse
    item = self._parse_item()
  File ".venv/lib/python3.8/site-packages/atoml/parser.py", line 298, in _parse_item
    return self._parse_key_value(True)
  File ".venv/lib/python3.8/site-packages/atoml/parser.py", line 370, in _parse_key_value
    key = self._parse_key()
  File ".venv/lib/python3.8/site-packages/atoml/parser.py", line 414, in _parse_key
    return self._parse_bare_key()
  File ".venv/lib/python3.8/site-packages/atoml/parser.py", line 466, in _parse_bare_key
    raise self.parse_error(ParseError, "Empty key found")
atoml.exceptions.ParseError: Empty key found at line 1 col 1

Empty key found at line 1 col 1

Please notice the standard explicit allows the usage of empty keys, and I believe the expected behaviour should be dumps({"": "src"})"" = "src".
(When providing the correct TOML form, loads seems to be able to work as expected: atoml.loads('"" = "src"'){'': 'src'})

I am using Python 3.8.0 and atoml 1.0.3 on Ubuntu 18.04.5 LTS.

Incorrect string returned by `dumps` when moving/renaming nested-table

Hello, this issue is more of a continuation of #24.

I tested the fixes implemented last week on the main branch and it seems that the dump-ed document gets the name of the nested tables wrong when we move the parent table. This can be seen in the example bellow:

>>> import atoml
>>> example = """\
... [a]
... x = 1
...
... [a.b]
... y = 2
... """
>>> doc = atoml.loads(example)
>>> doc
{'a': {'x': 1, 'b': {'y': 2}}}
>>> doc['c'] = doc.pop('a')
>>> doc
{'c': {'x': 1, 'b': {'y': 2}}}
>>> print(doc.as_string())

[c]
x = 1

[a.b]
y = 2

>>>

(Python 3.8.0, Ubuntu 18.04.5 LTS, atoml installed directly from the main branch on GitHub, i.e. 9dc2d6c)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.