Comments (2)
Update:
I hacked something real quick by modifying the common.py
:
class Index:
def __init__(self, initial_value=0):
self._value = initial_value
def increment(self):
self._value += 1
def decrement(self):
self._value -= 1
def __str__(self):
return f'{self._value}'
@utils.coroutine
def parse_basecoro(target):
path = []
while True:
event, value = yield
if event == 'map_key':
prefix = '.'.join(map(str, path[:-1]))
path[-1] = value
elif event == 'start_map':
if path and (indx := path[-1]) and type(indx) == Index:
indx.increment()
prefix = '.'.join(map(str, path))
path.append(None)
elif event == 'end_map':
path.pop()
prefix = '.'.join(map(str, path))
elif event == 'start_array':
prefix = '.'.join(map(str, path))
path.append(Index(0))
elif event == 'end_array':
path.pop()
prefix = '.'.join(map(str, path))
else: # any scalar value
prefix = '.'.join(map(str, path))
target.send((prefix, event, value))
Although it is not the best solution, it certainly achieves what I am looking for. Please consider adding something similar, but in the meantime, I will be using patch
to monkey-patch the library.
from ijson.
Hi @Minitour, thanks for taking an interest in improving ijson!
I think the idea is good in principle, but the suggested implementation is not going to fly. In particular:
- It breaks code for users of the
items
andkvitems
calls, and that's an absolute no. - It also breaks code for users of the
parser
calls, and that's also an absolute no. - I'm not sure if you're aware, but modifying
common.py
applies the changes to all backends exceptyajl2_c
, which is the default one (because it's 10x faster than the next one in the list).
If I implemented this, I'd do it at the items
/kvitems
level, where you could interpret the [n]
s in the given prefix and match them to the n
th appearance of item
in the underlying path. Also, maybe instead of a.b.[0].c
one could simply have a.b.0.c
? The brackets seem unnecessary.
In any case, I'm in no hurry to implement this. Maybe if more people somehow upvote this I could give it some attention. It would also be an incentive if someone (you?) presented a modified version of items_basecoro
that understood these numeric indices as indicated above, hopefully with tests -- then we could iterate into a final solution that covered all backends.
from ijson.
Related Issues (20)
- yajl2_c backend crashes on PyPy3 HOT 19
- Is there a way to recursively iterate the key? HOT 4
- ijson.items(file, prefix) waits for EOF HOT 8
- Wheels for Python 3.12 with yajl2_c backend HOT 4
- Iterate over more than one prefix? HOT 2
- How to parse a large gzip json file. HOT 2
- Make new release HOT 2
- yajl2_c backend for lambda function HOT 2
- How to use ijson to covert string to dict? HOT 3
- How to read json records in chunks using ijson? HOT 4
- Question: is it possible that returing bytes instead of str could speedup parsing? HOT 3
- Thread safety HOT 9
- Full support for byte stream generator HOT 9
- Allow to use ijson package by a relative import HOT 4
- How can I most-efficiently check for a key in the top-level of a json object? HOT 3
- Python3.12 compilation error: ‘PyGenObject’ has no member named ‘gi_code’ HOT 5
- Is it possible to use isjon with Jsonl, ndjson ? HOT 5
- Memory leak on exception handling with yajl2_c backend HOT 6
- _yajl2 backend broken with Python 3.12 HOT 9
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ijson.