Include array index about ijson HOT 2 OPEN

Update:

I hacked something real quick by modifying the common.py:

class Index:

    def __init__(self, initial_value=0):
        self._value = initial_value

    def increment(self):
        self._value += 1

    def decrement(self):
        self._value -= 1

    def __str__(self):
        return f'{self._value}'

@utils.coroutine
def parse_basecoro(target):
    path = []
    while True:
        event, value = yield
        if event == 'map_key':
            prefix = '.'.join(map(str, path[:-1]))
            path[-1] = value
        elif event == 'start_map':
            if path and (indx := path[-1]) and type(indx) == Index:
                indx.increment()
            prefix = '.'.join(map(str, path))
            path.append(None)
        elif event == 'end_map':
            path.pop()
            prefix = '.'.join(map(str, path))
        elif event == 'start_array':
            prefix = '.'.join(map(str, path))
            path.append(Index(0))
        elif event == 'end_array':
            path.pop()
            prefix = '.'.join(map(str, path))
        else:  # any scalar value
            prefix = '.'.join(map(str, path))
        target.send((prefix, event, value))

Although it is not the best solution, it certainly achieves what I am looking for. Please consider adding something similar, but in the meantime, I will be using patch to monkey-patch the library.

from ijson.

Hi @Minitour, thanks for taking an interest in improving ijson!

I think the idea is good in principle, but the suggested implementation is not going to fly. In particular:

• It breaks code for users of the items and kvitems calls, and that's an absolute no.
• It also breaks code for users of the parser calls, and that's also an absolute no.
• I'm not sure if you're aware, but modifying common.py applies the changes to all backends except yajl2_c, which is the default one (because it's 10x faster than the next one in the list).

If I implemented this, I'd do it at the items/kvitems level, where you could interpret the [n]s in the given prefix and match them to the nth appearance of item in the underlying path. Also, maybe instead of a.b.[0].c one could simply have a.b.0.c? The brackets seem unnecessary.

In any case, I'm in no hurry to implement this. Maybe if more people somehow upvote this I could give it some attention. It would also be an incentive if someone (you?) presented a modified version of items_basecoro that understood these numeric indices as indicated above, hopefully with tests -- then we could iterate into a final solution that covered all backends.

from ijson.