Hi, Can you explain the difference in <a href="https://pyorc.readthe

Hi, It affects both Reader and Writer. When the <code class="not

Question: pyorc.StructRepr difference between Tuple and Dict? about pyorc HOT 2 CLOSED

noirello commented on September 15, 2024

Question: pyorc.StructRepr difference between Tuple and Dict?

from pyorc.

Comments (2)

noirello commented on September 15, 2024

Hi,

It affects both Reader and Writer.
When the struct_repr parameter is set to pyorc.StructRepr.DICT, then the Reader returns the rows as dicts and the Writer's write method expects a dict with all the keys of the column names. (It can have other keys that are not represented in the schema, but they will be ignored.)

import pyorc
input = open('./deps/examples/TestOrcFile.test1.orc', 'rb')
r = pyorc.Reader(input, struct_repr=pyorc.StructRepr.DICT)
print(next(r))
{'boolean1': False, 'byte1': 1, 'short1': 1024, 'int1': 65536, 'long1': 9223372036854775807, 'float1': 1.0, 'double1': -15.0, 'bytes1': b'\x00\x01\x02\x03\x04', 'string1': 'hi', 'middle': {'list': [{'int1': 1, 'string1': 'bye'}, {'int1': 2, 'string1': 'sigh'}]}, 'list': [{'int1': 3, 'string1': 'good'}, {'int1': 4, 'string1': 'bad'}], 'map': {}}
r = pyorc.Reader(input, struct_repr=pyorc.StructRepr.TUPLE)
print(next(r))
(False, 1, 1024, 65536, 9223372036854775807, 1.0, -15.0, b'\x00\x01\x02\x03\x04', 'hi', ([(1, 'bye'), (2, 'sigh')],), [(3, 'good'), (4, 'bad')], {})
output = open('test.orc', 'wb')
w = pyorc.Writer(output, 'struct<a:int,b:float,c:string>', struct_repr=pyorc.StructRepr.DICT)
w.write({'a': 1, 'b': 2.0, 'c': 'test'})
w.write({'a': 1, 'b': 2.0})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'c'