bug: unable to deserialize using cloudpickle about ibis HOT 4 CLOSED

After a bit of hacking, I got this working:

In [1]: from ibis.interactive import *

In [2]: @ibis.udf.scalar.python
   ...: def add_one(x: int) -> int:
   ...:     return x + 1
   ...:

In [3]: import cloudpickle

In [4]: cloudpickle.loads(cloudpickle.dumps(add_one))
Out[4]: <function __main__.add_one(x: int) -> int>

from ibis.

From looking into this further, I think this is related to the fact that cloudpickle doesn't serialize functions with type annotations e.g. cloudpipe/cloudpickle#541.

I see the same error with the following

@ibis.udf.scalar.python
def add_one(x: int) -> int:
    return x + 1

with open("func.pkl", "wb") as f:
    cloudpickle.dump(add_one, f)

with open("func.pkl", "rb") as f:
    loaded_model = cloudpickle.load(f)

from ibis.

cloudpickle does seem to preserve annotations (I think cloudpipe/cloudpickle#541 is about annotating the cloudpickle library itself, not about supporting them in pickling/unpickling).

Perhaps there's an issue on our side.

In [1]: import cloudpickle

In [2]: def add_one(x: int) -> int:
   ...:     return x + 1
   ...:

In [3]: func = cloudpickle.loads(cloudpickle.dumps(add_one))

In [4]: func
Out[4]: <function __main__.add_one(x: int) -> int>

In [5]: func.__annotations__
Out[5]: {'x': int, 'return': int}

from ibis.

To add to this issue, I've been investigating using pickle instead, but have hit a blocker with a case statement

import pickle
from ibis import _
import ibis

c = ibis.case().when(_.s == 1, "EXACT").when(True, "DIFFERENT").end()

pickle.loads(pickle.dumps(c)) 

raises

PicklingError: Can't pickle <function _finish_searched_case at 0x78fdf8061900>: it's not the same object as ibis.expr.builders._finish_searched_case

I am able to pickle/unpickle this using cloudpickle however

from ibis.