Stack overflow when running Pytorch/Tensorflow about flor HOT 5 CLOSED

The stack overflow has to do with serialization causing an infinite loop. Pickling torch or tensorflow objects results in a call to a special serialization function. The serialization function then happens to call some sort of loading function that has been decorated by flor. Then, when flor writes and serializes the object, it goes right back into the serialization function.

from flor.

Blacklisting save/serialization functions only goes so far. Flor doesn't seem to stack overflow anymore once we blacklist serialization methods, but encounters variable used before assignment errors. We should be doing some sort of loop detection, perhaps in flog.serialize.

from flor.

Loop detection seems to be the correct way to address this and allows the pytorch example to finish running fully.

from flor.

Tensorflow bug appears to be resolved after modifying the file in two ways: blacklist the DeprecationWrapper class in deprecation_wrapper.py and the _new__init__ function in tf_should_use.py.
These snippets of code were where the attribute errors were happening, but blacklisting them should get rid of the errors. More testing is required to confirm this.

from flor.

I think loop detection is the right approach, rather than blacklisting. I will be trying a bug fix. @eecsliu can you share the workload that hits this problem so I can try to reproduce it?

from flor.