UnicodeDecodeError about places365 HOT 18 OPEN

Thanks @paffell for the trick.
For those of you who don't want to modify PyTorch's source code, you can do the following in your code just before loading the model:

from functools import partial
import pickle
pickle.load = partial(pickle.load, encoding="latin1")
pickle.Unpickler = partial(pickle.Unpickler, encoding="latin1")
model = torch.load(model_file, map_location=lambda storage, loc: storage, pickle_module=pickle)

This is for the run_placesCNN_unified.py example. For run_placesCNN_basic.py, replace the model_file in the last line with model_weight.
Note that your pickle namespace will be polluted after this operation. A better idea would be to copy the module first and modify the methods of this copy.

from places365.

I solved it with a small change in torch code in serialization.py lines 376-377 by adding the encoding there:
_sys_info = pickle_module.load(f,encoding='latin1')
unpickler = pickle_module.Unpickler(f,encoding='latin1')

I know it is not the cleanest solution, but it works :) To make it cleaner, it should be at least wrapped it by Try - Catch. But I do use Torch for this purpose only

from places365.

Indeed, it is the pickle library in python which cannot correctly load the CNN model trained in python2.7 into python3.6. Please let me know if there is some good solution for that.

from places365.

I solved in the same way as paffel did, and it works pretty well.

from places365.

No idea. maybe you could try useGPU = 1. Model is cached using python2.7.

from places365.

i met the same problem! Did you solve it?

from places365.

i use python 3.6, i think the version may be the key!

from places365.

yep, i just use virtualenv to create a pure environment for python2.7 and all is ok.
thank you for your reply! this is an excellent work!

from places365.

Thanks @soravux for cleaning the solution. I started to use it as well :)

from places365.

Thanks @soravux, it works for me too.

from places365.

@soravux Thanks so much!!! it's work for me

from places365.

https://discuss.pytorch.org/t/unicodedecodeerror-ascii-codec-cant-decode-byte-0xc3-in-position-918-ordinal-not-in-range-128/15356/7

from places365.

UnpicklingError: invalid load key, '\x5c'.

Getting the same error.Can anyone help?

from places365.

_pickle.UnpicklingError: invalid load key, '\x04'.

Getting the same error. Tried Soravux's method, still report error.

from places365.

I solved in the same way as paffel did, and it works pretty well.

will you please mention the line numbers and the file name. Thanks.

from places365.

from places365.

The hack suggested in this issue, i.e., the following line:

pickle.Unpickler = partial(pickle.Unpickler, encoding="latin1")

completely breaks toch.load since version 1.11 because in that version, PyTorch subclasses pickle.Unpickler but after this, pickle.Unpickler is a function and not a type which leads to the following error:

[...]
  File "lib/python3.9/site-packages/torch/serialization.py", line 820, in _legacy_load
    class UnpicklerWrapper(pickle_module.Unpickler):  # type: ignore[name-defined]
TypeError: the first argument must be callable

To load models saved in Python2, since PyTorch 1.1.0 (released May 2019), do

torch.load(...., encoding='latin1')

from places365.

from places365.