Preprocessing problem: TIFF reading error about raster-vision HOT 6 CLOSED

Are you running this inside the VM and Docker container using the develop branch? Could you post the stack trace? Also, do you mean that it generated a tiff file that is more than 4GB? I'm also surprised that the size of data in "gts_for_participants" and "dsm" increased a lot, since files in those directories shouldn't be affected by running that command.

from raster-vision.

There are some unrelated bugs on the develop branch that were introduced by a recent PR. I'm fixing that now.

from raster-vision.

I just merged a PR to develop that fixes some issues with the preprocessing. I think these issues are unrelated to your problem, but at least you should be able to use what's on the develop branch. I ran the command you tried without any errors, and don't have any ideas about what could be causing the error for you at the moment.

from raster-vision.

Thank you for replying. I am running this on a laptop locally using the develop branch. For the increased file size, it seems the processed the data were written back to the raw files. I guess I messed up the inheritance (super()) in the codes. When I ran the semseg.data.factory.py, I got an error like:

File "factory.py", line 18, in init
super().init([POTSDAM, VAIHINGEN], [IMAGE, NUMPY])
TypeError: super() takes at least 1 argument (0 given)

So I have to do modifications for all super() like: "super(SemsegDataGeneratorFactory,self).init([POTSDAM, VAIHINGEN], [IMAGE, NUMPY])".

For their super class I modify it like:
class DataGeneratorFactory() -> class DataGeneratorFactory(object)

I am not sure if these modifications are correct or not. Any ideas? Thanks!

from raster-vision.

I think I figured it out the problem. In the previous version, the preprocessed data were written back to the raw files in "vaihingen.py":
@staticmethod
def preprocess(datasets_path):
VaihingenImageFileGenerator(
datasets_path, [0, 1, 2, 3]).write_channel_stats(datasets_path)

This has been updated in the latest version as:
VaihingenImageFileGenerator(
datasets_path, options).write_channel_stats(proc_data_path)

Now I can run it without TIFF reading error. The processed file are stored in a separate folder.

from raster-vision.

Glad it works now. Thanks for alerting us to the problem.

from raster-vision.