hermannsblum / bdl-benchmark Goto Github PK

Hi,Thanks for a great benchmark!
When I follow the instruction on https://fishyscapes.com/dataset to download fishyscapes Static, I got some trouble.

Downloading and preparing dataset fishyscapes/Static/3.0.0 (download: Unknown size, generated: Unknown size, total: Unknown size) to ~/tensorflow_datasets/fishyscapes/Static/3.0.0...
Dl Completed...: 0 url [00:00, ? url/s]
Dl Size...: 0 MiB [00:00, ? MiB/s]

Dl Completed...: 0 url [00:00, ? url/s]
Extraction completed...: 0 file [00:00, ? file/s]
Extraction completed...: 0 file [00:00, ? file/s]
Traceback (most recent call last):
  File "/home/anaconda3/envs/s032_psac/lib/python3.6/site-packages/google/protobuf/json_format.py", line 519, in _ConvertFieldValuePair
    [f.json_name for f in message_descriptor.fields]))
google.protobuf.json_format.ParseError: Message type "tensorflow_datasets.DatasetInfo" has no field named "configDescription".
 Available Fields(except extensions): ['name', 'description', 'version', 'citation', 'sizeInBytes', 'downloadSize', 'location', 'downloadChecksums', 'schema', 'splits', 'supervisedKeys', 'redistributionInfo']

Seems this problem has discussed in this issue #8 ,
but this issue didn't give a clear answer, Could you please help me what could be a problem and how to solve it?

Additional info

OS -Ubuntu16
tensorflow-gpu==2.2.0
tensorflow-datasets==3.1.0

Good day,
I am trying to instal bdlb library and tensorflow-gpu via the instructions on the main page

The installation is successful but when I do

import bdlb

i get an error

ValueError: Invalid Version('3.0.0', 'january 2020: added cityscapes objects as negative test'). Description is deprecated. RELEASE_NOTES should be used instead.

Could you please help me what could be a problem and how to solve it?

Additional info

OS - Windows10
Running the notebook in google colab works for me but I need to install this library on my laptop.

the tensorflow-gpu seems to work because via

device_lib.list_local_devices()

I get

device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 9878896951696190928
, name: "/device:GPU:0"
device_type: "GPU"
memory_limit: 3129973145
locality {
  bus_id: 1
  links {
  }
}
incarnation: 10037955231168284208
physical_device_desc: "device: 0, name: NVIDIA GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1"
]

I have a question regarding the evaluation of results on Fishyscapes LAF in the leaderboard.

The labels differ between LAF and FS-LAF:

• the in-distribution part in Fishyscapes LAF is all 19 cityscapes classes.
• for LAF the in-distribution label is a coarse annotation of the road in front of the car.

I would expect all evaluation metrics for Fishyscapes LAF to be computed wrt the Fishyscapes LAF labels. However, in the code there is this statement

Benchmark for road obstacles
In the LostAndFound dataset, the obstacles are located on the road in front of the car. Using this prior knowledge, we can ignore the non-road part of the image - we require the method to find obstacles within the road area only. In practice, we limit the evaluation to pixels marked as "free space" or "obstacle" in the original LAF labels

Aren't metrics like FPR, AUROC, etc, very dependent on which type of labels are being used? Could you clarify this please, if possible? And in case just the LAF labels are used, then what is the intended purpose of the FS-LAF labels?

Hi,
thanks for the benchmark and code, i'm sure it will be super helpful to the community.

I'm having some trouble with tweaking the location where the fishyscapes dataset gets downloaded by default since that's abstracted out by tensorflow_datasets i assume and i can't figure out from the documentation which config file/function argument to change. It gets downloaded to my home directory for now, where i run into space issues.
Could you guide me to the exact location in code where i can specify my custom path?
Thanks!
Gunshi

Where can I get the road obstacle dataset？