mcordts / cityscapesscripts Goto Github PK

Hi, I use the annotation tool to annotate images, and find when the zoom window is open, the red isn't following the mouse, as shown in image below. I can't figure out the reason. Is this normal? Could you help to fix it? @mcordts

Hi! @mcordts , is there a doc for preparing the dataset for object detection task? do we just convert the polygonal to bounding boxes? Thank you!

in the gtbbox set, there are four numbers. What it is?
left_corner_x, left_corner_y, height, width?
why it is not written in the readme file?

Hi all,

i already trained some nets with your released dataset. everything worded fine so far for the training process. But i have a couple of questions if i understand everything correctly:

1. On your homepage it says there are 30 classes in 8 categories, so far so good. If i take a look into the labels.py file i count 35 classes (line 63 to 97). For me its not really clear what number to set as the number of output of my last conv layer. acutally after some experementing my suggestion is to set the number of outputs to 34. The problem is, i dont know exactly why... :-).
2. I saw some net architectures where there is a "ignore label" parameter. Regarding your dataset my understanding is that the void-category should be ignored (better sayed, void is not included for evaluation). How to handle this? on your Homepage void category contains "ground", "dynamic" and "static", so should i set ignore_labesl to 3? in labels.py there are 7 labels within the void-category. This is a little bit confusing for me.
3. If i have 'class_weighting' by 'weight_by_label_freqs: true' in a "SoftmaxWithLoss"-Layer, how are the weights calculated for the cityscapes labels?
4. Could you explain the iIoU metric in a little bit more "easy" detailed way than described in your official paper? I still have some understanding issues here.

Thank you very much in advance for your help!

Regards
Stefan

Dear authors,

When evaluating the result by running evalInstanceLevelSemanticLabeling.py, we could get AP and AP50%. I have a question about AP50%. 50% means the threshold of IoU is set as 0.50. However, I'm not sure what IoU represents, IoU of bounding boxes or IoU of masks ?

Thanks in advance!

If not, I will upload this project by myself later.

Hello,

I was trying the cityScapesViewer by python cityScapesViewer.py and was able to start up the UI and select a folder for display. But then no image was shown in the UI. Am I missing something?

any other ways to download in the remote servers?
i can only download the dataset in the chrome

Hello,

I am trying to compute the camera projection matrix from the intrinsic and extrinsic parameters following the description in: https://github.com/mcordts/cityscapesScripts/blob/master/docs/csCalibration.pdf
however the matrix I obtain is not correct and I suspect that I am misinterpreting some of the instruction.

Here is the snippet of code (based on opencv) I am using which should correspond exactly to the provided description.

    // assuming all the values are read from the file
    
    // cos/sin for yaw, pitch, roll
    double c_y = cos(yaw);
    double s_y = sin(yaw);

    double c_p = cos(pitch);
    double s_p = sin(pitch);

    double c_r = cos(roll);
    double s_r = sin(roll); 
    
    // camera to vehicle rotation
    cv::Matx33d R(  c_y * c_p,  c_y * s_p * s_r - s_y*c_r,      c_y*s_p*c_r + s_y * s_r,
                    s_y * c_p,  s_y * s_p * s_r + c_y * c_r,    s_y*s_p*c_r - c_y * s_r, 
                    - s_p,      c_p * s_r,                      c_p * c_r);

    // camera rotation
    R = R.t();

    // camera t
    cv::Matx31d t = - ( R * cv::Matx31d(x,y,z) );

    // skipping steps to copy the values in the 3x4 Rt 

    // Intrinsic and final Projection matrix
    K = cv::Matx33d( fx,  0,     u0,
                   0,  fy,     v0,
                   0,  0,      1);

     cv::Matx33d intrinsic_rot(      0,  -1,     0,
                                    0,  0,      -1,
                                    1,  0,      0);

      K = K * intrinsic_rot;
      P = K * Rt

For example, given camera position (1.7, 0.1, 1.22) (in train/aachen) projecting the 3d point (5.4, 0.1, 2.22) gives a 2d point at coordinate (u = 1052, v = -192). This point far in front of the camera (4 meters) should be visible in the image.

Many thanks
Aziz

I use the tool json2instanceImg.py to generate instance labels, but in the png image genereted by json2instancImg.py , different car instances(for example) seem to have the same label.
As well, after i colored the png image, i found that the color of different car instances have the same color.
Also with other classes such as person, different persons have the same instance label.

In json2instancImg.py, different instances are drawed with a unique id combined of class id and instance id, but in result png images, i can't find the defference between labels of instances.
Could you give some advice?
Thx a lot!

Hi, i want to label my images like as cityscape, i use cityscapesLabelTool, when I choose the catalog with images, they don't open...how can i fix it?

i load tool

choose dir

and images don't open

i can open it with my python script. In it i specify a way to image and tool works.But without label(color)map and i can't label my images.
open only one picture

label area

choose class

and than don't have results

similar situation with bbox

maybe someone can help me.

Hi again Marius,

everything works so far but another question came up i didnt find an answer for yet:

i trained some nets with data from your dataset. when i calculate IoU and iIoU then with your script evalPixelLevelSemanticLabeling.py i get for example the result like attached:

My question is, why do i get a "nan" for the result of iIoU of almost all classes?

When i take a look in your script there are following classes defined for calculating iIoU:

args.avgClassSize = {
"bicycle" : 4672.3249222261 ,
"caravan" : 36771.8241758242 ,
"motorcycle" : 6298.7200839748 ,
"rider" : 3930.4788056518 ,
"bus" : 35732.1511111111 ,
"train" : 67583.7075812274 ,
"car" : 12794.0202738185 ,
"person" : 3462.4756337644 ,
"truck" : 27855.1264367816 ,
"trailer" : 16926.9763313609 ,
}

So i expect at least for these classes not a "nan" as result?! And why did it work for "car" and "person" in my case? (well, its 0 but at least it does something :-) )

For the IoU, the "nan" for the train class means that there wasnt even one train-class in my training data set, correct?

But i dont understand all the other "nan", please help me out!

Thank you!
Stefan

Hello CityscapesTeam,

As your available data, you provided two kinds of data right and left images. If I want to match them to make a wide view. Does your script provide code for matching these two images? In additions, if it is available, do you provide labels (gtFine) for the matching image?

I ask the question because I found some papers that use a deep information to enhance accuracy but I do not know how they do if these labels are not available to download.

Thank all.

Hi, Marius：
Sorry to bother again！
Where is the download link for the leftImg8bit_sequence_trainvaltest.zip dataset?
I have been looking for two weeks and sent a message to your email address.
My account is mengfansheng16.
Please help me to reply to the email at your convenience.
I wish you all the best, thank you very much.
Fan-sheng Meng

I am trying to annotate an image from scratch. So, initially I don't have any label files for that image. But I am facing the following error after creating the new object and selecting its label from the dropdown list-

Traceback (most recent call last):
  File "cityscapesLabelTool.py", line 944, in newObject
    self.appendObject( label , self.drawPoly )
  File "cityscapesLabelTool.py", line 2347, in appendObject
    for obj in self.annotation.objects:
AttributeError: 'NoneType' object has no attribute 'objects'

Apparently it seems that self.annotation is of NoneType.

Could you please provide the color code for the groundtruth images, so that we can find which color corresponds to which class? Thanks!

Hi,
recently I downloaded gtFine_trainvaltest.zip from the website Cityscapes. However, when I unzipped it and looked into the test folder, I found all annotation images are almost empty without any labels on objects.

Is there something wrong?

Dear Marius Cordts

I want to use 'all_demoVideo.zip' and 'leftImg8bit_sequence_trainvaltest.zip'. How could I get these datasets? I sent an email to '[email protected]', but get no reply.

Thank you very much for your help.

Best wishes,

Ruihao

It's a known bug of PIL: the np.array cannot convert the bw png image.
http://stackoverflow.com/questions/2761645/error-converting-pil-bw-images-to-numpy-arrays

The solution is simple. Change the image to grayscale before being converted by np.array.
In your def assignGt2Preds(gtInstancesOrig, gtImage, predInfo, args):
# Read the mask
predImage = Image.open(predImageFile)
predImage = predImage.convert("L"); <---add this line pls, line 302
predNp = np.array(predImage)

Hi Mcordts,

Because I want to make a test on the FCN and I find you have trained the FCN on cityscapes in the page of the official benchmark suite , I want to ask you whether the model still keep.
If yes, can you release the model?

Thank you!

Hello,

is there an official script that converts the labels from >30 classes to 19?

Thanks a lot,
Filippo

Dear Marius Cordts

I try to evaluate my results, but when I run this script I get the following error:

Found no prediction for ground truth /home/timo/Cityscapes/gtFine/val/frankfurt/frankfurt_000001_065617_gtFine_labelIds.png

My call in the terminal: python evalPixelLevelSemanticLabeling.py

I created a folder "results" in the root directory. Inside is a image, where pixels encode the class IDs as defined in labels.py: frankfurt_000001_011835* .png
The image is from the ground truth values: /home/timo/Cityscapes/gtFine/val/frankfurt
The result should therefore be 100%, but the above error occurred.

If I use the following call: python evalPixelLevelSemanticLabeling.py /path/to/ground_truth/prediction
I get only a lot of "nan" as output.
Could you please give me a hint?

Best,
Timo

Hi, am running the instance seg evaluation and the code doesn't seem to pass with test image where there's no instance.
While the other images are passed and processed without an issue, it says KeyError with an absolute path to an image with no instance inside. Does anyone have any idea or similar encountering?

I want to know the camera parameters such as is the distance between two cameras and the focal length of camera. The README file says that csCalibration.pdf contains these parameters but I cannot find it. Can you tell me about this? Thanks very much.

My validation and training sets give images with colored labels, instance labels, and gray scale labels. But when I checked the test set, all three images are completely black. Am I supposed to process these or did my download result in an error?

Hello @krosaen @lucasb-eyer @MarvinTeichmann @mohomran @mcordts ,

When i check the labels in gtFine ,i found that the BitDepth of labelid is 8 bit, that of instanceId is 16 bit ,while the BitDepth of color is 32 bit.
Why is it 32 bit, and if i can just use it as 24 bit?

Thanks a ton!

I am a Ph.D. candidate of Xiamen University in China. My research is focused on depth estimation and semantic segmentation. Recently I need to use your packages named leftImg8bit_sequence_trainvaltest.zip and camera_trainvaltest.zip of cityscapes dataset. But the website shows "The following packages are available upon request". In order to complete my research, I need your permission to download these two package.

Hi again Marius,

i was able to train my network as discussed in the other topic. I created result pictures and now i want to evaluate them.
I still have a problem with the evalPixelLevelSemanticLabeling.py

First it says that it cant find a prediction file...then i created a folder named "result" in my Cityscapes root directory. Then it says that it find multiple predictions for ground truth in '../../gtFine/val/lindau/lindau_000001_000019_gtFine_labelIds.png'! why? i have only two images in the result-folder...

Why is it always pointing to the gtFine/val...path no matter what i do??? For first experiment issues i only want to evaluate one prediction image. So i have one picture named 'lindau_000001_000019.png' and another picture named 'lindau_000001_000019_gtFine_labelIds.png' in the 'result' folder like described above.
I just want to compare this two pictures with each other and calculate the IoU etc....it took me already hours to step behind this, i was not able to do it...

What am i doing wrong here?

Could you give my like a 'step by step' advice how to handle this? Like i said i would already be happy just to compare (evaluate) these two images before doing this with much more files.

Thank you

Stefan

My output is in the form of trainIds, but in your github, you say the evaluation is done in labelIds, so I have to convert the trainIds to labelIds first, but in the file helpers/labels.py, I find one trainId is mapped to multiple labelId, so I am confused that which labelId I should convert to, please give me a detail about it, which will help me a lot, thanks

when i use this matrix to get the real depth

disparity precomputed disparity depth maps. To obtain the disparity values, compute for each pixel p with p > 0: d = ( float(p) - 1. ) / 256., while a value p = 0 is an invalid measurement. Warning: the images are stored as 16-bit pngs, which is non-standard and not supported by all libraries.

I found the d is wrong. the object that near the camera have a big distance,the object that far away is small

hi,
I am confusing about how can i calculate the real depth, i use the formula d = ( float(p) - 1. ) / 256. to calculate the disparity value , and then i calculate the depth using : depth = baseline * focal length / disparity value , for example: depth = 0.22*2262/0.13=3828. i got a very large depth value, is that correct? how can i get a correct real depth?

Hello,
Everything is OK, except the function of "intersect/merge" does not work.
Are you OK using this function? I don't know if I have made some mistakes.

any tutorial to use this tool to annotating image data, I just download this repository and run the executable *.py file, there was same error warning like this:

then, I try to guess what that means, and do some configurations. But fail, if authors can shows us a simple tutorial. I believe that the most wasting time finding out how to using this tool can be saved for me and people like me.
my great respect to authors!

Hi all,

i want to use the Cityscapes Dataset to train a CNN with Caffe and make some experiments with it, actually i have no idea how to convert the Images and labeling Information of the dataset to the Caffe required levedb or lmdb format.

I successfully created lmdb files with the Caffe internal tool with my own pictures and a corresponding txt-file with the path to the Pictures and the associated labels. But how does this work with the cityscapes dataset?

Did anybody use this dataset with Caffe before? Any help is appreciated.

Thank you in advance!

Best regards
Stefan

Dear all,

I had downloaded the dataset:
gtFine_trainvaltest
gtCoarse

I could find the annotated images. The orginal rgb images are stored in leftImg8bit_trainvaltest.zip and leftImg8bit_trainextra.zip, right?

Milton

Hi authors,
Cityscape is an excellent work, however, we are strongly looking forward to the full video train-test dataset. Could you please tell us what does "available upon request" exactly mean? Are there any prerequisites?
Many many thanks and best wishes!

I add new image and open it by cityscapesLabelTool.py.
Then I draw one polygonal in image, but can not save it and generate JSON file.
Any suggestion?

Hi all,

I just use the Cityscapes Dataset to train a SegNet model and make some experiments with it.
And now, I want to evaluate my results on val set, and I also know that we need to have full resolution if I want to submit to the evaluation server.

But, I am confused that why I see the result of SegNet on official benchmark suite was evaluated on subsample with factor 4 at test time?
Is there another official code for the result which is subsample with factor 4 ?
Because I must compare my result with that in the same way.

And, I want to ask all that could anybody share the trained model on the Cityscapes dataset if you did?

Thanks all!

Best,
Gary

Hi, Cityscapes Team

I generated the *_gtFine_labelTrainIds.png files with two different approaches and got different results.

[1] Files are generated with createTrainIdLabelImgs.py supplied in this project.
[2] Files are generated from *_gtFine_labelIds.png supplied in dataset with matlab code below:

infile = 'gtFine/train/aachen/aachen_000000_000019_gtFine_labelIds.png';
outfile = 'gtFine/train/aachen/aachen_000000_000019_gtFine_labelTrainIds_matlab.png';
img = imread(infile);
outimg = uint8(ones(size(img))*255);
mapping = [7, 8, 11, 12, 13, 17, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 31, 32, 33];
for idx = 1:19
    outimg(img == mapping(idx)) = idx-1;
end
imwrite(outimg, outfile);

The *_gtFine_labelTrainIds.png results generated from the 2 methods above are different at the edge pixels of objects & staffs in the images. See diff image below for your reference (white pixels are those with different labels).

So how to deal with this inconsistency?

Thanks~

hello，

    I have some problem when I using this dataset. Should I use trainid for training and use ID for testing?
    It looks a little strange...

Thank you

hi, I used the cityscapesLabelTool to label new images, but I just draw only one polygonal in image, i can't add a new polygonal, should i change the 'us_us'?

Traceback (most recent call last):
File "cityscapesscripts/annotation/cityscapesLabelTool.py", line 943, in newObject
self.appendObject( label , self.drawPoly )
File "cityscapesscripts/annotation/cityscapesLabelTool.py", line 2364, in appendObject
obj.updateDate()
File "cityscapesscripts/helpers/annotation.py", line 102, in updateDate
locale.setlocale( locale.LC_ALL , 'us_us' )
File "/usr/lib/python2.7/locale.py", line 579, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting

I am new to this dataset, I met this problem when I run python cityscapesscripts/preparation/createTrainIdLabelImgs.py
Failed to find all Cityscapes modules, I set dataset path in my os environment.(I have install this package, also do the python build)

Hi,Marius Cordts.
Sorry to disturb you.I'm a postgraduate of Southeast University. Our team want to get cityscapes dataset for research learning, I've registered in the dataset website for several days, but I have not gotten the human approval.If you are convenient,please tell us how to get the dataset easily.I am looking forward for your reply.

Hi!

First, great dataset, thank you. And one question: apparently evalPixelLevelSemanticLabeling.py is for evaluating the class IDs(0-33). I trained a network using only the standard trainId-s from label.py(0-18). Is there a script I skipped or an easy way to edit evalPixelLevelSemanticLabeling.py for evaluating trainIds?

Hi, @mcordts ,

Just find the annotation mistakes in train ground-truth file "aachen_000053_000019_gtFine_labelIds.png", the car at the center is mistakenly annotated as id 16 (tunnel).

When I was trying to run python cityscapesLabelTool.py, I got error:

terminating with uncaught exception of type NSException.

The complete error message:

2018-07-13 14:01:27.796 python[36643:563603] -[QNSApplication _setup:]: unrecognized selector sent to instance 0x7fc2b581d650
2018-07-13 14:01:27.796 python[36643:563603] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[QNSApplication _setup:]: unrecognized selector sent to instance 0x7fc2b581d650'
*** First throw call stack:
(
0 CoreFoundation 0x00007fff27ce832b __exceptionPreprocess + 171
1 libobjc.A.dylib 0x00007fff4ee46c76 objc_exception_throw + 48
2 CoreFoundation 0x00007fff27d80e04 -[NSObject(NSObject) doesNotRecognizeSelector:] + 132
3 CoreFoundation 0x00007fff27c5e870 forwarding + 1456
4 CoreFoundation 0x00007fff27c5e238 _CF_forwarding_prep_0 + 120
5 libtk8.6.dylib 0x000000010d74946f TkpInit + 383
6 libtk8.6.dylib 0x000000010d6a221e Initialize + 2622
7 _tkinter.cpython-35m-darwin.so 0x000000010d4c8a06 _tkinter_create + 1174
8 python 0x000000010aa5c49e PyCFunction_Call + 62
9 python 0x000000010ab231e1 PyEval_EvalFrameEx + 15409
10 python 0x000000010ab1df2f _PyEval_EvalCodeWithName + 335
11 python 0x000000010aa2966a function_call + 106
12 python 0x000000010a9e5b45 PyObject_Call + 69
13 python 0x000000010aa08694 method_call + 148
14 python 0x000000010a9e5b45 PyObject_Call + 69
15 python 0x000000010aa82375 slot_tp_init + 117
16 python 0x000000010aa86a01 type_call + 209
17 python 0x000000010a9e5b45 PyObject_Call + 69
18 python 0x000000010ab234f4 PyEval_EvalFrameEx + 16196
19 python 0x000000010ab1df2f _PyEval_EvalCodeWithName + 335
20 python 0x000000010aa2966a function_call + 106
21 python 0x000000010a9e5b45 PyObject_Call + 69
22 python 0x000000010aa08694 method_call + 148
23 python 0x000000010a9e5b45 PyObject_Call + 69
24 python 0x000000010aa82375 slot_tp_init + 117
25 python 0x000000010aa86a01 type_call + 209
26 python 0x000000010a9e5b45 PyObject_Call + 69
27 python 0x000000010ab234f4 PyEval_EvalFrameEx + 16196
28 python 0x000000010ab232d2 PyEval_EvalFrameEx + 15650
29 python 0x000000010ab1df2f _PyEval_EvalCodeWithName + 335
30 python 0x000000010ab707d8 PyRun_FileExFlags + 248
31 python 0x000000010ab6ff6e PyRun_SimpleFileExFlags + 382
32 python 0x000000010ab94de6 Py_Main + 3622
33 python 0x000000010a9d6891 main + 497
34 libdyld.dylib 0x00007fff4fa60015 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Abort trap: 6