Error occurred when finalizing GeneratorDataset iterator | Trying to run image segmentation on custom dataset about advanced-deep-learning-with-keras HOT 6 CLOSED

Hi, For the OOM problem, pls adjust the batch size. The default is 4 for big memory GPUs like V100. For smaller GPUs like 1060Ti, pls use 1.
For the number of classes, the data generator reads the npy file and automatically computes the number of unique classes based on the GT mask.

Therefore, if you want to make n_classes=3 (inc bg), you have to prepare dataset with key=image_file_name and value=tensor_mask where the dim of tensor_mask is image_dim x 3. The tensor_mask defines the segmentation mask of each object in the image.

from advanced-deep-learning-with-keras.

Hi, The short answer is - unfortunately, it is not straightforward. You must change the image variable:

to something like:
image = np.ones((w, h, n_classes), dtype="uint8")

In your case, (w,h) is your image dim and n_classes = 3, or 2 classes + 1 bg class.

Assuming you have the polygon annotation using via tool, the mask will be set on the bg variable:

Rowel

from advanced-deep-learning-with-keras.

@roatienza Here are the codes used to convert to .npy

ef generate_dataset(args):
    data_dict = {}
    js = load_json(args.data_path, args.json)
    #js = js["_via_img_metadata"]
    keys = js.keys()
    rgb = rgb = [(255, 0, 0), (0, 255, 0), (0, 0, 255), (255, 255, 0), (255, 0, 255), (0, 255, 255)]
    images_no_objs = []
    for key in keys:
        entry = js[key]
        filename = entry["filename"]
        path = os.path.join(args.data_path, filename)
        regions = entry["regions"]
        masks = []
        for region in regions:
            shape = region["shape_attributes"]
            x = shape["all_points_x"]
            y = shape["all_points_y"]
            name = region["region_attributes"]
            class_id = name["Crimps"]
            #class_id = 1 if class_id == "Copper" else 2
            #print(class_id)
            fmt = "%s,%s,%s,%s"
            line = fmt % (filename, x, y, class_id)
            #print(line)
            xy = np.array([x, y], dtype=np.int32)
            xy = np.transpose(xy)
            xy = np.reshape(xy, [1, -1, 2])
            mask = { class_id : xy }
            #print(mask)
            masks.append(mask)

        image = plt.imread(path)
        #dim = (512, 420)
        #image = cv2.resize(image, dim, interpolation = cv2.INTER_AREA)
        if args.show:
            plt.xlabel('x')
            plt.ylabel('y')
            plt.title('Input image', fontsize=14)
            fname = os.path.splitext(filename)[0]
            fname = fname + "-input.png"
            path = os.path.join("images", fname)
            plt.imshow(image)
            plt.savefig(path)
            plt.show()
        else:
            image = np.zeros_like(image)

        shape = image.shape
        shape = (shape[0], shape[1])
        bg = np.ones(shape, dtype="uint8")
        bg.fill(255)
        i = 0
        image = np.zeros_like(image)
        #image[:] = [128, 0, 128]
        for mask in masks:
            name = list(mask)[0]
            #print(name)
            mask = mask[name]
            #cv2.fillPoly(image, mask, rgb[int(name)-1])
            cv2.fillPoly(image, mask, rgb[i - 1])
            i += 1
            cv2.fillPoly(bg, mask, 0)

        if args.show:
            name = os.path.splitext(filename)[0]

            plt.xlabel('x')
            plt.ylabel('y')
            plt.title('Ground truth semantic segmentation', fontsize=14)
            fname = name + "-semantic.png"
            path = os.path.join("images", fname)
            plt.imshow(image)
            plt.savefig(path)
            #plt.show()

            #plt.xlabel('x')
            #plt.ylabel('y')
            #plt.title('Background segmentation', fontsize=14)
            #fname = name + "-bg.png"
            #path = os.path.join("images", fname)
            #plt.imshow(bg, cmap='gray', vmin=0, vmax=255)
            #plt.savefig(path)
            #plt.show()


        shape = (*shape, 1)
        bg = np.reshape(bg, shape)
        #print(bg.shape)
        data = np.concatenate((bg, image), axis=-1)
        data = data.astype('float32') / 255
        data = data.astype('uint8')
        data_dict[filename] = data
        print(filename, len(masks))
        if len(masks) == 0:
            images_no_objs.append(filename)

    if not args.show:
        np.save(args.save_filename, data_dict)

from advanced-deep-learning-with-keras.

Hi @roatienza Thank you for your help. For the number of classes, I am bit confused. Can you please tell me in where of the code, I have to change? I am very new in Computer Vision.

from advanced-deep-learning-with-keras.

Hi @roatienza I am getting following error when trying to train Mask-RCNN model with customs images.I tried with different version of tensorflow, keras and open cv, but no luck yet. All it seems that all x values are under 1280 and y values are under 720. My image sizes are 1280 * 720. How may I solve this issue?

AssertionError: Assertion failed.
warning: source train not part of our classes, delegating to parent.
ERROR:root:Error processing image {'id': 'WIN_20210309_14_32_19_Pro.jpg', 'source': 'train', 'path': '/content/gdrive/MyDrive/vgg_promark/Image_Segmentation/train/train/WIN_20210309_14_32_19_Pro.jpg', 'width': 1280, 'height': 720, 'polygons': [{'name': 'polygon', 'all_points_x': [428, 357, 349, 347, 359, 339, 349, 340, 338, 346, 356, 349, 359, 360, 359, 354, 361, 372, 396, 415, 427, 429, 422, 410, 417, 423, 429], 'all_points_y': [190, 189, 189, 202, 220, 232, 255, 262, 280, 289, 291, 305, 320, 335, 357, 374, 385, 394, 398, 398, 381, 360, 325, 294, 276, 228, 214]}, {'name': 'polygon', 'all_points_x': [499, 442, 423, 419, 420, 439, 431, 423, 420, 416, 434, 426, 415, 419, 430, 457, 482, 506, 511, 527, 528, 519], 'all_points_y': [145, 139, 139, 155, 172, 184, 216, 254, 288, 303, 348, 397, 412, 423, 430, 422, 411, 404, 318, 249, 182, 151]}, {'name': 'polygon', 'all_points_x': [504, 463, 446, 445, 438, 445, 458, 475, 484, 490, 493, 498, 499, 496, 486, 483, 499], 'all_points_y': [504, 497, 501, 553, 609, 667, 665, 675, 680, 680, 669, 637, 616, 590, 547, 528, 524]}, {'name': 'polygon', 'all_points_x': [255, 233, 217, 204, 187, 154, 143, 140, 141, 155, 163, 159, 153, 154, 172, 261, 261, 256, 247, 253], 'all_points_y': [552, 552, 542, 536, 538, 541, 545, 568, 582, 592, 594, 607, 619, 632, 631, 642, 622, 602, 587, 568]}], 'r_object_name': ['Copper', 'Belmouth', 'Belmouth', 'Copper']}
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/mrcnn/model.py", line 1704, in data_generator
use_mini_mask=config.USE_MINI_MASK)
File "/usr/local/lib/python3.7/dist-packages/mrcnn/model.py", line 1261, in load_image_gt
hooks=imgaug.HooksImages(activator=hook))
File "/usr/local/lib/python3.7/dist-packages/imgaug/augmenters/meta.py", line 470, in augment_image
return self.augment_images([image], hooks=hooks)[0]
File "/usr/local/lib/python3.7/dist-packages/imgaug/augmenters/meta.py", line 603, in augment_images
hooks=hooks
File "/usr/local/lib/python3.7/dist-packages/imgaug/augmenters/meta.py", line 2816, in _augment_images
hooks=hooks
File "/usr/local/lib/python3.7/dist-packages/imgaug/augmenters/meta.py", line 603, in augment_images
hooks=hooks
File "/usr/local/lib/python3.7/dist-packages/imgaug/augmenters/geometric.py", line 620, in _augment_images
shear_samples, cval_samples, mode_samples, order_samples)
File "/usr/local/lib/python3.7/dist-packages/imgaug/augmenters/geometric.py", line 689, in _augment_images_by_samples
return_matrix=return_matrices,
File "/usr/local/lib/python3.7/dist-packages/imgaug/augmenters/geometric.py", line 956, in _warp_cv2
borderValue=cval
cv2.error: OpenCV(3.4.2) /io/opencv/modules/imgproc/src/imgwarp.cpp:2610: error: (-215:Assertion failed) src.cols > 0 && src.rows > 0 in function 'warpAffine'

Process Process-1:

StopIteration Traceback (most recent call last)
in ()
----> 1 history = train(model, dataset_path, 5)

3 frames
/usr/local/lib/python3.7/dist-packages/keras/engine/training.py in fit_generator(self, generator, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch)
2009 batch_index = 0
2010 while steps_done < steps_per_epoch:
-> 2011 generator_output = next(output_generator)
2012
2013 if not hasattr(generator_output, 'len'):

StopIteration:

from advanced-deep-learning-with-keras.

It appears that the input to your MaskRCNN model is None. Pls check your input data just before it is passed to the model.

This issue is no longer about the book content and example code so I am closing it. Thank you.

from advanced-deep-learning-with-keras.