Problem with train_generator: list index out of range

[thepate94227]

Hey Golbstein,
thank you for your code. I am trying to implement it, but i get an error:

Traceback (most recent call last):
  File "segmentation.py", line 65, in <module>
    rotation=False, zoom=0.1, validation_split = .15, seed = 7, do_ahisteq = False)
  File "utils.py", line 206, in create_generators
    validation_split = validation_split, seed = seed)
  File "utils.py", line 263, in __init__
    self.label_path_list = [self.label_path_list[j] for j in x]
  File "utils.py", line 263, in <listcomp>
    self.label_path_list = [self.label_path_list[j] for j in x]
IndexError: list index out of range

Do you know why?
In your Code in utils.py you used the folder "SegmentationClassAug" in line 247, but there is no folder with this name in VOC2012. There is one named "SegmentationClass" and "SegmentationObject". I used "SegmentationClass". And i created the folder "train", which you used in line 246, too. There i copied all the images which were in JPEGImages before.
Maybe this has to do with the error i got.

[Golbstein]

I think your PATH to the dataset is incorrect

[thepate94227]

I think your PATH to the dataset is incorrect

Thank you for your answer. But my PATH to the dataset is like yours. My Path is ..../datasets/VOCdevkit/VOC2012/'
And the folder structure you can see here:

Is it correct, that i have 17.125 Input Images, but 2.913 SegmentationClass Images?

Also, in the class "SegmentationGenerator(Sequence):" function in line 239:
if i print print(self.label_path_list) or print(self.image_path_list) in the init function, i get an long list back, but the image_list is longer than the label_list.
For example: One element from image_list is /.../tmp/datasets/VOCdevkit/VOC2012/JPEGImages/train/2010_000978.jpg'
One element from the label_list is: '/.../tmp/datasets/VOCdevkit/VOC2012/SegmentationClass/2007_000032.png'

The length of the self.image_path_list is 17125 and the length of the self.label_path_list is 2913, so the PATH has to be correct, right?

[Golbstein]

Obviously not.. You must have pairs of image and its mask

[thepate94227]

Thank you for your answer. Hm, thats weird. I downloaded it from the official site: http://host.robots.ox.ac.uk/pascal/VOC/voc2012/#devkit
I don't know why there are 17K Input Images, but only 2.9K Segmentation Images.

Is it correct that your label is a grayscale image with the value 0 for background and value 1 for class 1, value 2 for class 2 etc.? And for each object there is a shape with value 255?
Like this image:

When i want to train DeepLab with my own data, do my label image has to have the shape with the value 255? Because i have pictures with two ropes and a background, see my post here:
keras-team/keras#3416 (comment)
And as far as i understand your code, you ignore the value 255, right?

[thepate94227]

An another question: how can i save the model after the training? I don't know how to use google Colab with my own data, which are on my PC. Therfore i want to first train my data and after that i want to save the model. And then in another Python File i want to load the model and show the result the same way you did.

[peachthiefmedia]

If you use the VOC2012 development kit it won't be arrange as needed for the training, as it has all the jpegimages together without a split between train and test. You can split it yourself, I used the below code (set the data_dir to your directory with masks, set the image_dir to all your images, and image2_dir to where you want the train/val images to go).

import os
import glob
import numpy as np
import shutil

data_dir = 'Mask/'
image_dir = 'Images/'
image_dir2 = 'Train/'

names = [os.path.splitext(os.path.basename(x))[0] for x in (glob.glob(data_dir + "*"))]

names = np.unique(names, axis=0)
print(names)
print(len(names))

for name in names:
    shutil.copyfile(os.path.join(image_dir,name + ".jpg"),os.path.join(image_dir2,name + ".jpg"))

You'll have to alter the utils.py to match what you want, and the same holds if you want to train it on your own dataset.

        self.image_path_list = sorted(glob.glob(os.path.join(folder, 'Train', '*')))
        self.label_path_list = sorted(glob.glob(os.path.join(folder, 'Mask', '*')))

I've set my checkpointer to output the whole model (save_weights_only=False), that's the easiest way to do it probably.

[Mps24-7uk]

@thepate94227 Download the SegmentationClassAug folder from https://www.dropbox.com/s/oeu149j8qtbs1x0/SegmentationClassAug.zip