Label objects for deep learning

Доступно с лицензией Image Analyst.

Доступно с лицензией Spatial Analyst.

All supervised deep learning tasks depend on labeled datasets, which means humans must apply their knowledge to train the neural network on what it is working to identify. The labeled objects will be used by the neural network to train a model that can be used to perform inferencing on data.

Image annotation, or labeling, is vital for deep learning tasks such as computer vision and learning. A large amount of labeled data is required to train a good deep learning model. When the right training data is available, deep learning systems can be highly accurate in feature extraction, pattern recognition, and complex problem solving. The Label Objects for Deep Learning pane can be used to quickly and accurately label data.

The Label Objects for Deep Learning button is found in the Classification Tools drop-down menu, in the Image Classification group on the Imagery tab. The pane is divided into two parts. The top part of the pane is used for managing classes, and the bottom part of the pane is used for managing the collection of the samples and for exporting the training data for the deep learning frameworks.

Create classes and label objects

The top portion of the pane allows you to manage object classes and manually create the objects used for training the deep learning model. There are many tools available to help you create labeled objects.

Tool	Function
	Create a labeled object by drawing a rectangle around a feature or object in the raster.
	Create a labeled object by drawing a polygon around a feature or object in the raster.
	Create a labeled object by drawing a circle around a feature or object in the raster.
	Create a labeled object by drawing a freehand shape around a feature or object in the raster.
	Create a feature by selecting a segment from a segmented layer. This option is only available if there is a segmented layer in the Contents pane. Activate the Segment Picker by highlighting the segmented layer in the Contents pane, then select the layer from the Segment Picker drop-down list.
	Assigns the selected class to the current image. This is only available in Image Collection mode.
	Select and edit a labeled object.
	Create a classification schema.
	Select a classification schema option. Browse to an existing schema. Generate a new schema from an existing training sample feature class. Generate a new schema from an existing classified raster. Use the default 2011 National Land Cover Database schema.
	Save changes to the schema.
	Save a new copy of the schema.
	Add a class category to the schema. Select the name of the schema first to create a parent class at the highest level. Select the name of an existing class to create a subclass.
	Remove the selected class or subclass category from the schema.

1. Click one of the sketch tools, such as Rectangle, Polygon, Circle, or Freehand, to begin collecting object samples.
2. Using a sketch tool, delineate the image feature representing the object on the map.
   1. If you are creating a feature without a class specified, the Define Class dialog box appears. For more information about this dialog box, see the Define Class section.
3. Continue to create and label objects as specified in the steps above.
4. You can use the Labeled Objects tab (at the bottom of the pane) to delete and organize your labeled object samples.
5. Once you are satisfied with all your labeled objects, save your samples by clicking the Save button on the Labeled Objects tab.

Now that you have manually labeled a representative sample of objects, these can be used to export your training data.

Define Class

The Define Class dialog box allows you to create a new class or define an existing class. If you choose Use Existing Class, select the appropriate Class Name option for that object. If you choose Add New Class, you can optionally edit the information and click OK to create the new class.

Label image collections

If you have an image collection, or want to label individual images within a mosaic dataset, use the Image Collection tab. For more information about image collections, see Mosaic datasets.

Using the mosaic layer, you can label each of the images. Using the Image Collection tab, you can access the list of images in the drop-down list. The selected image will draw in the map. You can then label the image with the appropriate class. Use the arrow buttons to choose the next image you want to view and label.

When your image is in an Image Coordinate System (ICS), the image may be in an unusual orientation, especially when dealing with oblique or perspective imagery. To view your image in pixel space, check the Label in pixel space check box. This will draw the image in an orientation more conducive to intuitive image interpretation.

Label the entire image

For instances when you don't want to draw a boundary around an object, you can use the Label Image button to label the entire image with the selected class, irrespective of the spatial aspect of the object.

Labeled Objects

The Labeled Objects tab is located in the bottom section of the pane and manages the training samples you have collected for each class. Collect representative sites, or training samples, for each class in the image. A training sample has location information (polygon) and an associated class. The image classification algorithm uses the training samples, saved as a feature class, to identify the land cover classes in the entire image.

You can view and manage training samples by adding, grouping, or removing them. When you select a training sample, it is selected on the map. Double-click a training sample in the table to zoom to it on the map.

Tool	Function
	Open an existing training samples feature class.
	Save edits made to the current labeled objects feature class.
	Save the current labeled objects as a new feature class.
	Delete the selected labeled objects.

Export Training Data

Once samples have been collected, you can export them into training data by clicking the Export Training Data tab. The training data can then be used in a deep learning model. Once the parameters have been filled in, click Run to create the training data.

Parameter	Description
Output Folder	Choose the output folder where the training data will be saved.
Mask Polygon Features	Полигональный класс объектов, который обозначает области, где будут создаваться кусочки изображений. Будут создаваться только те кусочки, которые полностью находятся в пределах полигонов.
Image Format	Specifies the raster format for the image chip outputs. TIFF. This is the default. MRF (Meta Raster Format). PNG. JPEG. Форматы PNG и JPEG поддерживают до трех каналов.
Tile Size X	Размер кусочков изображений по измерению X.
Tile Size Y	Размер кусочков изображений по измерению Y.
Stride X	Расстояние для смещения по направлению X, при создании следующих кусочков изображения. Когда шаг равен размеру листа, наложения (перекрытия) не будет. Если шаг равен половине размера листа, величина перекрытия будет равна 50.
Stride Y	Расстояние для смещения по направлению Y, при создании следующих кусочков изображения. Когда шаг равен размеру листа, наложения (перекрытия) не будет. Если шаг равен половине размера листа, величина перекрытия будет равна 50.
Rotation Angle	The rotation angle that will be used to generate additional image chips. An image chip will be generated with a rotation angle of 0, which means no rotation. It will then be rotated at the specified angle to create an additional image chip. The same training samples will be captured at multiple angles in multiple image chips for data augmentation. The default rotation angle is 0.
Output No Feature Tiles	Определяет, будут ли экспортироваться кусочки изображения, не захватывающие обучающие выборки. Unchecked—Only image chips that capture training samples will be exported. This is the default. Checked—All image chips, including those that do not capture training samples, will be exported.
Metadata format	Задает формат, который будет использован для надписей выходных метаданных. Если ваши входные данные обучающей выборки являются слоем класса объектов, например, слоем зданий или файлом обучающей выборки стандартной классификации, используйте опцию меток KITTI или PASCAL Visual Object Classes (KITTI_rectangles или PASCAL_VOC_rectangles в Python). Выходные метаданные – это файл .txt или .xml, содержащий обучающие выборки данных, которые попадают в минимальный ограничивающий прямоугольник. Имя файла метаданных соответствует имени входного изображения. Если ваши входные данные обучающих выборок – карта классов, используйте опцию Классифицированные листы (Classified_Tiles в Python) в качестве выходного формата метаданных. KITTI Labels—The metadata follows the same format as the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) Object Detection Evaluation dataset. The KITTI dataset is a vision benchmark suite. The label files are plain text files. All values, both numerical and strings, are separated by spaces, and each row corresponds to one object. PASCAL Visual Object Classes—The metadata follows the same format as the Pattern Analysis, Statistical Modeling and Computational Learning Visual Object Classes (PASCAL VOC) dataset. The PASCAL VOC dataset is a standardized image dataset for object class recognition. The label files are XML files and contain information about image name, class value, and bounding boxes. This is the default. Classified Tiles—The output will be one classified image chip per input image chip. No other metadata for each image chip is used. Only the statistics output has more information on the classes, such as class names, class values, and output statistics. RCNN Masks—The output will be image chips that have a mask on the areas where the sample exists. The model generates bounding boxes and segmentation masks for each instance of an object in the image. It's based on Feature Pyramid Network (FPN) and a ResNet101 backbone in the deep learning framework model. Labeled Tiles—Each output tile will be labeled with a specific class. If you choose this metadata format, you can additionally refine the Blacken Around Feature and Crop Mode parameters. Multi-labeled Tiles —Each output tile will be labeled with one or more classes. For example, a tile may be labeled agriculture and also cloudy. This format is used for object classification. Export Tiles —The output will be image chips with no label. This format is used for image translation techniques, such as Pix2Pix and Super Resolution. CycleGAN —The output will be image chips with no label. This format is used for image translation technique CycleGAN, which is used to train images that do not overlap. Imagenet—Each output tile will be labeled with a specific class. This format is used for object classification; however, it can also be used for object tracking when the Deep Sort model type is used during training. Для формата метаданных KITTI создается 15 столбцов, но только 5 из них будут использованы инструментом. Первый столбец – значение класса. Следующие три столбца пропускаются. Столбцы c 5 по 8 задают минимальный ограничивающий прямоугольник, который состоит из четырех координат местоположений изображения: левый, верхний, правый и нижний пиксел, соответственно. Минимальный ограничивающий прямоугольник определяет обучающий кусочек изображения, который используется классификатором глубокого обучения. Остальные столбцы не используются.
Blacken Around Feature	Определяет, будут ли маскироваться пикселы вокруг каждого объекта или пространственного объекта в каждом листе изображения. Не отмечено — пикселы, окружающие объекты или явления, затемнены не будут. Это значение по умолчанию Отмечено — пикселы, окружающие объекты или явления, будут затемнены. Этот параметр применяется только в случае, когда в качестве Формата метаданных указаны Надписанные листы, а также указан входной класс объектов или классифицированный растр.
Crop Mode	Определяет, будут ли экспортированные листы обрезаться так, чтобы они все стали одинакового размера. Fixed size—Exported tiles will be the same size and will center on the feature. This is the default. Bounding box—Exported tiles will be cropped so that the bounding geometry surrounds only the feature in the tile. Этот параметр применяется только в случае, когда в качестве Формата метаданных указаны Надписанные листы или Imagenet, а также указан входной класс объектов или классифицированный растр.
Reference System	Определяет тип системы привязки, которая будет использоваться при интерпретации входного изображения. Указанная система привязки по возможности должна совпадать с привязкой, которая использовалась для тренировки модели глубокого обучения. Map space—The input image is in a map-based coordinate system. This is the default. Pixel space—The input image is in image space (rows and columns), with no rotation and no distortion.
Additional Input Raster	An additional input imagery source for image translation methods. This parameter is valid when the Metadata Format parameter is set to Classified Tiles, Export Tiles, or CycleGAN.

The exported training data can now be used in a deep learning model.