Classify Pixels Using Deep Learning (Image Analyst)—ArcGIS Pro

Available with Image Analyst license.

Summary

Runs a trained deep learning model on an input raster to produce a classified raster, with each valid pixel having an assigned class label.

This tool requires a model definition file containing trained model information. The model can be trained using the Train Deep Learning Model tool or by a third-party training software such as TensorFlow, PyTorch, or Keras. The model definition file can be an Esri model definition JSON file (.emd) or a deep learning model package, and it must contain the path to the Python raster function to be called to process each object and the path to the trained binary deep learning model file.

Usage

You must install the proper deep learning framework Python API (such as TensorFlow or PyTorch) in the ArcGIS Pro Python environment; otherwise, an error will occur when you add the Esri model definition file to the tool. Obtain the appropriate framework information from the creator of the Esri model definition file.
To set up your machine to use deep learning frameworks in ArcGIS Pro, see Install deep learning frameworks for ArcGIS.
This tool calls a third-party deep learning Python API (such as TensorFlow, PyTorch, or Keras) and uses the specified Python raster function to process each object.
Sample use cases for this tool are available on the Esri Python raster function GitHub page. You can also write custom Python modules by following examples and instructions in the GitHub repository.
The Esri model definition parameter value can be an Esri model definition JSON file (.emd), a JSON string, or a deep learning model package (.dlpk). A JSON string is useful when this tool is used on the server so you can paste the JSON string, rather than upload the .emd file. The .dlpk file must be stored locally.
For more information about deep learning, see Deep learning in ArcGIS Pro.

The following code sample uses the Esri model definition file (.emd):

{
    "Framework":"TensorFlow",
    "ModelConfiguration":"deeplab",

    "ModelFile":"\\Data\\ImgClassification\\TF\\froz_inf_graph.pb",
    "ModelType":"ImageClassification",
    "ExtractBands":[0,1,2],
    "ImageHeight":513,
    "ImageWidth":513,

    "Classes" : [
        {
            "Value":0,
            "Name":"Evergreen Forest",
            "Color":[0, 51, 0]
         },
         {
            "Value":1,
            "Name":"Grassland/Herbaceous",
            "Color":[241, 185, 137]
         },
         {
            "Value":2,
            "Name":"Bare Land",
            "Color":[236, 236, 0]
         },
         {
            "Value":3,
            "Name":"Open Water",
            "Color":[0, 0, 117]
         },
         {
            "Value":4,
            "Name":"Scrub/Shrub",
            "Color":[102, 102, 0]
         },
         {
            "Value":5,
            "Name":"Impervious Surface",
            "Color":[236, 236, 236]
         }
    ]
}

For information about requirements for running this tool and issues you may encounter, see Deep Learning Frequently Asked Questions.

Syntax

ClassifyPixelsUsingDeepLearning(in_raster, in_model_definition, {arguments}, processing_mode, {out_classified_folder})

Parameter	Explanation	Data Type
in_raster	The input raster dataset to classify. The input can be a single raster or multiple rasters in a mosaic dataset, an image service, or a folder of images.	Raster Dataset; Raster Layer; Mosaic Layer; Image Service; MapServer; Map Server Layer; Internet Tiled Layer; Folder
in_model_definition	The in_model_definition parameter value can be an Esri model definition JSON file (.emd), a JSON string, or a deep learning model package (.dlpk). A JSON string is useful when this tool is used on the server so you can paste the JSON string, rather than upload the .emd file. The .dlpk file must be stored locally. It contains the path to the deep learning binary model file, the path to the Python raster function to be used, and other parameters such as preferred tile size or padding.	File; String
arguments [arguments,...] (Optional)	The function arguments are defined in the Python raster function class. This is where you list additional deep learning parameters and arguments for experiments and refinement, such as a confidence threshold for adjusting sensitivity. The names of the arguments are populated from reading the Python module.	Value Table
processing_mode	Specifies how all raster items in a mosaic dataset or an image service will be processed. This parameter is applied when the input raster is a mosaic dataset or an image service. PROCESS_AS_MOSAICKED_IMAGE —All raster items in the mosaic dataset or image service will be mosaicked together and processed. This is the default. PROCESS_ITEMS_SEPARATELY —All raster items in the mosaic dataset or image service will be processed as separate images.	String
out_classified_folder (Optional)	The folder where the output classified rasters will be stored. A mosaic dataset will be generated using the classified rasters in this folder. This parameter is required when the input raster is a folder of images or a mosaic dataset in which all items are to be processed separately. The default is a folder in your project folder.	Folder

Return Value

Name	Explanation	Data Type
out_classified_raster	The name of the classified raster or the mosaic dataset containing the classified rasters.	Raster

Code sample

ClassifyPixelsUsingDeepLearning example 1 (Python window)

This example classifies a raster based on a custom pixel classification using the ClassifyPixelsUsingDeepLearning tool.

# Import system modules
import arcpy
from arcpy.ia import *

# Check out the ArcGIS Image Analyst extension license
arcpy.CheckOutExtension("ImageAnalyst")

out_classified_raster = ClassifyPixelsUsingDeepLearning
     ("c:\\classifydata\\moncton_seg.tif", "c:\\classifydata\\moncton_sig.emd", 
     "padding 0; batch_size 16", "PROCESS_AS_MOSAICKED_IMAGE")
Out_classified_raster.save("c:\\classifydata\\classified_moncton.tif")

ClassifyPixelsUsingDeepLearning example 2 (stand-alone script)

This example classifies a raster based on a custom pixel classification using the ClassifyPixelsUsingDeepLearning tool.

# Import system modules
import arcpy
from arcpy.ia import *

"""
Usage: ClassifyPixelsUsingDeepLearning(in_raster,out_classified_raster, 
       in_classifier_definition, {arguments}, {processing_mode})
                      
"""

# Set local variables
in_raster = "c:\\classifydata\\moncton_seg.tif"
in_model_definition = "c:\\classifydata\\moncton_sig.emd"
model_arguments = "padding 0; batch_size 16"
processing_mode = "PROCESS_AS_MOSAICKED_IMAGE"

# Check out the ArcGIS Image Analyst extension license
arcpy.CheckOutExtension("ImageAnalyst")

# Execute 
Out_classified_raster = ClassifyPixelsUsingDeepLearning(in_raster, 
                        in_model_definition, model_arguments, processing_mode)
Out_classified_raster.save("c:\\classifydata\\classified_moncton.tif")

Environments

Cell Size, Current Workspace, Extent, Geographic Transformations, GPU ID, Output Coordinate System, Parallel Processing Factor, Processor Type, Scratch Workspace, Snap Raster

Licensing information

Basic: Requires Image Analyst
Standard: Requires Image Analyst
Advanced: Requires Image Analyst