Detect Objects Using Deep Learning (Raster Analysis)

Summary

Runs a trained deep learning model on an input raster to produce a feature class containing the objects it identifies. The feature class can be shared as a hosted feature layer in your portal. The features can be bounding boxes or polygons around the objects found, or points at the centers of the objects.

Illustration

Usage

• Your raster analysis (RA) server Python environment must be configured with the proper deep learning framework Python API such as Tensorflow, CNTK, or similar.

• With the tool running, your RA server calls a third-party deep learning Python API (such as TensorFlow or CNTK) and uses the specified Python raster function to process each raster tile.

• The Input Model parameter will only use a deep learning package (.dlpk) item from the portal.

• After the Input Model is selected or specified, the tool will obtain the model arguments information from your raster analysis server. The tool may fail to obtain such information if your input model is invalid or your raster analysis server isn’t properly configured with the deep learning framework.

• Use the Non Maximum Suppression parameter to identify and remove duplicate features from the object detection.

• For more information about deep learning, see Deep learning in ArcGIS Pro.

Parameters

Dialog

Label	Explanation	Data Type
Input Raster	The input image used to detect objects. It can be an image service URL, a raster layer, an image service, a map server layer, or an internet tiled layer.	Raster Layer; Image Service; Map Server; Map Server Layer; Internet Tiled Layer; String
Input Model	The input model can be a file or a URL of a deep learning package (.dlpk) item from the portal.	File
Output Name	The name of the output feature service of detected objects.	String
Model Arguments (Optional)	The function model arguments are defined in the Python raster function class referenced by the input model. This is where you list additional deep learning parameters and arguments for experiments and refinement, such as a confidence threshold for fine tuning the sensitivity. The names of the arguments are populated by the tool from reading the Python module on the RA server.	Value Table
Non Maximum Suppression (Optional)	Specifies whether non maximum suppression, where duplicate objects are identified and the duplicate feature with a lower confidence value is removed, will be performed. Unchecked—All detected objects will be in the output feature class. This is the default. Checked— Duplicate detected objects will be removed.	Boolean
Confidence Score Field (Optional)	The field in the feature service that contains the confidence scores that will be used as output by the object detection method. This parameter is required when the Non Maximum Suppression parameter is checked.	String
Class Value Field (Optional)	The name of the class value field in the feature service. If a field name is not specified, a Classvalue or Value field will be used. If these fields do not exist, all records will be identified as belonging to one class.	String
Max Overlap Ratio (Optional)	The maximum overlap ratio for two overlapping features, which is defined as the ratio of intersection area over union area. The default is 0.	Double
Processing Mode (Optional)	Specifies how all raster items in a mosaic dataset or an image service will be processed. This parameter is applied when the input raster is a mosaic dataset or an image service. Process as mosaicked image—All raster items in the mosaic dataset or image service will be mosaicked together and processed. This is the default. Process all raster items separately—All raster items in the mosaic dataset or image service will be processed as separate images.	String

Derived Output

Label	Explanation	Data Type
Out Objects	The output feature service.	Feature Class

Python

arcpy.ra.DetectObjectsUsingDeepLearning(inputRaster, inputModel, outputName, {modelArguments}, {runNMS}, {confidenceScoreField}, {classValueField}, {maxOverlapRatio}, {processingMode})

Name	Explanation	Data Type
inputRaster	The input image used to detect objects. It can be an image service URL, a raster layer, an image service, a map server layer, or an internet tiled layer.	Raster Layer; Image Service; Map Server; Map Server Layer; Internet Tiled Layer; String
inputModel	The input model can be a file or a URL of a deep learning package (.dlpk) item from the portal.	File
outputName	The name of the output feature service of detected objects.	String
modelArguments [modelArguments,...] (Optional)	The function model arguments are defined in the Python raster function class referenced by the input model. This is where you list additional deep learning parameters and arguments for experiments and refinement, such as a confidence threshold for fine tuning the sensitivity. The names of the arguments are populated by the tool from reading the Python module on the RA server.	Value Table
runNMS (Optional)	Specifies whether non maximum suppression, where duplicate objects are identified and the duplicate feature with a lower confidence value is removed, will be performed. NO_NMS—All detected objects will be in the output feature class. This is the default. NMS—Duplicate detected objects will be removed.	Boolean
confidenceScoreField (Optional)	The field in the feature service that contains the confidence scores that will be used as output by the object detection method. This parameter is required when the NMS keyword is used for the runNMS parameter.	String
classValueField (Optional)	The name of the class value field in the feature service. If a field name is not specified, a Classvalue or Value field will be used. If these fields do not exist, all records will be identified as belonging to one class.	String
maxOverlapRatio (Optional)	The maximum overlap ratio for two overlapping features, which is defined as the ratio of intersection area over union area. The default is 0.	Double
processingMode (Optional)	Specifies how all raster items in a mosaic dataset or an image service will be processed. This parameter is applied when the input raster is a mosaic dataset or an image service. PROCESS_AS_MOSAICKED_IMAGE—All raster items in the mosaic dataset or image service will be mosaicked together and processed. This is the default. PROCESS_ITEMS_SEPARATELY—All raster items in the mosaic dataset or image service will be processed as separate images.	String

Derived Output

Name	Explanation	Data Type
outObjects	The output feature service.	Feature Class

Code sample

DetectObjectsUsingDeepLearning_ra example 1 (Python window)

This example creates a hosted feature layer in your portal based on object detection using the DetectObjectsUsingDeepLearning tool.

import arcpy
arcpy.DetectObjectsUsingDeepLearning_ra(
        "https://myserver/rest/services/Farm/ImageServer",
        "https://myportal/sharing/rest/content/items/itemId", "detectedTrees",
"score_threshold 0.6;padding 0", "NO_NMS")

DetectObjectsUsingDeepLearning example 2 (stand-alone script)

This example creates a hosted feature layer in your portal based on object detection using the DetectObjectsUsingDeepLearning tool.

#---------------------------------------------------------------------------
# Name: DetectObjectsUsingDeepLearning_example02.py
# Requirements: ArcGIS Image Server
# Import system modules
import arcpy
# Set local variables
inImage = "https://myserver/rest/services/coconutFarmImage/ImageServer"
inModel = "https://myportal/sharing/rest/content/items/itemId"
outName = "detectedTrees"
modelArgs = "score_threshold 0.6;padding 0"
runNMS = "NMS"
confScoreField = "Confidence"
classVField = "Class"
maxOverlapRatio = 0.15 
# Execute Detect Objects Using raster analysis tool
arcpy.DetectObjectsUsingDeepLearning_ra(inImage, inModel, outName, modelArgs,
runNMS, confScoreField, ClassVField, maxOverlapRatio)

Environments

Cell Size, Extent, Output Coordinate System, Parallel Processing Factor, Processor Type

Licensing information

• Basic: Requires ArcGIS Image Server
• Standard: Requires ArcGIS Image Server
• Advanced: Requires ArcGIS Image Server