Evaluate Point Cloud Classification Model (3D Analyst)

Summary

Evaluates the quality of one or more point cloud classification models using a well-classified point cloud as a baseline for comparing the classification results obtained from each model.

Usage

• To obtain the best evaluation results, the reference point cloud must be well classified for the objects of interest that are being evaluated.

• All input models being evaluated must have the same classification codes. If the reference point cloud does not have class codes whose values or meaning match the classes in the models being evaluated, use the Point Cloud Class Remap parameter to designate the reference point cloud's classification with the codes in the trained models.

• The PointCNN deep learning architecture is used to create the point cloud classification model. This architecture may yield slightly different results when processing the same set of input data. You can specify the same model as an input more than once to evaluate the consistency of the classification result from the point cloud. See PointCNN: Convolution On X-Transformed Points for more information on the potential for variability in the classification results.

• The evaluation process will create several output files in the Target Folder parameter value. The name of each file will begin with the text specified in the Base Name parameter value.

  • <base name>_ModelStatistics.csv—A table listing each model's overall accuracy, precision, recall, and F1 score.
  • <base name>_ClassCodeStatistics.csv—A table summarizing the evaluation results for each class code's accuracy, precision, recall, and F1 score.
  • <base name>_ConfusionMatrices.csv—A table summarizing how each point from each class code in the reference point cloud was classified by the input models. In addition to the number of true positives and false positives, this table also identifies the classes that the false positives were occurring in.
  • <base name>_ConfusionMatrix_<model number>.png—An image showing a chart that provides an easy to read depiction of the confusion matrix for a given model.

Parameters

Dialog

Label	Explanation	Data Type
Input Model Definition	The point cloud classification models that will be evaluated. Accepted inputs are Esri model definition (.emd) and deep learning package (.dlpk) files.	File; String
Reference Point Cloud	The point cloud that will be used to evaluate the classification models.	LAS Dataset Layer; File
Target Folder	The directory that will store the files that summarize the evaluation results.	Folder
Base Name	The name that will serve as the prefix for each of the files summarizing the evaluation results.	String
Processing Boundary (Optional)	The polygon feature that delineates the portions of the reference point cloud that will be used for evaluating the classification models.	Feature Layer
Point Cloud Class Remapping (Optional)	The way class codes from the reference point cloud will be remapped to match the class codes in the models being evaluated.	Value Table

Derived Output

Label	Explanation	Data Type
Output Confusion Matrices	The CSV format table storing the confusion matrix for each class code in each input model.	Text File
Output Model Statistics	The CSV format table that summarizes the overall statistics of the input models.	Text File
Output Class Code Statistics	The CSV format table that summarizes the statistics for each class code in each input model.	Text File

Python

arcpy.ddd.EvaluatePointCloudClassificationModel(in_trained_model, in_point_cloud, target_folder, base_name, {boundary}, {class_remap})

Name	Explanation	Data Type
in_trained_model [in_trained_model,...]	The point cloud classification models that will be evaluated. Accepted inputs are Esri model definition (.emd) and deep learning package (.dlpk) files.	File; String
in_point_cloud	The point cloud that will be used to evaluate the classification models.	LAS Dataset Layer; File
target_folder	The directory that will store the files that summarize the evaluation results.	Folder
base_name	The name that will serve as the prefix for each of the files summarizing the evaluation results.	String
boundary (Optional)	The polygon feature that delineates the portions of the reference point cloud that will be used for evaluating the classification models.	Feature Layer
class_remap [class_remap,...] (Optional)	The way class codes from the reference point cloud will be remapped to match the class codes in the models being evaluated.	Value Table

Derived Output

Name	Explanation	Data Type
out_confusion_matrices	The CSV format table storing the confusion matrix for each class code in each input model.	Text File
out_model_statistics	The CSV format table that summarizes the overall statistics of the input models.	Text File
out_class_code_statistics	The CSV format table that summarizes the statistics for each class code in each input model.	Text File

Code sample

EvaluatePointCloudUsingTrainedModel example (Python window)

The following sample demonstrates the use of this tool in the Python window.

import arcpy
arcpy.env.workspace = 'C:/data'
arcpy.ddd.EvaluatePointCloudUsingTrainedModel(
        ['Transmission_Power_Lines.dlpk', 'Distribution_Power_Lines.dlpk'], 
        'Classified_Power_Lines.lasd', 'D:/Evaluate_PointCNN_Models', 
        'Power_Line_Results_', 'test_boundary.shp', [[18, 14], [20, 14]])

Environments

Extent, Processor Type, GPU ID, Output Coordinate System, Geographic Transformations

Licensing information

• Basic: Requires 3D Analyst
• Standard: Requires 3D Analyst
• Advanced: Requires 3D Analyst