Evaluate Point Cloud Classification Model (3D Analyst)

Summary

Evaluates the quality of one or more point cloud classification models using a well-classified point cloud as a baseline for comparing the classification results obtained from each model.

Usage

  • To obtain the best evaluation results, the reference point cloud must be well classified for the objects of interest that are being evaluated.

  • All input models being evaluated must have the same classification codes. If the reference point cloud does not have class codes whose values or meaning match the classes in the models being evaluated, use the Point Cloud Class Remap parameter to designate the reference point cloud's classification with the codes in the trained models.

  • The PointCNN deep learning architecture is used to create the point cloud classification model. This architecture may yield slightly different results when processing the same set of input data. You can specify the same model as an input more than once to evaluate the consistency of the classification result from the point cloud. See PointCNN: Convolution On X-Transformed Points for more information on the potential for variability in the classification results.

  • The evaluation process will create several output files in the Target Folder parameter value. The name of each file will begin with the text specified in the Base Name parameter value.

    • <base name>_ModelStatistics.csv—A table listing each model's overall accuracy, precision, recall, and F1 score.
    • <base name>_ClassCodeStatistics.csv—A table summarizing the evaluation results for each class code's accuracy, precision, recall, and F1 score.
    • <base name>_ConfusionMatrices.csv—A table summarizing how each point from each class code in the reference point cloud was classified by the input models. In addition to the number of true positives and false positives, this table also identifies the classes that the false positives were occurring in.
    • <base name>_ConfusionMatrix_<model number>.png—An image showing a chart that provides an easy to read depiction of the confusion matrix for a given model.

Parameters

LabelExplanationData Type
Input Model Definition

The point cloud classification models that will be evaluated. Accepted inputs are Esri model definition (.emd) and deep learning package (.dlpk) files.

File; String
Reference Point Cloud

The point cloud that will be used to evaluate the classification models.

LAS Dataset Layer; File
Target Folder

The directory that will store the files that summarize the evaluation results.

Folder
Base Name

The name that will serve as the prefix for each of the files summarizing the evaluation results.

String
Processing Boundary
(Optional)

The polygon feature that delineates the portions of the reference point cloud that will be used for evaluating the classification models.

Feature Layer
Point Cloud Class Remapping
(Optional)

The way class codes from the reference point cloud will be remapped to match the class codes in the models being evaluated.

Value Table

Derived Output

LabelExplanationData Type
Output Confusion Matrices

The CSV format table storing the confusion matrix for each class code in each input model.

Text File
Output Model Statistics

The CSV format table that summarizes the overall statistics of the input models.

Text File
Output Class Code Statistics

The CSV format table that summarizes the statistics for each class code in each input model.

Text File

arcpy.ddd.EvaluatePointCloudClassificationModel(in_trained_model, in_point_cloud, target_folder, base_name, {boundary}, {class_remap})
NameExplanationData Type
in_trained_model
[in_trained_model,...]

The point cloud classification models that will be evaluated. Accepted inputs are Esri model definition (.emd) and deep learning package (.dlpk) files.

File; String
in_point_cloud

The point cloud that will be used to evaluate the classification models.

LAS Dataset Layer; File
target_folder

The directory that will store the files that summarize the evaluation results.

Folder
base_name

The name that will serve as the prefix for each of the files summarizing the evaluation results.

String
boundary
(Optional)

The polygon feature that delineates the portions of the reference point cloud that will be used for evaluating the classification models.

Feature Layer
class_remap
[class_remap,...]
(Optional)

The way class codes from the reference point cloud will be remapped to match the class codes in the models being evaluated.

Value Table

Derived Output

NameExplanationData Type
out_confusion_matrices

The CSV format table storing the confusion matrix for each class code in each input model.

Text File
out_model_statistics

The CSV format table that summarizes the overall statistics of the input models.

Text File
out_class_code_statistics

The CSV format table that summarizes the statistics for each class code in each input model.

Text File

Code sample

EvaluatePointCloudUsingTrainedModel example (Python window)

The following sample demonstrates the use of this tool in the Python window.

import arcpy
arcpy.env.workspace = 'C:/data'
arcpy.ddd.EvaluatePointCloudUsingTrainedModel(
        ['Transmission_Power_Lines.dlpk', 'Distribution_Power_Lines.dlpk'], 
        'Classified_Power_Lines.lasd', 'D:/Evaluate_PointCNN_Models', 
        'Power_Line_Results_', 'test_boundary.shp', [[18, 14], [20, 14]])

Licensing information

  • Basic: Requires 3D Analyst
  • Standard: Requires 3D Analyst
  • Advanced: Requires 3D Analyst

Related topics