Change Point Detection (Space Time Pattern Mining)

Summary

Detects time steps when a statistical property of the time series changes for each location of a space-time cube.

The tool can detect changes in the mean value or standard deviation of continuous variables as well as changes in the mean of count variables. The number of change points at each location can be determined by the tool or a defined number of change points can be provided that will be used for all locations. The change points divide each time series into segments in which the values in each segment have a similar mean or standard deviation. Change points are defined as the first time step in each new segment, so the number of change points is always one less than the number of segments.

Learn more about how Change Point Detection works

Illustration

Change Point Detection tool illustration

Usage

  • When you choose the Auto-detect number of change points option of the Method parameter, the Detection Sensitivity parameter is used to control the sensitivity of the detection. Higher values for the sensitivity result in more change points at each location. The choice of sensitivity is critical to the results of the analysis, and it is recommended that you try multiple values and compare results.

    Learn more about detection sensitivity

  • For analysis variables that represent counts, the Count option of the Change Type parameter is often most appropriate for detecting changes in the mean value of the counts. However, the Mean shift option may provide equivalent or better results for count data. This is because the model of the count change type assumes that the values of each segment follow a Poisson distribution in which the variance of the segment is equal to the mean value of the segment. The mean shift change type instead assumes that the values of each segment are normally distributed, so the mean value can be larger or smaller than the variance of the values.

    In a Poisson distribution, most counts are within approximately two square roots of the mean value. For example, for a Poisson distribution with a mean value equal to 100, approximately 95 percent of the counts will be between 80 and 120 (2 * sqrt(100) = 20). For a Poisson distribution with a mean equal to 1 million, most counts will be between 998,000 and 1,002,000 (the square root of 1 million is 1,000). The range of counts is comparatively more narrow for the larger mean of 1 million, where most counts are within 0.2 percent of the mean value. In the smaller mean of 100, however, counts vary up to 20 percent from the mean value. Compared to their mean value, if the values of your counts vary more than expected from a Poisson distribution, many time steps may be detected as change points. This is most common with large counts. In this case, it is recommended that you detect mean shift.

  • This tool accepts netCDF files created by the Create Space Time Cube By Aggregating Points, Create Space Time Cube From Defined Features, and Create Space Time Cube from Multidimensional Raster Layer tools.

  • The Output Features parameter value will be added to the Contents pane with rendering based on the number of change points detected at each location, along with the fields of the dates of the first and last change point. The pop-ups of the features include a line chart showing the time series values, change points, and estimates of the mean or standard deviation of the segments between each change point.

  • The layer time of the Output Features parameter value is based on the date of the first change point. You can use the time slider to filter locations based on the date of the first change point. The layer time can be changed to the date of the last change point in the layer properties.

  • The input space-time cube is updated with the results of the analysis and can be used in the Visualize Space Time Cube in 3D tool with the Time series change points option of the Display Theme parameter to display the results in 3D. The output will contain one feature per time step of the space-time cube with fields indicating whether the time step is a change point, along with estimates of the mean or standard deviation of the segment. Using the space-time cube in the Visualize Space Time Cube in 2D tool will re-create the output feature class of change point detection.

  • This tool supports parallel processing to analyze separate locations across different processing cores and uses 50 percent of available processors by default. The number of processors can be increased or decreased using the Parallel Processing Factor environment.

Parameters

LabelExplanationData Type
Input Space Time Cube

The space-time cube containing the variable for which change points will be detected. This file must have an .nc file extension and must have been created using the Create Space Time Cube By Aggregating Points, Create Space Time Cube From Defined Locations, or Create Space Time Cube From Multidimensional Raster Layer tool.

File
Analysis Variable

The numeric variable of the space-time cube containing the time series values of each location.

String
Output Features

The output feature class that will contain the change point detection results. The layer displays the number of change points detected at each location and contains pop-up line charts showing the time series values, change points, and estimates of mean or standard deviation of each segment.

Feature Class
Change Type
(Optional)

Specifies the type of change that will be detected. Each option specifies a statistical property of the time series that is assumed to be constant in each segment. The value changes to a new constant value at each change point in the time series.

  • Mean shiftShifts in mean value will be detected. This is the default.
  • Standard deviationChanges in standard deviation will be detected.
  • CountChanges in the mean of count data will be detected.
String
Method
(Optional)

Specifies whether the number of change points will be detected automatically or specified by a defined number of change points used for all locations.

  • Auto-detect number of change points (PELT)The number of change points will be detected automatically. The sensitivity of the detection will be defined by the Detection Sensitivity parameter. This is the default.
  • Defined number of change points (SegNeigh)The number of change points will be defined by the Number of Change Points parameter.
  • Auto-detect number of change points (PELT)The number of change points will be detected automatically. The sensitivity of the detection will be defined by the sensitivity parameter.This is the default.
  • Defined number of change points (SegNeigh)The number of change points will be defined by the num_change_points parameter.
String
Number of Change Points
(Optional)

The number of change points that will be detected at each location. The default is 1.

Long
Detection Sensitivity
(Optional)

A number between 0 and 1 that defines the sensitivity of the detection. Larger values will result in more detected change points at each location. The default is 0.5.

Double

arcpy.stpm.ChangePointDetection(in_cube, analysis_variable, output_features, {change_type}, {method}, {num_change_points}, {sensitivity})
NameExplanationData Type
in_cube

The space-time cube containing the variable for which change points will be detected. This file must have an .nc file extension and must have been created using the Create Space Time Cube By Aggregating Points, Create Space Time Cube From Defined Locations, or Create Space Time Cube From Multidimensional Raster Layer tool.

File
analysis_variable

The numeric variable of the space-time cube containing the time series values of each location.

String
output_features

The output feature class that will contain the change point detection results. The layer displays the number of change points detected at each location and contains pop-up line charts showing the time series values, change points, and estimates of mean or standard deviation of each segment.

Feature Class
change_type
(Optional)

Specifies the type of change that will be detected. Each option specifies a statistical property of the time series that is assumed to be constant in each segment. The value changes to a new constant value at each change point in the time series.

  • MEANShifts in mean value will be detected. This is the default.
  • STANDARD_DEVIATIONChanges in standard deviation will be detected.
  • COUNTChanges in the mean of count data will be detected.
String
method
(Optional)

Specifies whether the number of change points will be detected automatically or specified by a defined number of change points used for all locations.

  • AUTO_DETECTThe number of change points will be detected automatically. The sensitivity of the detection will be defined by the sensitivity parameter.This is the default.
  • DEFINED_NUMBERThe number of change points will be defined by the num_change_points parameter.
String
num_change_points
(Optional)

The number of change points that will be detected at each location. The default is 1.

Long
sensitivity
(Optional)

A number between 0 and 1 that defines the sensitivity of the detection. Larger values will result in more detected change points at each location. The default is 0.5.

Double

Code sample

ChangePointDetection example 1 (Python window)

The following Python script demonstrates how to use the ChangePointDetection function.

import arcpy
arcpy.env.workspace = "C:/Analysis"

# Automatically detect changes in population counts
arcpy.stpm.ChangePointDetection("Population.nc", "Pop_NONE_ZEROS", 
                            "Analysis.gdb/Population_Change_Points", 
                            "COUNT", "AUTO_DETECT", "", 0.5)
ChangePointDetection example 2 (stand-alone script)

The following Python script demonstrates how to use the ChangePointDetection function.

# Detect changes in air temperature.

# Import system modules.
import arcpy

# Set property to overwrite existing output.
arcpy.env.overwriteOutput = True

# Set workspace.
workspace = r"C:\Analysis"
arcpy.env.workspace = workspace

# Detect the largest mean shift of temperature measurements.
arcpy.stpm.ChangePointDetection("Air_Temp.nc", "MAX_DEGREES_F", 
                            "Analysis.gdb/Temperature_Change_Points", 
                            "MEAN", "DEFINED_NUMBER", 1, "")

# Create a feature class visualizing the change point results in a 3D scene view.
arcpy.stpm.VisualizeSpaceTimeCube3D("Air_Temp.nc", "Pop_NONE_ZEROS", 
                                    "VALUE", "Analysis.gdb/ForecastsFC")

Licensing information

  • Basic: Yes
  • Standard: Yes
  • Advanced: Yes

Related topics