The Time Series Smoothing tool smooths a numeric variable of one or more time series using centered, forward, and backward moving averages, as well as an adaptive method based on local linear regression.
Time series smoothing techniques are broadly used in economics, meteorology, ecology, and other fields dealing with data collected over time. Smoothing temporal data often reveals longer-term trends or cycles while smoothing over noise and short-term fluctuations.
Potential applications
Time series smoothing is applicable to any time series data that is known to contain noise or short-term fluctuations. For example, you can use the tool in the following applications:
- Daily influenza cases are commonly used in epidemiological research and planning. However, influenza cases that are detected on weekends often are not reported until Monday, making case counts from Monday appear larger than they should be and case counts on the weekend appear smaller than they should be. To correct for this, you can use a backward moving average with a time window of 6 days. Using 6 days will average the value of the current day and the previous 6 days for a total of 1 week.
- You have long-term temperature data measured hourly. When plotted in a time series, the data is too noisy and voluminous to see clear patterns and trends. You can capture the general trend of the data using adaptive bandwidth local linear regression to allow for clearer visualization and analysis. The adaptive bandwidth method will use wider time windows in some sections of the time series than others, depending on the amount of data needed to effectively smooth each section.
Smoothing methods
Four smoothing methods are available for the tool.
The Backward moving average method (also called simple moving average) is a widely used and simple smoothing method that smooths each value by taking the average of the value and all previous values within the time window. An advantage of this method is that it can be immediately performed on streaming data; as a new value is recorded, it can be immediately smoothed using previous data in the time series. However, this method has the drawback that the value being smoothed is not in the center of the time window, so all information comes from only one side of the value. This can lead to unexpected results if the trends of the data are not the same on each side of the value being smoothed.
The Forward moving average method is analogous to backward moving average, but the smoothed value is instead the average of the value and all subsequent values within the time window. It has the analogous drawback that all information used for smoothing comes from one side of the value.
The Centered moving average method smooths each value by averaging within the time window, where the value being smoothed is at the center of the window. For this method, the time window is split so that half of the window is used before the time of the value being smoothed, and half of the window is used after. This method has the advantage of using information before and after the time of the value being smoothed, so it is usually more stable and has smaller bias.
The Adaptive bandwidth local linear regression method (also called Friedman’s super smoother) smooths values using a centered time window and fitting linear regression (straight line) models to the data in multiple time window. The length of the time windows can change for each value, so some sections of the time series will use wider windows to include more information in the model. This method has the advantage that the time window does not need to be provided and can be estimated by the tool. It is also the method best suited to model data with complex trends. If a time window value is provided in the tool, a single time window is used to smooth all records, and the method is equivalent to local linear regression. For a complete description of the method, see the paper at the end of the Adaptive bandwidth local linear regression section.
The Apply shorter time window at start and end parameter is used to control the time window at the start and end of the time series. If a shorter window is not applied, smoothed values will be null for any record where the time window extends before the start or after the end of the time series. If the time window is shortened, the time window will truncate at the start and end and smooth using the values within the window. For example, if you have daily data and use a backward moving average with a two-day time window, the smoothed values of the first two days will be null if the time window is not shortened (note that the second day is only one day after the start of the time series). On the third day (two days after the start of the time series), the two day time window will not extend before the start, so the smoothed value of the third day will be the average of the values of the first three days.
Tool outputs
The primary output of the tool is a feature class or table containing the original values, smoothed values, and number of neighbors used to smooth the location. The alias of the field of smoothed values displays the smoothing method and the time window of the analysis (if an adaptive bandwidth is used, the time window is not displayed). If you append to the input data, these fields are appended to the input features or table. For adaptive bandwidth local linear regression, the number of neighbors might not be an integer. This is discussed further in the Adaptive bandwidth local linear regression section below.
Time series charts
You can use the Enable time series pop-ups parameter to create pop-up charts for each output record. For feature output, click a feature on the map to display the original values and smoothed values of the time series of that feature. To access the pop-ups for table output, right-click a record in the attribute table.
The output features or table also include a line chart showing the smoothed values of each time series.
Note:
You may experience performance problems when viewing the chart if the input data has a large number of time series.
Geoprocessing messages
The geoprocessing messages include a Summary of Smoothing section that contains information about the smoothing results for each time series. The information includes the R2 value and summary statistics for the number of temporal neighbors (minimum, maximum, mean, median, and standard deviation).
Adaptive bandwidth local linear regression
Adaptive bandwidth local linear regression builds local linear models at each time step using neighboring values in time, where the number of neighbors can vary for each time step. At each time step, several linear regressions are performed with varying numbers of neighbors, and the models are smoothed and mixed to provide the best fit to the data while still effectively smoothing.
The image below shows a time series with 200 time steps. The gray points are the original noisy time series values, and the red and blue lines each represent a smoothing result with a fixed number of neighbors. The red line uses 20 neighbors and does not effectively smooth short-term fluctuations in the data, which is especially apparent on the right side of the graph where the red line is jagged and unsmooth. The blue line uses 80 neighbors and is too smooth to reach the peak and valley of the data in the first half of the time series. The green line represents an optimal mixture of the red and blue lines that maintains appropriate levels of smoothness across the entire time series. The green line uses more neighbors in sections of the time series where the red line is jagged and fewer neighbors in sections where the blue line is too smooth.
In some sections of the time series, the green line is closer to the red line, and in other sections, the green line is closer to the blue line, depending on which fits the time series better at that time step. The image below shows the time series zoomed in around time step 134. The red line falls closer to the middle of the cloud of points than the blue line, so the green line is closer to the red line than the blue line.
The number of neighbors used at a time step is defined by a linear interpolation between the number of neighbors of the red and blue lines, weighted by the line that provides the better fit. The image below shows that the red line fits best for most time steps before approximately time step 150, and the blue line fits best for most time steps after time step 150. The optimal number of neighbors for time step 134 is 26.4, closer to 20 neighbors than 80 neighbors.
For a complete description of adaptive bandwidth local linear regression, see the following reference:
- Friedman, J. H. (1984). "A variable span smoother." USDOE Office of Science (SC). SLAC-PUB-3477. https://doi.org/10.2172/1447470