For large projects, the ability to scale is of utmost importance. Terrain datasets are designed to do just this. They can handle projects involving hundreds of millions, even billions, of points. Terrain tools facilitate the use of large point collections, such as lidar, that would normally pose a problem to databases. Scalability is achieved primarily through two means: terrain pyramids and the multipoint shape type.
Terrain pyramiding is used to improve performance. It does so by providing a scale-dependent means of data reduction. Pyramids reference only the data needed to construct a surface of an approximate accuracy. On-the-fly surface construction, display, and analysis are faster for smaller-scale applications because only a thinned subset of the data is required. The original data is not moved or averaged in any way. The exact positional information of the measurements is maintained. Two types of pyramids can be used to build a terrain dataset: z-tolerance and window size.
Using the z-tolerance pyramid type, pyramiding is accomplished through the application of a z-tolerance-based filter that is used to thin points. You eliminate noncritical points to produce derivative surfaces that are within an approximate vertical accuracy relative to the full-resolution data.
With the window size pyramid type, pyramiding is carried out through the designation of a window size filter. It thins points for each pyramid level by partitioning the data into equal areas (windows) and selecting just one or two points from each area as representatives. It essentially controls horizontal sample density with a controllable bias toward high points, low points, or average-height points.
Additionally, the enforcement of lines and polygons is controlled on a per-pyramid-level basis. For example, breakline enforcement can be restricted to the highest one or two resolution pyramid levels. Some features, such as study area boundaries and lake shorelines, might need representation through all scales but not at the same detail. Generalized representations can be used at coarse scales, while the full detail is only applied at larger scales.
Each pyramid level has an assigned vertical tolerance or window size and a scale threshold. This is used to control the scale range associated with each level when the terrain dataset is displayed on a map. The number of pyramid levels, their tolerances, and their thresholds are all user definable.
Terrain pyramids are cumulative. Each level within a pyramid does not contain a separate and independent set of all the measurements it needs. Rather, to go from a coarse-level pyramid to a more refined level involves adding measurements to those belonging to the coarse level. The full-resolution level is really the sum of all the lower-level measurements plus a few more. This helps to improve performance when using a terrain and reduces storage overhead.
Terrain pyramids are defined using the Create Terrain geoprocessing tool. For more information on creating a terrain dataset, see Creating a terrain dataset in ArcGIS Pro.
Z-tolerance pyramid type
The z-tolerance pyramid type controls the vertical accuracy of each pyramid level relative to the full-resolution data. The vertical accuracy of a pyramid level is always relative to the accuracy of the full-resolution source data. For example, if the source data has a known vertical accuracy of 0.5 feet and the z-tolerance of the first pyramid is 1 foot, the absolute accuracy of the first pyramid is 1.5 feet.
You must determine how many pyramid levels you require and also the z-tolerance for each. The primary factors that influence these decisions are the scale range of use of the terrain dataset, the z-range, and the variability of height in the terrain. One method you can use to define your pyramid levels follows the contour map model.
Defining z-tolerance pyramid levels
To define pyramid levels using the contour map model, do the following:
- Consider a standard set of map scales you would use for constructing contour maps from the terrain.
- Arrange the scales from largest to smallest. Note the contour interval that would be appropriate for each scale. Have your terrain dataset pyramid mimic this collection.
- Define a pyramid level for each map scale, setting the scale threshold for each level to the corresponding map scale. The z-tolerance should be set to half the contour interval that would be used at that scale.
Given the pyramid definition below, the full-resolution data would be used for display scales larger than 1:5,000. The pyramid level based on 0.5-unit z-tolerance would be used between 1:5,000 and 1:12,000, the 1.0-unit z-tolerance level between 1:12,000 and 1:24,000, the 2.5-unit z-tolerance level between 1:24,000 and 1:100,000, and the 5.0-unit z-tolerance level at scales smaller than 1:100,000.
Contour map series example on which to base z-tolerance pyramid levels
| Map scale | Contour interval (meters) |
|---|---|
1:5,000 | 1 |
1:12,000 | 2 |
1:24,000 | 5 |
1:100,000 | 10 |
Terrain dataset scale threshold levels and corresponding terrain dataset z-tolerance pyramid levels
| Scale threshold | Z-tolerance (meters) |
|---|---|
5,000 | 0.5 |
12,000 | 1 |
24,000 | 2.5 |
100,000 | 5 |
Window size pyramid type
Pyramid-level resolution is defined by the window size. The window size pyramid type thins points for each pyramid level by partitioning the data into equal areas (windows) and selecting just one or two points from each area as representatives.
Point selection for each window is based on one of the following criteria:
- The point with the minimum z-value
- The point with the maximum z-value
- Two points to capture both the z-minimum and z-maximum
- The point closest to the mean z-value
Pyramid-level resolution is defined by the window size. This is the length of the side of each square area defining the subdivision. Coarser-resolution pyramid levels are defined with large window sizes. A large window size results in relatively few areas from which to select points. Since only one or two points are picked for each area, there will be lots of thinning and generalization. Finer-resolution pyramid levels are defined using smaller window sizes. Smaller windows mean more areas, and thus more points, less thinning, and more detail.
Like the z-tolerance-based pyramid, the window size pyramid is cumulative. Points used for a pyramid level are the sum of all the points selected for coarser levels plus an additional set unique to the given level. Cumulative pyramids are storage efficient since a separate and complete copy of data is not needed for each pyramid level.
Point selection method recommendations
The highest-resolution pyramid level should use a window size that is equal to or larger than the average point spacing. If you know there are many points that are closer than the average, the z-mean would be a good value to use because it could effectively thin some points. Otherwise, use a value that is two times the average point spacing.
The exception is if you're using the z-minimum/z-maximum point selection method, in which you should use four times the average spacing. The coarsest pyramid level should have a window size that is based on the x- or y-extent of the terrain. Somewhere between 1/500 to 1/1000 the larger of the x- and y-extent is reasonable. The most efficient pyramids are made with window sizes that are a power of two from one another. Determine the smallest window size first and proceed from there.
The selection criteria are used to determine which points are chosen as representatives for the corresponding areas of the different pyramid levels. Each criterion offers a bias that is useful for a certain type of data or application. Note that the bias does not classify or have an impact on the full-resolution pyramid level.
| Method | Goals | Suggested applications |
|---|---|---|
Z Minimum | Bias toward local lows, rivers, valleys |
|
Z Maximum | Bias toward local highs, ridges, hilltops |
|
Z Minimum/Z Maximum | Captures extremes; does not thin as much as other options |
|
Z Mean | Avoids extremes |
|
Secondary thinning
When using the window size pyramid, there is an option to include secondary thinning. This can reduce the number of points for a pyramid level above and beyond the thinning accomplished through the window filtering. This works starting from the coarsest pyramid level window size by examining the data for each window. If the range of z-values for points in the window is within a user-defined threshold, the area is considered flat. One or two points are selected for the area, as is normal for window size processing, but all the remaining points are assigned to the full-resolution pyramid level rather than being refiltered by the remaining levels. Since the area is flat, there's no need to select additional points with smaller window sizes.
Secondary thinning method recommendations
When enabled, secondary thinning reduces the number of points used over flat areas. An area is considered flat if the heights of points within the area are within a user-specified secondary thinning threshold. Its effect is more evident at higher-resolution pyramid levels, since smaller areas are more likely to be flat than larger areas.
The secondary thinning threshold should be set at least as large as the vertical accuracy of the data to get over its noise floor. As you specify larger values, you'll thin more points and will realize some performance gain, but your ability to resolve or distinguish surface features will decrease.
- Mild Thinning—Works best to preserve linear discontinuities (for example, building sides and forest boundaries). It is recommended for lidar that includes both ground and nonground points. Mild thinning will thin the fewest points.
- Moderate Thinning—Provides a good trade-off between performance and accuracy. It does not preserve as much detail as mild thinning but comes nearly as close, while eliminating more points overall. Moderate thinning is a good thinning method for all types of data.
- Strong Thinning—Removes the most points but is less likely to preserve sharply delineated features. Its use should be limited to surfaces where slope tends to change gradually. For example, strong thinning would be efficient for bare-earth lidar and bathymetry.
Window size pyramid level creation example
Base the pyramid levels on the following information:
- The average point spacing of the point data is 1 meter.
- There is not a large variance in the point spacing, so most points are around 1 meter apart.
- The data extent is 20 kilometers east to west and 10 kilometers north to south.
- Start with a window size of 2 (meters) and increase by powers of two: 2, 4, 8, 16, 32. Stop at 32 since it falls between 1/500 and 1/1000 the extent of 20 kilometers.
- For each window size, use a scale threshold that is twice the size of the previous scale threshold. You should end up with a pyramid definition as indicated below.
Sample pyramid definition for 1-meter point spacing data
| Window size | Scale |
|---|---|
2 | 3,000 |
4 | 6,000 |
8 | 12,000 |
16 | 24,000 |
32 | 48,000 |