Terrain datasets are designed to scale projects involving hundreds of millions, even billions, of points. Terrain tools facilitate the use of large point collections, such as lidar, that would normally pose a problem to databases. Scalability is achieved primarily through terrain pyramids and the multipoint shape type.
Terrain pyramiding improves performance by providing a scale-dependent means of data reduction. Pyramids reference only the data needed to construct a surface of an approximate accuracy. On-the-fly surface construction, display, and analysis are faster for smaller-scale applications because only a thinned subset of the data is required. The original data is not moved or averaged in any way. The exact positional information of the measurements is maintained. Two types of pyramids can be used to build a terrain dataset: z-tolerance and window size.
Using the z-tolerance pyramid type, pyramiding is accomplished through the application of a z-tolerance-based filter that is used to thin points. You eliminate noncritical points to produce derivative surfaces that are within an approximate vertical accuracy relative to the full-resolution data.
With the window size pyramid type, pyramiding is carried out through the designation of a window size filter. It thins points for each pyramid level by partitioning the data into equal areas (windows) and selecting just one or two points from each area as representatives. It essentially controls horizontal sample density with a controllable bias toward high points, low points, or average-height points.
Additionally, the enforcement of lines and polygons is controlled on a per-pyramid-level basis. For example, breakline enforcement can be restricted to the highest one or two resolution pyramid levels. Some features, such as study area boundaries and lake shorelines, may need representation through all scales but not at the same detail. Generalized representations can be used at coarse scales, while the full detail is only applied at larger scales.
Each pyramid level has an assigned vertical tolerance or window size and a scale threshold. This is used to control the scale range associated with each level when the terrain dataset is displayed on a map. The number of pyramid levels, their tolerances, and their thresholds are all user definable.
Terrain pyramids are cumulative. Individual levels within a pyramid do not contain separate and independent sets of all the measurements they need. Rather, to go from a coarse-level pyramid to a more refined level involves adding measurements to those belonging to the coarse level. The full-resolution level is the sum of all the lower-level measurements plus a few more. This improves performance when using a terrain and reduces storage overhead.
Terrain pyramids are defined using the Create Terrain geoprocessing tool. For more information on creating a terrain dataset, see Create a terrain dataset with geoprocessing tools.
Z-tolerance pyramid type
The z-tolerance pyramid type controls the vertical accuracy of each pyramid level relative to the full-resolution data. The vertical accuracy of a pyramid level is always relative to the accuracy of the full-resolution source data. For example, if the source data has a known vertical accuracy of 0.5 feet and the z-tolerance of the first pyramid is 1 foot, the absolute accuracy of the first pyramid is 1.5 feet.
You must determine how many pyramid levels you require and also the z-tolerance for each. The primary factors that influence these decisions are the scale range of use of the terrain dataset, the z-range, and the variability of height in the terrain. One method you can use to define your pyramid levels follows the contour map model.
Define z-tolerance pyramid levels
To define pyramid levels using the contour map model, do the following:
- Consider a standard set of map scales you will use for constructing contour maps from the terrain.
- Arrange the scales from largest to smallest. Note the contour interval that is appropriate for each scale. Have your terrain dataset pyramid mimic this collection.
- Define a pyramid level for each map scale, setting the scale threshold for each level to the corresponding map scale. Set the z-tolerance to half the contour interval that is used at that scale.
In the pyramid definition example below, the full-resolution data can be used for display scales larger than 1:5,000. The pyramid level based on 0.5-unit z-tolerance can be used between 1:5,000 and 1:12,000, the 1.0-unit z-tolerance level between 1:12,000 and 1:24,000, the 2.5-unit z-tolerance level between 1:24,000 and 1:100,000, and the 5.0-unit z-tolerance level at scales smaller than 1:100,000.
Contour map series example on which to base z-tolerance pyramid levels
| Map scale | Contour interval (meters) |
|---|---|
1:5,000 | 1 |
1:12,000 | 2 |
1:24,000 | 5 |
1:100,000 | 10 |
Terrain dataset scale threshold levels and corresponding terrain dataset z-tolerance pyramid levels
| Scale threshold | Z-tolerance (meters) |
|---|---|
5,000 | 0.5 |
12,000 | 1 |
24,000 | 2.5 |
100,000 | 5 |
Window size pyramid type
Pyramid-level resolution is defined by the window size. The window size pyramid type thins points for each pyramid level by partitioning the data into equal areas (windows) and selecting one or two points from each area as representatives.
Point selection for each window is based on one of the following criteria:
- The point with the minimum z-value
- The point with the maximum z-value
- Two points to capture both the z-minimum and z-maximum
- The point closest to the mean z-value
Pyramid-level resolution is defined by the window size. This is the length of the side of each square area defining the subdivision. Coarser-resolution pyramid levels are defined with large window sizes. A large window size results in relatively few areas from which to select points. Since only one or two points are picked for each area, there will be lots of thinning and generalization. Finer-resolution pyramid levels are defined using smaller window sizes. Smaller windows mean more areas, and thus more points, less thinning, and more detail.
Like the z-tolerance-based pyramid, the window size pyramid is cumulative. Points used for a pyramid level are the sum of all the points selected for coarser levels plus an additional set unique to the given level. Cumulative pyramids are storage efficient since a separate and complete copy of data is not needed for each pyramid level. The images below show how the points are selected in each window.
  |
  |
Point selection method recommendations
The highest-resolution pyramid level must use a window size that is equal to or larger than the average point spacing. If you know there are many points that are closer than the average, use the z-mean because it may effectively thin some points. Otherwise, use a value that is two times the average point spacing.
The exception is if you're using the z-minimum/z-maximum point selection method, in which you must use four times the average spacing. The coarsest pyramid level must have a window size that is based on the x- or y-extent of the terrain. Somewhere between 1/500 to 1/1000 the larger of the x- and y-extent is recommended. The most efficient pyramids are made with window sizes that are a power of two from one another. Determine the smallest window size first and proceed from there.
The selection criteria are used to determine which points are chosen as representatives for the corresponding areas of the different pyramid levels. Each criterion offers a bias that is useful for a certain type of data or application. Note that the bias does not classify or have an impact on the full-resolution pyramid level.
| Method | Goals | Suggested applications |
|---|---|---|
Z Minimum | Bias toward local lows, rivers, valleys |
|
Z Maximum | Bias toward local highs, ridges, hilltops |
|
Z Minimum/Z Maximum | Captures extremes; does not thin as much as other options |
|
Z Mean | Avoids extremes |
|
Secondary thinning
When using the window size pyramid, you can include secondary thinning. This can reduce the number of points for a pyramid level above and beyond the thinning accomplished through the window filtering. This works starting from the coarsest pyramid level window size by examining the data for each window. If the range of z-values for points in the window is within a user-defined threshold, the area is considered flat. One or two points are selected for the area, as is normal for window size processing, but all the remaining points are assigned to the full-resolution pyramid level rather than being refiltered by the remaining levels. Since the area is flat, there's no need to select additional points with smaller window sizes.
Secondary thinning method recommendations
When enabled, secondary thinning reduces the number of points used over flat areas. An area is considered flat if the heights of points within the area are within a user-specified secondary thinning threshold. Its effect is more evident at higher-resolution pyramid levels, since smaller areas are more likely to be flat than larger areas.
The secondary thinning threshold should be set at least as large as the vertical accuracy of the data to get over its noise floor. As you specify larger values, you'll thin more points and will realize some performance gain, but your ability to resolve or distinguish surface features will decrease.
- Mild Thinning—Works best to preserve linear discontinuities (for example, building sides and forest boundaries). It is recommended for lidar that includes both ground and nonground points. Mild thinning will thin the fewest points.
- Moderate Thinning—Provides a good trade-off between performance and accuracy. It does not preserve as much detail as mild thinning but comes nearly as close, while eliminating more points overall. Moderate thinning is a good thinning method for all types of data.
- Strong Thinning—Removes the most points but is less likely to preserve sharply delineated features. Its use should be limited to surfaces where slope tends to change gradually. For example, strong thinning would be efficient for bare-earth lidar and bathymetry.
Window size pyramid level creation
Base the pyramid levels on the following information:
- The average point spacing of the point data is 1 meter.
- There is not a large variance in the point spacing, so most points are around 1 meter apart.
- The data extent is 20 kilometers east to west and 10 kilometers north to south.
The following example starts with a window size of 2 (meters) and increases by powers of two: 2, 4, 8, 16, 32. It stops at 32 since it falls between 1/500 and 1/1000 the extent of 20 kilometers. For each window size, the example uses a scale threshold that is twice the size of the previous scale threshold, resulting in the following pyramid definition.
Sample pyramid definition for 1-meter point spacing data
| Window size | Scale |
|---|---|
2 | 3,000 |
4 | 6,000 |
8 | 12,000 |
16 | 24,000 |
32 | 48,000 |