Available with Geostatistical Analyst license.
Geostatistics assumes that all values in your study area are the result of a random process. A random process does not mean that all events are independent, as with each flip of a coin. Geostatistics is based on random processes with dependence.
In a spatial or temporal context, such dependence is called autocorrelation.
Prediction for random processes with dependence
In geostatistics, there are two key tasks: to uncover the dependency rules and to make predictions. The predictions come from first knowing the dependency rules.
Kriging is based on two tasks: (1) semivariogram and covariance functions estimation of the statistical dependence values (called spatial autocorrelation) and (2) prediction using generalized linear regression techniques (kriging) of unknown values. Because of these two distinct tasks, it has been said that geostatistics uses the data twice: first to estimate the spatial autocorrelation and second to make the predictions.
In general, statistics rely on some notion of replication, where it's believed estimates can be derived and the variation and uncertainty of the estimates can be understood from repeated observations.
In a spatial setting, the idea of stationarity is used to obtain the necessary replication. Stationarity is an assumption that is often reasonable for spatial data. There are two types of stationarity. One is mean stationarity, where it's assumed that the mean is constant between samples and is independent of location.
The second type of stationarity is second-order stationarity for covariance and intrinsic stationarity for semivariograms. Second-order stationarity is the assumption that the covariance is the same between any two points that are at the same distance and direction apart no matter which two points you choose. The covariance is dependent on the distance between any two values and not on their locations. For semivariograms, intrinsic stationarity is the assumption that the variance of the difference is the same between any two points that are at the same distance and direction apart no matter which two points you choose.
Second-order and intrinsic stationarity are assumptions necessary to get the replication to estimate the dependence rules, which allows you to make predictions and assess uncertainty in the predictions. Notice that it is the spatial information (similar distance between any two points) that provides the replication.