Geoprocessing considerations for co-locating analysis processing and data

Web layers from an ArcGIS portal and services from ArcGIS Server are supported data formats in ArcGIS Pro. You can perform most of the same tasks with web layers as you can with layers residing in a local folder or geodatabase, including visualization and exploration in a map, editing, and analysis. Web feature layers and web image layers from your portal and feature services and image services from ArcGIS Server are supported as input to many geoprocessing and analysis tools that support local feature classes and raster datasets.

ArcGIS Pro provides a number of geoprocessing tools, called portal tools, that can be used with input data in your portal, as the tools run on the analysis servers in your enterprise, and create new layers (services) as output of the tools. These portal tools work best with hosted web layers, and desktop geoprocessing tools work best with desktop data sources.

Generally speaking, the data to be used for analysis should be co-located with where the processing will take place. This means that if the data is in your ArcGIS Enterprise, it is best to run portal analysis tools. If the data is on your desktop, it is best to run desktop tools.

Note:

If you have a large web layer and want to perform analysis using desktop geoprocessing tools, it is recommended that you download the web layer to your desktop once, and perform the analysis on the local copy. This will ensure that if you need to run the analysis multiple times or use the same data as input to multiple tools, you will have the local copy and performance will be best with that local data. Use one of the following methods to download a web layer as a desktop GIS dataset:

  • In a web browser, on the portal item page, use the Export command.
  • In ArcGIS Pro, use the Copy Features or Copy Raster tool copy a web layer into a folder or geodatabase. Both of these tools support the Extent geoprocessing environment to apply a spatial filter to the download.

While web layers are supported by many of the desktop geoprocessing tools in ArcGIS Pro, and you can use desktop GIS data sources as input to the portal tools, consider the following before running an analysis in which the data and the analysis are not co-located:

  • Performance
  • Scalability
  • Output location
  • Permissions

Performance

When data and processing in an analysis are not co-located, there is a negative performance impact due to data transfer. Geoprocessing requires that data be in the same location as the processing, so the data will be automatically downloaded or uploaded if it is not co-located with the analysis. This data transfer takes time.

Using nonhosted web layers as input to portal tools increases this performance problem, as the data must be downloaded from one server to the desktop, and uploaded to another server to perform the analysis.

Note:

If you require one of the portal tools to perform analysis but your input data is in a local folder or geodatabase, share your local data as a web layer, and perform the analysis using the web layer as input. This will improve performance.

Scalability

Similar to performance, scalability will be impacted due to data transfer. If data download is required to run a desktop process on a web layer, the request to download the data may time out or fail part way through. If the data is large, the service may time out or reach a maximum amount of data to extract from the service. A tool running without co-located data may not be able to process as large a dataset compared to a properly co-located analysis.

Note:

The impact of co-location on performance and scalability is more significant with large datasets. With smaller datasets (for example, less than a few hundred features), co-location may not severely impact performance and scalability.

Output location

Regardless of the location of the input dataset, desktop tools create their outputs in a desktop GIS workspace (local folder or geodatabase), and portal tools create their outputs in the portal (as a new web layer representing services on an ArcGIS server). If you need an output to be shared with your enterprise but you ran a desktop analysis tool, it can introduce significant time to share your desktop tool output. It may be faster to run a portal tool, which will automatically create the output in the enterprise where it can be shared with your organization.

Permissions

A number of desktop geoprocessing tools modify the input dataset. For these tools to run using web layers as input, those layers often require specific capabilities or permissions to be able to modify the data. If you are not the owner of the portal item or do not have permission to update the data, the tool will fail.