Batch Import Data (Intelligence)

Summary

Imports directories of KML, KMZ, shapefiles, Excel worksheets, tabular text files, GeoJSON, and GPX files to feature classes stored in a single geodatabase.

Usage

  • The Input Data location will be recursively searched, by default, for KML (.kml), KMZ (.kmz), shapefiles (.shp), Excel workbooks (.xlsx and .xls), tabular text files (.csv, .txt, and .tab), GeoJSON files (.geojson), and GPX files (.gpx). The Filter parameter can be used to limit which files are imported.

  • Rasters, or ground overlays contained in a KML or KMZ file, are converted into a mosaic dataset within the Target Geodatabase. The source rasters referenced by the mosaic dataset are stored in their native format in a subfolder named after the mosaic dataset. This is located at the same location as the target geodatabase. These files are named after the Target Geodatabase and appended with _Rasters. For example, if your Target Geodatabase is C:\Data\OperationalData.gdb, a folder named OperationalData_Rasters will be created under C:\Data.

  • If you are importing files into an enterprise geodatabase, make sure the database connection file (.sde) is specified as a UNC path in the Target Geodatabase parameter and the path is accessible by the enterprise geodatabase.

  • KML and KMZ input up to KMZ version 2.2 of the Open Geospatial Consortium KML standard is generally supported. Point locations that use the address tab (by way of geocoding) are not supported. A valid latitude and longitude location is required inside the source KML.

  • The standard delimiter for tabular text files with .csv and .txt extensions is a comma, and for files with a .tab extension, a tab. To use an input table with a nonstandard delimiter, the delimiter must be specified using a schema.ini file. See Schema.ini File (Text File Driver) in the Microsoft documentation for more information.

  • For input Excel worksheets and tabular text files, only point geometry is supported.

  • The GeoJSON specification supports multiple geometry types. A separate feature class will be created for each geometry type in the input GeoJSON file using the naming convention SourceGeoJSONFile_GeometryType. For example, if the GeoJSON file is named World Airports.geojson and contains point, multipoint, polyline, and polygon geometry types, the feature classes will be named World Airports_Points, World Airports_Multipoint, World Airports_Polylines, and World Airports_Polygons.

  • Coordinate systems have the following considerations when running this tool:

    • For shapefiles, KML, KMZ, GeoJSON, and GPX files, the coordinate system of the output dataset will be the same as the input dataset. The KML and GPX specifications mandate the coordinate system of the input dataset to be GCS WGS1984.
    • For Excel workbooks and tabular text files, the coordinate system of the input is assumed to be GCS WGS1984 and the longitude and latitude coordinate values are in decimal degrees.

  • If any of the input longitude or latitude coordinates for a feature are invalid (null or nonnumeric values), the corresponding feature will have a null geometry. The Check Geometry tool can be used to check for null geometries in the output.

  • Tabular text files (.csv, .txt, and .tab) and Excel worksheets must have a longitude and latitude column. Valid longitude columns are longitude, long, lon, x_coordinate, x_coord, and x. Valid latitude columns are latitude, lat, y_coordinate, y_coord, and y. If the source has multiple valid longitude and latitude columns, the first valid column is used.

  • All worksheets in an Excel workbook will be imported.

  • Symbology is applied to the output layers added to the map based on the symbology in the source data files as follows:

    Note:
    An active map must exist in the ArcGIS Pro project at the time of tool execution to maintain the symbology. If the tool is running outside an ArcGIS Pro project—in other words, from a stand-alone Python script—the symbology will not be maintained.
    • For output layers created from KML or KMZ files, the symbology is extracted from the source KML or KMZ file and applied to the output layers. This symbology can be overridden by providing a layer file (.lyrx or .lyr) in the same folder as the source KML or KMZ file. The layer file must have the same base name as the source data file, appended with an underscore (_), and the geometry type (points, polylines, or polygons) of the layer to be overridden. For example, if the KML file is named World Airports.kml and contains point, line, and polygon features and you want to override the symbology for all geometry types, the layer files would be named World Airports_Points.lyrx, World Airports_Polylines.lyrx, and World Airports_Polygons.lyrx. The geometry type values are case sensitive. If a layer file does not exist for a given geometry type, the symbology from the source KML or KMZ file is applied to the output layer.
    • For output layers created from sources other than KML or /KMZ files, symbology is applied to the output layer if a layer file (.lyrx or .lyr) exists in the same folder as the source data file. The layer file must have the same base name as the source data file, except Excel worksheets and GeoJSON files. For Excel worksheets, the layer file must be named the same as the base name of the source data file, appended by an underscore (_) and the worksheet name. For example, if the Excel file is named World.xlsx and contains a worksheet named Cities (2018), the layer file must be named World_Cities (2018).lyrx or World_Cities (2018).lyr. For GeoJSON files, the layer file must be named the same as the base name of the source data file, appended by an underscore (_) and the output geometry type—points, multipoint, polylines, and polygons). If a layer file does not exist, a default symbol will be applied to the output layer.

Syntax

arcpy.intelligence.BatchImportData(in_data, target_gdb, {filter}, {include_sub_folders}, {include_groundoverlay})
ParameterExplanationData Type
in_data
[in_data,...]

The directories containing the data files to convert to geodatabase feature classes.

Folder
target_gdb

The target geodatabase where output feature classes will be stored.

Workspace
filter
(Optional)

A filter for the data being added to the geodatabase. The wildcard characters for the filter work on the full path to the input data and are listed below:

  • *—Matches any characters
  • ?—Matches a single character
  • [ range ]—Matches a single character in the range
  • [! range ]—Matches any character not in the range
Multiple patterns can be added to the filter by separating each pattern with the vertical bar/pipe delimiter (|); no spaces should exist before or after the delimiter. Pattern comparisons are not case sensitive, so using the *airport.shp, *AIRPORT.SHP, or *Airport.shp pattern will import the same shapefile.

The following are filter examples:

  • To import all shapefiles, use *.shp.
  • To import all shapefiles and all KML files, use *.shp|*.kml.
  • To import all files that have airport in the file path or file name, use *airport*.
  • To import all GeoJSON files that have airport in the file path or file name, use *airport*.geojson.
  • To import all KMZ files that have airport appended with any two characters in the name, use *airport??.kmz.
  • To import all files that have 1990 through 1997 in the file path or file name, use *199[0-7]*.
  • To import all shapefiles that have the exact folder name airfacilities in their file path, use *\airfacilities\*.shp.

String
include_sub_folders
(Optional)

Specifies whether subfolders will be recursively explored.

  • SUBFOLDERSAll subfolders will be explored This is the default.
  • NO_SUBFOLDERSOnly the top-level folder will be explored.
Boolean
include_groundoverlay
(Optional)

Specifies whether KML or KMZ ground overlay (raster, air photos, and so on) is included in the output.

Use caution if the KMZ points to a service that serves raster imagery. The tool will attempt to translate the raster imagery at all available scales. This process could be lengthy and possibly overwhelm the service.

  • GROUNDOVERLAYGround overlays are included in the output. This is the default.
  • NO_GROUNDOVERLAYGround overlays are not included in the output.
Boolean

Derived Output

NameExplanationData Type
out_geodatabase

The updated target geodatabase.

Workspace

Code sample

BatchImportData example 1 (Python window)

The following Python window script demonstrates how to use the BatchImportData function in immediate mode:

import arcpy 

arcpy.BatchImportData_intelligence("c:/data/sourcedata", "c:/data/output.gdb", "", "SUBFOLDERS", "NO_GROUNDOVERLAY")
BatchImportData example 2 (stand-alone script)

The following Python script demonstrates how to use the BatchImportData function in a stand-alone script:

# Name: BatchImportData_Example2.py
# Description: Imports all shapefiles with 'airport' in the file name or file
#              path within a directory to feature classes in a single geodatabase.
# Requirements: Standard or Advanced license

# Import system modules
import arcpy

# Set local variables
in_data = ["c:/data/sourcedata"]
target_gdb = r"\\data\operationaldata.sde" 
filter = "*airports*.shp" 
include_sub_folders = "NO_SUBFOLDERS" 

# Execute BatchImportData
arcpy.BatchImportData_intelligence(in_data, target_gdb, filter, include_sub_folders)

Environments

This tool does not use any geoprocessing environments.

Licensing information

  • Basic: No
  • Standard: Yes
  • Advanced: Yes

Related topics