Create geoprocessing modules

Custom and Python toolboxes can be distributed as Python packages. The process for building and distributing a toolbox involves creating a Python package, and extending the package structure with an esri folder to include your own toolboxes. Once the package is installed in a Python environment, your toolbox will appear as and be accessible in all the ways that a system toolbox within ArcGIS Pro would, accessible through the Geoprocessing pane and from arcpy. In addition, a custom geoprocessing module can take advantage of the well-established methodology that ArcGIS Pro system toolboxes have for message distribution, language-based help, and respond to localized settings.

Create a Python package

For illustrative purposes, we will create a Python package called foo that contains the module bar.py

bar.py

Sample code for the bar Python module.

import os 
 
def hello(): 
    return f'Hello {os.getenv("username")}'

For the package to be built and distributed, a specific directory structure must exist, a directory named foo must be created to store the bar module. Since distribution requires that the directory storing the bar module be within a parent directory, a directory named src is created to house the foo directory and the bar module. The directory structure will be as follows:

src
└───foo
     └  bar.py

For the bar module to initialize and execute specific code once it has been imported, it requires an __init__.py file. By using an __init__.py file, the foo directory is treated as a package by Python.

__init__.py

Sample code for __init__.py in foo.

from foo import bar

The directory structure will now be the following:

src
└───foo
     ├  __init__.py 
     └  bar.py

Note:

Throughout the process, additional __pycache__ folders may appear in the directories. These folders are caches of compiled versions of modules generated by Python to speed up the loading of modules. To simplify the folder structure diagrams, they are not shown here.

With the files and directory structure, the bar module can be imported through import foo, and the foo.bar.hello() function can be executed.

Note:

To import a module in Python, the module must be within the current directory, a directory listed in sys.path, or a directory listed with the PYTHONPATH environment variable.

Extend a Python package with geoprocessing tools

To extend a Python package with custom or Python toolboxes, the toolboxes and accompanying files are organized into a module in an esri folder. The standard directory structure for a Python package extended with custom geoprocessing functionality is the following:

Src
└──foo
    ├  __init__.py 
    ├  bar.py
    └──esri
        ├──arcpy
        ├──help
        │   └──gp
        │       ├──messages
        │       └──toolboxes
        └──toolboxes

Custom toolboxes are placed in the esri\toolboxes directory along with any supporting .py files. The toolbox and tool metadata (.xml) files are stored in the esri\help\gp folder. The naming convention for the toolbox .xml file is the toolbox alias followed by _toolbox.xml, and for tools, the naming convention is the tool name and toolbox alias separated by an underscore and with an .xml extension. Any geoprocessing message .xml files are stored in the esri\help\gp\messages directory. The message files are used within the Python toolboxes for messages that need to be internationalized.

The process of extending geoprocessing through Python packages can be demonstrated through the creation of a Python toolbox named SamplePythonToolbox.

Learn more about creating a Python toolbox

SamplePythonToolbox.pyt

Sample code to create a Python toolbox.

# -*- coding: utf-8 -*-

import arcpy
import os
import foo

class Toolbox(object):
    def __init__(self):
        """Define the toolbox (the name of the toolbox is the name of the
        .pyt file)."""
        self.label = "Sample Python Toolbox"
        self.alias = "SamplePythonToolbox"

        # List of tool classes associated with this toolbox
        self.tools = [SampleTool]

class SampleTool(object):
    def __init__(self):
        """Define the tool (tool name is the name of the class)."""
        self.label = "Sample Tool"
        self.description = ""
        self.canRunInBackground = False

    def getParameterInfo(self):
        """Define parameter definitions"""
        parameters=[arcpy.Parameter(displayName='Msg', 
                                  name='msg',
                                  datatype='GPString',
                                  parameterType='Derived',
                                  direction='Output')
                                  ]
        return parameters

    def isLicensed(self):
        """Set whether tool is licensed to execute."""
        return True

    def updateParameters(self, parameters):
        """Modify the values and properties of parameters before internal
        validation is performed.  This method is called whenever a parameter
        has been changed."""
        return

    def updateMessages(self, parameters):
        """Modify the messages created by internal validation for each tool
        parameter.  This method is called after internal validation."""
        return

    def execute(self, parameters, messages):
        """The source code of the tool."""
        result = foo.bar.hello()
        messages.AddMessage(f"{result}, welcome to the sample tool")
        parameters[0].value = result
        return

Note:

To test the toolbox at this stage, temporarily place it into the root src folder. Since foo is being imported in the toolbox, the tool will likely not execute properly unless it is in the same directory as the foo package.

In the SamplePythonToolbox.pyt toolbox, the foo package is imported and the execute method of the SampleTool class calls the hello()  function from the bar module.

This approach is an effective way of distributing custom Python code as a package and exposing its functionality through ArcGIS Pro geoprocessing tools. Once the SamplePythonToolbox.pyt has been created, the parameter help can be configured for the toolbox through the metadata. To edit a tool's metadata, from the Catalog pane, right-click the tool in the toolbox and choose Edit Metadata.

After the metadata is updated for the tool and toolbox, the arcpy.gp.createtoolboxsupportfiles function can be run to generate the supporting .xml files found within the esri folder structure described above. This command also produces an arcpy wrapper, SamplePythonToolbox.py named after the tool, that allows the toolbox to be accessible from arcpy. Place the SamplePythonToolbox.pyt in a separate staging folder, then run arcpy.gp.createtoolboxsupportfiles(<path to toolbox>).

Note:

Make sure that the toolbox has an alias before running arcpy.gp.createtoolboxsupportfiles. The alias is needed to generate the help files.

Within the esri folder, create a toolboxes folder and place SamplePythonToolbox.pyt and its accompanying .xml files into it. The staging folder should now look like the following:

Staging
 └──esri
     ├──arcpy
     │   └  SamplePythonToolbox.py
     ├──help
     │   └──gp
     │       ├  SamplePythonToolbox_toolbox.xml
     │       ├  SampleTool_SamplePythonToolbox.xml
     │       ├──messages 
     │       └──toolboxes
     │           └  SamplePythonToolbox.xml
     └──toolboxes
         ├  SamplePythonToolbox.pyt
         ├  SamplePythonToolbox.pyt.xml
         └  SamplePythonToolbox.SampleTool.pyt.xml

Note:

The messages folder is not generated by arcpy.gp.createtoolboxsupportfiles; it must be created manually to make the error messages used in the script tools or Python toolboxes localizable, but can be omitted if there is no plan for internationalization.

The esri folder must be copied over from the staging directory where it was created into the src\foo directory. The directory structure for the distribution will now be as follows:

Src
└──foo
    ├  __init__.py 
    ├  bar.py
    └──esri
        ├──arcpy
        │   └  SamplePythonToolbox.py
        ├──help
        │   └──gp
        │       ├  SamplePythonToolbox_toolbox.xml
        │       ├  SampleTool_SamplePythonToolbox.xml
        │       ├──messages
        │       └──toolboxes
        │           └  SamplePythonToolbox.xml
        └──toolboxes
            ├  SamplePythonToolbox.pyt
            ├  SamplePythonToolbox.pyt.xml
            └  SamplePythonToolbox.SampleTool.pyt.xml

For English language distributions, no additional steps are required. For more information on extending geoprocessing for distribution in languages other than English, see Internationalization of geoprocessing modules. To distribute your module, see Distribute a geoprocessing module.