Replicate Workflow Manager (Classic) repositories

Available with Workflow Manager license.

ArcGIS Workflow Manager (Classic) repository replication allows organizations to maintain multiple Workflow Manager (Classic) repositories locally in different geographic regions for performance reasons, while the users at each location continue to work on the same set of jobs. Workflow Manager (Classic) replication is not an extension of geodatabase replication. It is accomplished with Workflow Manager (Classic) services or configuration files. Workflow Manager (Classic) replication is a two-way replication.

You'll define a collection of Workflow Manager (Classic) repositories that will have identical Workflow Manager (Classic) contents after replication. This collection of repositories is known as a repository cluster.

One Workflow Manager (Classic) repository is designated as the parent repository and is tasked with coordinating the synchronization between each repository. After the parent repository is specified, each child repository is added. The child repositories can be designated as connected repositories, where the synchronization happens automatically, or disconnected repositories, where the synchronization occurs by creating files that can be manually imported.

• Repository Cluster—A cluster is a collection of Workflow Manager (Classic) repositories that participate in repository replication. Each repository is identified as a node. Nodes can be added to the cluster at any time. For example, a cluster might be created for the Workflow Manager (Classic) repositories at Redlands, California; Washington, D.C.; St. Louis, Missouri; and Denver, Colorado. The repositories at these locations must have Workflow Manager (Classic) installed, and the Create Workflow Database tool must have been executed to create the necessary tables. For the purpose of replication, the minimum configuration can be imported.
• Parent node—Each cluster must have a parent node. In connected replication, the parent node initiates all replication and synchronization processes. The parent node must be a repository that has the configuration you want to distribute to users in the other locations.
• Child node—A cluster can have more than one child node. The children have identical elements when replicas are created and changes are synchronized.
• Elements—Elements are considered configuration items like job types and step types. However, they are not limited to just configuration items. Your jobs are also elements, and they all get replicated and synchronized.

Note:

Workflow Manager (Classic) replication does not replicate and synchronize data workspaces and spatial notification rules between repositories.

• Last Sync date—This date is a property of each node in the cluster. It's used when you create a replica or synchronize changes. As changes are made to each of your elements, the date and time that it occurred are updated. If there is an element with a newer date and time stamp, the application imports that to the parent and pushes this change to all the children in the cluster.
• Connected or Not—This is the status of the node. A connected node means that it is online, and the replication and synchronization changes relay immediately. If the node is disconnected, specify a file location to export the configuration file and manually import it later using tools in the Workflow Manager (Classic) geoprocessing toolbox.

Replicas

Create replicas is the process of making all child repositories an identical copy of the parent repository. It exports the configuration from the parent repository, deletes existing configuration in child repositories, and imports the parent repository configuration in child repositories. Replicas can be created in Workflow Manager (Classic) Administrator or by using Workflow Manager (Classic) geoprocessing tools.

Synchronization of changes

Synchronization is the process of making the changes match between the parent and child repositories. Changes from child repositories are sent to the parent repository, and consolidated changes are sent to all child repositories. Changes can be synchronized in ArcGIS Workflow Manager (Classic) Administrator or by using Workflow Manager (Classic) geoprocessing tools.

Connected replication

Connected replication is ideal for locations that have network access as it requires ArcGIS Workflow Manager (Classic) Server and is accomplished using Workflow Manager (Classic) services. When using connected replication, the information is automatically exchanged using Workflow Manager (Classic) services with minimal user interaction. ArcGIS Workflow Manager (Classic) Server must be installed at each node, and the parent and child repositories must be published as Workflow Manager (Classic) services. There are two ways to set up connected replication:

• Workflow Manager (Classic) Administrator—Use the Manage Replication dialog box to create replicas and synchronize changes.
• Geoprocessing tools—Use Workflow Manager (Classic) geoprocessing tools to create replicas and synchronize changes.

Note:

Parent and child services must be online for connected replication.

Create Workflow Manager (Classic) services

Create Workflow Manager (Classic) services for all repositories participating in your cluster.

1. Start ArcGIS Pro.
2. Run the Publish Workflow Service geoprocessing tool to publish a workflow service for the parent repository.
3. Run the Publish Workflow Service geoprocessing tool to publish a workflow service for each child repository that participates in your cluster.

Create and add nodes to the cluster

Clusters are created on the parent repository in Workflow Manager (Classic) Administrator.

1. Start Workflow Manager (Classic) Administrator and connect to the parent repository.
2. Right-click the parent database connection and click Manage Replication.

   The Manage Replication dialog box appears and the repository name for the parent is automatically populated.

3. Leave the Connected check box checked.
4. Provide the URL of the parent repository’s Workflow Manager (Classic) service in the Workflow Manager Server REST URL text box.
5. Click the Add button to add a new child to the cluster.
6. Specify the repository name.
7. Click the Connected check box.
8. Provide the URL of the child repository’s Workflow Manager (Classic) service in the Workflow Manager Server REST URL text box.

   Note:

   Click the Save button at any time to store the added information.

9. Repeat steps 5 through 8 for all the other child nodes.
10. Click Save when you are finished adding repositories.

Create a new replica

The Create New Replica option sends the contents of the parent repository to all the other nodes in the cluster. This operation deletes the contents of all the child nodes; therefore, run this operation initially when defining the cluster.

1. Start Workflow Manager (Classic) Administrator and connect to the parent repository.
2. Right-click the database connection and click Manage Replication.
3. Click the Create New Replica button.

   This process may run for several minutes, depending on the size of the parent repository.

4. Click OK when replication completes.

Synchronize replicas

The synchronize process compares the differences between all the children in the cluster, imports them to the parent node, and sends the changes to all the other nodes in the cluster.

1. Start Workflow Manager (Classic) Administrator and connect to the parent repository.
2. Right-click the database connection and click Manage Replication.
3. Click the Synchronize Replicas button.

   This process may run for several minutes, depending on the number of changes in the parent and child repositories.

4. Click OK when synchronization completes.

Disconnected replication

Disconnected replication is ideal for locations where the network connectivity is a problem or when ArcGIS Workflow Manager (Classic) Server is not available at each location. It can be used when the parent repository is on a server but the child nodes are not connected to a server, or if both the parent and child repositories are not connected to a server. There are two ways to set up disconnected replication:

• Workflow Manager (Classic) Administrator—Use the Manage Replication dialog box to create a configuration file consisting of all elements and jobs in the parent repository.
• Geoprocessing tools—Use Workflow Manager (Classic) geoprocessing tools to create configuration files to create replicas and synchronize changes.

Disconnected replication—parent repository connected

Occasionally, in disconnected replication, the parent repository is connected to a server but the child nodes are not. In this case, the parent repository is published as a service and the configuration from the parent repository is stored as a configuration file. This configuration file is used to create replicas in the child nodes and the changes are also synchronized using configuration files. This scenario also uses replication tools in Workflow Manager (Classic) Administrator as well as geoprocessing tools.

Create the Workflow Manager (Classic) service for the parent

If the parent repository is connected to a server, run the Publish Workflow Service geoprocessing tool to publish the Workflow Manager (Classic) service for the parent repository.

Create and add nodes to the cluster

If the parent repository is connected to a server, the disconnected replication can be partially managed in the administrator. Clusters can be created through the administrator on the parent repository and the Manage Replication tool can be used to add the nodes.

1. Start Workflow Manager (Classic) Administrator and connect to the parent repository.
2. Right-click the parent database connection and click Manage Replication.

   The Manage Replication dialog box appears and the repository name for the parent is automatically populated.

3. Leave the Connected check box checked.
4. Provide the URL of the parent repository’s Workflow Manager (Classic) service in the Workflow Manager Server REST URL text box.
5. Click the Add button to add a new child to the cluster.
6. Specify the repository name.
7. Uncheck the Connected check box.
8. Click the Workflow Manager Server REST URL browse button (...) to specify where to save the parent repository’s JXL file.
9. Repeat steps 5 through 8 for all the other child nodes.
10. Click Save when you are finished adding repositories.

Create new replicas using administrator

In disconnected replication, if the parent repository is connected to a server, the Create New Replica option creates a configuration file with the elements and jobs of the parent repository at the specified folder location. This operation will not delete the contents of the child nodes, as the process cannot communicate with them. Use this to run initially when defining the cluster.

1. Start Workflow Manager (Classic) Administrator and connect to the parent repository.
2. Right-click the database connection and click Manage Replication.
3. Click Create New Replica.

   This process may run for several minutes, depending on the size of the parent repository.

4. Click OK when replication completes.

Synchronize replica using geoprocessing tool

The synchronize process compares the differences between all the children in the cluster, imports them to the parent node, and sends the changes to all the other nodes in the cluster.

1. Start ArcGIS Pro.
2. Open the Export Job Data tool.
3. Specify the folder location for the Folder to export to parameter.
4. Specify the folder location of the child repository connection file for the Input Database Path (.jtc) parameter.

   If no connection file is specified, the current default Workflow Manager (Classic) database is used.

5. Specify the repository name of the child repository for the Repository Name parameter.

   If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

6. Specify the date when replicas were created or changes were last synchronized for the Export Since parameter.
7. Click Run.

   The child repository configuration consisting of changes exports to the specified folder location as a .jxl file. The changes are exported only for the specified time after the export since parameter.

8. Open the Import Job Data tool.
9. Select the file created in step 7 as the input for the Input JXL/Acknowledgement parameter.
10. Check the Merge check box.
11. Specify the folder location of the parent repository connection file for the Input Database Path (.jtc) parameter.

    If no connection file is specified, the current default Workflow Manager (Classic) database is used.

12. Specify the repository name of the parent repository for the Repository Name parameter.

    If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

13. Click Run.

    The child repository configuration with changes is imported to the specified parent repository and merged with the contents of the parent repository.

14. Repeat steps 2 through 13 for all child repositories to send changes to the parent repository.
15. Open the Export Job Data tool.
16. Specify the folder location for the Folder to export to parameter.
17. Specify the folder location of the parent repository connection file for the Input Database Path (.jtc) parameter.

    If no connection file is specified, the current default Workflow Manager (Classic) database is used.

18. Specify the repository name of the parent repository for the Repository Name parameter.

    If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

19. Specify the date when the replicas were created or changes were last synchronized for the Export Since parameter.
20. Click Run.

    The parent repository configuration consisting of changes from the parent repository as well as all child repositories is exported to the specified folder location as a .jxl file. The changes are exported only for the specified time after the export since parameter.

21. Open the Import Job Data tool.

    Select the file created in step 20 as the input for the Input JXL/Acknowledgement parameter.

22. Check the Merge check box.
23. Specify the folder location of the child repository connection file for the Input Database Path (.jtc) parameter.

    If no connection file is specified, the current default Workflow Manager (Classic) database is used.

24. Specify the repository name of the child repository for the Repository Name parameter.

    If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

25. Click Run.

    The parent repository configuration with all consolidated changes is imported to the specified child repository and merged with the contents of the child repository.

26. Repeat steps 21 through 25 for all child repositories.

Disconnected replication—all repositories disconnected

In cases where none of the repositories participating in the cluster are connected to a server, all information exchanged is through configuration files. The configuration files are created using the Export Job Data and Import Job Data geoprocessing tools.

Create new replicas using geoprocessing tools

Creating replicas is the process of making all child repositories an identical copy of the parent repository. It requires exporting the configuration from the parent repository, deleting the existing configuration in child repositories, and importing the parent repository configuration in child repositories.

1. Start ArcGIS Pro.
2. Open the Export Job Data tool.
3. Specify the folder location for the Folder to export to parameter.
4. Specify the folder location of the parent repository connection file for the Input Database Path (.jtc) parameter.

   If no connection file is specified, the current default Workflow Manager (Classic) database is used.

5. Specify the repository name of the parent repository for the Repository Name parameter.

   If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

6. Do not specify any value for Export Since parameter.
7. Click Run.

   The parent repository configuration is exported to the specified folder location as a .jxl file.

8. Open the Import Job Data tool.
9. Select the file created in step 7 as the input for the Input JXL/Acknowledgement parameter.
10. Uncheck the Merge check box.
11. Specify the folder location of the child repository connection file for the Input Database Path (.jtc) parameter.

    If no connection file is specified, the current default Workflow Manager (Classic) database is used.

12. Specify the repository name of the child repository for the Repository Name parameter.

    If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

13. Click Run.

    The parent repository configuration is imported to the specified child repository, and the entire contents of the child repository are replaced with the contents of the parent repository; therefore, the child repository becomes identical to the parent repository.

14. Repeat steps 8 through 13 for each child repository.

Synchronize replica using geoprocessing tools

In cases where none of the repositories participating in the cluster are connected to a server, all information exchanged and synchronized is through configuration files. The configuration files are created using the Export Job Data and Import Job Data geoprocessing tools.

1. Start ArcGIS Pro.
2. Open the Export Job Data tool.
3. Specify the folder location for the Folder to export to parameter.
4. Specify the folder location of the child repository connection file for the Input Database Path (.jtc) parameter.

   If no connection file is specified, the current default Workflow Manager (Classic) database is used.

5. Specify the repository name of the child repository for the Repository Name parameter.

   If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

6. Specify the date when the replicas were created or changes were last synchronized for the Export Since parameter.
7. Click Run.

   The child repository configuration consisting of changes is exported to the specified folder location as a .jxl file. The changes are exported only for the time specified after the export since parameter.

8. Open the Import Job Data tool.
9. Select the file created in step 7 as the input for the Input JXL/Acknowledgement parameter.
10. Check the Merge check box.
11. Specify the folder location of the parent repository connection file for the Input Database Path (.jtc) parameter.

    If no connection file is specified, the current default Workflow Manager (Classic) database is used.

12. Specify the repository name of the parent repository for the Repository Name parameter.

    If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

13. Click Run.

    The child repository configuration with changes is imported to the specified parent repository and merged with the contents of the parent repository.

14. Repeat steps 2 through 13 for all child repositories to send changes to the parent repository.
15. Open the Export Job Data tool.
16. Specify the folder location for the Folder to export to parameter.
17. Specify the folder location of the parent repository connection file for the Input Database Path (.jtc) parameter.

    If no connection file is specified, the current default Workflow Manager (Classic) database is used.

18. Specify the repository name of the parent repository for the Repository Name parameter.

    If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

19. Specify the date when replicas were created or changes were last synchronized for the Export Since parameter.
20. Click Run.

    The parent repository configuration consisting of changes from the parent repository as well as all child repositories is exported to the specified folder location as a .jxl file. The changes are exported only for the time specified after the export since parameter.

21. Open the Import Job Data tool.
22. Select the file created in step 20 as the input for the Input JXL/Acknowledgement parameter.
23. Check the Merge check box.
24. Specify the folder location of the child repository connection file for the Input Database Path (.jtc) parameter.

    If no connection file is specified, the current default Workflow Manager (Classic) database is used.

25. Specify the repository name of the child repository for the Repository Name parameter.

    If no repository name is specified, the current default Workflow Manager (Classic) database repository name is used.

26. Click Run.

    The parent repository configuration with all consolidated changes is imported to the specified child repository and merged with the contents of the child repository.

27. Repeat steps 21 through 26 for all child repositories.

Note:

The export and import of configuration files using geoprocessing tools can be scripted to automate the workflow.

Deleting nodes from a cluster

Nodes added to the cluster can also be deleted using the tools available in the Workflow Manager (Classic) Administrator. When a node is deleted, a message is sent to the parent and other nodes to ensure that the existing relationship clears from the system tables.

1. Start Workflow Manager (Classic) Administrator and connect to the parent repository.
2. Right-click the database connection and click Manage Replication.
3. Click the node that you want to delete from cluster.
4. Click the Remove a Repository button.

There are geoprocessing tools available in the Workflow Manager toolbox that create replicas, synchronize replicas, delete nodes, and export and import data. This gives you the option to run these as scheduled tasks through a Python script. See An overview of the Workflow Manager toolbox for more information.