ArcGIS Enterprise 10.8.1: Content Migration tools


Migrating content from one ArcGIS Enterprise deployment to another can be one of the biggest challenges for organisations and their GIS Administrators. There are many different circumstances where migrating content would be beneficial over manually re-creating, for example migrating content between development, test, and production environments, as well as between disconnected Enterprise environments.

Read on to understand the different approaches that can be followed when planning to migrate content.

There are number of tools and resources available to use within the ArcGIS API for Python when transferring items, for instance the clone_items() function. An example script which utilises this API is available to view and download at the end of the article.

However, for those of you who are not confident directly jumping in to writing a Python script (either within your own text editor, IDE, or using an ArcGIS Notebook), ArcGIS Enterprise also offers Content migration tools to move and manage content between two ArcGIS Enterprise environments. These tools can be used to move items between development and production Portals that belong to the same organization, or even two Portals hosted in different environments completely disconnected and unaware of each other.

Starting from ArcGIS Enterprise version 10.8.1, the export group content and import group content operations are available in the ArcGIS REST API that allows migration of supported items. A set of supported items that can be used with the migration tools is listed here.

Following a workflow utilising these operations is relatively simple:

  • Share all the items you wish to export to a group within Enterprise Portal A.
  • Export the group content into an “export package” file (.epk) through the Portal A’s REST API endpoint – /community/groups/[groupID]/export.
  • Import the “exported package” file from Portal A into ArcGIS Enterprise Portal B through Portal B’s REST API endpoint – /community/groups/[groupID]/import.
Content migration process between ArcGIS Enterprise Portals.

Once completed, new content will be created in the ArcGIS Enterprise Portal B shared with the specified group in ArcGIS Enterprise Portal B.

With that workflow in mind, let’s take a closer look at this process in a practical sense. We have an ArcGIS Enterprise Portal 10.8.1 deployed, where number of items need to be moved across to a different portal. Keep in mind that this workflow is only supported starting from ArcGIS Enterprise 10.8.1, which means that both of our Portals need to be at least at this version, and that the receiving portal has to be at least the same, or later, version as the sending portal.

Step 1 – Share the items to be exported to a specific group in the Source Portal.

The first step is to share the items to the target group created for purposes of this demo – ExportGroupContent. We are going to call this environment the “Source Portal”. There are couple of items of interest and in total we have 4 items that we are going to transfer to another Portal – which we will call the “Target Portal”. It’s important to note that these two portals are hosted in different environments totally unaware of each other.

Step 2 – Generate the EPK export file from the Source Portal by using the /export REST API endpoint.

Once all data is shared to the group, we will need to generate export package that contains all items. Since this step is performed through the REST API, we will copy the Group ID for ExportGroupContent (as noted in the URL of the screenshot for Step 1) and access the Sharing REST API of Source Portal to get to the group content.

The URL to access will follow a pattern similar to:

https://{domain.com}/{portal_web_adaptor}/sharing/rest/community/groups/{group_id}

In order to generate export package we are going to call export group content operation.

Clicking “Export Group Items” will trigger a synchronous job and will result in an export package file (.epk) downloaded to your local machine. This export package will also consider item dependencies. Therefore, its important to note that all related data must be shared to the same group as well for this workflow to succeed.  For example, you should not just share a web application item with the group in the Source Portal, you must also share the associated web map and hosted feature layer used within the web application as well.

Step 3 – Confirm the group in the Target Portal to receive the imported items.

In our Target Portal, we have a group called MigrateContentTest which currently has no items shared with it. We will take note of the group ID, as highlighted in the URL.

Step 4 – Upload the EPK export file as an item into the Target Portal.

Using the /addItem REST API endpoint, add the EPK as an Item to the Target Portal. The URL to access this page will follow a pattern similar to:

https://{domain.com}/{portal_web_adaptor}/sharing/rest/community/content/users/{username}/addItem

The “Add Item” button can be found on the bottom left hand side of the form.
Here we can see the EPK has been added to Portal for ArcGIS.

Step 5 – Import the EPK export file into the Target Portal by using the /import REST API endpoint.

Now that the EPK item has been added to the Target Portal, we will now need to import that EPK file into the MigrateContentTest group (also within our Target Portal). Since this step is performed through the REST API, we will copy the Group ID for MigrateContentTest and access the Sharing REST API of the Target Portal to get to the group content.

The URL to access will follow a pattern similar to:

https://{domain.com}/{portal_web_adaptor}/sharing/rest/community/groups/{group_id}

In order to import the export package we are going to call import group content operation. If the items to be exported already exist in the target Portal, there is the option to overwrite those items. 

Clicking “Import Package” will trigger a synchronous job and will result in the EPK file being extracted and the items within to be imported within the Target Portal. These items will be automatically shared with the MigrateContentTest group. Options are also available to instead trigger an asynchronous job, or to trigger a job where only a ‘package preview’ will show (in case you want to check the contents of the package). When the import job is complete, the EPK file item will be deleted from the Target Portal.

When items are imported into the Target Portal, their item IDs are maintained, URLs are updated to reflect the new environment, and the export package will be deleted by Portal.

Step 6 – Refresh the group details page in Target Portal and confirm that items have been successfully migrated.

Looks like the items have been successfully imported! Items have been successfully created within the MigrateContentTest group in the Target Portal, and their URLs have been updated to reference their new environment.

But what about Python?

You may have reached the end of this post and be wondering, “how is this approach covered using Python?”

Introducing… migrate_content_script.py!

The file can be downloaded to your local machine through the following link.

Provided a range of parameters/arguments, this script will follow the below process:

  • Connect to Portal A (let’s call it “Source Portal”)
  • Find the group specified
  • Create an export (EPK) file for that group
  • Download to your machine
  • Connect to Portal B (let’s call it “Target Portal”)
  • Find the group specified, or create a group if one is not specified
  • Upload the EPK file to the target portal
  • Share the EPK file with the group in the Target Portal
  • Extract the EPK file and import the applicable items within the target portal.

It’s at this time I would like to stress that this has been something written over a period of free time, and does not constitute ‘official’ software in any way that will be supported. Running this on any machine, in any environment, carries its own risk. It has been tested on Windows 10 machines running either ArcGIS Enterprise 10.9 and 10.8.1, but your mileage may vary.

Example 1 – Running the script with the help command

python .\migrate_script_content.py -h

Example 2 – Running the script with an error output

python .\migrate_content_script.py -spurl “https://{domain}.com/portal/home” -spun “initialadmin” -sppw “{password}” -spgrp “ExportGroupContent” -tpurl “https://{domain}.com/waportal/home/” -tpun “portaladmin” -tppw “{password}” -tpgrp “EPK Testing”

Example 3 – Running the script successfully

Same command format as Example 2

Note – When running the script, in some circumstances, the error highlighted in the screenshot below may appear. Although this is raised as an ‘error’ in the script, based on testing conducted, the import process of the EPK into the target portal has completed successfully.

Additional Resources

Written by Irina Rakhimova and Rhys Walsh-Tindall.

Got something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s