Determining the interdependencies of items in Portal for ArcGIS

With almost any Portal there comes a time when you have dozens of items, created over time by different people, and you may be looking to perform a bit of a clean-up.

The problem now arises that you do not know if the Web Map made by a colleague, who since left the organisation, can be deleted or if it is being used by any other items? You might find that deleting an insignificant-looking item brings your organisation’s most-used application down.

The best approach is to avoid ending up in this situation, by making sure your organisation has good conventions in place to leverage the following to keep your content structured and clean:

• Tags
• Categories
• Delete Protection
• Mark data as authoritative
• Mark data as deprecated
• Naming conventions
• Folders
• Groups
• Thumbnails

Unfortunately, if you are already in the scenario of having dozens or hundreds of items, it can be very difficult to make sense of which items relate to others. The below Python script will assist in extracting the dependencies between items from your Portal and present them in an interactive HTML diagram. The script is written for Python 3.7.

It makes use of the following libraries that you may need to install into your Python environment:

1. NetworkX
2. MatPlotLib
3. PyVis

To install these packages, use the pip installer to add them to your environment:

1. Open command prompt and enter cd C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3
2. Run pip network
3. Run pip matplotlib
4. Run pip pyvis

Once these have been installed, copy the below script into your Python IDE of choice.

Modify the line

source = GIS(r’https://<machine fqdn>/<web adaptor name>’, ‘<portaladmin login>’, ‘<portaladmin password>’)

to reflect your Portal’s url and Administrator login. Administrator level is required to be able to list items that are owned by other users. Currently, ArcGIS Online does not support the item.dependent_upon() function which is required by this script.

Now when you run the script (updated 8 Sept 2021 for refactoring and adding arrows), you should get an output like below which graphically shows the different items in your Portal and their relationships to each other.

import arcpy
import requests

import pandas as pd
import networkx as nx
import matplotlib.pyplot as plt

from matplotlib.pyplot import figure
from urllib3.exceptions import InsecureRequestWarning
from pyvis.network import Network
from pyvis.options import Options

# Suppress only the single warning from urllib3 needed.
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)

from arcgis.gis import GIS
from IPython.display import display

def addDependency(item, dependency, pdArray, includeUrlDependencies= False):
    guidMap[item.itemid] = str.format("{2} \r\n {0} \r\n {1}", item["title"], item.itemid, item["type"])
    if dependency['dependencyType'] == 'id':
        pdArray.loc[len(pdArray)] = [item.itemid,dependency['id']]
    elif (includeUrlDependencies == True and dependency['dependencyType'] == 'url'):
        pdSource.loc[len(pdSource)] = [item.itemid,dependency['url']]



source = GIS(r'https://<fqdn>/<webAdaptorName>/', 'portaladmin', '<password>')

pdSource = pd.DataFrame(columns=['source','dependent_on'])
guidMap = {}

source_users = source.users.search(query='', sort_field='username', sort_order='asc', max_users=10000, outside_org=False, exclude_system=True)

arcpy.AddMessage(source_users)
print('User list: Role')
for user in source_users:
    print(user.username + "\t:\t" + str(user.role))
    arcpy.AddMessage(user.username + "\t:\t" + str(user.role))
for user in source_users:
    num_items = 0
    num_folders = 0
    print("Collecting item ids for {}".format(user.username))
    arcpy.AddMessage("Collecting item ids for {}".format(user.username))

    user_content = user.items()
    source_items_by_id = {}

    # Get item ids from root folder first
    for item in user_content:
        print(item.itemid)
        #guidMap[item.itemid] = str.format("{2} \r\n {0} \r\n {1}", item["title"], item.itemid, item["type"])
        source_items_by_id[item.itemid] = item
        if (item.dependent_upon()['total'] > 0):
            print("\n\n Item: \t {2} \t\t\t id {0} is dependent on these items: \t {1}".format(item.itemid,item.dependent_upon(), item.title))
            for dependency in item.dependent_upon()['list']:
                addDependency(item, dependency, pdSource, False)

    # Get item ids from each of the folders next
    folders = user.folders
    for folder in folders:
        num_folders += 1
        folder_items = user.items(folder=folder['title'])
        for item in folder_items:
            num_items += 1
            #guidMap[item.itemid] = str.format("{0} ({1})", item["title"], item.itemid)
            source_items_by_id[item.itemid] = item
            if (item.dependent_upon()['total'] > 0):
                print("\n\n Item: \t {2} \t\t\t id {0} is dependent on these items: \t {1}".format(item.itemid,item.dependent_upon(), item.title))
                for dependency in item.dependent_upon()['list']:
                    addDependency(item, dependency, pdSource, False)


    print("Number of folders for {} {} # Number of items checked {} ".format(user.username,str(num_folders), str(num_items)))

print(pdSource)

figure(figsize=(10,8))

G = nx.DiGraph()
G = nx.from_pandas_edgelist(pdSource, 'source', 'dependent_on', create_using=nx.DiGraph)
G = nx.relabel_nodes(G, guidMap)

nx.draw_networkx(G)

import json

options = {
  "edges": {
    "arrows": {
      "to": {
        "enabled": True
      }
    },
    "color": {
      "inherit": True
    },
    "font": {
      "size": 8
    },
    "smooth": False
  },
  "interaction": {
    "hover": True
  },
  "manipulation": {
    "enabled": False,
    "initiallyActive": False
  },
  "physics": {
    "barnesHut": {
      "gravitationalConstant": -20700,
      "springLength": 110,
      "avoidOverlap": 0.4
    },
    "minVelocity": 0.75
  }
}

net = Network(height="100%", width="100%")
net.set_options(json.dumps(options))
#net.show_buttons(True)
net.from_nx(G)
net.show("Dependencies.html")