ArcREST: Simplifying ArcGIS REST API scripting with Python


Does your organisation share content through ArcGIS Online, ArcGIS Server or Portal for ArcGIS? Ever thought it would be handy to be able to script everyday tasks to better manage your content across your ArcGIS platform?

Whether you want take a weekly snapshot of the data behind a Feature Service used in Collector for ArcGIS, update the data behind one of your web maps, administer your ArcGIS Server, or add a published service to your Portal – all this and a whole lot more has always been possible to script through any language you like.  The catch is that hasn’t always been very easy to do, particularly for those of us without coding experience outside of the GIS world. This is where Esri’s open source ArcREST python module comes in.

In plain English, REST (which stands for Representational State Transfer), is a commonly used approach for enabling applications to interact with each other, typically over a network or the Internet. Every time you load a web map in your browser, there is code running in the background that asks the server what should be visible in the current map extent. The server will process this request by looking up what needs to be displayed, then sends the results back to your to browser to be displayed on your screen.

Interacting with maps and services is one way in which ArcGIS uses REST, however a valuable point to recognise is that the vast majority of administrative and data management operations we might want to perform in ArcGIS Online, ArcGIS Server or Portal for ArcGIS are all governed by this same process. From adding content to your Portal, to adding users to your ArcGIS Online organisation to querying the logs on your ArcGIS Server – your browser is requesting specific URLs to tell the platform to do something, then interprets the responses.

The trick here is knowing firstly how to format these requests, then where to send them, and finally how to interpret the results. The specifications of how to do all of this is laid out in Esri’s documentation of the ArcGIS REST API, though it can be time consuming to translate this into code.

Essentially the creators of the ArcREST python module have taken the specifications of the ArcGIS REST API and turned much of this into easy to use python classes and functions. This frees us from dealing with the nuts and bolts of the HTTP requests, and by exposing the platform’s capabilities through python objects and functions we can then focus on the core functionality of the script.

The implementation is quite comprehensive and you can expect to see it exposing functionality for most common tasks, and many not so common ones too. Some capabilities I’ve found particularly interesting include being able to script the management of services, security and machines for your ArcGIS Server site. ArcGIS Online administrators are able to update their user’s details, assign credits to users, manage which folders services are stored in, work with groups, add/remove uploads, and manage sharing permissions. Portal administrators will see much of the same functionality available for managing their content, with functions added for administrative tasks like managing identity stores, web adaptors and logs.

And that’s just scratching the surface of ArcREST’s capabilities.

Whether you’re taking your first steps into the world of scripting or a seasoned developer experienced in the administration of ArcGIS, the ArcREST module exposes the functionality you need to get the job done and takes care of the lower-level details involved in formatting REST requests and interpreting responses.

To get set up with ArcREST ideally we would have ArcGIS already installed – this gives python access to arcpy and the geoprocessing tools involved in tasks like publishing services. It is still possible to use ArcREST without arcpy, just be aware that some extra functionality such as publishing services will be unavailable.

You’ll want to download and extract the ArcREST zip file from https://github.com/Esri/ArcREST . After extracting the contents of the zip file, running the setup.py script from the src sub-directory installs ArcREST into the default python environment.

If you have multiple python environments installed – say for example having both 32 and 64 bit versions of Python installed, running the install script from a specific python.exe will install ArcREST for that particular Python environment.

With ArcREST installed, we can confirm everything is working with a short test script. This one will log into ArcGIS Online using the credentials you provide through the username and password variables and list all the items that display on your ‘My Content’ page of ArcGIS Online.


import arcrest
username = ""
password = ""
sh = arcrest.AGOLTokenSecurityHandler(username=username, password=password)
admin = arcrest.manageorg.Administration(securityHandler=sh)
content = admin.content
# Get the logged in user
currentUser = content.users.user()
# List titles and sharing status for items in users' home folder
for item in currentUser.items:
    print "[%s]\t%s" % (item.sharing['access'], item.title)

Alternatively, if we wanted to run this script against Portal, the only change required would be replacing the call to arcrest.AGOLTokenSecurityHandler. Instead we’d use arcrest.PortalTokenSecurityHandler(url, username, password) where the url parameter provides the web address to your Portal.

Assuming your script runs successfully, you’re all set to go forth and explore what can be done with scripting the ArcGIS REST APIs! A great place to start is with the example scripts included in the ‘samples’ folder when downloading ArcREST. Stay tuned to the Esri Australia Tech blog for some practical applications of the ArcREST module.

James

4 thoughts on “ArcREST: Simplifying ArcGIS REST API scripting with Python

  1. cafulele

    I’m so much in love with your informative blog. I’m currently working on a wetlands project (landscape). The idea is to derive wetland condition from land cover classes. I am doing this on ArcGIS model builder, the challenge I have now is using calculate value by multiplying (weighting rankings) to the extent of landcover. Please advise

    Reply
    1. jamesbrander Post author

      URLError is a general urllib2 exception class. This exception could be raised for a multitude of reasons relating to either the specified URL and/or the environment’s network configuration. One of the more common possible reasons for seeing this error is that a forward proxy on your network requires authentication. Many of the ArcREST samples show how to work with proxy and supply credentials. ArcREST Samples

      Other possible problems could include things like issues with firewalls, connecting via HTTPS where the server is using an untrusted certificate, connecting to the server using HTTPS via its hostname only where a trusted certificate used for HTTPS has instead been issued to the fully qualified domain name, issues with DNS, and trying to connect to a port the server isn’t listening on.

      Along with the exception class, there should be a message stating why the exception was raised which provides an avenue for further investigation. And if required, a HTTP debugger like Fiddler can often be useful in further narrowing down the cause of problems in scripts working with HTTP.

      Reply

Got something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s