Spatial analysis for the rest of us


Since May of this year, when I demonstrated a very early version of the new Insights for ArcGIS product at the Directions LIVE events Esri Australia staged around the country, it has been the topic of many conversations I have had – both internally and with customers.

Everyone is keen to understand where Insights fits in to the ArcGIS platform, and where it sits in relation to other similar products in the broader market.

There’s a buzz about this that I haven’t witnessed for some time in Esri circles, and I’ve got to say – it’s infectious. For me personally, Insights, and the GeoAnalytics Server that is also in the pipeline for ArcGIS 10.5, stand to be highlights of my work over the next year or so.

I’d like to share my early thoughts on Insights, and I hope that leaves you curious enough to find out more.

My perspective is clearly a little biased as I work for Esri Australia, but it’s worth noting my background is not what you might call a GIS professional. I discovered the intersection of IT and geography well into my career after pursuing a personal side interest in environmental science at university. During that time, my day job was focused on designing and building large-scale data-driven web applications.

It would be true to say I’ve written more SQL statements than I’ve made maps, and as a result when I look at a map, even now, I see it firstly as a unique and valuable expression of data rather than a cartographic masterpiece. The ‘magic’ that geographic thinking and geographic science brings to data is what drew me to work for Esri Australia, and that continues to fuel my interest in what I do.

The Esri tagline for Insights is ‘spatial analysis that rewards your curiosity’. I think that’s appropriate as a marketing line, but I would re-phrase it in my terminology as ‘exploratory data analysis through a spatial lens’. Maybe not so catchy, but I think my version sums up what Insights does, and why it’s different from other tools that offer similar capabilities.

Exploratory spatial data analysis (ESDA) is not a new thing – it’s been around for a long time. In fact, it’s part of the toolset featured in the Geostatistical Analyst extension for ArcGIS Desktop. If you want to get to grips with the science behind ESDA, there’s a great resource available online in the form of Geospatial Analysis – A comprehensive guide.

Pic 1

If you have the appetite to dive into either of those links, you’ll observe they’re not for the faint-hearted, and I think that kind of sums up what is an ongoing challenge for Esri. Spatial analysis is awesome – you can do amazing things with it, and the capability in ArcGIS is, I would argue, unmatched after decades of honing this aspect of the platform.

However, a pretty steep learning curve has to be climbed for most users to realise the benefits. In my opinion, the surfacing of some of these capabilities in a simpler form as analysis tools in ArcGIS Online and Portal for ArcGIS is a step in the right direction. For someone like me though, I think there’s still a significant barrier to entry which is this: how to associate a question I want answered with the appropriate spatial analysis tool for the job. Should I use ‘find similar locations’ or ‘choose best locations’? If I pick one and it’s wrong, will I know I’m chasing the wrong outcome?

My more spatially enlightened colleagues will be sighing at this point I’m sure, but look, this is reality for me. I’m someone who is very resourceful when it comes to exploring what’s going on with data. I’ll use SQL, I’ll use R, I’ll use Excel or whatever gets me to where I need to be, but the spatial analysis element of exploratory data analysis is the tough bit.

If it were just putting data on a map for visualisation purposes, that would be simple, but that’s not what this is about. What I really want to do, as someone tasked with turning my organisation’s data into useful information and insights, is to interact with that data in a fluid fashion, where I can explore hypotheses, seek patterns and, importantly, sometimes discover perspective I did not necessarily set out to find or even know about ahead of time – the surprise element.

I want to do all of this in an environment where simplicity and impunity are key. What I mean by that is if I screw up and ask a stupid question, or I go down a rabbit hole that yields nothing, then (a) I didn’t have to spend hours learning how to make the mistake, and (b) things happened at pace so my mistakes are not time-costly – in fact, they’re a completely normal and valid part of my exploration of data. Along the way I want to have been using spatial analysis, but not have been aware of a context switch to something so specific.

While I’m sketching out my version of ESDA utopia, let me add something else to the picture. If I’m going to be able to try different analytical approaches at pace and with impunity, then help me out by letting me trace and re-trace my analytical path so that if (when?) I strike gold, I can actually figure out how I got to that point and capture the magic.

I can say from bitter experience that I have on more than occasion ended up with what I believe to be a high-value outcome, but with poor, or non-existent reproducibility. That’s not good – it leaves your results open to question as to their pedigree, and severely limits your ability to share your work with others.

Pic 2

It’s important to note in its foundation release later this year, along with ArcGIS 10.5, Insights for ArcGIS will be an application available through Portal for ArcGIS – part of an on-premises deployment of the ArcGIS platform. So this isn’t a standalone BI tool – it’s a web app that is part of the whole ArcGIS platform, and just like some of the other ArcGIS apps, it can be both a consumer and a contributor to the body of geographic knowledge curated by an organisation.

For example, I might draw on a web map of sales regions published by my sales team as the basis for a simple drag-and-drop spatial aggregation step in my analysis, and then later on I might share my analysis outcomes back to the organisation in the form of charts, maps, tables or complete Insights workbooks, so others can use them in their work.

I might even take just a couple of exploratory analytical steps in Insights and then choose to take my intermediate outcomes and explore them further in another part of ArcGIS, or hand them over to a GIS professional to refine – that’s a totally legitimate thing to do. Analysis is not necessarily going to be an end-to-end-raw-data-to-perfect-visualisation thing every time for every user.

What I believe makes Insights different is that it elevates geography and spatial analysis to be first-class citizens in the data analysis workflow, and it does it in a way that abstracts the complexity of the spatial science to a point where everyone can use it to great effect, whether they’re clued into GIS or not. I talked a lot about what someone like me wants out of a tool for exploratory data analysis and that was intentional – I honestly believe there are many out there who share a similar profile to me when it comes to approaching data analysis.

I know my data pretty well, I know it’s in many places potentially, but I know where to find it. I might know the business questions/hypotheses that I want to explore (or maybe I don’t…), and I know enough about data analysis and geography to be confident that if suitably prompted through the user experience, I can use the geography unique to my organisation (think sales territories for example) to great effect and generate real insight.

In a geoenlightened organisation where location is already being exploited, Insights will be a potent addition to existing systems of geographic record and engagement. I believe existing GIS professionals in these types of organisations will find Insights to be an exciting new channel to showcase their capabilities. Equally, data analysts with limited or no exposure to GIS will be able to lift their analytical game, and take their use of location past simple visualisation.

Insights for ArcGIS is expected to be available in public beta sometime during July or early August, and final release is slated for late 2016. I would encourage you to visit the Insights for ArcGIS page on the Esri website for more information, and to sign up for the beta program.

Josh V

Got something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s