GDA2020, ArcGIS Online and the Web Mercator Dilemma


In ArcGIS Online Map Viewer, all layers in a web map inherit the coordinate system of the basemap. WGS 1984 Web Mercator (auxiliary sphere) is the spatial reference of basemap services in the default Esri basemap gallery, meaning this is the default web GIS coordinate system. Layers also have their own spatial reference and transformations defined at a service level. When you add layers to the Web Map Viewer, an on-the-fly reprojection is performed to align your data with the coordinate system of the basemap.

When publishing data to web GIS you need to consider what spatial reference will be used to store the data at a service level, how the publishing process may affect it, as well as what transformations (or lack of) will be applied when the layer is displayed in a web map with a datum.

It is also important to understand GDA2020 and its relationship with web GIS. The de facto WGS84 in web GIS brings some challenges not unique to Australia or ArcGIS. Failing to address these challenges may result in data misalignment.

The Web Mercator Dilemma

During the early 1990s, the governing bodies defining the transformation parameters between GDA94 and WGS84 set the transformation as ‘null’. At this period both GDA94 and WGS84 were considered coincident as they were based on the same spheroid, with essentially the same origin. That ‘Null’ parameter now creates a problem when projecting GDA2020 data to WGS84 Web Mercator as the differences between GDA94 datum (a static datum) and WGS84 (effectively, a dynamic datum) have increased over time. Although this has not presented significant issues to the end-user as data mapped in web maps typically had low accuracy – this is changing.  Essentially, the problem presents itself as a misalignment of GDA94 or GDA2020 data when viewed in a WGS84 based projection.

Unfortunately, with the modernisation of the Australian datum to GDA2020 there is not a definitive relationship between GDA2020 and WGS84. The following diagram gives a representation of the relevant issue:

At the time of writing in March 2021, there is no transformation between GDA2020 and WGS84 directly. A composite transformation is available of WGS84 to GDA94 and GDA94 to GDA2020. This introduces challenges where a composite transformation is not allowed, such as applying transformations to mosaic datasets.

Data in the web map environment can be displayed against Esri basemaps and web scenes and other WGS84-based data. In most instances this will not be an issue as the generic basemap data is derived from many sources with varying positional accuracies. From a user data perspective on GDA2020 Esri Australia recommends:

  • Create your own basemap in GDA2020 and use layers published in GDA2020.
  • If publishing for use in WGS84, project the data first using GDA2020 > GDA94 > WGS84. Data will be published with GDA94 accuracy and coordinate transformations between WGS84 and GDA94 can be performed.
  • Do NOT mix GDA94 and GDA2020 data in a web GIS as there will be an approximately 1.8m alignment error. Transform all data to GDA2020 and republish to Web GIS in GDA2020 or a GDA2020-based projected coordinate system.
  • If a coordinate conversion is required in Web GIS from WGS84 to GDA2020 then a transformation path of WGS84 > GDA94 > GDA2020 should be utilised.

Publishing Hosted Feature Layers from ArcGIS Pro – Coordinate Systems and transformations at a service level

Layers added to the ArcGIS Online Web Map Viewer and Scene Viewer will be reprojected on the fly to line up with the basemap. But what coordinate system is stored against the layer being added, and what transformation is being applied? This is dictated by the coordinate system and transformations set in the Map Properties in ArcGIS Pro at the time of publish. If you publish Hosted Feature Layers with a different datum to your Web Map, the data may not align if a transformation cannot be correctly applied.

When you publish a hosted feature layer from ArcGIS Pro, and you know people will use the hosted feature layer in maps that contain ArcGIS Online basemaps, you can do one of the following:

Option 1: Preserve the projection of your data but define a transformation that ArcGIS Online can use when performing on-the-fly reprojections of the hosted feature layer.

With this option, your source data and hosted feature layer data remain in the same projection. You define what transformation should be used when on-the-fly reprojection is required i.e. when you add the layer to a web map with a basemap in a different datum.

The following steps summarise how to keep your data’s original projection and define a transformation to control how the layer is reprojected when viewed on an ArcGIS Online basemap.

  1. In ArcGIS Pro, open the map you want to publish as a hosted feature layer.
  2. Open the Map Properties and set the data frame’s coordinate system to match the coordinate system of the source data in your map and apply your changes.
  3. In the Map Properties > Transformations, add a transformation. Choose a transformation path between the source data and the expected web GIS spatial references (i.e. to/from WGS84). From ArcGIS Pro 2.7 you can add transformations for coordinate systems not currently in your map – for earlier releases you’ll need to add a layer to include such transformations.
  4. Go to Share > Share As > Web Layer to publish all the layers in the map as a Hosted Feature service in the coordinate system of the Map with the transformations retained.

What this means is you can publish your data in a projection other than Mercator and add a transformation to go from your layer projection into another projection such as WGS84.  When that data is displayed on top of a basemap based on WGS84, it will use the transformation you picked to reproject the data.  This method should work with any transformation that is supported out of the box in Esri’s desktop software.

User-defined/Custom transformations or grid-based transformations that typically require downloading a third-party grid shift database (GSB) are not supported at this time. Please remove them from your data frame prior to publishing to avoid any unexpected issues. This includes the NTv2 grid-based transformation methods for GDA2020 to GDA94

Option 2: Configure the map so that the feature class data is reprojected into WGS 1984 Web Mercator (auxiliary sphere) when it is copied to ArcGIS Online.

With this option, the hosted feature layer data is stored in ArcGIS Online in the same coordinate system as the ArcGIS Online basemaps; therefore, no on-the-fly reprojection is needed when viewed on these basemaps. However, this means the hosted feature layer data and the data in your data source are now in different projections.

The following steps summarise how to configure the map so the publishing process reprojects the hosted feature layer data so it is stored in WGS 1984 Web Mercator (auxiliary sphere):

  1. In ArcGIS Pro, open the map you want to publish as a hosted feature layer.
  2. Set the data frame’s coordinate system to WGS 1984 Web Mercator (auxiliary sphere). This is the projection in which the hosted feature layer data will be stored.
  3. Define a geographic transformation between WGS84 and the coordinate system of your data source as required.
  4. Publish your layer to ArcGIS Online. As part of the publishing process, your data will be reprojected to match the Map’s projection using the transformation you have specified. This method even works if you have your own custom projections and associated custom transformations. Depending on your workflows, though, this method creates some extra steps if you need to merge the published layer back into the original dataset.

Checking the coordinate system and transformations of published layers

To identify the spatial reference of an ArcGIS Online Hosted Feature Layer:

  1. Open the Hosted Feature Layer Item Details page via Content
  2. On the bottom-right of the page click the ‘View’ button next to URL

A new browser page will open showing service details, including the Spatial Reference WKID.

To identify datum transformations stored in the service:

Continuing from the ArcGIS Rest Services Directory above, move to the Admin REST API by adding ‘admin/’ to the browser URL between rest and services.

https://services.arcgis.com/j3JVMGycAs95su2E/arcgis/rest/services/Map1_CC/FeatureServer

to:

https://services.arcgis.com/j3JVMGycAs95su2E/arcgis/rest/admin/services/Map1_CC/FeatureServer

The ArcGIS REST Administrator Directory will have a distinctive red

From the REST Administrator Directory, click the JSON link at the top of the page to view the service definition with any datumTransformation parameters

The Transformations in the ArcGIS Pro Map Properties have carried across to the service

With care, advanced users can use the Admin Service endpoint Update Definition operation to directly update the datumTransformations included in the service definition. The transformations must be supported for Hosted Feature Layers (at this time, no NTv2 grid-based transformations).

Changing the basemap layer, and coordinate system, of your Web Map

In addition to the basemap gallery, you can also use your own basemap by searching for an existing layer or adding a layer from the web. Several layer types are supported. When you use your own basemap, your map uses the coordinate system of that basemap instead of Web Mercator, the coordinate system of Esri basemaps.

The following layer types are supported for custom basemaps: imagery, map image, tile, vector tile, WMS OGC, WMTS OGC, Bing, and OpenStreetMap.

Please see the help documentation to use your own basemap for more information.

Moving to GDA2020 and looking for a strategy built on a thorough analysis of your requirements? Find out more about Esri Australia GDA2020 service packages and book a free consultation here.

Continue Reading

  1. Understanding Spatial References in Australia
  2. Introduction of GDA2020 to ArcGIS and how to install additional transformations
  3. Working with coordinate systems and transformations in ArcGIS Pro
  4. GDA2020, ArcGIS Online and the Web Mercator Dilemma (current page)

4 thoughts on “GDA2020, ArcGIS Online and the Web Mercator Dilemma

  1. Margaret Maher

    Hello, Margaret Maher here, Esri Support in Redlands, CA. Reading through this very informative blog, I just question the statement that custom geographic transformations, including NTv2, are not supported. If possible I’d like to see a little more explanation since that blanket statement might be confusing to users. We definitely support custom transformations that use NTv2 GSB files, just maybe not in this context. Thanks and have a great day.

    Reply
    1. Chris Counsell Post author

      Hi Margaret,
      The wording on that section is directly lifted from Esri’s blogs.

      What I have found is that users will publish grid-based transformations to ArcGIS Online receive no warning and assume that they are being used. As the grid-based transformations are recommended to Australian GIS users, I think it is necessary to highlight that these default/preferred transformation paths are not supported when publishing Hosted Feature Layers.

      I encourage users to read all four blogs in the series to best understand transformations in ArcGIS.
      Cheers,
      Chris

      Reply
  2. James O'Kane

    Hi,
    I’m considering the implications on migrating from GDA94 to GDA2020 and data misalignment is a real concern. Your article rightly points out that OTF projections of HOSTED feature layers can live happily together in a WGS84 basemap. What you haven’t mentioned is REFERENCED feature layers. From my own testing it seems that the only option for referenced feature layers is to build and your my own GDA2020 basemaps. Is that correct or have I got something wrong?
    Cheers,
    James

    Reply
    1. Chris Counsell Post author

      Hi James,
      The intention was to write additional blogs on ArcGIS Server, Vertical transformations and ArcGIS Field apps. Please also note that new transformations were released that better facilitate transformations with Web GIS.

      Otherwise in principle the information in these blogs remains largely applicable to ArcGIS Server based workflows. The key differences I am aware of are:

      ArcGIS Server may support grid-based transformations for non-hosted services (for compatible versions with the NTv2 files installed).

      Viewing service endpoints to view datumTransformations is slightly different; the below URL construct will work:
      https://xxxxxx.services.esriaustralia.com.au//rest/admin/services/Hosted/Map_Heatmap_test_WFL1/FeatureServer?f=pjson&token=abcdefg12345

      The referenced source data can be in a different spatial reference to the service itself. For an overview of this, please refer to this timestamped video:

      The recommendations to use the same spatial reference where possible or to use an appropriate transformation remain true for ArcGIS Server workflows.

      Cheers,
      Chris

      Reply

Got something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s