Author Archives: Will B

ArcGIS Server and GDA2020

Background

Esri has been working for to incorporate time-based geographic transformations into the software. As you can imagine, these transformations are hugely complex, since the process involves the actual position of the features on the surface of the earth, taking continental drift, and plate tectonic motion into consideration. These time-based transformations are based on a so-called “epoch”, which is the date on which all features are at located in relation to an absolute location.

There are several moving parts involved here.

1] The computer programming to perform the math to implement these geographic transformations.

2] The transformation parameters (14 for time based transformations) to transform data between an original, static datum like GDA94, which assumes that the features of the earth are in one position and not moving and a datum like GDA2020.  Later datums will be dynamic so as to take  plate motion into consideration, this is discussed here, software vendors are already planning for this new approach.

3] The transformation parameters to transform between one time-based datum, and another more current time-based transformation – GDA2020 to say, GDA2030 for example if such a thing were to exist someday.

What the ArcGIS Software does not do is calculate these transformation parameters. ArcGIS checks them and verifies that the transformation parameters provided by the national government agency perform to the accuracy provided, but does not calculate them.

Therefore, even if ArcGIS has the transformation method in place to perform a 14-parameter time-based transformation, ArcGIS needs the transformation parameters, from the national government agency that has created the specific transformation for their area of interest.

ArcGIS software does support GDA2020 Z point transformations ‘out of the box’ but not the NTv2 grid files for the mathematical and mathematical plus distortion transformations.  The NTv2 grid files will have to downloaded (which include some content from Geoscience Australia) for the appropriate ArcGIS version from;

 My Esri > Product Components > Data and Content > ArcGIS Coordinate Systems Data

Description:
Contains the data files required for the … transformation method and vertical transformation files for … the world … for use with either ArcMap, ArcGIS Enterprise, ArcGIS Engine, or an ArcGIS Pro per machine install

and install on each ArcGIS machine, this will install to C:\Program Files (x86)\ArcGIS\CoordinateSystemsData.

This installer is specific to the ArcGIS version, eg:

  • ArcGIS_Coordinate_Systems_Data_Windows_1061_163829.exe
  • ArcGIS_Coordinate_Systems_Data_Windows_1071_169282.exe

The NTv2 transformation between GDA 1994 and GDA 2020 is one small moving part of this major upheaval in the world of geodesy, and the pursuit of ever-more-accurate and ever-more-precise coordinates for features on the earth’s surface.
Esri publish documentation specific to each ArcGIS version which have included GDA2020 for some time, for instance in 10.5.1 they put the transformation GDA_1994_to_GDA_2020_NTv2_CD in 10.5.1 assuming that they knew the names of ntv2 files, then Geoscience Australia changed the file names so this transformation was deprecated in the _C versions. As far as this naming is concerned _C is for Conformal and _CD is Conformal_and_Distortion. Each of these transformations will get a unique Well Known ID (WKID) value each time the transformation changes. For example:

geographic_transformations_1060

geotran_1071_24

How To: Select the correct geographic (datum) transformation when projecting between datums

Esri Australia have been publishing blogs on GDA2020 for a while;

https://esriaustraliatechblog.wordpress.com/tag/gda2020/

Up to version 10.6.1 for some situations you needed to rename the GSB file as mentioned here;

https://esriaustraliatechblog.wordpress.com/2018/01/04/gda2020-arcgis/

The transformation 108065 was added to the Projection Engine at 10.6.0, but was removed from 10.6.1, so that transformation never existed at 10.6.1. The same transformation, but with a new name for the GSB file was added again at 10.7.1.
The conclusion here is that you should upgrade to the latest version of ArcGIS to get the most GDA2020 transformations.
A map of the approximate shifts from GDA94 to GDA2020 locations across Australia is here; https://www.icsm.gov.au/gda2020

ArcGIS Server (does not support time-dependent transformations yet)

Expect in 2020 many agencies will be publishing services natively GDA2020.
If all data is stored in GDA2020 and published in GDA2020 there should not be any issues, apart from later on taking into account the epoch of the data, as mentioned here;

https://esriaustraliatechblog.wordpress.com/2018/12/24/understanding-gda2020-and-its-relationship-with-web-gis/

If you are upgrading your servers and need to change the coordinate systems of your data or services, please republish the services. The version of the ArcGIS Coordinate Systems Data should be identical on the desktop client and server should be the same.

You can test a server’s support for a particular transformation if the GeometryServer is running;

https://localhost/server/rest/services/Utilities/Geometry/GeometryServer/project?inSR=4283&outSR=7856&geometries={“geometryType” : “esriGeometryPoint”,”geometries”: [{“x”: 153.30141143,”y”: -27.90831298}]}&transformation=108447&transformForward=true&vertical=false&f=html

should return (which is less than 1mm away from the expected ordinates);

{“geometries”: [{ “x”: 529661.2445906086, “y”: 6912919.071889246}]}

For instance, trying;

url1will return the message;

Invalid or missing input parameters.
the specified wkid or wkt is not a geotransformation

As;

url2

is at ArcGIS version 10.4.1 so is not capable of doing GDA2020 transformations.

 

 

 

 
If you access a more recent ArcGIS Server version like;

url4

 

and you get an error like;

Unable to complete operation.
Cannot load the data file Dataset_australia/GDA94_GDA2020_conformal_and_distortion for the geographic transformation GDA_1994_To_GDA2020_NTv2_2_Conformal_and_Distortion.

Then the server is capable of that transformation but the ArcGIS Coordinate Systems Data installer has not been run on the server.
What this means is that if you query the following to add a layer to a web map of yours which is going to be in a particular coordinate system like outSR=7856 https://epsg.io/7856 and you get geometry from this layer;

 

url5

{“displayFieldName”:”PROP.QLD_SURVEYCONTROL_SCDB.MRK_ID”,”fieldAliases”:{“PROP.QLD_SURVEYCONTROL_SCDB.GDALATITUDE”:”GDA latitude”,”PROP.QLD_SURVEYCONTROL_SCDB.GDALONGITUDE”:”GDA longitude”,”PROP.QLD_SURVEYCONTROL_SCDB.AHDHEIGHT”:”AHD height”},”geometryType”:”esriGeometryPoint”,”spatialReference”:{“wkid”:7856,”latestWkid”:7856},”fields”:[{“name”:”PROP.QLD_SURVEYCONTROL_SCDB.GDALATITUDE”,”type”:”esriFieldTypeDouble”,”alias”:”GDA latitude”},{“name”:”PROP.QLD_SURVEYCONTROL_SCDB.GDALONGITUDE”,”type”:”esriFieldTypeDouble”,”alias”:”GDA longitude”},{“name”:”PROP.QLD_SURVEYCONTROL_SCDB.AHDHEIGHT”,”type”:”esriFieldTypeDouble”,”alias”:”AHD height”}],”features”:[{“attributes”:{“PROP.QLD_SURVEYCONTROL_SCDB.GDALATITUDE”:-27.9083129838,”PROP.QLD_SURVEYCONTROL_SCDB.GDALONGITUDE”:153.3014114276,”PROP.QLD_SURVEYCONTROL_SCDB.AHDHEIGHT”:30.34},”geometry”:{“x”:529660.65561161563,”y”:6912917.6855818881}}]}

Or this (where you also asked for the datum transformation);

url6

{“displayFieldName”:”PROP.QLD_SURVEYCONTROL_SCDB.MRK_ID”,”fieldAliases”:{“PROP.QLD_SURVEYCONTROL_SCDB.GDALATITUDE”:”GDA latitude”,”PROP.QLD_SURVEYCONTROL_SCDB.GDALONGITUDE”:”GDA longitude”,”PROP.QLD_SURVEYCONTROL_SCDB.AHDHEIGHT”:”AHD height”},”geometryType”:”esriGeometryPoint”,”spatialReference”:{“wkid”:7856,”latestWkid”:7856},”fields”:[{“name”:”PROP.QLD_SURVEYCONTROL_SCDB.GDALATITUDE”,”type”:”esriFieldTypeDouble”,”alias”:”GDA latitude”},{“name”:”PROP.QLD_SURVEYCONTROL_SCDB.GDALONGITUDE”,”type”:”esriFieldTypeDouble”,”alias”:”GDA longitude”},{“name”:”PROP.QLD_SURVEYCONTROL_SCDB.AHDHEIGHT”,”type”:”esriFieldTypeDouble”,”alias”:”AHD height”}],”features”:[{“attributes”:{“PROP.QLD_SURVEYCONTROL_SCDB.GDALATITUDE”:-27.9083129838,”PROP.QLD_SURVEYCONTROL_SCDB.GDALONGITUDE”:153.3014114276,”PROP.QLD_SURVEYCONTROL_SCDB.AHDHEIGHT”:30.34},”geometry”:{“x”:529660.65561161563,”y”:6912917.6855818881}}]}

Then you are not going to see the points transformed onto your map as you expect (as the server does not have the NTv2 files) and the layer will be not shifted as needed.

Make it your 2020 resolution to bring your ArcGIS Servers up to date, install ArcGIS Coordinate Systems Data and discuss GDA2020 with your users!

 

 

 

 

Free and easy ArcGIS Online from Desktop – Part 3 – Giving (not so freely)

A good relationship will require a bit of give, somethings can be given for free.

Many types of content can be saved from Desktop as a file and then uploaded into ArcGIS Online, such as file types of Map documents and Layer files.
Add Item

All ArcGIS for Desktop users on maintenance are not only entitled to a named user account, but also get 100 credits a year as well.

However some giving can come at a cost with ArcGIS Online.  We are going to have a look some of the ways we can add to our ArcGIS Online organisation.

We can publish our map document as a hosted service.

Publish a Service Continue reading

Free and easy ArcGIS Online from Desktop – Part 2 – Taking

The first thing you may want to try is to see what other named users have already added and allowed you access to (including those items that are public).

ArcGIS online dialog

If you Open an item from ArcGIS Online in ArcGIS Desktop you will have a new ArcMap Document and Table of Contents with the same name, and this will look and act very similar to the same Web Map opened in a browser in ArcGIS Online, Office, Sharepoint, or in ArcGIS Explorer on an iPhone etc.  That is as long as you meet any security constraints, for instance if a data source is on a different network or is secured by ArcGIS Server and you do not have credentials then it just will not show.

Open Web Map Continue reading

Free and easy ArcGIS Online from Desktop – Part 1 – Establishing the relationship

For a while now ArcGIS Desktop has had a cosy relationship with ArcGIS Online and it all started like an arranged marriage configured by default via your GIS Administrator.

ArcGIS Manager Manage Portal for ArcGIS Connections

However this relationship will only work if you have an internet connection (to http://www.arcgis.com) , if not and you have a local Portal for ArcGIS server then you can Add a different portal URL and continue in a more secretive place.

So you can see if there is a relationship when you start ArcGIS Desktop which puts a globe icon in the system tray so you can see at a glance what the connection is like.

Globe icon in the system tray Continue reading

In search of Doctor Who (travels with ArcGIS server, episode 4)

In the prior episodes we have been staying mostly within the safe and familiar corner of the universe called ArcGIS, we have ventured tentatively forth using OGC KML which may have been maintained by some alien technology.  We have relied on many other technologies which we find pervasive throughout our travels such as HTTP, HTTPS and digital certificates.  To get this far you may have needed to tinker yourself or call for help as some of this pervasive technology sits at the edge of the ArcGIS universe, but as our universe expands it will become more familiar.

We will now tentatively search with Geoportal Server. Sounds like a parallel universe perhaps, with similar sounding naming of Portal and Server, if so it is time for disambiguation.

If you want to have a play with the big Portal for ArcGIS in the sky there is obviously ArcGIS Online, your Organisation may have already rented a tenancy there in the cloud.

Continue reading

In search of Doctor Who (travels with ArcGIS server, episode 3)

Last episode galactic federation had stopped with the one server but there is still more to explore in the box.  We have seen our ArcGIS for Server published from and managed by both ArcGIS Desktop and Server Manager in a browser and used from various Portal for ArcGIS clients.  It was noted that we also chose a federated server to act as our portal’s hosting server.

With Space to spare in the box, what Relatively new discoveries And functions can we explore this Time by adding In the extra Dimension of a hosting server?

How

Continue reading

In search of Doctor Who (travels with ArcGIS server, episode 2)

Last episode we got as far as creating a map service to view a few of the nearby places Doctor Who has been seen but it is now time to find out what galactic federation has taken control of our ArcGIS Server security.

Knowing my server box was a lot bigger on the inside, I decided when Portal for ArcGIS is installed to give its Web Adaptor the name ‘portal’ as I wanted to leave the more familiar ‘arcgis’ application name for the ArcGIS Server which was also installed.

How

Continue reading