Check and repair the geometry for features in an Enterprise geodatabase.

ArcGIS applications are built with the assumption that the feature’s geometry follows certain specifications. When the processing algorithms encounter data that does not follow the specifications, the software can perform poorly, generate errors, crash, or the operation may succeed without an apparent problem but generate incorrect results.

Repair Geometry Geoprocessing tool

The Repair Geometry tool inspects each feature in a feature class for problems with its geometry. Upon discovery of a problem, a fix will be applied, and a one-line description will identify the feature, as well as the geometry problem that was fixed.

After applying a fix, the tool will re-evaluate the resulting geometry and if another problem is discovered, the relevant fix will be performed for that problem as well.

Origins of bad geometry

Enterprise geodatabases automatically check the validity of each geometry when they are uploaded; therefore, the Check Geometry and Repair Geometry tools have not been available for feature classes in an Enterprise geodatabase. Geometry errors can however occur for features in an enterprise geodatabase as DBMS systems provide native geometry field types that can be uploaded and modified through manual SQL expressions or from other software packages. These systems can create geometries that are invalid for ArcGIS.

Support for feature classes stored in an enterprise geodatabase

At ArcGIS Pro 2.7, the check geometry and repair geometry geoprocessing tools now support feature classes stored in an enterprise geodatabase. For feature classes stored in an enterprise geodatabase, the following spatial types are supported:

  • Microsoft SQL Server Geometry and Geography
  • PostgreSQL PostGIS Geometry and Geography
  • Oracle SDO Geometry

The tools do not support Esri Spatial Types. Features created using Esri Spatial types are created with ArcGIS Software and hence won’t have any geometry errors.

Below is the list of geometry problems that may occur from data stored in an enterprise geodatabase and the corresponding fix that will be performed by the tool:

  • NEEDS_REORDERING—The shape will be reordered and duplicate points removed.
  • SE_INVALID_ENTITY_TYPE—Not repairable (feature must be deleted).
  • SE_SHAPE_INTEGRITY_ERROR—Not always repairable.
  • SE_INVALID_SHAPE_OBJECT—Not always repairable.
  • SE_COORD_OUT_OF_BOUNDS—Not repairable.
  • SE_POLY_SHELLS_OVERLAP—Overlapping shells are merged.
  • SE_TOO_FEW_POINTS—Not repairable.
  • SE_INVALID_PART_SEPARATOR—Not always repairable.
  • SE_INVALID_POLYGON_CLOSURE—Unclosed shells will be discarded (the resulting polygon may be left empty).
  • SE_INVALID_OUTER_SHELL—An attempt to repair the feature’s outer shells will be made; shells may be discarded in the process.
  • SE_ZERO_AREA_POLYGON—Converted to empty shape.
  • SE_POLYGON_HAS_VERTICAL_LINE—The vertical line is removed (the shape may be converted to 2D).
  • SE_OUTER_SHELLS_OVERLAP—Overlapping parts are merged.
  • SE_SELF_INTERSECTING—Intersection points are added as needed.

Related topics

Created by: Sam P

GNSS support in ArcGIS Pro 2.7

A new feature introduced in ArcGIS Pro 2.7, is GNSS (GPS) location support. You can now connect a GNSS device to ArcGIS Pro to assist with feature creation and navigation within a map or scene. In this article, I’ll provide an overview of this new feature, then discuss some hardware considerations for working with GNSS.

Continue reading

Multi-tier base deployment of ArcGIS Enterprise 10.8.1 using ArcGIS Enterprise Cloud Builder for Microsoft Azure.

ArcGIS Enterprise Cloud Builder for Microsoft Azure is an application you install on your local Microsoft Windows machine to deploy ArcGIS Enterprise and stand-alone ArcGIS Server sites on Microsoft Azure. Depending on what role you want the site to fill, Azure Cloud builder provides several deployment options.

This blog will discuss the workflow of deploying multi-machine ArcGIS Enterprise (non-HA) with each component of base Enterprise deployment – Portal for ArcGIS, ArcGIS GIS Server, Data Store and App Gateway – installed on its own dedicated servers. One thing to note, starting from Azure Cloud Builder 10.8 for Microsoft Azure,  the base ArcGIS Enterprise deployment is no longer accessed through a load balancer and reverse proxy. The 10.8 version of ArcGIS Enterprise Cloud Builder for Microsoft Azure creates ArcGIS Enterprise deployments that use a single Azure Application Gateway to access the portal and all federated servers – this is referred to as a version 2 (V2) deployment type.

CONTINUE READING

Basic Enterprise Geodatabase Maintenance

So, you’ve got your new database set up by IT or your vendor, you create an Enterprise Geodatabase on top of it and you’re good to go! Not quite. There is a tendency for a gap to form between the maintenance tasks required for a RDBMS and a Geodatabase. Normally, the IT department will not be aware of the specific requirements of an Enterprise Geodatabase.

Administrative users

There is a hierarchy of users in the database. When the geodatabase is created, two users are important: the database administrators and geodatabase administrators.

Geodatabase Roles

These administrators each have their own subset of roles when it comes to doing maintenance on a database.

RoleDatabase AdminGeodatabase Admin
ResponsibilitiesThe database administrator is always created when you install the RDBMS.   Owns the database instance and databases inside it.Creates databases and schemas.Creates database users and roles.Grants database access permissions. Database-wide maintenance tasks: Backup and recovery.Configure storage in the database.  The Geodatabase administrator is created when the Enterprise Geodatabase is created/enabled on top of an RDBMS. Owns the schema.Creates users and roles.Grants table access permissions.Manages the ArcSDE configuration.Configures DBTune parameters.Perform various maintenance tasks:Post/reconcile versions.Compress.Update indexes.Update statistics.Manage the top-level version.  
WhoIT department/ RDBMS Vendor/ Esri Australia Managed Cloud SolutionsGIS Administrator
Geodatabase Administrative Roles

Some of these activities are performed ad-hoc (e.g. creating users, assigning roles), while others should be performed routinely as part of good system maintenance.

Neglecting routine geodatabase maintenance could result in degraded performance, overly-complicated lineages and excess consumption of disk space.

The below script is a Python script which performs the following routine maintenance:

  • Reconcile all versions
  • Compress the geodatabase
  • Rebuild indexes on all tables
  • Update database statistics on all tables

This script can be scheduled to run every weekend as a good start to keeping the database in good shape.

import logging
import arcpy
from datetime import date

# Set variables
#First connection should connect using the OWNER of the data. If multiple owners exist this script will need to be modified.
daConnectionString = r'D:\temp\sql_dev_db.gisowner.sde' 

#This should connect using the Geodatabase admin user.
sdeConnectionString = r'D:\temp\sql_dev_db.sde.sde'

logFileLocation = r'D:\temp\GDBMaintenanceLog'
reconcileLogFileLocation = r'D:\temp\reconcilelog_{}.txt'.format(date.today().strftime("_%d_%m_%Y"))
targetVersion = 'sde.DEFAULT'

# Setup logging
logging.basicConfig(filename=logFileLocation + date.today().strftime("_%d_%m_%Y") + '.log',
                        filemode='a',
                        format='%(asctime)s,%(msecs)d %(levelname)s %(message)s',
                        datefmt='%Y-%m-%d %H:%M:%S',
                        level=logging.INFO)

logging.info("Maintenance script starting")

try:
    # Set the workspace
    arcpy.env.workspace = sdeConnectionString

    workspace = arcpy.env.workspace
    arcpy.env.overwriteOutput = True

    logging.info("Environment set to {0}".format(sdeConnectionString))

    #Sometimes a stale connection can cause issues with versioning.
    arcpy.ClearWorkspaceCache_management()

    # Block new connections to the database.
    logging.info("The database is no longer accepting connections")
    arcpy.AcceptConnections(sdeConnectionString, False)

    # Disconnect all users from the database.
    logging.info("Disconnecting all users")
    arcpy.DisconnectUser(sdeConnectionString, "ALL")

    # Get a list of versions to pass into the ReconcileVersions tool.
    # Only reconcile versions that are children of Default
    logging.info("Compiling a list of versions to reconcile")
    verList = arcpy.da.ListVersions(sdeConnectionString)
    versionList = [ver.name for ver in verList if ver.parentVersionName == targetVersion]

    # Execute the ReconcileVersions tool.
    # We are not posting any edits to default or deleting any versions here
    logging.info("Reconciling all versions")
    arcpy.ReconcileVersions_management(sdeConnectionString, "ALL_VERSIONS", targetVersion, versionList, "LOCK_ACQUIRED",
                                       "NO_ABORT", "BY_OBJECT", "FAVOR_EDIT_VERSION", "NO_POST", "KEEP_VERSION",
                                       reconcileLogFileLocation, 'PROCEED', 'RECONCILE')

    # Run the compress tool.
    logging.info("Running compress")
    arcpy.Compress_management(sdeConnectionString)

    # Allow the database to begin accepting connections again
    logging.info("Allow users to connect to the database again")
    arcpy.AcceptConnections(sdeConnectionString, True)

    # Update statistics and indexes for the system tables
    # Note: to use the "SYSTEM" option the user must be a geodatabase or database administrator.
    # Rebuild indexes on all (including system) tables
    logging.info("Rebuilding indexes on the system tables")
    arcpy.RebuildIndexes_management(sdeConnectionString, "SYSTEM", "", "ALL")

    # Update statistics on all (including system) tables
    logging.info("Updating statistics on the system tables")
    arcpy.AnalyzeDatasets_management(sdeConnectionString, "SYSTEM", "", "ANALYZE_BASE", "ANALYZE_DELTA", "ANALYZE_ARCHIVE")

    '''
    *********************
    Data Owner(s) Section
    *********************
    '''
    # Set a variable for the workspace
    arcpy.env.workspace = daConnectionString

    # Set a variable for the workspace
    workspace = arcpy.env.workspace

    logging.info("Environment set to {0}".format(daConnectionString))

    # Get a list of datasets owned by the data owner user

    # Get the username for the workspace
    # this assumes you are using database authentication.
    # OS authentication connection files do not have a 'user' property.
    userName = arcpy.Describe(arcpy.env.workspace).connectionProperties.user
    logging.info("Connected with user {0}".format(userName))

    # Get a list of all the datasets the user has access to.
    # First, get all the stand alone tables, feature classes and raster datasets owned by the current user.
    oDataList = arcpy.ListTables('*') + arcpy.ListFeatureClasses('*') + arcpy.ListRasters('*')

    # Next, for feature datasets owned by the current user
    # get all of the feature classes and add them to the master list.
    for dataset in arcpy.ListDatasets('*'):
        oDataList += arcpy.ListFeatureClasses(feature_dataset=dataset)

    logging.info("Tables to be included {0}".format(oDataList))

    # Rebuild indexes and analyze the data owner tables
    arcpy.RebuildIndexes_management(workspace, "NO_SYSTEM", oDataList, "ALL")
    logging.info("Rebuild indexes complete")

    arcpy.AnalyzeDatasets_management(workspace, "NO_SYSTEM", oDataList, "ANALYZE_BASE", "ANALYZE_DELTA", "ANALYZE_ARCHIVE")
    logging.info("Analyze datasets complete")

    '''
    *************************
    End Data Owner(s) Section
    *************************
    '''

except Exception as e:
    print(str(e))
    print(arcpy.GetMessages(2))
    logging.error(arcpy.GetMessages(2))
    logging.error(str(e))

finally:
    logging.info("Script completed")
    print("Done.")

Input also provided by Sam Pienaar and Greg Briggs

Esri Basemaps – December update for Australia

Parcels, parcels, parcels…

Big thank you to the following agencies, who between them contributed over 10 million property parcels into the Esri Basemaps:

State Agency Sync or Upload
VIC DELWP Upload
TAS DPIPWE Upload
NSW DCS SS Upload
ACT ACTGov Sync
QLD DNRME Upload

Jump into the Esri Topographic basemap and have a look for yourself.

Burnie – Parcel data from DPIPWE in the Topographic Style – preview
Continue reading

How to connect to an ArcGIS License Manager using IPv4 and IPv6 connections.

Are you having trouble connecting to the ArcGIS License Manager (LM) when using IPv4 and IPv6 connections?  When using Dual-IP stacks, your computers and other devices can run both protocols however there can be issues when connecting to an ArcGIS License Manager.  The connection is usually set to be IPv4 (e.g. internal network) or to IPv6, for example MS DirectAccess, that uses IPv6 connections; not to both protocols.

Continue reading

How to compartmentalise content in ArcGIS Online

Do you have guests or temporary workers in your organisation’s ArcGIS Online or Portal for ArcGIS?

Their accounts can often see anything that is shared to the organisation. This can be undesirable.   How do you enable your organisation’s core-staff to have access to a wide range of content but have that content unavailable to these temporary users? 

If this is something that your organisation has concerns about, it’s a great time to start using Groups to divide and organise your content into sections or categories for your different types of users. This process is called Compartmentalization.

Continue reading

Using Calculate Field Tool to calculate a value in a table based on another field using Python

In some cases, you may want to calculate a field based on the value of another field. A good example of this would be when you may want to create or calculate domain code values from a description field.

In the following example we are using some Tree data to calculate the Risk_Code associated with the Risk description which can then later be used as a Domain.

  • To begin, open the Attribute Table for the data in ArcGIS Pro
  • Select applicable field to calculate (i.e. RISK_CODE)
  • Right click field (RISK_CODE) > Field Calculator
  • Select Python 3 as the Expression Type

Code Block

def calcRiskCode(riskDesc):
     if riskDesc == "High":
         return 1
     elif riskDesc == "Medium":
         return 2
     elif riskDesc == "Low":
         return 3
     else:
         return 0

Expression:

calcRiskCode(!RISK_DESC!)

  • Click OK.

For example:

Result:

Further Calculate Field Examples:

https://pro.arcgis.com/en/pro-app/tool-reference/data-management/calculate-field-examples.htm

Sam Pienaar.

How do I add Bing Maps to the Basemap collection in ArcGIS Pro?

Would you like to add Bing Maps as basemaps in ArcGIS Pro?  Here are the steps that will get you up and running!

To add Bing Maps as a basemap in ArcGIS Pro, you will need to do two things:

1 – Configure ArcGIS Online to access Bing Maps.
2 – Create the Bing Maps Basemaps and share with a group that is configured as the organisations default basemap group

Once these configurations are complete, users will be able to access the Bing Maps from the Basemap gallery in ArcGIS Pro.

Things to keep in mind, you will need to be logged in as an ArcGIS Online Organisation Admin to make the following changes on the Organisation settings page.

Steps to add Bing Maps to the Basemap collection in ArcGIS Pro

Continue reading