Is ArcSDE dead? Part II


The term ArcSDE/SDE is being gradually replaced with the term “Multiuser Geodatabase” and this is probably creating some confusion among the users of the ArcGIS platform. In order to understand the difference between ArcSDE and multiuser geodatabase, let’s start clarifying what is and what is not ArcSDE.

What is not ArcSDE:

ArcSDE is not a product. It’s a technology. In the same way ArcGIS is not a product, is a platform. Only prior to ArcGIS 9.2, ArcSDE was a standalone software product. At the ArcGIS 9.2 release, ArcSDE was integrated into both ArcGIS for Desktop and ArcGIS for Server.

ArcSDE is not only the application sever connection, a piece of software that packs the giomgr.exe and a gsrv.exe processes. This is a common mistake that I have mentioned in my previous post but from a more business perspective. I’ll try to do the same exercise now but from a technical perspective.

So, what is ArcSDE  – also called “the database enhanced with the ArcSDE technology” or simply the “ArcSDE geodatabase” or the “Multiuser geodatabase”?

Let’s answer this question step by step.

First of all, ArcSDE is a hybrid technology built up from different components. These components are implemented in two places, the client (ArcGIS for Desktop/Pro or ArcGIS Server, mobile, etc.) and in the relational database management system (any of the supported ones).

The components that make the ArcSDE technology are those shown in the slide number 3 and 4 below.

cmgd

Note that, the above are the slides of the official geodatabase training course version 10.2, taught all around the world therefore, at least until version 10.2, we have been using, explaining and discussing what ArcSDE is.

The slides below are from the same course at version 10.3. Note that the content is the same although the term ArcSDE has been replaced by “Multiuser geodatabase” (except that the translator component is only in the client side reflecting the deprecation of the application server connection).

CMGD_103

What are the components of the ArcSDE/Multiuser geodatabase technology? – Note that the lack of any of the components described below would mean that the multiuser geodatabase wouldn’t work.

  • The client (ArcGIS for Desktop (32 bits) , ArcGIS Pro and ArcGIS Server (64bits), etc.)
  • The repository of ArcSDE/multiuser geodatabase, the geodatabase system tables, stored procedures and functions.
  • The management tools:
    • Geoprocessing tools
    • Database tools – for example, the functionality to backup and restore is considered part of the ArcSDE technology.
    • And until version 10.2.x, the command line tools. Deprecated in version 10.3.
  • And finally, the translator, this is the component that translates every map interaction (clicks on the map performed by the user) into SQL statements – Since the database doesn’t understand “clicks on a map”, it only understands the SQL language, we need something that acts as an “interpreter” between the user and the database.

Notice that in the slide above (version 10.2) the translator component is floating in the middle of the client and the server. There is a reason why this is so… the reason is related with how the client connects to the multiuser geodatabase. Up to version 10.2, there were two different connection types, application server connection and direct connections – first released with version 9.2 -. Let’s review some of the particularities of each connection type:

  • Direct connections (two tier connection): If the client connects using direct connections, the translator component is located in the client side and requires to have the database client installed in the machine where the ArcGIS client is installed -. This is the default connection method.

transl

  • Application Service connections (3-tiers connection). The application server connection has been the traditional way to connect to an enterprise geodatabase. It requires a service (commonly called “sde” or “arcsde”) which is normally installed and configured in the same machine where the database server is installed.

transl22

The sde service packs two processes– the giomgr and the gsvr – the giomgr.exe is in charge of listening for incoming connections (clients creating a connection to the enterprise geodatabase using the sde service), when the connection is accepted, the giomgr fires up the “gsvr.exe”, one per client connection. Each gsvr.exe will then translates the clicks on the map performed by the user into SQL statements that are then passed to the database, the database interprets the SQL, generates the answer which is finally sent to the client through the gsvr.exe process, as it is shown below.

With this connection type all the translation load is placed on the server side.

sde_service

As I mentioned before, the application server connection method has been deprecated in version 10.3 and with it, the command line tools.

I’m sure there are many reasons why the application server connection has been deprecated. From my point of view there is one reason that deserves to be discussed more in detail. Performance.

When the application server connection was released the processing capacity of the servers was much higher than the processing capacity of the workstations. The application server connection was a good approach to put all the translator processing load into the server machine and so to use the server processing capacity to perform the translation.

Nowadays however, there are reasons why we wouldn’t want to use application server connections:

  • Today, we can use client’s machines processing capacity to perform the translation and in this way, to share the translation load among all the clients connecting to the multiuser geodatabase. We just need to install the database client and to connect to the multiuser geodatabase using direct connections. In this way, the translation will happen in the client machine, leaving the server to use the processing capacity for other purposes.
  • Application server connections also create single points of failure, if the sde service fails, no one can connect to the geodatabase whereas with the direct connection this risk doesn’t exist.
  • Application server connections put much more load on the server if we compare it with direct connections.

With regards to the last point, the capacity planning tool is an excellent tool to model each connection type and to see how much server processing capacity is required. Below I have done a quick exercise that shows the amount of server processing power required to perform the same workflow using application server connections versus direct connections.

I have modelled a simple workflow, ArcGIS for Desktop 10.2 connecting to a multiuser geodatabase using direct connection [1] and application server connections [2]. Both workflows generate 50.000 transactions per hour (140 concurrent users approx. with a productivity of 6 displays per minutes as average per user (a standard user).

DC

[1] Workflow: ArcGIS Desktop connecting to a multiuser geodatabase using direct connection (DC).

ASC

[2] Workflow: ArcGIS Desktop connecting to a multiuser geodatabase using application server connection (ASC).

Notice that using direct connection the workflow requires 14,7% of the server processing capacity whereas using application service connections it requires 29,3% of the server processing capacity, almost double. This is another reason  to reconsider the use of application server connections, if you are still using them. 

I hope I was able to bring some light into this topic. If you are interested in expanding your understanding about enterprise geodatabases, I highly recommend to attend the following training courses (in particular the second one):

Deploying and Maintaining a Multiuser Geodatabase (10.3)

Implementing Versioned Workflows in a Multiuser Geodatabase (10.3)

Regards, Walter Simonazzi

 

Got something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s