Which ArcGIS Online items are consuming the most credits through storage?

As your ArcGIS Online organization grows, so does the amount of content, and subsequently the credit consumption to store it. It is important for Creators to understand which types of items consume storage credits at a higher rate, and for administrators to identify which users and items are incurring the highest usage.

Understanding credits

Credits are the currency used across ArcGIS and are consumed for specific transactions and types of storage. As this blog post specifically looks at item storage, I would encourage readers new to ArcGIS Online and credits to review the resources below to better understand their application more broadly:

Credits used for storage

The ArcGIS Online help documentation lists credits by capability, including the different storage rates based on the item:

ArcGIS Online > Understand Credits > Credits by capability

Storage costs are associated with size. The rates are shown per month and calculated hourly to account for changes as content is created. The documentation goes into more detail but we can see different item types incur different rates to reflect the way they are stored in the ArcGIS Online backend (database storage is more resource intensive than file).

What we can see is that there are essentially two cost rates depending on the storage type:

2.4 credits per 10 MB stored per month, calculated hourly:

  • Feature storage. e.g. a Hosted Feature Layer. (excluding feature attachments i.e. photos)

1.2 credits per 1 GB stored per month, calculated hourly:

  • Feature attachments
  • Feature collections
  • Features associated with location tracking
  • imagery storage
  • All other content except for hosted feature layers and content in ArcGIS notebooks.

This means it costs roughly 200x less per mb to store content that is not a Hosted Feature Layer (excluding size from their attachments, such as photos). It can also help in identifying which items are incurring more credits than others as the storage type will be reflected in ArcGIS Online usage dashboards and reports.

How many credits are being used daily for storage in my organization?

ArcGIS Online administrators can explore credit utilization by going to their Organization > Status page, pictured below. There are tools available to export this summary or define the usage period.

The Credit Utilization Summary allows us to view daily and period based consumption of credits.

Observing the photo above we can identify that approximately 20 credits are being consumed daily for storage (green bars). There are a few days where we have consumed credits through Analysis (blue bars) but this is relatively low. If they were higher we could explore the usage by interacting with the graph.

Scrolling down the page we then see a Usage Aggregation by type.

Usage Aggregation by Type. We can see that storage is the relatively largest consumer of credits for this organization.

By clicking on the Storage (green) section of the Chart, we can get a more detailed breakdown of the credit utilization based on type:

Storage Usage Chart. We can now see that Feature Storage uses the most credits relative to other storage types, such as file or tile.

It is is now clear that feature storage is accounting for the majority of credits used for storage in this period. We can bring up a list of items by clicking again on the chart, selecting Standard Feature Data Store in the list and clicking View item details:

Important Note: The ‘Size’ column in this summary shows the total size of the item, i.e. both the feature storage and attachments. This is misleading and has been lodged with the development team under BUG-000141394. If this affects you, please reach out to technical support and asked to be attached to the issue. The feature vs file attachment sizes are distinct at the Item details page and in the Item Report.

By exploring the credit status dashboards we have now identified which capabilities (storage) are incurring the most credits and gotten a quick summary of the items by combined size. For a more accurate breakdown of item feature vs file size, continue reading the blog and create an Item Report.

You could take a similar approach to explore other credit capabilities that may be using credits, such as analysis.

Are these items being used or just taking up space?

Moving from the Credits status dashboard to the Content Dashboard, we can explore how members are creating, using, and sharing content. The charts, tables, and map support drill-down interactions that allow you to refine the details. Adjust the time at the top of the app to change the time period. You can see content activity up to the past 12 months. The default is two weeks.

Using this dashboard I can search for the largest item or owners and see its usage

We can also navigate to the item directly in ArcGIS Online by searching for it, managing the members items (requires admin privileges) or by clicking the URL from an Item Report.

The Usage tab on the Item Details shows the number of requests against this service

We can now identify that this item is using 99 credits a month in storage, was created in March, has not been used much since, and subsequently reach out to the owner to discuss potential removal.

I would generally advocate touching base with the item owner – and backing it up offline – before removing anything to ensure that live content is not inadvertently deleted! This is where good content management helps. Items with metadata, thumbnails and clear naming conventions help administrators understand the items purpose and usage.

Generating Item Reports – a detailed storage breakdown

Now that we know feature storage is using a lot of credits, we need a more detailed breakdown to understand the feature and file storage per item, as well as explore which users own the items. An Item Report will help explore this.

We can create Reports by going to Organization > Status > Reports > Create Report. There are a few useful reports we can create, including Activity, Members and Credits. To identify which items are the largest – and consuming more credits through storage – we can generate an Item report.

Credit consumption is calculated by the total organization database size, so we won’t see a credits field in the output CSV. We can infer the approximate cost from the item size columns. Remember there are different rates for different item types – feature storage will consume approximately 200x more credits per mb than files – so we’ll need to apply different estimates for each field.

To calculate the monthly storage rate for feature storage:

  • (convert bytes to kb to mb) * applied rate of 2.4 per 10mb)
  • = (cell / 1024 / 1024) / 10 * 2.4)

To calculate the monthly storage rate for file storage:

  • (convert bytes to kb to mb to gb) * applied rate of 1.2 per gb)
  • = (cell / 1024 / 1024 / 1024) * 1.2)

Insert extra columns in the Item Report and add the cells.

The Item Report shows feature and file storage sizes for each item in your organization in bytes. You can convert this to mb or gb and apply the credit rates to get an estimate per month, and thus understand which items are consuming the most credits.

Important Note: These estimation tools are not officially endorsed by Esri. They are provided as a guide by the author to help you explore your content and identify more larger items. There are a number of additional variables, such as changing content size, that need to be considered.

The other fields in the report will help explore content in your organization. For example, filter by User , or use the URL column to view the Item Details page. The below documentation page gives more information on the fields generated in Item and other Status reports:

ArcGIS Online help > Report Fields

What contributes to Hosted Feature Layer size?

There are a number of variables that may increase the size of a Hosted Feature Layer, including:

  • Number of features
  • Number of fields
  • Complexity of the geometry (think of 1 point, vs millions of vertices that make up a complex feature)
  • Editing and editor tracking settings
  • Layer optimization
  • Sync
  • Attachments (stored and costed separately)

There is an apparent correlation of size with features (number of, spatial complexity and attribute data). This can be reduced accordingly by removing unnecessary fields and features on publish. Simplifying the geometry will also reduce size and improve display performance:

Other size contributors are manifested in backend tables and less apparent to users. Layer optimization improves drawing times in web clients but users additional storage space to do so:

ArcGIS Online Help > Manage Data > Manage Hosted Feature Layers > Optimize drawing of complex lines and polygons

Enabling Sync (required for offline use and collaboration) can also increase the feature storage size over time:

ArcGIS Online Help > Manage Data > Manage Hosted Feature Layers > Manage editor settings

If you have a Hosted Feature Layer with very few features relative to the size, it could be tied to one of the above settings. For example, a Hosted Feature Layer with less than 100 features could be hundreds of mb in feature storage size if it has editing and sync enabled, and then been taken offline extensively over a long period. Every time a user takes the layer offline it creates entries in the sync tables and accumulates over time. You can immediately decrease the size by disabling (and re-enabling) the setting. Just make sure to check-in (sync) all your offline data collection first – otherwise they won’t be able to sync changes!

Best practice, budgeting tools and further reading

To stay on top of credits in your ArcGIS Online organization:

  • Enable Credit budgeting per user
  • Work with users:
    • Create custom member roles with privileges to limit access to credit consuming tools
    • Educate the users on how credits are consumed. Privileges can then be provided to users who have demonstrated understanding.
  • Monitor organizational usage as explored in this blog

For more information on credits and best practices:

Have a lot of users and content? A Premium Feature Data Store comes with dedicated database resources for your organization and allocated 1TB+ storage limits.

Got something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s