How to Do More with ArcGIS Online

Creating a web map in ArcGIS Online is a great way to share and analyze data in the cloud.

For example, 1Spatial uses it for a live fleet positioning system that updates vessel locations every two minutes, the City of Sandy Springs uses it to provide public access to utility permits, and the City of Oshkosh Child Abduction Response Team uses it to share critical “Place Last Seen” information pulled from a real-time SQL database.

In all of these scenarios, ArcGIS Online is the platform of choice for sharing complex data in a simple, visual way.

It’s easy to build an ArcGIS Online map that displays data from supported file formats or feeds—but what if you want to display data from an unsupported type? What if you want to blend multiple data sources, filter, process, and validate it, before viewing the customized result in a web portal? And how do you keep the map up to date as the source datasets change?

Let’s look at how you can use data integration workflows to publish data to ArcGIS Online (or Portal for ArcGIS) like in the above examples. We’ll discuss how to create maps that incorporate a variety of data sources, how to leverage bulk loading and repeatable workflows, and how to keep your map up to date in real time using FME.

Building a Data Integration Workflow for Publishing to ArcGIS Online

When using FME Desktop or the FME-powered ArcGIS Data Interoperability extension, there’s no such thing as a format that’s “incompatible” with ArcGIS Online. Data is just data, whether you’re working with GIS, CAD, web services, databases, cloud systems, or any other type. If it contains spatial information, then you can massage it into the right format to be viewed in an ArcGIS Online map.

Add readers and transformers to your workspace to integrate, process, filter, and validate the data. Then add an ArcGIS Online Feature Service Writer to publish data to an ArcGIS Online feature service or the ArcGISOnlineConnector transformer to manage the files stored in ArcGIS Online.

Watch this demo to see how to use this transformer to upload files for sharing, update existing files, and list and download files.

This FME workspace integrates two source datasets and processes the data through a series of transformers before publishing it to ArcGIS Online. Screenshot from this demo.

For example, Victoria International Airport maintains a map of trees around the airport with cost estimates for removing ones that grow too high. They use an FME data integration workflow to integrate LiDAR scans and raster imagery, determine which trees are too high, calculate cost estimates for their removal, and publish the results to ArcGIS Online.

Making the Workflow Repeatable

In FME, you can easily set up the bulk migration of data into ArcGIS Online, as well as repeatedly run your workflow as new source data arrives. To make a workflow repeatable, it’s often helpful to use dynamic readers and writers. This means your workflow won’t be tied to a static schema. For example, in this demo, we set up a workflow that migrates arbitrary MITAB files to ArcGIS Online, allowing for dynamic schemas so it can be run repeatedly. You can also accomplish batch processing using the WorkspaceRunner transformer.

Keeping an ArcGIS Online Map Up To Date

FME Server Automations is key to running your data integration workflow as needed, for example, when working with data that frequently changes. Set up a trigger, like a schedule, directory watch, or webhook, and whenever this event occurs, the Automation will launch your workspace to process the data and publish it to ArcGIS Online. It’s often helpful to run workflows on a schedule, or to process everything in a folder whenever new files are placed there.

Keeping an ArcGIS Online map up to date often involves tracking changes and applying incremental updates so you don’t have to re-publish the entire dataset every time. Read more about this in the blog about change data capture.

This demo shows how to keep an ArcGIS Online Feature Service up to date as new information arrives via JSON feed. The workflow uses the ChangeDetector transformer to identify changes in the JSON data, and then updates only the changed records in the ArcGIS Online map.

As an example of a real map displaying live data, Iowa DOT created a web portal that shows cam images and data from snowplows, which the public uses to check live road conditions before driving. Underneath, FME retrieves plow data from Oracle every minute, creates a KML file that is uploaded to Microsoft Azure, then publishes that to ArcGIS Online.

Bonus: Extract Data from ArcGIS Online

You can also use FME to extract data from ArcGIS Online and write it out to any format or system, say to bring it into a data warehouse for further analysis or to generate a report. For example, PNM uses FME to back up their ArcGIS Online data. Watch this demo in our webinar.

Automating this type of workflow is also possible using webhooks. Use an ArcGIS Online webhook as a trigger, and then send it through the ArcGISOnlineWebhookDecoder for processing. See this in action in this demo.

Real-world Examples: Mapping COVID-19

con terra GmbH maintains an up-to-date COVID-19 dashboard at a district and municipal level. The visualization displays information like case numbers and vaccination rates to facilitate decision-making and crisis management.

Three FME Server Automations retrieve and process case data from different sources at intervals of 15 minutes, 6 hours, and daily. The data is stored in SQLite, including historical data to produce timeline views, and the visualization is implemented as ArcGIS Online Feature Services in map.apps. They also use FME and Survey123 for ArcGIS to optimize case tracking and automate contact tracing in Kreis Steinfurt.

con terra’s interactive map displaying critical COVID-19 information, powered by ArcGIS Online and FME and updated continuously.

BlueDot also uses ArcGIS Online as one of the outputs for their early warning system for disease outbreaks like COVID-19. Their workflow is powered by FME and you can learn more about it in this blog post.

• Did you know: We’re offering free FME licenses for COVID-19 projects. Learn more & apply

Have you used ArcGIS Online and FME or the Data Interoperability extension to create an interactive map? Let us know in the comments!

Want to learn more? Here are some related resources:

• Tutorial: How do I write updates to an existing ArcGIS Online feature service?
• How to Use the ArcGISOnlineConnector
• FME & ArcGIS Online FAQ
• Download a free trial of FME

Tiana Warner

Tiana is a Senior Marketing Specialist at Safe Software. Her background in computer programming and creative hobbies led her to be one of the main producers of creative content for Safe Software. Tiana spends her free time writing fantasy novels, riding her horse, and exploring nature with her rescue pup, Joey.