safe.com blog community

Enterprise Integration tackles two main problems that make it difficult for an organization to freely share data:

  1. Data is stored in a format or system that restricts users from gaining access at the source.
  2. Data is complex and expert knowledge is required to navigate the system or understand the data itself.

Data Integration and Application Integration help make information physically accessible and easier to understand, but total Enterprise Integration is a tall order for many organizations. This is where Automation comes into play. Automation is the process of building a task or service that can run with minimal human interaction. Whether you are a team of 10 or 10,000, the ability to employ automation is essential to enterprise growth.

Why Automation?

Who am I to tell you that automating processes is key to a successful business? Instead, I’ve decided to let real organizations do the talking. I’ve sifted through our stack of customer stories and can say with confidence that the power to ‘Automate’ is a common theme throughout. I’ve identified the top five common trends (in no particular order) that businesses highlighted as the major benefits of automating their tasks.

1.   Enhanced Data Quality

“The world’s most valuable resource is no longer oil, but data” – The Economist, 2017. This is one of our go-to quotes at Safe Software, we can probably all agree that data is an abundant resource, but it is only of value if it provides an accurate representation of the information it is intended to deliver.

One of the biggest challenges any organization that works with data will face is handling erroneous datasets and ensuring the data they output is high quality.

Automation gives you the power to create and streamline workflows that can be run again and again…and then again! This ensures the quality of the result will be consistent, precise and accurate every time, while removing the chance to introduce human error. California CAD Solutions and Environment and Climate Change Canada both remarked on improved data quality through automation.

2.   Provide Data in Real Time

Automation allows you to deploy workflows to run on a schedule at a recurring interval, or react to events as they happen in real time and process new information. This flexibility gives the individual access to the most up-to-date information, enabling them to make more informed decisions.

Powerlink Queensland created a fully automated, data-driven emergency management system for extreme weather events including bushfires, cyclones, flooding, thunderstorms, and earthquakes. A real-time monitoring and notification system provisioning the newest weather information is critical to enabling rapid response in emergencies.

3.   Eliminate Data Silos

No enterprise is free from data silos, often introduced by use of legacy systems and disjointed departments. Enterprise integration is all about performing complex data and application integrations to consolidate information so that it is consistent and easily accessible to any team member through whatever tools they use.

The City of Coquitlam created fully automated integration processes between multiple applications and noted considerable improvements in data integrity and accessibility between its internal teams and external vendors.

4.   Save Time

The time savings given through automation are a win-win for all parties involved and this section could probably be divided into at least five further points if we really wanted to!

First, automation improves productivity. In business, a person’s time is a valuable resource. Automation removes a series of tasks from a person/team’s to-do list, and not only does this eliminate constant interruptions*, but just like at Grafton Technologies, it allows your team to allocate the time once spent manually processing integration tasks towards other pieces of work.

Second, task efficiency and hence the overall throughput is greatly increased when relying on a machine rather than a human. Tasks that once took days or hours to complete, through automation can take only minutes, enabling you to deliver twice the product (or much more!) in half the time.  Utah Department of Transportation reduced processing times by almost 80% by implementing an automated application integration e-ticketing system.

Lastly, and arguably the most important time savings, is that reducing the overall delivery time of your service can have a hugely positive impact on the end user experience, and a happy customer makes for a happy organization! The City of Burnaby provides an underground utilities information package, and not only did automating this package delivery lend to a much faster lead time, but the City can now host the service 24/7, providing their service to customers outside of their regular business hours.

*Numerous studies have found interruptions at work can be hugely costly to your productivity, from do-overs to momentum loss.

5.   Save Money

Perhaps saving time and saving money are one in the same, but this is worthy of being highlighted in its own right because, well, who doesn’t want or need to save themselves some cash! There are countless reports of cost savings, I think it’s best to let the numbers speak for themselves in support of this point.

Geoscience Australia – The creation of Elevation Information System (ELVIS), an automated system for data delivery, reduced costs by 600%.

Arkansas Geographic Information Systems Office – After migrating their open data portal to the cloud, their three-year financial projections estimated a 73.4% savings just to infrastructure costs.

Typical Enterprise Tasks that Can Be Automated

Now we’ve established automation is a necessity to scale your business to meet growing demands. In case the customer stories weren’t inspiration enough, I’ve identified some high-level tasks that are ideal for automation:

FME aka. The Enterprise Integration Solution

Safe Software’s FME is a data integration platform that has built-in support for hundreds of formats, applications and transformation tools. More than that, it provides a no-code solution to build and automate custom integration workflows, sharing information seamlessly to keep teams connected.  

Before we take a look at how you can automate tasks, I wanted to give a special shout-out to a single FME Workbench transformer, the HTTPCaller, that is integral to building any enterprise integration workflow.

HTTPCaller: Submit API requests to fetch, post, update or delete information from a system without being logged in to the user interface. This transformer provides endless opportunities for the applications you can communicate with. Read our Ultimate Guide to APIs for a complete introduction to this topic.

Workflow automation takes place in FME Server, using the oh-so-conveniently named functionality “Automations”. Automations aid the implementation of Enterprise Integration Patterns – a series of steps that provide the framework for enterprise integration solutions. Now let’s take a closer look at some of the features that give you the power to Automate.

Triggers: To automate a workflow, you have to control what is going to kick start the process. Automation Triggers run a workflow in response to an external or system event—to name a few, this includes creating schedules, reacting to an email, a file upload to a Directory, FTP Server (and more) or even configure a Webhook to receive messages from almost any other application whenever a specific function is performed!

 

External Actions: Once a transformation is complete, upload the output to your file server (e.g. Dropbox, Amazon S3) or set up informative notifications to message channels (e.g. Azure Event Grid) or to relevant persons (e.g. email, Slack).

 

 

Guaranteed Delivery: Prepare for failures ahead of time by configuring jobs with retry attempts. If something interrupts your automation there is still no manual intervention needed! Learn more.

 

 

 

Automation Writer: Provides the ultimate flexibility to build complex integration workflows. Use the writer to create multiple output ports to workspace nodes and bring data out of a workspace into an Automation for further downstream processing. Learn more.

 

 

 

Split-Merge Block: Brand new in FME 2021.1, used in combination with the Automation Writer, group multiple workspace actions together and wait for all jobs contained within the block to complete processing before any further downstream action is triggered. Learn more.

Get your free trial of FME Server today and check out the Getting started with Automations tutorial series to bring you up to speed with all things Automation.

“Get FME! You’ll save yourself a staff member! Seriously, you’ll automate everything.” – Tara Preston, City of Bozeman

Additional Resources

Enterprise Integration Patterns:

APIs:

Application Integration Automation Data Integration Enterprise Microsoft Azure Slack

Holly Coxon

Holly is an FME Server Technology Support Team Lead at Safe Software. Among other things, she provides technical support, creates content for the Community, presents webinars and contributes to product development through functional testing and sharing customer feedback. Holly spends her weekends exploring the great backyard of British Columbia.