If you have data stored in multiple systems, then enterprise integration patterns are relevant to you. It’s possible that, without realizing it, you are already making use of these patterns within data and application integration workflows.
Here we will explore why enterprise integration patterns are necessary to your organization, demonstrate some common use cases, and briefly touch on how you can begin applying these patterns to your business processes.
Before we dive in, let’s take a moment to define the two main concepts I’ll be introducing:
-
- Enterprise Application Integration (EAI): the process of connecting disparate systems to allow for “communication” and data sharing between them.
- Enterprise Integration Pattern (EIP): a series of steps that provide the framework for enterprise integration solutions. EIPs are not tied to specific implementations but provide a method of “communication” between systems.
The concept of an EIP was made popular by Gregor Hohpe and Bobby Woolf in their book ‘Enterprise Integration Patterns’. Together they created a catalogue in which they explain 65 patterns that fall under what they call the messaging integration style. They define this type of integration as using “messaging to transfer packets of data frequently, immediately, reliably, and asynchronously, using customizable formats.”
The Importance of Enterprise Integration Patterns According to FME Experts
Many organizations inadvertently create data silos as a result of different departments needing to make use of best fit-for-purpose applications for their work. Data silos exist when data is stored in a way that makes it inaccessible or unusable to others within an organization.
Business impacts that can derive from data silos include:
- Key decision-makers do not have access to the full picture
- There is limited ability for teams to work together and share their data or information
- More time spent on performing repetitive administrative tasks, with an increased likelihood of human error
- As a result, data may be poorly maintained, leading to inaccuracies
Making use of EIPs effectively can bridge data gaps and mitigate negative business impacts. Data integration and application integration are necessary for users to maximize the value of their datasets.
Whilst some alternative solutions are available, such as developing complex data conversion scripts or forcing multiple departments onto the same application, these have their own shortcomings. In particular, these alternatives often require specialized developers to maintain custom-built solutions, making fixes and update requirements extremely costly.
5 Common Enterprise Integration Pattern Use Cases
The use cases I comment on below are all solved using EIPs originally defined by Hohpe and Woolf. Depending on the integration challenge, different pattern logic is applicable. All patterns published in their book are described in detail on their website.
The example use cases listed below are all processes or challenges that can be solved using EIPs in FME. After building workflows in FME Desktop, you can use Automations in FME Server to effectively build your own enterprise integration patterns.
1. Data Migration
Data migration is the process of moving data from one or more systems to another. Migration may be necessary if your organization manages a data warehouse (a large data repository to store data from multiple systems). One-off data migration tasks may come up when an organization chooses to undergo a complete system change. In this case, it is important to restore historical data records to the new system.
Safe Software has recent experience with a data migration task as we opted to change the platform hosting the FME Community. To provide our users with the best customer experience possible, it was integral to preserve all the online knowledge resources without losing data and having as minimal downtime as possible.
If you are performing data migration tasks for enterprise integration, there are a variety of patterns and combinations of patterns you can use. The way you choose to migrate data with EIPs is dependent on the data transformations required.
2. Data Synchronization
Data synchronization is the bi-directional exchange of data across systems which helps maintain consistent datasets across multiple applications. Data synchronization tasks are necessary for tackling siloed systems, especially in cases where companies are merged or acquired. Data silos are practically unavoidable because teams can and should use applications most appropriately designed for their daily tasks. However, as mentioned earlier, this brings about a host of challenges.
Application integration often uses APIs to enable systems to communicate with one another programmatically. To maintain data accuracy, when one system is updated the other system will be updated immediately to reflect this change, ensuring the entire organization has access to the most up-to-date information in the system they interact with (no matter where the update was made).
To perform data synchronization, the message channel pattern and message bridge pattern are ideal. The message channel pattern proposes using a dedicated channel in which systems can communicate. The message bridge pattern looks to replicate messages from one system to another.
 |
 |
| Message Channel Pattern | Message Bridge Pattern |
As each system is designed for a specific target audience, how the data is stored may differ. A key element in any data synchronization workflow is to transform data into the format used by the specific system receiving the data.
3. Data Analytics
Data analytics are essential for decision making because they provide quick insight into business processes and company performance. Using data analytics and business intelligence tools, information can be presented in an easily understandable manner, using visual components like charts and diagrams, enabling key stakeholders to identify trends and potential problems.
4. Receiving Data from External Sources
Many organizations collect data from external sources such as suppliers, partners, customers or members of the general public. For example, many cities rely on the local population to report potential road or street hazards, such as a pothole. Once reported, they can use automated processes to validate this data and parse it over web services into the appropriate system(s) to investigate, implement, and invoice the solution.
The forms created for data submission are usually designed to only allow specific inputs but can be compiled of multiple different sections that can be processed individually. A combination of the content-based router and splitter patterns can be used to design a workflow that follows a series of configured rules to divide each element up before processing.
 |
 |
| Content-Based Router Pattern | Splitter Pattern |
The City of Coquitlam has been able to improve information sharing and accessibility with stakeholders by integrating various business applications and web services using Automations in FME Server. They receive data from both the public and third-party vendors to ensure that all city services are carried out in a timely manner.
5. Consolidated Customer Overview
At every organization, a customer will interact with many departments, such as sales, customer support, training, or marketing, depending on their present needs. Therefore, each department will have different data based on their interactions with each customer. The ability to summarize all data into a single view not only provides each team a complete history of the customer, allowing them to make more informed decisions, but it will likely provide a better overall customer experience as there is increased collaboration between the teams.
Two patterns that can aid in these workflows are the normalizer pattern and content enricher pattern. The normalizer pattern, which behaves similarly to the aggregator pattern, brings together information from different systems stored in different formats to be presented in a single standard way. The content enricher pattern allows you to enhance message content by retrieving data from other related sources.
 |
 |
| Normalizer Pattern | Content Enricher Pattern |
Be Your Own Solutions Architect
One-to-one solutions requiring specialist skills can now be considered a thing of the past. Technology and integration software like FME are constantly being upgraded to empower the everyday data user to embrace self-service integration techniques. These users have come to be known by Gartner as ‘ad hoc’ or ‘citizen’ integrators.
To apply these messaging style EIPs, Hohpe and Woolf touch on the Canonical Data Model, a workflow that is designed independently of any single application. Working in this way greatly reduces an organization’s maintenance costs because integrators can work with the existing model to complete integration projects in a more timely fashion. Integrators can implement patterns most easily by introducing message-oriented middleware. The FME Platform includes capabilities like Automations that enable independently developed software components to run on different platforms to send and receive messages using APIs.
I’ll leave you with this question to ponder:
Is there any data you regularly need access to, but currently, it is stuck in a place that makes it difficult to access, and the task to gain access is manual and painfully time-consuming?
Perhaps it’s time you became an integrator!
If you’d like more information on how FME can be used as the middleware component to implement enterprise application integration workflows check out the ‘Getting Started with Enterprise Integration’ webinar or our ‘Getting Started with Enterprise Integration Patterns’ tutorial series, with more articles coming soon.
Holly Coxon
Holly is an FME Server Technology Support Team Lead at Safe Software. Among other things, she provides technical support, creates content for the Community, presents webinars and contributes to product development through functional testing and sharing customer feedback. Holly spends her weekends exploring the great backyard of British Columbia.
