FME Server 2020 and Enterprise Integration Patterns

With the recent release of FME 2020.0, I want to share the work we’ve done to take FME Server Automations to the next level and bring support for Enterprise Integration to the FME platform. 

More than ever, organizations are deploying specialized, fit-for-purpose applications to provide business solutions. The problem is that these applications are often unable to communicate with each other, resulting in simple tasks requiring duplicate data entry and extra manual effort. 

One of the most common challenges faced by organizations is how to get their varied systems connected so that they can work together in harmony.

Enterprise Integration is all about providing this connection between disparate systems.

What are Enterprise Integration Patterns? 

In Enterprise Integration, a pattern is a repeatable solution to a commonly occurring problem encountered when integrating applications or systems. A pattern is a template for how to solve a problem that occurs in many different situations. 

Enterprise Integration Patterns (EIPs) were made famous by Gregor Hohpe and Bobby Woolf and are popular amongst IT architects and application integration specialists. The patterns provide a common language for communicating solutions. To learn more about EIPs, visit www.enterpriseintegrationpatterns.com.

FME 2020.0 delivers Enterprise Integration

Using new capabilities introduced in FME 2020.0, Enterprise Integration Patterns can be implemented using FME’s visual interface, no coding required.

Whereas other Enterprise Integration tools have been built with the programmer as the primary user, we have built FME Server and Automations for general technology professionals – both programmers and non-programmers – to enable an unprecedented level of productivity. 

Here’s how you can implement Enterprise Integration Patterns using the FME Platform.

From Orchestration to Integration

FME Server 2019 introduced Automations.  Built on top of and relying on the notifications framework of FME Server, Automations provided a graphical interface making it easy for users to build workflows consisting of Event Triggers and Actions. These workflows were able to orchestrate workspaces.  Controlling the sequence of actions that responded to events.

FME Server 2019 Example Orchestration

Now, in FME 2020, we have extended Automations to deliver Enterprise Integration capabilities.  

Meet the FME Server Automations Writer

The first step to achieving Enterprise Integration is the FME Server Automations Writer. This Writer acts as a communication conduit, connecting the FME Engine (FME’s processing core) to Automations in FME Server. 

When you add the workspace; created in FME Desktop containing the FME Server Automations Writer to an Automation, the outputs of the writer become output ports inside the FME Server Automation. 

It is this connection that gives you the ability to build Enterprise Integration solutions with unrivalled format support.

For a full tutorial on how to use the FME Server Automations Writer, click here. 

There are several exciting properties of the connection between the FME Engine and the Automations that are provided by the FME Server Automations Writer.

• Multiple Messages: You can output multiple messages out of any port.  An Automation message is created for each record/feature that is written by the FME Server Automation Writer. You can output many such messages within a single Automation workspace each triggering separate downstream actions. 
• Any Number of Ports: A workspace can write records/features to different output feature types/tables with the FME Server Automation Writer.  Each record is output via the associated port triggering different flows downstream.  
• No Message: A workspace with the FME Server Automation Writer may output no messages to any output feature type/table.  In this case “no messages” at all come out of any of the defined ports.  Workspaces without an FME Server Automation Writer are also supported as before. 
• Completion Ports Fire Last: The completion ports “✓” and “X” are guaranteed to be the last messages that come out of the workspace and one and only one message will come out of either one but not both.
• Ability to connect to any node: New for Automations in FME 2020 and crucial for Enterprise Integration Patterns such as Process Manager (see below).  A message port can now be connected anywhere in an Automation.  This gives you the flexibility needed for the most complex and dynamic Automations.   To create a loop you must first “enable looping” which you will find as an option under the “Menu”.

Benefits of using Enterprise Integration

These platform enhancements change how FME workflows are built, providing increased scalability. Previously, all of the complexity involved with integrating data and applications had to be housed inside large FME workspaces. Now, FME 2020.0 allows users to build small, focused workspaces to run in series or in parallel. This new approach allows organizations to build more scalable integration solutions. 

Sample Enterprise Integration Patterns in FME 

The pattern names used below are taken from the above referenced Enterprise Integration Patterns resources that we consider to be the definitive source.

With these platform enhancements coupling Automations to the power of the underlying FME Engine you can now implement over 35 Enterprise Integration Patterns, including the following:  

Content Based Router

This message based routing pattern is used to examine a message’s content and routes the message based on what data is inside. In FME,  this pattern can be built  with the Filter tool in Automations. More complex Routing based on message content or on data identified by the message can be done with FME Server Automation Writer and the power of the FME Engine. 

Content Enricher

This pattern uses the information inside of the incoming message to retrieve data from an external source and append the data to the message. This pattern is simple to build with the FME Server Automation Writer. Typically a key or other information is part of the message that is passed to the FME workspace.  The workspace simply looks up the information in one of the data stores supported by FME and writes the enriched content to an output feature type/table using the FME Server Automation Writer.  

Process Manager

This is one of the most powerful patterns and of course FME can help you build it with no code!  The looping capability of FME Server 2020 enables you to connect a custom output port to a previous input port in an Automation workflow.

This enables FME to create the Process Manager pattern which is a pattern in which  the processing steps and their order is determined at run time. Again the power of the FME Engine and Automations make this possible. In the example below, the brains of the  Process Manager pattern are encapsulated in a single workspace called “ProcessManager”.  In FME Server 2020 the Automation fragment for Process Manager looks as follows:

You can see the controlling workspace “ProcessManager” and the associated “Task” workspaces that execute and then give control back to ProcessManager.  Once the task is complete ProcessManager outputs the message via the Complete output port and the Automation continues downstream. To learn how to create looping workflows in FME Server, you can read this Knowledge Base article. 

The list of patterns that we address in FME 2020 is over 35 but I will stop here as I am sure you get the idea.

The Enterprise Integration Journey

Even as development for FME 2020.0 has wrapped up, we are continuing on our Enterprise Integration journey with FME Server and Automations for future releases.   

In 2019 the FME Platform delivered Automations and Enterprise Orchestration.

In 2020 the FME Platform delivers Automations for Enterprise Integration.

Equally exciting is that FME 2020 is not the end of Automations or Enterprise Integration development.  Many exciting features are still to be implemented. Let us know what you would like to see next!

At Safe, we continue to push the FME Platform beyond integration. Our ultimate goal is to deliver nothing short of complete enterprise transformation capabilities. 

Try FME Server Again for the First Time

You can see why we are so excited by FME Server 2020! 

If you haven’t tried FME Server recently then you really haven’t tried FME Server.  The Enterprise Integration capabilities of FME 2020 change the FME Server experience.   

You can get FME Server 2020.0 here.  We are happy to give you an extended trial so you have the opportunity to experience it before you buy it. 

Let us know how it goes and what you would like to see us do next!