One of the strengths of FME is that it is extremely flexible: It can connect to almost any data format, transform it almost any way, and automate the process however you would like through automations. Hence, as one of many FME solutions, building more usable custom transformers, is one of them!
As a result, FME workspaces are highly customizable, and can quickly become complex.
At the same time, if you’ve ever listened to one of our webinar demos with Don (one of our co-founders), you’ll know us Safers have a well-known ‘rule’ that “workspaces should never have more than 10 transformers”.
So… somehow we ideally use less than 10 transformers, when there are 450+ transformers we could be using? What if there is no single ‘ideal’ transformer for my use case? What If I need multiple transformers in a series to accomplish a single logical step!? Surely it can’t be done in less than 10!
Easier said than done, I know. But, if you’ve ever tried to collaborate, reuse or repurpose workspaces, you can see why this sentiment is really important!
FME is a visual programming tool. We tuck the complex code into transformers so that citizen integrators are able to easily build a solution for their data challenges. But, like any quality piece of programming, building with simplicity in mind means treating workspaces as solving a single scenario, ideally in such a way that you can bundle it up as a tool that can be re-deployed and combined with other workspaces to solve larger organizational data challenges.
That’s one way we can work around the 10-transformer rule: Pack a multi-transformer solution into a special FME object known as a custom transformer.
You can think of usable custom transformers as mini workspaces representing some reusable logic, that can be shared across your organization, or with other FME users through FME Hub.
To quote Trent and Jovita, two of our FME Experts:
“Authoring workspaces for reusability is core to maximizing the value of FME. The key is to identify repeatable data processing tasks within your organization, and pick out reusable logic across these tasks allowing you to author workspaces as tools that can be utilized over and over again.”
(☝️ Shameless plug for their upcoming lightning talk for the World Fair!)
Other folks have discussed the value of custom transformers (including our own FME Evangelist, Mark Ireland), so I won’t expound too much more on that here.
What I’d like to focus on is what we changed about the experience of creating a usable custom transformer in FME 2021.0 and where we plan on going in the future.
Part of making a great custom transformer is having parameters (aka. user prompts). Parameters allow you to pass data into the custom transformer, as well as control custom transformer behaviour. Parameters abstract and simplify what the end user needs to provide for a workspace to run successfully. They are easy to understand for the end user, but flexible enough to handle the various workspaces and underlying logic where the transformer will be deployed.
Here at Safe, we have a lot of tools at our disposal (namely, an entire development team!) to create the rich dialogues you see in the transformers we ship with FME. Before 2021.0, custom transformer authors were limited in their ability to create similarly rich and intuitive interfaces, which made the experience of building a user-friendly custom transformer frustrating, compared to the kind of control ‘shipped’ transformers gave you.
Let’s take the example of parameter groups in a ‘shipped’ (Safe-built) transformer:
Here is the parameter editor for the Bufferer. You’ll see that the fields (parameters) are grouped in a way that makes sense to the end user: General, Area Parameters, Generate Dissolve List. These groups make it easy to show, hide, or disable sets of parameters from the user depending on what they are trying to accomplish (flexible, right?).Before 2021.0, there was no way to create these same groups in a custom transformer. All of these parameters would just be shown in a flat list, with no control over whether they were hidden or shown. This didn’t necessarily limit the functionality of a custom transformer, but it made it quite difficult to make a user-friendly interface.
Plus, it meant that custom transformers didn’t have the same look and feel as our own shipped transformers. It was also difficult to easily reorganize or edit your parameters in one place—you would need to edit them one by one!
So, what did we change?
We recognized that as more and more FME users want to share solutions within and across organizations, we needed to make the experience of building usable custom transformers easier and give workspace authors the power to build interfaces that can achieve parity with shipped transformers.
In 2021.0, users now have the ability to see all of their user parameters, reconfigure them in a drag-and-drop interface, and control their default values all in one place. Additionally, we provided the ability to create parameter groups, making things much tidier for the end user.
To access the user parameter dialog builder in FME Workbench, right-click User Parameters in the Navigator pane and choose Manage User Parameters or select the Tools menu in the ribbon and choose Manage User Parameters.
Sharing the Love
Let’s assume now that you’ve built a groundbreaking custom transformer that you know the FME Community will love. You’ve also built a beautiful parameter dialog for your custom transformer. How can you share this with the world? Enter FME Hub!
FME Hub is a place for FME users to share their custom solutions with others, in the form of custom transformers, custom formats, workspace templates, and web connections.
Us Safers are often contributing new content as well! Recently we have been deploying Python-based formats and connectors in the form of FME Packages, which allow us to support new formats asynchronously with the full FME release (this is becoming increasingly important, in the advent of frequently changing APIs).
We have a more detailed article on how to contribute to FME Hub at, but at a super high level the steps are:
Step 1 – Sign Up
You need an account to upload items. Sign up and create a free FME Hub account or use your FME Community login if you already have one.
Under each account you can share items either as an individual user publisher or under an organization with the broader community.
Step 3 – Upload Item
You can upload your item by dragging it into the drop panel or via the Select File button.
That’s it! Fill out the remaining details about your item, and it’ll be accessible on FME Hub.
If your item is publicly available, you can share your solution with your colleagues and the wider FME Community via a direct download link, or by searching for the item in FME Desktop.
Here’s how to find these goodies in-product:
Custom transformers can be found by searching the Transformer Gallery or Quick Add (Quick Add is brought up by simply typing on the canvas):
And available packages can be found through FME Options and Quick Add:
They say sharing is caring, so please share your solutions on FME Hub! Help other data integrators save time!
What’s Next?
We know this is just the beginning of the user parameter story. In working with FME users, we learned that being able to support parameter dependencies is one of the most powerful ways we could simplify complex workflows. The idea behind dependent parameters is that the value or visibility of a parameter could be controlled by another.
For example, maybe you only want to show the ‘Flowers’ parameter group, if the data type parameter is set to ‘Flowers’. This is just a mockup, but you can see on the right hand side how that logic can be applied in the parameter builder:
And if we preview what that looks like to the end user, you’ll see that the ‘Flowers’ parameter group is enabled based on the data type being ‘Flowers’ (and ‘Trees’ is disabled)Dependent parameters help guide end users logically through the steps and data needed to execute a workspace, which is particularly useful if the workflow is complex and a lot of parameters are required.
The ability to hide and show parameters based on conditions like this also opens up enormous potential for custom transformer utility. Usable custom transformers no longer need to be limited to one function. Custom transformer authors will be able to build more complex logic into their transformers backed by this powerful user interface, and custom transformer users will be able to easily and intuitively make use of this power in their own workflows.
But, as the title says, this is “What’s Next”. You’ll have to stay tuned and keep your eyes and ears open to hear more about this functionality in the future.
In Summary…
FME can solve almost any data problem, but the magic is in not having to solve the same problem more than once through building simplified, flexible and reusable solutions. User parameters are the key to unlocking “doing more with less”, which is why we wanted to make them easier to build and navigate than ever.
Here’s to making more usable custom transformers. If you’ve never built a custom transformer before, give it a shot in FME 2021.0 and let us know how it goes! We’d love to hear your feedback.
Other fun reads about FME Solutions:
FME Golf Challenge Part 1: My Solution
FME Golf Challenge Part 2: Your Solutions and a New Challenge
Annabelle Glass
Annabelle is a Product Owner for FME Desktop Authoring. She originally studied geography, GIS, and psychology which she likes to think gives her a unique skillset for product management at Safe! Outside of the office, she can be found training for triathlons (she completed her first Ironman last year!) or hanging out with her retired greyhound, Honey Bun.
