Skip to content

FME 2011 Special: (Published) Parameter Updates

Mark Ireland
January 19, 201111 min
With FME2011 now upon us - it was released yesterday - it's good timing for an evangelist post. This time I'm going to write about changes to parameters in 2011; particularly published parameters. Actually, one fundamental difference is a change in wording from "Published Parameters" to simply "Parameters", so forget all that you knew about the subject, and read on as I describe parameters, step-by-step, from the very basics...

Hi all,
With FME2011 now upon us – it was released yesterday – it’s good timing for an evangelist post.

This time I’m going to write about changes to parameters in 2011; particularly published parameters. Actually, one difference is a change in wording from “Published Parameters” to simply “Parameters”, and you’ll see why that change has been made as I continue.

In fact, because of terminology changes it’s easier for me to describe parameters step-by-step from the very basics. It’s a little long-winded, but the best way to understand the full logic; so forget all that you knew about the subject, and read on…

Parameters to Control Translations
Every component of a translation – workspace, reader, writer, feature type – has a set of parameters to control how it operates. These parameters are usually set in the Navigator window of Workbench before running the translation.

These are parameters that directly control a translation; I’m going to call them FME Parameters.

Here, for example, are the FME Parameters to control a MicroStation DGN Reader.

Parameters for User Input
If the author of a workspace needs it to accept some form of custom user input, then this can be achieved by with a different type of parameter.

These are parameters that may indirectly control a translation; to differentiate them I’m going to call them User Parameters.

Linked User Parameters
Quite often, user input relates directly to an FME parameter.

For example, in the FME Parameters screenshot above, one parameter is ‘Read Reference Files’. The workspace author might want an end-user to decide whether to read reference files or not.

So, the author creates a User Parameter (to accept user input) and then links it to the FME Parameter, to apply the user preference to that action.

That’s why I wrote that user parameters may indirectly control a translation; they can be used to accept user input and that input used to control the FME parameter itself.

Creating Linked User Parameters
The simplest way to create a user parameter and apply it to an FME parameter (all in one process) is to locate the FME parameter in the Navigator window, right-click it and choose “Create Parameter” (prior to 2011 this used to be “publish”).

This will open a dialog in which to create a new “user” parameter, and apply it directly to that particular “FME” parameter.

Editing Linked User Parameters
Prior to FME2011, any user parameter created in the above way was restricted in its definition by the FME parameter to which it was linked.

For example, in a database writer the Transaction Interval would always create a parameter of type ‘text’; allowing any user input:

However, in FME2011,when I create a user parameter from this, I can now change the parameter type; for example I can change it to ‘Choice’ and create a picklist of values for a user to choose from:

This capability is new for FME2011, and makes the use of parameters more flexible. However, be aware that there is no checking mechanism. For example I could tie a text type user parameter to an FME parameter that requires an integer as input, and cause the workspace to fail. Remember – with great power comes great responsibility!

Creating Unlinked Parameters
A second way to accept user input is to make a new user parameter from scratch. We sometimes call these custom or standalone parameters, as they aren’t automatically linked to an FME parameter.

You create a parameter like this by right-clicking the Parameters section of the Navigator window and choosing “Add Parameter”:

Then you need to define the parameter.

Using Unlinked Parameters
At this point there is a new user parameter, that is unlinked to any FME parameter: so what might that user input be used for?

Well, one use is to retrieve the value in the workspace using a ParameterFetcher transformer, and to direct data to different parts of a workspace on the basis of that value; but that hasn’t really changed for 2011.

The other option is to then manually apply that user parameter to a real FME parameter. You do this by right-clicking the parameter, choosing “Apply to”…

…and then selecting which FME parameter to apply it to.

You’ll probably say that there isn’t any difference between creating a linked user parameter, and creating an unlinked one and then applying it: and to be honest you’re correct. There really isn’t much difference now that (in 2011) I can edit any directly linked parameter anyway. One nice part is that the “apply to” dialog lets me apply my new user parameter to multiple FME parameters at the same time, speaking of which…

Shared Parameters
The “Apply To…” option (above) is interesting; once you’ve applied a user parameter to an FME parameter, you can still keep on applying it to other FME parameters.

This is called a Shared Parameter, as each FME parameter to which a User parameter is linked, will share the value of that user parameter at runtime.

A good use case is a database password parameter: there might be several FME parameters in a workspace, all of which need the same database login information. By sharing the same user parameter with each FME parameter, the end-user only has to enter that information once, rather than once for each FME parameter.

Navigator Window
When you create a user parameter it gets added to a section in the Navigator. As this functionality has been refined, so the terminology has come to take on slightly different meanings, and so the section of Navigator window where these are listed is now (in 2011) simply titled “Parameters”, rather than “Published Parameters” as it used to be.

Published Parameters
It’s obviously useful if the end-user can be prompted to enter values on running the workspace, rather than having to make edits to the workspace.

One of the settings when a parameter is created is a checkbox labelled “Published”:

Whenever a user parameter is published like this, the user is prompted for a value automatically, when the workspace is run:

This is obviously useful, because the end-user can now pass values into the translation, without having to make edits to the workspace.

Private Parameters
So under the Parameters part of the Navigator there are now two sub-sections. One is called Published, for user parameters defined as being Published, and another for Private Parameters.

Private parameters are new for FME2011. If the ‘Published’ check box is unchecked when you create a user parameter, then you are creating a private parameter.

The difference is that a published parameter will prompt the user to enter a value at run-time; a private parameter will not prompt the user. They may not even know it exists!

Using Private Parameters
So, what is the point of creating a Private parameter, that doesn’t prompt the user for anything?

Well, one reason is that the workspace author might want to create a shared parameter, but just for internal use, and not for the user to enter a value. A published parameter would prompt the user; a private one will not.

For example, consider a workspace containing 1,000 transformers; 10 of which are Snappers that all use the same tolerance, but that we don’t want the user to set.

In FME2010, to change the tolerance the workspace author would need to track down each Snapper (in a workspace of 1,000 transformers) and change it manually.

In FME2011, an author can create a private shared parameter that is linked to each Snapper; that way a change of tolerance is achieved by simply changing the user parameter value.

Here (click to enlarge) is that exact scenario:

A single parameter is used to share a tolerance value among multiple Snapper transformers. To change the tolerance, the author will change the user parameter value; all FME parameters sharing that will then also be changed. But – since this is a private parameter – the user is not prompted to enter a tolerance value at run-time.

Private Parameters have an obvious role for FME Server, since the workspace author won’t want to prompt the end user for more parameters than is necessary. By making parameters private they won’t show up on the Server web interface.

Concatenated Parameters
Concatenated parameters are new for FME2011. Technically you could try this trick in earlier versions of FME, but only in 2011 is it actually an intended capability!

Concatenated parameters are when the value of one parameter references the value of one or more other parameters. This is done by using the syntax $()

For example, the result of this parameter (MyConcatParam) includes the value of a prior parameter (MyNameParam)

You would typically use this for getting input from the user, and then concatenating it with a constant value to apply to an FME parameter.

One example is creating a specific file path – for example “C:myDataoutput$(MyFileName) – where the root of the path is always the same (so doesn’t need user input), but the name of the file to create is user-defined (and so a published parameter).

Notice in the above screenshot that the concatenated parameter is private, not published. That’s because the author wouldn’t want the end user to enter a value for MyConcatParam, just MyNameParam. Then MyConcatParam could be shared with an FME parameter (like Source Dataset if it were a path/filename).

Scripted Parameters
Scripted parameters are another new feature for FME2011. They are a way to incorporate a Python or Tcl script into the generation of a parameter value. Scripted parameters are simply a new type of parameter available in the Add/Edit Parameter dialog:

The value option opens a text editor into which a script can be inserted. The value calculated and returned by that script is the value given to the published parameter.

Note that the script is executed at run-time, so you can’t reference an attribute created in the workspace, even when the published parameter is retrieved/used after that point. But then, of course, in that scenario you could just use a PythonCaller or TclCaller.

Scripted parameters can’t be published, because they aren’t ones you would want the end-user to be able to edit or enter values to. However, to get user input I suspect you could reference other published parameters from within that script.

Conclusion
The updates for 2011 are so important they are getting a new section in the FME Server training course (as well as updates to the Desktop course). But, whether or not you’re an FME certified trainer you’ll want to familiarize yourself with this functionality. You can summarize it, like me, by stating that user parameters are a way to fetch and share user-defined values amongst FME parameters. For existing users updating to 2011, the three big updates are:

  • Revised terminology
  • Private Parameters
  • Concatenated Parameters

Scripted parameters may be a big deal to some, but I doubt the majority of users will be as interested as in the other updates (I am ready to be proved wrong!)

Hope you found this interesting.

Safe product icons
Reach out and get started with FME today

Real change is just a platform away.

FME is ready to put your data to work and transform your business today. Are you?