Hello FME’ers,
There are a number of sayings and phrases that deal with the concept of change.
In Latin there is “tempora mutantur, nos et mutamur in illis” (the times change and we change with them). If you prefer your philosophy to have a Greek bent, Heraclitus said “Everything flows, nothing stands still”, and if you prefer it to be more contemporary (though not that contemporary), Bob Dylan put it even more succinctly: “The times, they are a-changin'”.
Well, FME is always a-changin’, and Workbench in both 2015.0 and 2015.1 got some updates that will definitely affect the way you work. They are changes to Reader and Writer feature types; so let’s investigate them further.
What is a Feature Type?
Just to cover the terminology I’m using here, let’s discuss this first. An FME Feature Type simply represents a layer of data.
Of course this is very format-specific, which is why we needed our own term for the concept. Some formats and software will refer to it as a layer, others as a level, a class, a table, a category, a sheet, or an object; and yet other formats (Shape and CSV for example) don’t have the concept of layers and you use a separate file for each.
Feature Types are represented as light brown objects on the Workbench canvas. In this screenshot we have either a series of layers in a source dataset, or a single layer in a series of datasets.
Click on the little cog-wheel icon and you open up a dialog of settings. Notice one thing in FME 2015 is that we’ve replaced all use of the term “Feature Type” for the format-specific terminology, like here in this Excel dialog:
Notice how the label is “Sheet Name”, because a sheet is the feature type for Excel. If the format was AutoCAD then it would be labelled “Layer Name” instead. This labelling is new in FME2015 and is to reduce confusion for users not familiar with FME terminology.
Attribute Definition Mode (2015.0)
Now we’ve covered the terminology, let’s look at the functionality. The Attribute Definition mode first appeared in FME2015.0 and was covered in both the Desktop Deep Dive webinar and a previous Sneak Peek article, so you may already be familiar with it. But it’s such a useful tool that I think it’s worth covering anyway.
So, open the feature type properties dialog for a Writer feature type by clicking on the cog wheel icon, and then click on the Attribute tab. This is what you will see:
Now, Manual Mode is what we always have had in FME. You can manually make changes to the list of attributes, such as changing their names and types (char, int, etc.). And Dynamic Mode is the same as before too; attributes will be defined dynamically by an incoming schema.
But Automatic Mode is new. When you choose this you start out with no attributes on your feature type:
But make a connection, and attributes are automatically added:
So in automatic mode, the schema is automatically updated to reflect whatever is being connected to it. As demonstrated in the Deep Dive webinar, if you put AttributeCreator or AttributeRemover transformers (for example) into the workspace, then the output schema is automatically updated to reflect the changes. And if you connect multiple Reader feature types, then the Writer feature type becomes an amalgamation of the two:
The important thing is to not confuse this with the Dynamic mode. This creates the schema automatically while authoring the workspace; dynamic mode creates the schema automatically at run-time.
Feature Type Dialog (2015.1)
For 2015.1 the Writer feature type dialog got refreshed in more or less two ways. Firstly the feature type Fanout mode got replaced by new functionality and secondly the dynamic parameters got an overhaul.
So, first to the feature type fanout. Notice that in 2015.1, the fanout option has been removed:
Recall that in previous versions there was a checkbox and a drop-down parameter to select an attribute from. Well now that checkbox isn’t needed. Like many transformer parameters there is a drop-down menu for the Feature Type name that you can use to define it in any number of ways:
In the above I am defining the feature type name by selecting the ParkName attribute; in essence this is the same thing as a fanout.
However, the big advantage now is that I can also set the feature type name from a User Parameter (and think about it – there are many types of parameter like text, choice, or scripted) or I can open a text editor and construct an output feature type name. I think this update is great. It gives workspace authors way more flexibility in defining feature type names.
Notice that we’re not calling it a fanout any longer, so, yes, the feature type fanout is no more! I know. Even though it’s replaced by new and improved functionality, it still feels like losing an old friend. But do not despair fanout fans! [Try this Dalek relaxation therapy if you’re really stressed] The Dataset Fanout still remains in the Navigator window (although now it’s just labelled plain Fanout).
Now to the dynamic functionality in 2015.1. Mostly we reorganized the dialog to make it easier to use, but in some cases there is a bit of new functionality.
Here’s a dynamic workspace feature type as it was in FME2015.0:
Notice how the Fanout and feature type name parameters are greyed out, and the dynamic properties dialogs (for feature type name, geometry type, and schema source) are accessed through a set of browse buttons in the bottom right.
Here’s the same feature type in 2015.1:
The first obvious thing is that the feature type name parameter is not greyed out any more; that’s because it’s where the dynamic feature type name is now defined. So you don’t have to go into a sub-dialog to select a feature type name attribute, you can do it within this dialog.
Also you’ll notice that the dynamic geometry parameter is also merged into the existing Allowed Geometry field; again that’s so you can define that part of a dynamic translation without having to delve into a series of other dialogs. The two main options are From Schema Definition (i.e. the schema selected will define the geometry type) or First Feature (i.e. the geometry of the first feature defines what geometry will be allowed) – but of course you can also hard-code geometry type by selecting a specific geometry from the list:
Obviously, outside of dynamic mode the two lower options are not available, and you just define geometry as a fixed type. And where geometry is not an issue (i.e. the format doesn’t have geometry type restrictions) this parameter is greyed out completely.
The final new dynamic part is Schema Definition. This is where we can select a Reader dataset or Reader resource that defines the output schema. There is a sub-dialog here, which looks like this:
Notice I can select either an existing Reader (or Reader Resource) or I can just let the first feature that arrives at the Writer become the definition for the entire output dataset. I’m not sure every format supports that behaviour yet, but a lot did get that added in 2015.0: the What’s Great list denoted this with the phrase “Added support for writing feature-based dynamic schemas”.
Basically there have been a lot of updates to the dynamic functionality in 2015.1; it gives a few extra possibilities through the use of feature-based dynamic schemas, but the main improvements are around usability. Dynamic translations have always been among the hardest pieces of technology to understand and use. We hope that these new updates will make it much, much easier.
Merge Filters on Databases (2015.1)
On the Reader side of things, the opposite parameters to the fanout are Merge Feature Type and Merge Filter:
This is for when you have multiple source feature types (i.e. multiple layers) and you wish to read them all through a single canvas object:
Up until now, this has never applied to databases. It makes sense. You want to pick which tables you want to read; you don’t just want to read entire databases of information. But – as it turns out – many users wanted to do exactly that. Without this it was very hard to replicate an entire database.
So, for 2015.1 the merge filter WILL now apply to databases; that’s just about all FME-supported databases. The ones I know that it doesn’t apply to are Oracle, SDE, and Raster Geodatabase (and they will all be supported in 2016)
For example, if I have a single File Geodatabase table on my canvas, I can flip the Merge Feature Type switch and suddenly read all of the tables in the Geodatabase.
Dynamic and Merge Filters
The question now becomes what the difference is between this and a dynamic translation. Well, it’s pretty much the same as a non-database format: a dynamic translation will read all of your tables (or the ones you select) and all of your attributes. The Merge Feature Type will read all of your tables, but not necessarily all of the attributes.
It gets confusing because database formats are a bit inconsistent. Some will read all attributes, some will read attributes only where they occur in all tables, and some will fail with an error. If you’re having problems then try deleting all of the attribute definitions from your Reader feature type; that may help.
In short, there are some natural limitations, plus some usability issues that we’re working to improve. But the functionality is there, and usable, so give it a try. If you have any feedback please do contact our support team and let us know.
Conclusion
All of the changes I’ve mentioned will naturally lead you to change too: FME mutatur, nos et mutamur in id! But they should be good changes, that help you to become more productive with FME.
Oh, and look out for another change coming in 2016; we already have a FeatureReader transformer and 2016 promises to deliver a FeatureWriter too. With that you’ll not even need feature types on your canvas – a big change indeed!
Mark
Mark Ireland
Mark, aka iMark, is the FME Evangelist (est. 2004) and has a passion for FME Training. He likes being able to help people understand and use technology in new and interesting ways. One of his other passions is football (aka. Soccer). He likes both technology and soccer so much that he wrote an article about the two together! Who would’ve thought? (Answer: iMark)
