You’ll probably have noticed that many of our blog posts have titles like “6 Examples from FME Cloud Early Adopters“, “5 Tips for Providing Data to Europe’s SDI“, and “The Top Ten FME Tips of All Time” (guilty)!
That’s great for marketing and SEO purposes, but not so good for when a new FME version is released; not unless you’d like to read an article titled “760 Noteworthy Updates to FME2015” or “280 Developer Updates for FME2015 the FME Evangelist Flagged as Great”.
That would certainly be a big block of imposing text. And yes, those numbers are the real figures for FME2015. I flagged (so far) 760 items to go into the What’s New file, and 280 items to be mentioned in the What’s Great article. And that’s just for Desktop, not Server.
So, for this article I’m going to cut back that list and give you just a flavour of what we at Safe are excited about for FME2015. It’s not the full list by any means, just a taste of what is to come…
Of all the updates to FME for 2015, I think this is going to have the biggest impact. At the moment we call it “Named Connections”. Let me explain by showing you how it works. Select Tools > FME Options and there are now two additional sections for setting up connections:
As you can see, you can make connections to either a database or a web service. For example, here I make a database connection (click to enlarge):
OK. That gets me a database connection, but what can I do with it? Well, I can use that connection instead of entering my authentication parameters in a Reader, Writer, or transformer; like in this Joiner:
So, obviously I can re-use that connection within a workspace. But wait! There’s more! Because these connections are FME-wide options (for any one user) I can use them in any of my workspaces.
Plus – because the parameters are being stored as part of FME, and not inside the workspace – this functionality makes my workspaces more secure. If I pass on the workspace to another user then I’m not including my authentication info. Of course, when I do pass it to another user, they’ll need to provide their own connection information. If they have a named connection with the same name then it will be used automatically. Otherwise they will be prompted to select a new connection:
And if they – or you – don’t want to use a named connection, then you can revert back to the old parameters method of authentication, like so:
The one “gotcha” about all of this functionality is that it’s not yet available on FME Server. When you publish a workspace to Server, if there is a connection defined, you’ll be prompted to replace it with parameters (as above). However, we’ll get to implementing that in time, plus I expect we’ll also have connections like this that can be shared among multiple users, not just a single person.
Like I mentioned, at the moment we call this “Named Connections”, but we might rename that to just “Database Connections” and “Web Service Connections” by release time; we’ll have to wait and see.
Feature Type Updates
We did a whole bunch of updates around feature types for FME2015. The idea is to help demystify what feature types are for, what they define, and what their output is. Firstly, check out this screenshot of an AutoCAD DWG feature type:
Do you see how the label isn’t “Feature Type” and the default name isn’t “FeatureType1”? Instead it uses “Layer”. That’s because we’ve gone through all our formats, found the proper term for a feature type in that format, and substituted it where appropriate. Of course, in the above it says “Layer” because AutoCAD deals in Layers; for a database format (like PostGIS) it will be “Table” instead:
Neat eh? Where there isn’t a format-specific term we’ll just stick to Feature Type as before.
Hopefully that functionality is something that new users especially will appreciate. For more long-term users of FME, how about this new setting?
The issue we’re trying to solve here is how a feature type’s attributes are defined. Manual Definition is what you’re used to, and dynamic you (hopefully) know about. But Union of Inputs is a new mode. Without wishing to confuse you too much, I’d say it’s “semi-dynamic”. It’s dynamic in Workbench, but not in the translation. Take a look at the following example, then come back to read that sentence, and it will make more sense!
When I added a Reader and Writer separately in Workbench, I got a new “Unionized” feature type, like so:
You’ll know it’s in Union mode because it has no attributes listed, but it does have a different icon. Now, when I connect the Reader and Writer feature types, this happens:
Yes! The Writer feature type automatically gets its attribute schema from the Reader feature type. Now it’s time to go back and read that above sentence again…. OK. Got it? So it’s not fully dynamic (it won’t take a different schema if the source dataset changed at run time) but it does get the schema dynamically from whatever is connected to it! And it’s “unionized” because…? That’s right! It takes the union of whatever is attached to it, like so:
So with this you can just connect anything to an output feature type, and those attributes will get written. No more need to manually define attributes or copy them from the Reader as a separate action. Of course I may not always want to do this, so I can go into the dialog and switch back and forth between manual and unionized any time I like. Just keep an eye on the icon on each feature type and you’ll know what mode you’re working in.
Personally, I love this update because nothing makes me happier than standardizing different bits of functionality. Yes, I’m an interface geek. Who isn’t?
Anyway, the reason for this update is that a lot of FME transformers merge attributes together, and that requires some decisions to be made. For example…
…if I merge features A and B, then what attributes do I end up with? A’s attributes? Or B’s attributes? Or both sets? And what happens if both features have an attribute with the same name?
In FME2015, controlling this has all become a lot easier. Look at the parameters for the FeatureMerger (for example) and you’ll now see a special section labelled Attribute Accumulation:
This gives you the option to merge supplier attributes (keeping the requestor), have the supplier attributes replace the requestor’s, or add the supplier attributes with a prefix. You can also decide what to do when there is a conflict: do you keep the supplier’s version or the requestor’s?
Another problem with the old method was that very few data-merging transformers operated in a consistent way. The FeatureMerger might behave differently to the NeighborFinder, which also behaved differently to the Clipper.
It was that old “comparing apples to oranges” scenario. But not any more! In FME 2015 the parameters for attribute accumulation have been added to a whole bunch of transformers; for example the NeighborFinder, Clipper, ListExploder (because exploding a list has the same issues as merging), and all of the overlay transformers to which merging applies.
So now all of these transformers operate in a consistent way, and when you become familiar with the parameters from one, you now know how all the other transformers work too.
Do you remember this poll of Workbench improvements from the 2014 user conference (click to enlarge)?
Well, we managed to get several of those done for 2015, starting with the most-requested. So here they are…
This feature was at the top of the list, so having it in 2015 should make many of you very happy. It is a new system for undo/redo commands in FME Workbench based on a tree-view of operations. The information is held in a new window on the left-hand side of the canvas, called History (click to enlarge):
Above you’ll see that the top part of the window is made up of a list of edits made to the workspace. Click on an edit in the list and the bottom part updates to show the workspace as it looked at that point. Double-click on an edit and the workspace in the canvas reverts to that particular state. So you’ll be able to carry out a whole series of edits and, if the changes perhaps don’t work as expected, you can undo all of them and revert back to the original point.
Also notice that the list of edits above shows a branch. Presumably the two changes were made in the branch on the right, and then undo applied. Now any further changes appear in a new branch. But the original branch still exists in case you need to refer to that.
Additionally, I just found that you can switch to one version, copy an object from there, switch back to the first version, and paste the object into it. I’m sure that behaviour will be very useful.
To be honest, for individual undo commands I’ll probably stick to Ctrl+Z. But for bulk changes (sort of reverting to a checkpoint) or for when I’ve messed something up and can’t remember how to get back to the working state, this functionality is going to be very, very useful indeed.
Feature Type Updates
Well, as per the poll, we implemented resizable feature types and transformers, like so:
You can resize individual objects or – like I am doing above – you can resize multiple objects at the same time. I wasn’t sure I liked this functionality at first, but it’s really growing on me.
Also, did you notice in the above screenshot, there is some sort of pop-up dialog? That’s another new feature we added for 2015. Basically, click on a Reader feature type and you get this pop-up dialog appear:
Notice that there are three options. The first is a shortcut to disable the Reader (very useful). The second is a shortcut to “Open Containing Folder” (again, very nice to have a one-click option). The third is a shortcut that opens up the Source Dataset dialog.
I think this may change a bit by release time (for instance, the folder icon seems to me more appropriate to Source Dataset than Open Containing Folder) so keep an eye out. But basically the behaviour will be there and I think it’s going to be one of those functions that are so obvious that you can’t quite believe we hadn’t implemented it before!
Grids and Guides
We took the grid functionality out of Workbench 2014 because it was expensive to maintain and we didn’t think many people really used it. Boy, were we wrong! From the negative feedback it turns out that quite a few people did use these capabilities.
So, for FME2015 we did the only thing possible and added the functionality back. In fact, we did better than that and implemented a really cool system for organizing workspace objects.
Firstly under View > Grids and Guides there is a dialog in which you can turn the grid back on in the Workbench canvas (click to enlarge):
But, more awesomely, there’s also this (click to enlarge):
See the lines on the left-hand side? Basically, whenever you move an object on the canvas, guidelines appear to automatically align the object with others. That way you can get objects lined up vertically or horizontally, with no effort at all. You can use the above dialog to turn on either grids or guides (but not both at once). It’s all excellent, and the only downside for users is that the certification standards for neat workspaces just went up a level or two!
As for the rest of the items in that poll, I believe we’re working on non-modal dialogs for FME2016. Non-modal means that you can keep a transformer dialog open and still be able to go and work on something else; i.e. you don’t ever have to click OK/Cancel to close that dialog before doing anything else.
As for Frankenbench… for the uninitiated, Frankenbench was the idea of incorporating the FME Data Inspector as a separate window inside Workbench. That way it’s better integrated and there’s no need to switch between applications as much. I can’t say I expect that to get implemented any time soon; but I could be utterly wrong. If I see us hooking up lightning rods to our development teams’ computers, I’ll let you know!
The Data Inspector hasn’t had too many obvious features added to it, but there are some “hidden” items that are definitely worth mentioning. Firstly, the options for adding background maps got a few additional data sources:
I think GME, Bing, and Nokia, are the new ones there. Additionally the Feature Information window underwent a nice redesign (it also went into a 2014 Service Pack, so you may already be working with this) and we added some funky new geometry icons:
And finally, the same information window got some new functionality on the context (right-click) menus, allowing you to save/copy text, or send the feature directly to the log window:
The log window was never really that useful in the Data Inspector – but now I expect to use it quite a bit more.
New Transformers and Formats
Just to finish off, let me list a few of the new transformers that have been added to FME2015:
- TweetStreamer – causes a workspace to run continuously listening for tweets (matching a set of filters)
- GoogleMapsEngineStyler – defines style information for features to be written to GME
- S3xxxx – uploader and downloader transformers for Amazon S3 (Simple Storage Service)
…and a list of some new formats
- OSG – Open Scene Graph format
- Minecraft – both a Reader and Writer for Minecraft files
- GeoPackage – an OGC standard based on SQLite
Working in the software industry is very interesting. I would never have realized how much of a team effort it is to create a product such as FME. If you are interested in being part of such a team, well… we’re hiring! Safe is an awesome company to work for and, because FME is a team effort, we’re not just looking for software developers. We need to fill all sorts of positions such as QA/test engineers, technical writers, and sales persons.
The technical writing position we have right now will be working with me on creating and maintaining our FME training materials. If you know FME, have a passion for writing, and – especially – if you have knowledge of technical writing tools like Madcap Flare, then I definitely want to hear from you!
In the meantime, don’t be afraid to give the 2015 beta a try – we’re past feature freeze now, so it’s nearly complete and very stable – and let us know what you think.
Mark IrelandMark, aka iMark, is the FME Evangelist (est. 2004) and has a passion for FME Training. He likes being able to help people understand and use technology in new and interesting ways. One of his other passions is football (aka. Soccer). He likes both technology and soccer so much that he wrote an article about the two together! Who would’ve thought? (Answer: iMark)