My Favourite Things: By Mark Ireland
It’s that time of year when we all start to look forward to the next release of FME.
As usual there are many, many updates. I sometimes think our developers are part cyborg the amount of work they get through. But don’t worry about keeping up with them, because I’m here to help.
As FME Evangelist I comb through every single change made to our products (I reviewed about 10,000 for 2012). From that I create the What’s New file, the What’s Great page, and – most importantly – I pick out the tastiest treats to serve to you here.
So, as a flavour of what is to come, here’s a list of my highlights so far for FME2013. Just a brief list really as I’m sure to go into detail later, and there may be other updates still to come.
Some are big changes; some are just minor updates. But all of them in some way or other make me really happy.
Here’s a small, but useful, update to start with. For a long time I’ve been using Ctrl+W to close items in most software packages; so long in fact that I’ve unconsciously tried to do it in FME more than once. Well in 2013 this shortcut will now work.
Hurrah! Not necessarily the biggest update, but it works for me.
Place a transformer and you’ll notice many now carry an output port labelled REJECTED. That’s where features that aren’t compatible with the transformer – the wrong geometry type or otherwise invalid – will emerge.
This is great because it’s more consistent. Past behaviour was a bit variable (Error? Warning? Continue?) but now a workspace author can be more confident handling bad features.
To stop a translation add a Terminator transformer to that port. To log bad features, add a writer or Logger transformer. To continue – keeping those features – just connect them to the next INPUT port.
Simple! Not every transformer will have this REJECTED port yet (some just don’t need it) but I’m sure we’ll be adding them in where possible in the very near future.
If you’re like me, the PolygonBuilder transformer is often preceded by a Snapper to get the data cleaned up. Now you don’t have to bother; the PolygonBuilder has built in snapping options.
Again, not the biggest update, and for experienced users it will be little more than a minor time saver. But new users gain because it incorporates functionality they might not otherwise know about.
There’s a lot of scripting capability in FME (Tcl, Python, SQL, XQuery) but the editing tools have been quite basic until recently. FME2012 improved things enormously with the new editor dialogs. Now FME2013 adds search and replace functionality.
Simple press Ctrl+F to search for text (or Ctrl+H to replace). For me this is going to be a real time saver and I’m sure it will be for many users too.
Incidentally, SQL users get an added bonus. The left-hand part of SQL dialogs now includes shortcuts to most SQL commands (specific to the database being used) and a link to documentation for that format. Nice!
This is one of my favourite transformers, but it was always difficult to do multiple actions. As designed you had to create a new network model for every path you wanted to calculate.
Not any more! In 2013 this transformer has been redesigned so that the same network can be used for multiple network calculations.
I hear that one user’s workspace now takes 18 minutes to run – when it used to be an overnight process! So if you’ve felt limited by this transformer before, give it a try in 2013. It’s exponentially more awesome!
The Joiner parameters used to be set through a wizard. But we don’t like wizards and are sure you don’t. So in 2013 we changed it to a single dialog.
There’s also a new option for handling 1:M results, but I’ll let you explore that for yourself.
Many (many) users have asked for us to be able to write Shape datasets with a spatial index. Now we can! We created a new parameter and plugged it into ArcObjects:
So, as long as you have ArcGIS installed you can now write Shape spatial (and attribute) indexes. And yes, Shape is still the #1 format of choice for FME users!
I could (and will) write a whole post about the GeometryValidator transformer. For now, let’s just say that it incorporates a whole bunch of different validation techniques and also includes functionality for repairing many of the problems.
In particular it’s very good at picking up and fixing “degenerate” geometry. By degenerate we mean any geometry that doesn’t really match its supposed type; for example a line with only 1 point, or an arc without a radius.
Degenerate geometry is the main reason for writer failures so, if you have problems with a writer failing on a particular feature, run the data through this transformer first and you may well find the problem resolved.
Updates are still ongoing, so the dialog above might have changed by the time you read this. But once it’s settled down I’ll write more about all the options in a separate post…
Zip File Read/Write
This might actually be my favourite – and the most influential – update for FME2013. To put it simply, FME can now use zip files as the source for a reader, and can create zipped output datasets too.
I’ll let you ponder on all the potential uses, but my first thought was reading datasets stored on the web (or in the cloud if you prefer). That’s because this capability includes reading from URLs too. Now that’s #intriguing!
Last, but never least, my favourite Server updates are all the new protocols that we now support for notifications.
If you weren’t aware, notifications are automated responses to some sort of trigger event. In FME2012 an event could trigger Server to run a workspace, or even ‘hit’ a pre-defined URL, and send a simple notification message.
However, in 2013 the range of protocols (methods of communication) has increased dramatically!
Firstly we have a wider range of trigger (event) protocols, so that notifications can be set off by the receipt of an email (Server 2013 includes an email server!) or by the presence of a file in a folder.
But 2013 also includes a set of notification protocols, to issue notifications in different ways. Notifications can be sent by email (they could before, but now it’s simpler and supports HTML), or to an Apple device, an Android device (Google Cloud Messaging), an FTP server, and even through a UDP (User Datagram Protocol) connection.
Yes, by having Google and Apple protocols it means that we can communicate directly with apps on mobile devices. In fact the screenshot above was taken from an FME app that you can already find in the Apple store.
As if that weren’t enough, there are TCPIP transformers too, which can be incorporated into Server workspaces to handle events/notifications that need to run at very high speed (multiple events per second).
All in all, it’s a great set of functionality. I’ll be posting about these very shortly (especially the Apple Push protocol) and you can expect to see some great demos turning up soon on the FME YouTube channel.
So that’s my list of favourites for FME2013 and, amazingly, it is a nice, neat list of ten items. That’s not to say there aren’t more exceptional updates (HTTPMultipartUploader would also be up there, and I didn’t even mention what happens when you delete a transformer in 2013!) but these are the ten that I feel will have most impact on how I use FME.
Please do feel free to download the beta version and see what updates you like the most. Of course, there are still three months of development to go before FME2013 is released, but most of that will be testing and minor fixes. I don’t expect to see many more major updates.
Incidentally, another task we have for this time of year is to review what updates we want to go into FME2014. So if there’s new tool, function, task, or whatever, that you would like to see in FME, now is a great time to let us know. You can contact me directly or go through the support team at www.safe.com/support
Mark IrelandMark, aka iMark, is the FME Evangelist (est. 2004) and has a passion for FME Training. He likes being able to help people understand and use technology in new and interesting ways. One of his other passions is football (aka. Soccer). He likes both technology and soccer so much that he wrote an article about the two together! Who would’ve thought? (Answer: iMark)