1) Workspace Example: CSV Writing
2) Transformer Updates: The Sampler, The Visualizer, His ListSearcher, Her SurfaceModeller
3) Safe Science Lab: Multi-Threading
4) KML Format: Where’s Google?
If there’s a common theme to this issue, it’s how little changes – or little bits of knowledge – can have a disproportionately large impact on the performance of our users. The section on transformer changes in FME2009 is a good example. Each one of the highlighted updates was initiated by a user request, and is a relatively small enhancement that is a big saver in either performance or user effort.
And that’s why we have this technical bulletin: to spread the word about these updates and, by doing so, maximize our users’ performance. That way we’ll give everyone the chance to look their boss in the eye and say modestly “Don’t thank me. We’re all part of the same team”.
PS: Notice the slight change in format, with the expanded title and contents section first – that’s to give a better experience for our RSS subscribers. Hope it works.
Workspace Example: CSV Writing
One of the most common support questions is how to turn plain text x/y coordinates into proper spatial data, but sometimes we get the reverse question: “how do I write out a set of spatial features as a file of x/y coordinates?”
As with many FME tasks, achieving this requires a sequence of transformers; the snag being that the first of these – the CoordinateConcatenator – is relatively unknown. What this transformer does is extract the coordinates of a feature as a long string of (comma separated) numbers and attaches them as an attribute. From there it’s a relatively simple task to split off each coordinate pair into a list (AttributeSplitter), explode that list into a separate feature per pair (ListExploder) and then write the data to a file using the Textfile writer.
A second complication is that quite often you’ll want a header record in front of each features’ coordinates, in order to identify each feature within the text file. This is where you need to get creative in adding an extra “stream” to your workspace – attendees of the Advanced Training at the 2008 FME User Conference will understand what I’m talking about – and making sure that everything gets back into the correct order.
When you’ve done all that the final workspace will look something like this:
But don’t worry if you can’t quite understand what’s happening – it’s my fault for not using Best Practice – because the workspace and detailed explanation are now available as an fmepedia workspace example. Hope you find it useful.
Recent FME2009 builds have seen a number of very useful transformer updates.
The Sampler transformer now has an extra output port through which non-sampled features are output. This could be useful in a number of situations, but particularly where you want to skim off the first x features while retaining the rest.
When you send data from Workbench to Viewer using a Visualizer transformer, what happens is that the data is simply written to a temporary FFS format dataset which is subsequently opened in the Viewer. A recent update to the Visualizer allows you to specify the name and location of this FFS file, meaning that a user can save a more permanent copy of the output without having to add a writer.
Previous versions of the ListSearcher allowed the search string to be specified by an attribute value by using the format &<attributename> . A recent update to the ListSearcher turned the setting into a combination of text entry and drop-down list (what we at Safe call STRING_OR_ATTR), meaning that the same effect can now be achieved by selecting an attribute directly rather than having to enter the FME-specific syntax.
Surface modeling operations in FME work by converting a number of input features (3D lines, points, breaklines, etc) into an internal FME Surface Model from which various outputs such as contours and triangles can be generated. Considerable work has to take place to create the initial surface model, which leads to large overheads when the same surface model is required in different workspaces.
A new option in the SurfaceModeller reduces this overhead by allowing the user to read and write FME Surface Model datasets (.fsm files). Now a user can save the surface model from one workspace for use in another; a useful benefit when (for example) needing to drape different sets of features over the same surface.
Above: When this translation re-used a surface model it was over 50% quicker.
I’ve asked to have the GUI adjusted slightly (to me it’s not quite intuitive enough) so in the meantime my advice is to read the SurfaceModeller help very carefully. Plus, I’m hoping to find the same options exposed in related transformers (such as the SurfaceDraper) in the near future.
Above: Overwrite mode creates a new surface model file. Append mode reads the data back – when processed it will write it back with any additional features that entered the transformer appended into the model.
Safe Science Lab
The latest news from Safe’s top scientists is that work is slowly beginning on applying multi-threading to FME operations. For those not in the know, a thread is a computer task, so multi-threading simply means executing multiple tasks at the same time. For example, measuring the length of ten line features could be achieved by ten separate, but simultaneous, tasks, rather than one task which processes each feature in order.
It sounds easy, but the analogy of juggling ten balls at once – instead of one ball ten times – holds good: it’s a much more complicated manoeuver and we certainly want to practice before going live in public! So expect small steps on this one, rather than a single, massive improvement.
As of build 5572 the format “KML21 (Google Earth KML)” has been renamed “OGCKML (OpenGIS® KML Encoding Standard)”. Two obvious reasons: 1) OGC now ‘own’ the format and 2) the ’21’ part was confusing since our KML support is up-to-date and we do actually support version 2.2.
The big gotcha here is for users who type “Google” into the formats gallery – since Google has been removed from the format name you’ll no longer find the format. Type “KML” and you’ll have more success.
This Edition of the FME Evangelist…
…was written to the tune of “Taking on the World” by Amsterdam.
Amsterdam are a great band from Liverpool. Check out this live performance, then go buy their latest album.
Mark IrelandMark, aka iMark, is the FME Evangelist (est. 2004) and has a passion for FME Training. He likes being able to help people understand and use technology in new and interesting ways. One of his other passions is football (aka. Soccer). He likes both technology and soccer so much that he wrote an article about the two together! Who would’ve thought? (Answer: iMark)