The FME Desktop training course has a section titled Best Practice that contains a number of recommendations for using FME. Taken together these recommendations form a sort of philosophy of the fundamental nature of FME.
I believe the word for this is Tao. I’m calling it the Tao of Mark because it’s definitely a personal philosophy. It’s the way I try to work; but there are always different points of view.
Of course, to be fully enlightened, be sure to attend an FME training course. There are many throughout the year, either in-person or online.
The Best Practice chapter of our training is split into five sections:
So I am organising these recommendations along the same lines. I hope you find them useful.
A good design starts from the very moment you begin the workspace! This example is not a good design:
Can you tell which of the following recommendations would have helped here?
Recommendation: When creating a workspace you’ll be prompted which feature types to add to the canvas. Don’t add feature types you don’t need, as it will clutter the canvas.
Recommendation: When source data has many Feature Types it can be difficult to manage. If all source types are being routed into the same transformer then you can (and should) replace them with one dynamic feature type.
Recommendation: When you delete all the Feature Types for a particular Reader, be sure to delete the Reader too, unless you have a specific reason for keeping it.
Recommendation: Hide any attributes you don’t need to use in the translation. It will make the workspace tidier and Workbench may even respond faster.
Recommendation: If you create the same workspace repeatedly, then use the Defaults options to avoid having to enter the same parameter values each time.
Being able to structure a workspace to carry out the required task is also an art. Knowing how to embed Functions and spread the translation over several workspaces, doesn’t necessarily mean you should! Here’s another frightful section of workspace:
Again, can you tell which recommendation would help here?
Recommendation: If you’re tempted to use an FME function inside a workspace, then contact the support team at Safe who will either be able to find an alternate method, or file an enhancement request to have the same functionality supported directly.
Recommendation: If you’re in a situation that requires the use of two workspaces to carry out one translation, then reconsider your decision and consult other FME users for advice. It may not be necessary.
Recommendation: If you’re unsure as to what transformer to use, or are worried that your workspace has multiple streams of data for no good reason, then consult other FME users or the Safe Software support team.
As you know, under Tools > FME Options there are a number of different settings. These are what I prefer:
Here are my key recommendations:
Recommendation: Turn off the FME option “Open Attributes when Linking” The times you won’t want the transformers expanded far outweigh the times you will.
Recommendation: Turn on the FME option “Draw Bookmarks with a Filled Background”. It adds better visibility to bookmarks.
Recommendation: Increase the FME Option “Number of Recent Workspaces” to the maximum. This is a personal preference more than anything else, but the lengthier menu is offset by an ability to locate workspaces in a more efficient manner.
Recommendation: Turn off the FME option “Automatically Generate Header Annotation”. Anyone who reads this blog shouldn’t really need it.
Recommendation: Turn on the FME option “Use Transparent Annotations”. It’s useful to be able to view objects through annotation.
Style refers to the look and feel of a workspace. Is the workspace well laid-out and easy to understand? If not, tidy it up. You’ll appreciate the result when you come back to make edits in a few months. This workspace does not have style; it has built-in confusion!
In fact, this workspace breaks nearly all of these recommendations:
Recommendation: Create user annotation on nearly all transformers, to help future edits and updates go more smoothly.
Recommendation: If you use bookmarks for nothing else, use them to divide your workspace into easily identifiable sections. It really makes the canvas more readable.
Recommendation: Rename transformers to help identify them both on the canvas and in the navigator window.
Recommendation: Try to avoid adding extra vertices to a connection line, because their behavior is unpredictable when you move one of the attached transformers.
Recommendation: Don’t cross the streams! It’s worse than letting your anti-matter chopsticks touch!
OK, we make a ton of performance improvements with every release. But by adjusting your workspace slightly, you can help your translations run faster too.
This workspace reads an entire dataset without filtering. Why? Would you go into a restaurant and order everything off the menu, and only decide what you want once it has all been delivered to the table? Then why read an entire database when you only want a small part of it?
Recommendation: When you want to filter source data, and can use a specific reader parameter to do so, it is more efficient than reading all the data then using a transformer.
Recommendation: For maximum performance, determine which writer is receiving the most data, and ensure it is top of the writers list. But you knew that, right?
Recommendation: Only carry through the translation any geometry and attributes you need for transformation or that you intend to be available on the output. Otherwise remove excess data as early as possible in the translation. An AttributeRemover helps. Hiding (unexposing) attributes doesn’t.
Here’s a no-brainer: If a translation fails, look in the log window first. Seriously. Whether it fails with a crash, or just unexpected output, the log file will usually tell you why. Then start looking to the workspace itself for feature counts:
Here are some other debugging recommendations:
Recommendation: Use Feature Inspection when a transformation is going wrong and you can’t tell where, or when you suspect one particular feature is causing a problem. It’s likely to help less when the problem is a crash or ERROR in the log window.
Recommendation: Keep log timings turned on when you are trying to debug performance issues.
Recommendation: Identify bad features using the spatial log (.ffs) dataset. It will probably be easier than trying to locate the feature within the full source dataset.
Recommendation: Use a Workspace Search to locate items identified as problems in the log. This is especially useful when the workspace is very large.
Organization is probably the least considered issue. It means being efficient, co-operating, standardizing, sharing items, and much more. Shared folders are good for this, as are templates:
Recommendation: Use templates when you have a fixed design (a layout, schema, or transformation) that you (or colleagues) will use again and again in a series of workspaces.
Recommendation: For a large scale FME project, set up a project structure and stick to it. Keep copies of all FME workspaces, maybe in a revision control system like Subversion.
So there you are. About 25 different recommendations for my philosophy of FME. Follow these rules carefully and you too may be enlightened about the true nature of Best Practice!
PS: Don’t forget, the FME 2012 world tour takes place starting April but continuing for quite a while. There are many, many global locations and it’s always a good chance to catch up on new FME functionality, meet Safe Software staff, and network with other users.
Mark IrelandMark, aka iMark, is the FME Evangelist (est. 2004) and has a passion for FME Training. He likes being able to help people understand and use technology in new and interesting ways. One of his other passions is football (aka. Soccer). He likes both technology and soccer so much that he wrote an article about the two together! Who would’ve thought? (Answer: iMark)