Over the past few weeks I have presented in many different cities as part of the FME 2011 World Tour (here’s a summary of an event in the UK). Each year we look for common themes. Last year it was the growing presence of LiDAR data which led to our LiDAR (Point Cloud) initiative that was introduced in FME 2011.
This year, what struck me was people being forced to access to a large amount of data while only actually being interested in part of it. People kept saying things like “I have more data than time!”
The point being that they don’t want to, or have the need to, work with all of the data. In short, sometimes you only need some of the data – the part that’s important to you.
Separating the Signal from the Noise
In the early days of FME the big concern of users was: “Can we move all of our data?” As my colleague Dale recently talked about, data storage has come a long way. And so while moving the entire dataset used to be an obsession, and the number one question we would get asked – now we rarely get asked that question.
More and more, people only want to extract the information out of the data that is important to them for the task at hand. Since different people are concerned with different types of data, the challenge that now exists is to make it easy for people to access the information that they want.
To see this, take a look at our LiDAR scenarios. Many of these scenarios are about how to reduce file sizes or select pertinent data. Thinning, splitting, tiling, and clipping are front and center to make it easy for users to work with these datasets. The goal for the user is simple. Extract only the information that they “need” for the task at hand. Here’s a clipping example.
BIM scenarios are similar as well. We are working hard with partners on BIM to/from GIS. Again here users definitely do not want to move an entire BIM model into their GIS. The data volumes for a complete BIM model are huge as it contains everything from HVAC, to electrical and plumbing. People we talk to want to move just a small portion of the BIM model to their GIS. Again, they want to move just enough to solve their problem, whether that problem is space allocation and management of a large facility, interior routing, or otherwise.
Integrating a Wide Variety of Datasets
It is not just the size of the individual datasets that are growing, but it is also the number of datasets that are available. The story here is very similar. People don’t want entire datasets but rather an efficient method of integrating data from a wide variety of different datasets to produce a new derived dataset that meets their needs. This is the new norm for organizations.
This new norm is in stark contrast to the early days when the vast majority of uses of our tool were for running one-time data moves. Now, moving and integrating the latest and greatest (and always changing) data is done on a weekly, daily, hourly, or even on an as-needed basis.
While there are many ways to do this, one of my favourite examples of integrating multiple datasets is in Adobe 3D PDF using our FME technology. The greatest bit is that because the output is in PDF, even your grandmother can understand it. On the world tour, I showed an “uberdraping” example, which takes a variety of GIS and CAD vector data (roads, rivers, and parks) places it onto a raster image, and then drapes it all over a DEM before adding a BIM model. Feel free to download the below PDF yourself and pan around, or check out some similar examples from other people here or here.
This explosive growth of data is very exciting. Enabling people to get the data they want, when they want, to the application that they want is what our FME technology is all about. Often the challenge is not moving the massive amounts of data, but rather “precision selecting” or “data mining” to get data that is needed. What challenges do you and your organization have with efficiently extracting only the data you need?
Don Murray
Don is the co-founder and President of Safe Software. Safe Software was founded originally doing work for the BC Government on a project sharing spatial data with the forestry industry. During that project Don and other co-founder, Dale Lutz, realized the need for a data integration platform like FME. When Don’s not raving about how much he loves XML, you can find Don working with the team at Safe to take the FME product to the next level. You will also find him on the road talking with customers and partners to learn more about what new FME features they’d like to see.
