“Large LiDAR datasets usually have power over their users, but with what you’ve done with FME 2011, users will now have power over their LiDAR data.” That’s what Gene Roe of LiDARNews.com said to me last week during a quick conversation about the new point cloud functionality in FME 2011. It’s true. You can now do a lot with your massive point cloud datasets and I’ll get to that after sharing what we’ve learned about point clouds.

The Right Way to Approach Point Clouds
Regardless of whether you call it LiDAR or point clouds, this data type poses substantial data management issues. The key breakthrough comes when you began to see point clouds for what they truly are – misbehaved (dare I say “evil”?) rasters. Yes, rasters can be large – horribly large even. Yes, rasters can have many bands. Yes, the raster pixel values can have different data types. But even though it took us multiple years to finally beat rasters into submission with FME, I’m sorry – rasters just aren’t very evil. You can always easily find the value for a particular coordinate by doing some simple math. And really, how scary is that?point-clouds

But point clouds? You have files with even more horribly-huge numbers of samples in them. And these points aren’t spaced in any regular way. In fact, there’s no way of really knowing how they’ll be laid out spatially until you read them all (thank goodness for the spatial index that the libLAS folks provided, is all I’ll say). For a given spot on the earth, you could have multiple readings! Each reading is a trove of information, potentially including color values, intensity, classification, return #, as well as a precise x, y, and z. So what we have is a very rich dataset, increasingly less expensive to collect and increasingly more common, but very unruly to manage.

So how did we wrestle down the point cloud menace? For one thing, we applied the hard earned lessons we learned from our raster experience and modeled point clouds in FME just as we model rasters – handling each point cloud as an atomic unit in our workflow. We went out and talked with a large number of our users to find out just where their point cloud pain points were. We documented these scenarios, then went away and built the infrastructure. We also called in expert help – in particular, Howard Butler of libLAS (which we used for its spatial index and file I/O and chipping), as well as the folks at Pointools (who worked with us to add POD file reading and writing).

And we worked very hard to make sure this new data type integrated with all the other data types of FME in a reasonable fashion, which turned out to be a much larger task than we imagined at first, but also opened the door to a wide range of powerful data integrations (like the one in the video below where I read, clip, reproject, and create a surface model from LiDAR data, and then combine it with imagery to create an interactive 3D PDF).

GIS Folks: Take Control of Your Point Cloud Data
The net result? Well, I hadn’t thought of it this way until after speaking with Gene, but I believe that in GIS shops everywhere, the once mighty and powerful point cloud files are now cowering with fear, knowing that at any time their new masters may be sending them out on a transforming date with FME 2011 and that they’ll be coming back cut into pieces, split into parts, vertically reprojected, scaled and offset far from home, or worst of all, thinned into a pale shadow of their former selves; at last completely ready to have their intrinsic value wrung out of them by the application of their users’ choosing. Point clouds, say goodbye to your PhD In Horribleness.

Do you have unruly point cloud data in your shop? If so, let me know if the functionality described in my video above, or in Dmitri’s Point Cloud Lab, addresses your point cloud pain points. I’d love to hear what you plan to do with your point cloud data – drop me a line.


Bad Horse had allowed LAS files into the Evil League
of Evil Data solely on the basis of their sheer evil size.

About Data LIDAR Point Clouds Raster Spatial Data Interoperability

Dale Lutz

Dale is the co-founder and VP of Development at Safe Software. After starting his career working spatial data (ranging from icebergs to forest stands) for many years, he and other co-founder, Don Murray, realized the need for a data integration platform like FME. His favourite TV show is Star Trek, which inspired the names for most of the meeting rooms and common areas in the Safe Software office. Dale is always looking to learn more about the data industry and FME users. Find him on Twitter to learn more about what his recent discoveries are!

Comments

10 Responses to “How to Put LiDAR and Point Clouds in Their Place”

  1. […] This post was mentioned on Twitter by Matt Ball and Dale Lutz, Safe Software. Safe Software said: "How to Put LiDAR and Point Clouds in Their Place" a new blog post from @DaleAtSafe http://ow.ly/3KV0M […]

  2. First, hats off to you guys for putting LiDAR front and center in this release! I am continually shocked at how prevalent LiDAR is, but how few tools are out there in the mainstream GIS software packages to make the LiDAR accessible to the GIS community. I completely agree that rasterizing LiDAR is the single best way to make LiDAR data accessible to the GIS community. Of course the down side is that one looses a tremendous amount once a surface is created, but such a sacrifice is warrented if it opens up the data for wider use.

    One of the typically issues with rasterizing LAS files is that doing it tile by tile leads to nasty little gaps between the tiles, a result of the interpolation process. A way around this is to do a simple point to raster conversion without interpolation, but one has to either up the grid cell size or resample the raster, effectively degrading the data, to avoid gaps. Of course this is probably more of a concern for us LiDAR data wonks than the average GIS user.

    We (University of Vermont) were just accepted into the FME grant program so I am really looking forward to making the leap from the ArcGIS Data Interoperability extension to FME and testing out these LiDAR tools.

  3. Dale Lutz says:

    Thanks very much for the feedback, and welcome aboard the FME Grant Program! Can’t wait to see what you do with it.

    One way that could work to avoid the tile edge effects would be to make a surface model for an area larger than the tile, and then cut it out afterwards. In fact, the way FME works, I suspect we could run the LiDAR points all into a large SurfaceModeller, output a single raster feature, and then cut it into tiles. Under the covers we make only one Surface and there should be no edge effects. Definitely something to explore — we’ll be in touch.

    Thanks again,

    Dale

  4. Henry says:

    Hey Dale
    Would the same techniques work with terrestrial point cloud data? I would like to create some 3d models with the point cloud but the viewer is not rendering the 3d very well. Thanks for the post.
    Henry

  5. Dale Lutz says:

    Hi Henry,

    Many/most of the things we do should work just fine, but in terms of surface modelling, that is more of a 2.5 D operation and as such better suited to airborne collected point clouds. The “FME Data Inspector” should show your data just fine though.

    I’d really love to have a look at some of your terrestrial point clouds, if you’re able to share and they are small enough, please upload a sample to ftp://ftp.safe.com/incoming and drop me an email at dale.lutz AT safe.com and we’ll give it a look.

    Thanks,

    Dale

  6. […] will venture. To be sure, though, we did touch on what’s new and great in FME 2011 including LiDAR / Point Clouds, FME Server improvements, and XML enhancements – just to name a […]

  7. […] is changing how we model our world in 3D, bringing with it new data transformation challenges and huge data volumes. It also highlights some old and new challenges with coordinate systems. […]

  8. […] use cases, I was reminded again of Gene Roe’s comment to me some months ago when we unveiled our LiDAR support, saying – if I may paraphrase: “The data used to have power over the people, but now the […]

  9. siju chacko says:

    Hi,
    I am a student from Indian Institute of Technology Kanpur ,India.
    We were working on an issue wherein a scene is captured using Terrestrial Laser Scanner (ILRIS36D).

    The scene is captured at fov of 3 degrees paused then next scene is captured and so on in a sequential manner from left to right.

    I want to convert the point cloud data captured as mentioned above into raster of size 128 x 128 showing the range and intensity information for each pixel.

    How should we go about it.

    Thanks and regards
    Siju Chacko

  10. […] year ago on this blog I wrote about putting LiDAR in its place with FME 2011. One year (and a whole bunch of hard work) later we’re back with FME 2012, and we’ve got […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts