At Safe we often talk about how applications crave data. Without data, applications are useless. However, the relationship between data and applications is symbiotic, as data without applications is also useless.

What’s the point of an organization collecting data if it’s not going to use it? What organizations really need to do, is to get the all of their data moving faster and faster to the people and applications that need it.

The 5 Rights of Effective Spatial Data Delivery

In the past, we used to consider four main components for the effective use of spatial data: making sure that the Right Data, gets to the Right People, at the Right Place, and at the Right Time. These four have been evolving, and with the emergence of push capabilities, there is now a fifth – the Right Way.

    1. The Right Data – The volume of data and the number of its sources are rapidly growing – leading to a two-fold challenge. First, identifying the best available data for the task at hand is a real concern. Second, to be used effectively the data must be delivered to the user’s application in a way that it is immediately usable. The less post-delivery processing that has to be performed on the data before it can be used, the better.
    2. The Right People – In today’s organizations everyone needs access to data. However, in many cases not everyone should be allowed to see the same data. Privacy and security need to be taken seriously.
    3. The Right Place – With the proliferation of mobile devices, the right place increasingly means whatever location a person is currently at. Today people expect to be able to access their systems from wherever they are, so long as they have a web connection.

 

This means “whenever I need it” or ASAIIP (“as soon as it is available”). In short, people expect to be able to easily use the most up-to-date data, all the time. (Stewart recently touched on the growing expectation of real-time delivery of data and how new web-based HTML technology can make this happen.)

 

“Data Push” versus “Data Pull”

Today spatial data is mostly delivered using a “pull” paradigm. “Pull” is when the receiving person or application initiates the flow of data. While this works fine in many cases, it can also result in a sub-optimal situation where the recipient (a person or application) is forced to continually check to see if there is any new or changed data available from the producer. In this situation, the resources of both the sender and recipient are wasted.

A better paradigm for this situation is when the sender sends (“pushes”) data to the receiver when data of interest is available. With this approach there is no need for the recipient to continually “ask” the sender anything – the data is delivered as soon as it’s available.

(Aside: Now the interesting part becomes how the recipient is able to specify what data they are interested in, and conversely how the data pushing is triggered. Stay tuned for more on this in January when FME 2012 is released – see the countdown clock here. If you have any thoughts or examples on how you would like to use this new capability, let me know.)

To be clear, the data push approach is not always the right solution as in some cases, it’s more desirable that the recipient trigger the data flow. The introduction of “data push” does however open a whole new way for users to keep the data moving. That is to say: getting the Right Data, gets to the Right People, at the Right Place, at the Right Time, and – now – in the Right Way.

[Note: this is the last post on It’s All About Data for 2011. See you again in the new year!]

About Data Interoperability Spatial Data Distribution

Don Murray

Don is the co-founder and President of Safe Software. Safe Software was founded originally doing work for the BC Government on a project sharing spatial data with the forestry industry. During that project Don and other co-founder, Dale Lutz, realized the need for a data integration platform like FME. When Don’s not raving about how much he loves XML, you can find Don working with the team at Safe to take the FME product to the next level. You will also find him on the road talking with customers and partners to learn more about what new FME features they’d like to see.

Comments

10 Responses to “The Importance of Spatial Data in Motion!”

  1. Bala K. says:

    Nice article. These “5 Rights” are what I call Cloud(with exception that “data pull” is also important as “data push”).
    Take a scenario where multiple users working on the data (like a shape file) getting managed by a central Cloud(where translation or transformation services are available). The Cloud should be able to
    1) Let Client(s) subscribe or request services.
    2) Get the shape file(or files of other formats) with “data pull”.
    If needed do some Client side pre-processing.
    3) Perform translation or transformation on Server.
    4) Do a “data push” to client or subscriber(s).

    And Congratulations for the exciting next release of FME.

  2. Don Murray says:

    Bala,

    Thanks for your comment. You make a good point. By making it very easy to move data around with FME, it is easier for users to collaborate and share data and ideas.

    The topic of “real-time” services/data feeds is something we will be talking about here in the new year. Suffice it to say that this is part of the FME 2012 story. As a pre-release teaser on what is possible, check out this video which I put together that shows how easy it is for FME Server 2012 to consume sensor data. http://youtu.be/Qy4SgH8jbiQ

  3. Bala K. says:

    This looks good. I was also impressed by your demo 3 years back where you used KML to show the Vehicle route.
    Sensor data is huge and it would have to be pre-processed or filtered before FME can process it.

  4. Bala K. says:

    I confused Sensor(change detector) with Video stream. But latter may be possible with some App or if HTML5 allows for that.

  5. Don Murray says:

    Bala,

    This is early days of our sensor technology and demo scenarios and you hit the nail on the head with respect to the moving KML demo with vehicles. Once we identify a sensor on the move (from http://www.pachube.com or elsewhere) we will put together a demo like that. It could be very much like the moving asset demo of a few years ago. The number of sensors out there are just exploding.

    It would be good to discuss this in the new year. I would like to understand more what sabre systems.com does and your interest in sensors and video streams.

    Watch an upcoming blog where I talk about the proliferation of sensors and the importance of real-time from that perspective.

  6. Bala K. says:

    Sure, I’ll wait for the new year as to what are possible with FME 2012. My company does GIS contract work and my client uses FME technology. Here I’m just trying to see the bounds of FME and inform my client as to what is possible.

    Ultimate real-time is like the stream(Video frames or some equivalent GIS) keeps flowing (eg: like from the client to server an back), and sensors are implemented along the pipeline.

  7. Don Murray says:

    Bala,

    Have a Merry Christmas and I look forward to following up with you in 2012! Thanks for your insights and one of the topics I would like to discuss is video streams and what your thoughts are on them.

  8. Bala K. says:

    Thanks Don. Here is a working group which is looking to standardize various real-time data from client devices. By the time the implementation is ready (around 2013), it would be nice to have FME support scripting with Javascript(instead of just TCL/Python) given it integrates well with the Web (and is most closer to data and can be moved around the network easily). The earlier article about “WebSockets/Node.js” makes me think that such effort is on the way. Who knows, maybe Sensor would become another object (just like Transformer/Reader) on the design interface.

  9. Don Murray says:

    Bala,

    Thanks for the information and the link. We are looking at many different ways of integrating FME with many different data sources. We just completed UDP support and are looking at other ways to make it easy for folks to get data into the FME environment. Email being one of them where you could send an email to @ and FME Server would automatically process the email or attachments. Javascript has also come up before in discussions here as we map out FME 2013 and beyond. I will create a PR enhancement with this information.

    Stay tuned for a coming blog article more about sensors and real-time capabilities.

  10. […] interconnected world this doesn’t cut it. To support the latest applications the latency is expected to be real-time; measured in seconds or less. Data consumers increasingly expect to be able to subscribe and then […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Related Posts