Whitepaper: Spatial ETL
The Essential Data Pipe To Better Geospatial Intelligence
Business intelligence. It is what spatial data extract, transform and load (ETL) tools have been bringing to both the private and public industry for a decade. Spatial ETL tools aim to bring geospatial intelligence to organizations by bringing the potency of the geographic position to the desktop.
From ETL to Spatial ETL
Simply put, the objective of ETL tools is to transfer data from one datastore to another. To reach that objective they perform three separate functions. First the extract function reads data from a specified source datastore, extracting the desired data. Next, the transform function processes the acquired data - transforming it and even perhaps combining it with other data - to package it into the correct structure for the destination datastore. Finally, the load function writes the resulting data to a target datastore.
ETL tools are used either to acquire a temporary subset of data for a customized report or a specific business application, or to obtain a more permanent data set to populate a data warehouse in order to convert from one datastore to another or to migrate from one datastore or platform to another.
The power of ETL tools is that they provide an economical mechanism to quickly combine information from very large data repositories, displacing disjointed databases with windows that are open to all business-related information and operations. The seamless interconnectivity gives organizations the opportunity to improve productivity and better leverage the information they already have to ultimately make more informed business decisions. Bottom line? ETL tools help companies extract dollars from data.
Safe Software coins Spatial ETL
Safe Software coined the term Spatial ETL, which simply means ETL for spatial data. Spatial ETL products are designed to provide the same bottom-line benefit as traditional ETL systems, but with one distinct difference - they also include the spatial world of an organization, either in helping it create spatial data or in capitalizing on the data it already has. Here too the goal is to leverage existing spatial data - often the most untapped key corporate asset - to enhance business analysis and ultimately influence business decisions.
What Spatial ETL Does
A Spatial ETL tool such as Safe's FME will extract data from a desired datastore, transform it to the projection, format or style requested and load it into another datastore. Again the intended data can be obtained for temporary use or as part of a permanent data migration/translation project.
Often companies employ data translators to migrate or translate one particular datastore to another. Similar to translators, Spatial ETL tools will restructure specific data sets or an entire datastore into another datastore. However, unlike traditional data translators that are typically used when the source system is being abandoned, Spatial ETL systems can create a mirror of the data on both systems. This allows organizations to still use both systems during the migration process - a feature that is particularly relevant for organizations with legacy systems.
As a practical business tool, Spatial ETL will transform databases of tables, lines, points and polygons into a seamless spatial composite of business operations on a neighborhood scale, regional scale, national scale or global scale. For the nationwide retail chain, for example, a Spatial ETL system can integrate locations of depots and stores with street network data and real-time traffic reports, and in conjunction with GIS software, dispatchers can more effectively assign drivers routes and then monitor those routes, warning drivers in advance of traffic accidents or road construction and re-routing deliveries on the fly.
In short, Spatial ETL will do for the spatial world what the traditional database ETL tool has done for the corporate world: provide the ability to integrate disjointed data stores and leverage that enhanced business knowledge to compete with corporate cunning.
The Spatial ETL Trend
If the spatial data pundits are correct in that 80 percent of all corporate data is location-based, then the case for using a Spatial ETL system to help access, integrate and apply that geospatial data is indeed a strong one.
Just as companies over a decade ago began to view and to employ the traditional ETL tool as a facilitator to better corporate information, so too have private industry and public organizations begun to adopt the Spatial ETL tool as the de facto facilitator to harnessing the intelligence held within spatial information.
After all, business is so tied to the question of "Where is?" that the application of geospatial data and GIS are becoming unmistakable elements of daily business life. At the same time, however, effective spatial data management has also been dogged with data interoperability issues - proprietary formats rather than open formats have dominated, making it difficult for companies to easily and efficiently integrate and apply spatial information. Spatial ETL systems aim to resolve this issue by empowering a GIS package or similar spatial data management tool with the inherent interoperability to provide seamless interaction with a multitude of datastores, and to share that information across the enterprise in real-time. It is this invaluable data pipe to spatial information that is positioning Spatial ETL on the strategic-intelligence map of an impressive number of business and government sectors.
Spatial ETL Applied
With Spatial ETL as a core data extraction and transformation tool, organizations have the opportunity to adopt a holistic approach to business operations, to streamline business processes and achieve cost transparencies, without sacrificing data integrity. Gone is the idea of individual department productivity and the IT structure that can limit the company's potential to truly work as one.
As a data transport mechanism, Spatial ETL tools find their strengths in:
- Tying disparate database systems together
- Simplifying data migration/translation projects
- Distributing data across an organization
- Providing back ends for Web-based mapping tools
Spatial ETL systems, like Safe's FME, are designed to scale with an organization's data integration needs - whether it's maintaining a small database environment or a mammoth database structure. To read more about how this is being used in organizations, be sure to visit our library of Case Studies.