Proactive Schema Design and Management

FME has long been known for its powerful data integration and ETL capabilities. Creating data workflows often requires expertise in database design and schema management; this is especially true when multiple disparate inputs are processed to match a common schema. This level of complexity and expertise can often result in less technical users placing the task in the “too hard basket”. This can result in substandard schemas or even the schemas not being adjusted to better suit the new use case. The latter often causing cascading problems as other systems look to use the data. In many cases, the ownership of the data and schema aren’t the responsibility of the GIS team. By developing processes that allow individuals external to this team to better understand, access and ultimately own the data and schemas. This presentation will cover how we have developed a process that allows users to define a schema; indexes, domains, lookups and aliases included, and use this to map incoming data. The process also makes use of the hundreds of formats that FME can read and write to, providing almost unlimited combinations of input and output formats. This process has enabled Abley to rapidly ingest municipal infrastructure data into civil engineering companies infrastructure to ensure the data is complete and appropriate for managing maintenance and repairs. It has provided the civil engineers, who’s expertise is not data, schemas and GIS, ownership over the process and has engaged them to a point where they have understanding, confidence and ownership of the processes involving their data.