Author:
Kris Majury

All Blog Posts    >    About Our Customers    |   October 6, 2015   |    By Kris Majury

In the Scrum: XML Feeds and Tweets at Donegal County

FMEgoalpostsRugby madness is on! With the 2015 Rugby World Cup running through the end of October in England, fans of all stripes are glued to the matches.

And one of them, Daragh McDonough, is busy spatializing tweets, mauling XML feeds, and coming up with rugby analogies for FME’s role at Donegal County Council in Ireland. But we’ll get to that last bit later on…

The First Half: Geocoding Gritter Tweets

grittertweets

Click to enlarge.

Donegal County started out with their first FME license in 2004 – and today in 2015, it’s considered essential software. And it was during a particularly stormy winter that their roads department wanted to communicate with the public about their nightly gritting schedule via Twitter. Maps weren’t part of the initial conversation, until Daragh heard about the idea and thought, “Hey… isn’t there something in FME for that?”

With a bit of manipulation, he was able to geocode the tweets based on parsed route numbers. The workspace generates a GeoRSS feed, which is available to the public on an online map, and shows them which roads will be gritted every night.

grittertwitterfmw

The aptly-named GritterTwitter workspace parses tweets, spatializes them, and produces a GeoRSS feed. Click to enlarge.

There was an unexpected benefit from adding the mapping component to the workflow. The tweets needed to be in a predictable format structure so that the route numbers could be extracted. The engineers doing the tweeting ended up with pre-defined tweets they simply re-use, rather than having to write them each time. In the end, says Daragh, “Everyone’s happy! The engineers are happy, the public is happy, and we have maps to communicate the information as clearly as possible.”grittermap

The Second Half: Add XML Weather Feeds

With the public map online and doing its job, Daragh started looking for ways to make it even more useful. With storm after storm coming in, weather was the next logical step, and he went looking for an open data weather source.

The Norwegian Met office produces weather reports for the entire world as XML feeds, one per town. Daragh started with the feeds for all of the towns in Donegal, set up a workspace to transform the data into a clean and consumable form for the Esri-driven map, and added forecasts to the map for the entire county.

But people in Donegal often travel to the airports in Dublin and Belfast, so it seemed logical to add the weather data along those commonly traveled routes. And at that point, the map looked a little odd without the rest of Ireland covered… so they ended up with in the neighborhood of 180 individual feeds coming in, and coverage of most of the country. “This process was very addictive: I just couldn’t stop adding in more feeds and seeing FME churn out the results consistently,” he says.

windchill

This windchill map includes attributes calculated by FME, derived from wind and temperature data in the XML feeds. Click to enlarge.

Having the weather data available on the map where users could go and look for it was great. But with all the work already done to bring all of those feeds in, what else could they do with it? “Added value from existing data is one of my GIS mantras,” says Daragh. So could they alert people when there was bad weather forecasted– snow, winds, storms, and so forth? Of course they could! Again more twitter transformers to the rescue.

alerttweetsHe added another workflow, this time checking the weather forecasts for a specific set of conditions. Twice a day in the winter (and once per day the rest of the year), a workspace checks all of the feeds for forecasted conditions. And if the forecast includes bad weather in the next 12 hours, the workspace tweets alerts for predetermined regions in the County. The SpatialRelator transformer does the heavy lifting for this process.

The Game in Review

You might be thinking by now that these workflows are happening with the assistance of FME Server. But it’s FME Desktop, scheduled with Task Manager, doing the job every hour.

“In rugby there’s a saying – forwards win matches; backs determine by how much. FME is like the forwards,” says Daragh. “It’s doing the hard work, grabbing the feeds, transforming it all, and presenting nice clean, quick datasets for our online maps to consume. The webmaps are like the backs – running the ball, getting the glory, but without FME, the ball/data would still be stuck in the ruck! It’s one of the most powerful GIS tools we have.”

Thanks to Daragh for sharing his work with us. To (mis)quote beloved “Voice of Rugby” Bill McLaren, it’s as cunning as a bag o’ weasels!

Kris@Safe

@DonegalMaps on Twitter
Donegal’s Online Maps