Download OpenStreetMap with FME

One of the problems with obtaining OpenStreetmap data is that it's not possible to get large chunks of data in one go, unless you download state- or countrywide files from Geofabrik or Cloudmade (but they don't seem to offer the full OSM datamodel). So I've set out to come up with an easier solution, using FME.

It's a combination of 3 FME workbenches:
osm_manager.fmw is where you set the parameters. These include the bounding box coordinates of the area you'd like to download the data for as well as a 'chunk size'. Since the XAPI that is used is limited in the number of objects it can return you can only get larger areas by cutting them up into smaller chunks. How high the chunk size should be is a bit hard to guage, the default of 0.05 (that's degrees) seems to work. The manager divides the area you've specified up in those smaller chunks and then fires them off to a second workbench one by one, using a WorkspaceRunner.

This second workbench, osm_fetcher.fmw is doing the actual downloading. Basically it's a HTTPFetcher that gets fed urls and writes out the result as text files (containing the raw OSM XML data). Once that's all done control is handed back to the manager, who then fires off the last workbench, osm_writer.fmw. This workbench reads in all the OSM files, clips them to your original bounding box coordinates and writes the result out to shapefiles.

Since the OSM XAPI will not perform any clipping, the osm_writer workbench removes any duplicate objects it can find. The added benefit of that is that there will not be any noticable artifacts of the tiling process.

Some possible improvements of this process include reprojection upon output (it's all being done in WGS84) and adding in parameters to direct the output to another directory (currently all .osm files are being stored in the workbench directory, all shape output in a subdirectory)

You can download it here. Unzip it and open osm_manager.fmw in FME Workbench, set the parameters and run it.