| Home | |
| Consultancy / services | |
| Projects | |
| Development | |
| Training | |
| Support | |
| Products | |
| Portfolio/media | |
| Download | |
| Links | |
| Contact | |
 |
|
 |
|
 |
For quite some time now I have been collecting geographic data from various sources, so that I have immediate access to at least some basic data in case I need to produce a map for a certain country or region. Even if I don't end up using that data, it would be beneficial for me to be able to load it up quickly and use it as placeholder data until something more detailed comes along.
In my daily work, I use FME, Adobe Illustrator with MAPublisher and Manifold a lot. Fortunately this setup doesn't limit me to a particular format, thanks to FME I can handle just about anything.
The mainstay of the data I intend to use this way is the 4-cd VMAP collection and the data that came with my copy of Manifold (like VMAP, partly derived from DCW), but there's also other sets that I collected along the way, as well as data that was produced or upgraded manually. Currently, the data is stored in a number of different formats (VPF, MapInfo Mid/mif, Esri Shape, Manifold native to name but a few). If I need to make a small clip out of those datasets, I often have to run several FME workbenches to get to the desired end result, a very time-intensive process.
A spatial database is obviously a good way to store this data. However, wanting a low-cost solution, I decided against Oracle. They do have a free express edition, but there is a limit to how much data you can store in it, and you would still need a more expensive FME license to get the best result out of it. Also, not being very familiar with Oracle myself, I decided to go with an open-source solution. The MySQL database has had a spatial option since version 4.1 and it is supported (both reading and writing) by FME Professional. Admitted, it is not supported by MAPublisher and support in Manifold is tricky, but with the help of some FME workbenches that shouldn't be a problem at all.
I started out by installing the XAMPP package from apachefriends.org. (to be honest, I was already using that for local testing of my website) This is a complete Apache-PHP-MySQL environment delivered in the form of a Windows installer. If you're not a Linux user, this is probabely the easiest way to get yourself a nice working MySQL database. Installation is a breeze, there is good documentation and support for it available.
(XAMPP control panel)
XAMPP comes standard with phpMyAdmin, a web-based admin module for MySQL databases. Using that, I set up a separate database, with a separate user, for my test. There's no reason why you can't do this with the pre-installed root account, but it's simply not considered to be a smart idea.
The next step was going to be loading the data. I had already pulled the most important VMAP feature types (which were going to form the bulk of my data collection) from the cd's and stored them on my harddisk as MapInfo mid/mif files. So I set up a number of FME workbenches to write all of that to my MySQL database. In order to write to MySQL, FME does need some access information:
That's all there is to it. Server name, which is the same computer, database name, username and password. After some initial tests, I did find one other setting that was going to be important:
The "Operations Per SQL Statement" (known as transaction interval in other databases) parameter is set to 1000 by default, but that caused some time-out errors so I dropped it back to 100. Also, probabely somewhere during all the previous processing, my data had lost the coordinate system settings, so even though it was in WGS84 lat/lons (LL84 in FME terms), it didn't know that anymore. A CoordinateSystemSetter quickly solved that.
After adding a few feature types, and have FME churn away at the data for a couple of hours (which was mostly reading data from the mid/mif files), I ended up with a grand total of 1.5 Gb of data stored in my database. I then fired up the FME Viewer to do a quick reading test. Input settings are similar to the output settings, but there's the option of selecting one or more tables to read and specifying a region to clip to. This is where the power of spatial indexing becomes apparent. If I were to perform a clip on say all the contour lines using the mid/mif files as source, FME would spend 10 minutes reading in the entire mid/mif file and then do the clipping. By setting up the MySQL Reader to do the clipping for me, I save a lot of time. The same operation that way is done in a matter of seconds.
So, after setting up a few more FME workbenches, I now have a system in place where I can produce a clip of a number of feature types within a minute, load that data into Manifold or MAPublisher and have a basic map done in no-time.
Part two - building a control interface