Simple Example of Mapmaking with GIS

For a while, I have had it in mind to produce a physical map of stores and resources one might wish to go to before or during working on a project at Dallas Makerspace.  The map would consist of places one would go to procure raw materials, consumable supplies, or tools to finish a project in electronics, arts, woodworking, scientific endeavors, or you name it.  After spending a while scouring message board threads for local stores and resources previously mentioned by members, I called out for suggestions for any additional ones beyond those I scouted out.  After this, I sought out each place's address, geocoded it, and then visited each point using Web tools in order to test the accuracy of the geocoded list.

Here is a diary of my trials and tribulations throughout this journey.  Not having used GIS software before, you should know that many people have spent years building in all sorts of intricacies for dealing with many situations, from different datums one can select to orient a map all the way to accounting for atmospheric conditions when using aerial or satellite imagery as map layers.  As such, mentions of as much riffraff and unnecessary steps and settings as possible will be kept to a minimum, but you would need to learn these advanced settings in order to really build a map from scratch using all your own measurements and imagery.

Starting with QGIS on Linux

At first, QGIS seemed like a natural place to start.  I could use it on my Linux box with a huge 4K monitor, and it was easy to install from the Ubuntu Software Center.  There is a nice plugin for it called OpenLayers that really makes it easy to add nice raster map imagery from various OpenStreetMap sources.  However, I ran into 3 problems with QGIS version 2.10 "Pisa" on Linux:

1. Won't download all map imagery at once.

If the map is big and/or to be printed at a high DPI, this of course requires highly detailed map imagery which can take a long time to download.  Unfortunately, QGIS does not wait for all of the map tiles to finish downloading before it begins to render, so you will see only a rendered circle (imagine a Japanese flag where the red area is now a map) if you haven't waited long enough.

Here, I tried to show the same area with four different map servers.  You can see how fast (or not) the various servers responded to my request.

An example of a single map tile, © OpenStreetMap contributors.  Each map tile is designed to be 256*256 pixels.  All of the tiles containing imagery near this tile could be used to create a large map of the United Kingdom.

2. After upgrading from 2.08.0 to 2.10, it started giving me weird Python errors.  (Turns out these errors weren't really a big deal, at least for my use case.)

3. The print composer orients the points differently than the image renderer -- the image rendering is inaccurate and useless, especially for map insets.

Can GRASS GIS do any better?

To work around the problems faced in Linux, I installed GRASS GIS for Windows.  It makes you set up some things in advance, whereas QGIS lets you start going to town right away.  The interface can be a bit confusing and intimidating at first, but once you realize that most windows need to be expanded for you to see everything, and that there is more than just one window, it becomes easy to navigate.

I wanted to import my CSV file until I realized the Address field had commas in it too.  This was throwing off the import wizard.  I re-exported the CSV file as a tab-delimited file and then at least that problem was fixed.  However, GRASS still gave me a fair share of problems:

1. Relatively obscure error messages that don't exactly tell you why things fail.

2. Column names as you set up your database can't have spaces in them, or if they can, then you need to possibly surround them with single quotes or escaped double quotes in your command.

3. GRASS GIS kept emptying out the contents of the points file at some particular step.  I had to leave the file open and make sure to Save it each time the text editor told me the contents had changed.

4. I'm not sure if it likes file names with spaces in them.  Either it wasn't reading the file because it wasn't putting quotes around the "in=filename.txt" part of the command string, or it was trying to read from an empty file.  (I thought the whole reason of hitting the "Load" button was so the program would actually parse the points data from the Preview textbox or from allocated memory rather than having to reread it again once you hit "Run".)

5. Given all the frustrating failures I was having with importing my points file, I actually tried Command Line mode for a while.  Naturally, the instructions on this page didn't work for me because, of course, they'd released a whole 'nother major revision of GRASS GIS since I'd installed it, but it gave me a good starting point nonetheless.  After successfully importing my data, I tried to switch back to GRASS GUI but could not see the data I'd just imported!!! Why not?!?  I ended up having to re-import it through the GUI, carefully making sure it reconstructed the exact query I needed in order to get it to import correctly.

6. GRASS GIS can't handle special characters when parsing database data because their Python script hasn't been set up to handle ASCII characters above 127.

Obviously, as a new user myself, there are probably questions and blocking issues in here that someone more familiar with the program could address.  Maybe I just need to ditch the old version and try the newer one for a better user experience.  Nevertheless, with all these headaches, I finally gave up and embarked on the last frontier: Mac OSX.

Something that actually worked

There is, fortunately, a QGIS version built for Mac OSX.  It does require the manual installation of some dependencies, but it comes out just like the Linux version.  I quickly set up my desired map style and layers, then built the print composer to specify where to put the map, scale, titles, grid marks, and other text on the final rendering.  I checked to make sure the points came out the same between the map view and the Print Composer view, and sure enough, they came out OK.  It was time to hold my breath, not try anything different or unusual, and render the map.

For these specific steps, assuming you have QGIS for Mac and OpenLayers already installed:

1. Add your Base Map layer.  In the menu, go to Web -> OpenLayers plugin and then select your map provider and map style.  Use your mouse to position the map in the window as desired.
2. Add your data layer(s).  Go to Layer -> Add Layer -> Add Delimited Text Layer... (assuming you have a CSV-formatted list of points), then follow the prompts to guide it to your attribute names, which columns are latitude & longitude, and other such settings as desired.
3. Specify the Coordinate Reference System (CRS) of choice if it hasn't prompted you to do so yet.  Check with your coordinates provider to see which CRS/datum they base their coordinates off of.  I typically use one of the World Geodetic System 1984 (WGS84) sets.  To specify this, go into Projects -> Project Properties -> Coordinate reference systems of the world, and make sure QGIS confirms your selection as the one to use in your project.
4. Fine-tune the symbols used for your placemarks.  Notice in the bottom left-hand portion of the QGIS window, in the Layers panel, where you should now have at least two layers: one for your base map, and at least one per layer you added.  Right-clicking on one of your data layers and hitting Properties.  You can select several different ways to assign colors to your placemarks in the Style tab, from the dropdown list box on the top left.  Graduated is good for data on a continuous numeric scale.  Categorized is good for points associated with non-numeric categories, such as the type of store it is.  After you choose what "Column" contains the data on which you wish to index, use this panel to select the color, point type, and point size to associate with each category you specify.
5. Ideally, your very simple map is now positioned within the window just as you imagined it.  Now, it's time to export it into an image file for uploading or printing.  Go to Project -> New Print Composer and enter a name for your new print composer.  All this represents is one specific arrangement for which you wish to export the map.  Imagine if you want to make a version of a map to hang in a police station, a version to use in the car (if you're mobile-app averse :-P), and a version to hand out to citizens on a pamphlet; you could set up several print composers in your project to format it just perfectly for the different media you're using for each purpose.
6. Set the media size.  On the right, there is a spot where you can pick between three different panels: "Composition", "Item properties", and "Atlas generation".  Choose "Composition", then specify your paper size, resolution in DPI, and orientation.
7. Take time to learn the toolbars in the Composer Editor.  There are very few words in this view, so the icons will really help you out.  The most important one is the "Add new map" icon.  Click this icon and drag along the area which you wish to add the new map.
8. Set up your map.  Notice the "Item properties" panel on the right side.  Hopefully you have your map object selected in the print composer.  If so, you should be able to click "Set to map canvas extent" in the "Item properties" pane so it shows exactly what you intend it to.  Chances are the aspect ratio of your map canvas (the original window you started working with) is not exactly the same as the media you chose for the Print Composer, so you may need to scroll up a bit and adjust the Scale (zoom level) of the map.  You can also tweak the map data placement by adjusting the map in the map canvas and clicking "Set to map canvas extent" again.
9. Add other features, such as text labels with the map's name, your name, attribution information, scale bars, grids, shapes, legends, or whatever makes you happy.
10. Export your image.  In the menu bar, go to Composer -> Export as Image... and choose the name of your output file.  Take careful note of the output; just because it finishes (i.e. unfreezes itself) does not mean it will render what you expect.

Unfortunately, for several minutes after opening the print composer, it renders just a filled circle of map imagery, not the whole map, similar to the Linux version above.

I continued attempting to render the map, and each time, the print composer took a little longer to finish rendering, but at least the rendered circle got a little bigger too.  Eventually, after about the 13th attempt, the whole map was filled in.  Great!  Now it's time to make the one inset I need to put in to show a far-away town.

Oh crap, now it needs to re-download the imagery for both maps once again... and eventually, only certain random tiles from the original map are showing.  I restarted QGIS, then took off the original map so that only the inset is supposed to be rendered.  (To "take off" items in this context means I excluded them from the render without necessarily deleting them from the print composer.  To do this, click on the desired item, go to the "Item properties" view, and scroll down to the "Rendering" options.  Simply check the box for "Exclude item from exports".)  Now when I try to render the inset, it comes out with a completely different part of the map than what it's supposed to be rendering.  Not sure what that's all about, but it's highly annoying.  I eventually give up on doing the inset, leaving it blank and saving a spot for it so I can print it and paste it on later.

Each step of the way, I'm careful to save the final output with an incremental number, and am diligent in deleting useless copies I won't be able to send to the print shop.  After the fiasco with the inset map, I simply leave a rectangle for where it should go, and then re-enable rendering of the original map.  Now it has to spend a bunch of time re-downloading map imagery before I finally get the desired output, which can now get sent to the print shop.

A Physically Interesting Final Result

I went through some A/B testing exercises with folks at the Dallas Makerspace (really, more like A/B/C/D and A/B/C testing) regarding which map style and what type of points would be preferred.  I may have forgotten to take into account their preference for map style when creating the final output (oh well, that's what Version 2.0 is for), but among the ways to present placemarks, there was a clear winner.  The choices presented were to print the placemarks directly on the map, use strong magnets as placemarks, or use standard pushpins.  User feedback was that printed placemarks would get out of date and would obsolete the map in the event that places had to be added or (especially) removed; magnets could get disturbed by accidental or malicious activities; and pushpins would leave holes where they once existed, leaving the map to look tattered after years' worth of updates.  Weighing all these alternatives, the pushpin approach came out ahead, but I still sought to keep compatibility with magnetic markers for temporary placemarks.

For the placemarks, I considered what colors tend to be evoked when one thinks of a particular type of material or supply.  Given the number of categories, and my disdain for graphs with so many lines on them that the colors start to run together and become very hard to distinguish, I also made up shapes to pair with colors to give the final result; the shapes actually represent "category groups" including "Consumables/Materials" and "Tools".  Since I am looking out for those with disabilities too, I installed Color Oracle on my Mac to test out what all these colors would look like for colorblind people.  Since it is a very common disability mostly seen in men, and since the DMS membership is overwhelmingly men, this is an especially important situation to be aware of given the population.  I spent time rotating between the three forms of color blindness one can test for: deuteranopia, protanopia, and tritanopia.  Among these, I found deuteranopia and tritanopia to be quite ugly, so if I had to choose what colorblindness to get, I would definitely pick tritanopia (the rarest form).  It would still suck to have, but at least you don't have so many ugly yellow colors.

Magnetic backings are available in several forms.  The most common is, of course, a whiteboard, but whiteboards are impenetrable by common push pins -- not to mention expensive per square foot.  Another approach recommended to me in the Dallas Makerspace Forums was to use magnetic paint.  I'd never heard of that before, and after reading some reviews, was a bit skeptical.  I obtained a scrap piece of foam core and applied three coats of magnetic paint to one side.  Here are my hints for optimal magnetic paint application:

• Stir it well, as the globs of magnetic ore tend to clump together over time, leaving you with nothing but a heavy oil.  If stirring it seems risky (because you're wearing your nice slacks and shirt), just grab one of the big globs with your stir stick and mush it down with your paint roller.
• Apply several coats.  One won't do it for you.  Even after several coats, areas where you may have spread around clumps of ore will come out stronger than where you applied just the paint by itself.
• You may have had the Paint Department at your hardware store stir it up for you, but you will need to stir it again.  Even after spending time in their machine, it was still really clumpy by the time I got it back home.

The paint is also extremely oily, so bear that in mind when you choose your brush/roller.  Nevertheless, I was very satisfied with the outcome, and I can actually hold up the whole piece of sample foam by gripping just one neodymium magnet stuck to it.

After getting the printout and proper foam core delivered from the shop, I had just a basic map with really small dots guiding where I needed to place pin markers indicating each location I chose to show and exactly what type of place it is.  It took about 1 hour to place all the pins in their right location.  Without clipping all these pins, it's impossible to put it in a conventional frame and mount it against the wall, so currently it's balanced standing up on a table against the wall.

The index of all these places took me a little while to construct, as it's the heart and soul of what makes this map actually meaningful.  The places listed are formatted into 2 columns, and organized into categories with a Table of Contents.  The next step is to add color-coded tabs along the side so people can flip directly to the desired category.

While this map only has one or two instances of chain stores listed for this entire metro region, we are planning on a Web-based version of this Makers' Markers project that will allow people to filter by location as well as who is open now hours-wise.  Nevertheless, this experience has broadened my horizons in the endeavor of producing nice static maps with more flexibility than afforded by simply Google Maps or Open Street Maps alone.