Spatial Data Analysis in R: Lightning Demo!

At this years NEARC meeting I decided to give a lightning talk on using R as a GIS. As I was working on this I thought, “why not try a lightning demo?” That would be better than five minutes of slides on packages and commands. But, as anyone who has done a live demo will know, they often provide unexpected challenges. Add a 5 minute limit to that, and well, some level of failure is sure to occur. Becuase of this I have decided put everything into a web page so that the attendees (and others) can access the full demo at a later date. The full text and code is at

I finally got quickmapr on CRAN!

A little over 7 months ago I posted about a package that I had been working on, quickmapr. That was the pre-release version, the one that I finished up today and submitted to CRAN is a bit more polished, plots rasters a bit quicker, and is, I think, ready for wider release. It is now availble from CRAN.

The README on GitHub provides details plus some examples using a small dataset included with the package. I would be thrilled to get some feedback from people on the package, ease of use, suggestions for improvements, etc. I would be even more thrilled if you try it out on different datasets. Any thoughts just add them as issues.

Download Shapefiles - Take 2

So back in 2013 I posted a little function I wrote for grabbing all the relevant files that make up a shapefile from a URL. Turns out it doesn’t play so well with Windows 7 or Windows 8 (HT: John Lewis). Below is a reprised version that at least works on Ubuntu 14.04 and Windows 7. Haven’t tested it beyond that and supressing the warnings to get httr::GET to not complain too much about FTP seems a bit unclean. Well, you get what you pay for.

For all this to run you’ll need RCurl, httr, sp, and rgdal.

download_shp<-function (shape_url, layer, outfolder = ".") 
  if (length(grep("/$", shape_url)) == 0) {
    shape_url <- paste(shape_url, "/", sep = "")
  shapefile_ext <- c(".shp", ".shx", ".dbf", ".prj", ".sbn", 
                     ".sbx", ".shp.xml", ".fbn", ".fbx", ".ain", ".aih", ".ixs", 
                     ".mxs", ".atx", ".cpg")

  xlogic <- NULL
    xurl <- RCurl::getURL(shape_url)
    for (i in paste(layer, shapefile_ext, sep = "")) {
      xlogic <- c(xlogic, grepl(i, xurl))
  } else if(substr(shape_url,1,4)=="http"){
    for (i in paste(shape_url,layer, shapefile_ext, sep = "")) {
      xlogic <- c(xlogic,httr::HEAD(i)$status==200)
  shapefiles <- paste(shape_url, layer, shapefile_ext, 
                      sep = "")[xlogic]
  outfiles <- paste(outfolder, "/", layer, shapefile_ext, 
                    sep = "")[xlogic]
  if (sum(xlogic) > 0) {
    for (i in 1:length(shapefiles)) {
      x <- suppressWarnings(httr::GET(shapefiles[i], 
                                                       overwrite = TRUE)))
      dwnld_file <- strsplit(shapefiles[i], "/")[[1]]
      dwnld_file <- dwnld_file[length(dwnld_file)]
      print(paste0("Downloaded ", dwnld_file, " to ", 
                   outfiles[i], "."))
  else {
    stop("An Error has occured with the input URL or 
              name of shapefile")

And to see that it works again:

#Download the NH State Boundaries
## [1] "Downloaded NHSenateDists2012.shp to ./NHSenateDists2012.shp."
## [1] "Downloaded NHSenateDists2012.shx to ./NHSenateDists2012.shx."
## [1] "Downloaded NHSenateDists2012.dbf to ./NHSenateDists2012.dbf."
## [1] "Downloaded NHSenateDists2012.prj to ./NHSenateDists2012.prj."
## [1] "Downloaded NHSenateDists2012.sbn to ./NHSenateDists2012.sbn."
## [1] "Downloaded NHSenateDists2012.sbx to ./NHSenateDists2012.sbx."
#Read shapefiles in SpatialPolygonsDataFrame
## OGR data source with driver: ESRI Shapefile 
## Source: ".", layer: "NHSenateDists2012"
## with 24 features
#Plot it

plot of chunk run_it

quickmapr: An R package for mapping and interacting with spatial data

I do a lot of GIS. That used to mean firing up any one of the esri products, but over the course of the last couple of years I have done that less and less and instead fired up R.

When I first started using R for my spatial analysis work I often was left struggling with viewing the results of my analysis and could only do so with a clunky workflow of pushing my sp or raster objects out to shapefiles or tiffs and then pulling those into Arcmap. In short, spatial data visualization was severely lacking in R.

Fast forward to now, and that has all really started to change. Most of the work in this space has been on incorporating the slew of javascript tools (e.g. D3, leaflet, Crosslet) for visualizing spatial data. This has resulted in some really cool packages like:

These all result in great looking maps with nice interactivity; however, they all have two things in common. One, it is expected that your data are unprojected (i.e. Longitude and Latitude) and two that the data are simple text or in JSON (either GeoJSON or TopoJSON). This works for many use cases, but not for mine.

I usually start with small(ish) spatial data that are stored in GIS formats (e.g. shapefiles, esri rasters, file geodabase, etc.) and are projected. I use rgdal or raster to pull those into R and then do whatever it is I am doing to those and get sp and raster objects as output. At this point all I want to be able to do is quickly visualize the resultant data (usually less than 3 or 4 layers at a time), interact with that data by zooming, panning, and identifying values in the data interactively. I want to be able to this without having to convert to JSON or without having to un-project the data. The result of this desire is quickmapr.

With quickmapr you set up a qmap object by passing as many sp and raster objects as you’d like. There are some very basic controls on draw order and color. There are several zoom functions, a pan function, an identify function (which also returns the selected sp object or raster value), and a (currently very clunky) labeling function. This package is still a work in progress and I am hoping to keep working on quickmapr and tweaking how it works. I would love feedback so if you have thoughts, comments, complaints, etc don’t hesitate to leave some comments here, or better yet post issues on github, or fork the repo and make changes yourself. I will try and get up some contributing guidelines in the not too distant future.

Automatically Organizing Photographs on Google Drive: Why isn't this already a thing?

Over the last year or so, my wife and I have been moving most our files onto Google Drive. This has worked great as we have access across all of our devices. The last vestiges of local storage have been our photos. With the recent purchase of a Chromebook (which I’ve been impressed with but that is a different post…) and the 1TB of Google Drive storage that came with it, I decided to take the plunge and get our photos up their too.

Now, I thought that getting the photos moved and organized would be a snap because there certainly had to be a solution for keeping photos on Drive and organized. There were many options but none of them perfect. Many of them relied on Google+ Photos (I am not a fan of Google+), struggled with organizing large numbers of photos (nearly 20k), or were not able to be automated. It quickly became clear that I had to more less roll my own.

The solution I settled on in the end was to use a simple bash script set up as a cron job on an Ubuntu box that called grive and exiftool.

Update 3/13/2015: I was going to have more in this post, but didn’t get around to it. In the meantime, grive started giving me problems and wasn’t syning correctly. I think I have a solution that uses google-drive-oacmlfuse to just mount google drive directly on the linux box. I need to update my bash scripts and once I get that done, I’ll post those scripts here.