OSGeo Live 6.0 for teaching – a saga

4 02 2013

I don’t often blog (to say the least) but I thought I’d write up a little saga that I’m actually still in the middle of (but I think is sorted). I’m going to start just by writing it all down while I remember – I’ll hopefully come back to put links in, and later to write this up more formally (a FOSS4G paper, maybe!).

Last year I ran a new module on our MSc GIScience for the first time call Geospatial Information Services (GIServices from here on). The aim of the module is to introduce students to OGC web services, interoperability, Google mashups, etc. and the new, Web-based ways of “doing GIS”. As part of this, the practicals are to populate a spatial database (PostGIS), connect an OWS server (Geoserver) to create WMS and WFS services and then connect desktop (QGIS) and web (OpenLayers) clients to the services. As you can see, all done with an open source, OSGeo stack. The demonstration data was all gathered from data.gov.uk and so is Open Data too – a fully redistributable practical set.

To support this, I need each student in the class to have access to a machine that’s set up as a web server with PostGIS & Geoserver, and a way of testing clients. Initially, last year, I decided that a neat way to do this would be to create an Oracle VirtualBox virtual machine (VirtualBox is also open source, and pretty solid) that each student could have a copy of. I managed to create this is such a way that each lab machine had the original source VM image which was not updated – the VM differences were written to the student’s directory on the School of Geography’s SAN. In theory then they should be able to switch machines and still pick up where they left off. The VM I used was the OSGeo Live 5.0 system which is fantastic as it comes already configured with the services I needed (and a lot more).

This was only partly successful. Firstly, it work ok with one person in the lab who sticks to the same machine. It doesn’t scale very well with multiple students (network bandwidth to the SAN – I should have seen that coming). There’s also another subtle problem that each VM source image on each different machine ended up with a different UUID (because of how I installed it, I presume) so swapping machines didn’t work as VirtualBox didn’t recognise the source images as the same.

An issue that I also didn’t solve was the network access in the VM. I wanted each VM to get its own IP address so the physical host machine could be used as a client to the VM’s web server. However getting the VM to acquire an address from the university’s locked down DHCP services was a battle too far, and we stuck to localhost testing of the VM services – a little disappointing but not the end of the world. (I’ve had some success with this since, ask me if you’re interested).

The UUID problem I could probably have fixed but the network to the SAN wasn’t easily fixed and I felt that this was not reliable enough. So half way through last year’s course we swapped to using bootable USB memory sticks. We bought a stack of Kingston DataTraveller 100 G2 16GB memory sticks for the purpose (lots of room for the data – you can squeeze OSGeo Live 5.0 onto a 4GB stick).

So, how to set up the memory sticks? Well thankfully there were instructions as to how to create an OSGeo Live 5.0 bootable USB stick. I had some problems making this work at first (which are now irrelevant so I won’t go into here). Eventually I achieved this. Slightly annoyingly the “persistence file” that allows the Live Xubuntu linux on which OSGeo Live is built to save data is capped at 4GB because of FAT32 file limits on the USB stick, so a lot of the 16GB of the stick was left unusable from OSGeo Live. However this was enough for the practicals (just – as long as download ZIP files were deleted as the students went on with the work). I also had to set the university’s proxy settings in the running OSGeo Live system (unfortunately this is a bit of a hassle in Xubuntu (as opposed to plain Ubuntu) as it involves editing linux config files), and I copied some of the data from the previous steps in the VM into the memory stick system to give the students a leg-up towards where they had already got to in class.

At the end of this I had a “master” USB drive prepared for the class. Then it was a matter of cloning this to the rest of the drives. I tried “Clonezilla” but settled on another package, OSFClone to do the job. It could do direct drive-to-drive USB cloning, preserving the bootability of the target drive. I spent a day cloning USB drives in the background to other work.

And it all worked! There was the odd problem in class when students filled the persistence files by not deleting ZIP files but overall it was pretty good – the OSGeo stack all worked well. What suffered however was really student confidence (not marks, interestingly – about the usual histogram for such a course). There was too much technology in the way of the lessons, between setting up the VM just right and then switching to the USBs. And I had a lot of work mid-semester to construct the USBs – quite a number of late nights!

This year…

The plan for GIServices this year is to repeat the practical content but sticking with the USB sticks from the start. Last year the USB sticks were given to the students in exchange for a deposit for roughly the value of the stick (10 pounds!). The students had the choice of returning the stick & getting the money back, or keeping the stick and forfeiting the money. In the end no-one tracked me down to get their money back. I see this as hopefully a good thing: the students go away with a full, bootable “GIS in a box” with example data too.

This year therefore we’ve bought a stack more DataTraveller 100 G2 sticks. Same stick, same process, n’est ce pas? Non.

It seems that for some reason this year’s batch of sticks are not all of exactly the same capacity (possibly I should have complained but I’m out of time for that). The variation is a fraction of a GB (though I remember when 100MB was a lot of disk space!) but it’s enough that drive-to-drive cloning won’t reliably work as sometimes the target is smaller than the source!


I also wanted to recover the “missing” space of the USB stick to be useable in the OSGeo Live system, on top of the persistence file.

As a result of all this I’ve created a new “master” USB stick this year. And since I’m doing that I’ve upgraded to OSGeo Live 6.0.

After some experimentation, the partition map for the USB sticks using an MBR / MSDOS boot sector, it has an ~9GB primary FAT32 partition (for the OSGeo Live system + 4GB persistence file which contains an ext2 file system), blocked at the start of the drive. It has a ~5GB extended partition containing a FAT32 logical partition, blocked at the end of the stick’s drive map. This leaves a small unallocated space between the primary and extended partitions that can account for the varying stick capacities.

Here’s the partition map in gparted (I have to say, I’m not an expert at partitioning and copied the partition flags from a working partitioned disk – I’m not sure if I need ‘lba’ on the first partition or elsewhere. parted will warn about poor alignment of partitions when you create them, and in this case I get no warnings. I used parted and not gparted to create the partitions as it could be scripted and gave better feedback on the choices I was making. I check what’s been created in gparted):


The OSGeo Live 6.0 system is then installed in the first, 9GB partition according to the updated instructions for this version of the Live system (in this case, I used OSGeo Live 6.0 burnt to a DVD-ROM to do the installation)

In the OSGeo Live 6.0 system, I’ve made three adjustments on this occasion (by booting the master USB stick and making changes before cloning). I’ve copied in some source data; I’ve set up the proxies, and I automount the 5GB logical partition under “/giservices” to make it automatically accessible from OSGeo Live. Another advantage of the 5GB partition is that it can be simply accessed both in OSGeo Live and when the stick is accessed from a Windows machine. (The persistence file’s ext2 system is not simply accessible from Windows). This means that results and data can be transmitted simply from OSGeo Live to Windows (and back).

So, that’t the “master” drive. Now I need to clone this drive to all the others, handling the difference in stick capacities. Well for this I’m back to using a two step process. I’ve used Clonezilla to first take images of the two FAT32 partitions (the primary and logical partitions), and stored these on the internal hard drive of the PC. To create a clone, I boot into an OSGeo Live system (could be any Ubuntu-derived live system), and used “parted” to set up the same partition structure with empty FAT32 file systems as on the master stick (the unallocated space will vary in size with the target stick’s capacity). I then use Clonezilla to restore the partition images to the target stick. This overwrites the empty FAT32 partitions and in fact restores the UUIDs of the original partitions too (handy for that automount). It’s a little slow – about 30 minutes per stick. It also makes sense to create the partition maps for all the sticks first, then boot into Clonezilla and do all the restoring.

At the end of it though, I do have a stack of USB drives with OSGeo Live 6.0, with the extra 5GB partition automatically mounted at boot. For some reason the OSGeo Live 6.0 boot seems to be a lot slower than for 5.0 (several minutes, versus about 1 minute) but we can live with that – it seems to be fine when it’s running.

I’ll add an update when the class has been using them, and when the bugs have crawled out of the woodwork. Now to rewrite the practical documents!…

PS: If anyone wants more details (e.g. of a little script to feed into parted to automate the USB drive partitioning), let me know.

UPDATE (5 Feb)

Well, there’s one small problem. The partitioning scheme doesn’t quite do what I wanted. It’s fine in the OSGeo Live system – the logical partition automounts fine. However Windows 7 won’t mount that extra partition, only the primary. It’s visible in & understood by the Win7 Disk Management tool but just won’t mount – it seems that Win7 doesn’t support any more than the first partition on a removable flash drive. MacOS 10.7 (Lion) mounts both partitions. I’ll add a note about Win XP (I expect this will be ok – XP is less fussy about partitions.)

If there’s no way round this in Windows 7 (as it seems) then it may actually be better to have a single partition, create a data directory on it and find a way to mount that directory in the OSGeo Live file system. (Normally, the physical file system on the first partition (as opposed to the persistence file’s virtual file system) is mounted read-only under /cdrom in the OSGeo Live system).

GISRUK 2010 abstract submissions

7 12 2009

Well we’ve just pushed past 100 abstracts submitted for the next GISRUK conference at UCL next April. We defined several themes in line with the interests of UCL, the London location for the conference and the “Global Challenges” overall theme. This is how the abstracts have split out:

  • Crime and Place (7 submissions)
  • Environmental Change (6 submissions)
  • Geodemographics and population (8 submissions)
  • Human-Computer Interaction, Usability and Geovisualisation (8 submissions)
  • Intelligent Transport (6 submissions)
  • London as a global city (4 submissions)
  • Migration and Identity (1 submissions)
  • Open GIS and Volunteered Geographic Information (9 submissions)
  • The geoweb and neo-geography (13 submissions)
  • Public Health and Epidemiology (8 submissions)
  • Simulation and Modelling (27 submissions)
  • Other (5 submissions)

We’ll actually close the submission tomorrow (as I write), 7th December, so if you’ve a paper ready to go, there’s still the chance to get in there! It seems that webGIS in its various forms is a popular topic still.

GISRUK 2010 – paper deadline approaching

3 11 2009

UCL is hosting this year’s GIS Research UK (GISRUK) conference, mostly because I landed them with it. As a result I’m co-chairing this with Muki Haklay at UCL. We hope to attract the regular GISRUK crowd but also to bring in a slightly wider audience of people whose disciplines use or connect to GIS. The paper deadline is approaching at the end of this month, so now’s the time to get writing.

GISRUK is an annual, academic conference series that has been running since 1993. It’s a relatively informal conference and aims to be a good place for PhD students and others to make their first presentations in academic GIS. GISRUK traditionally just takes extended abstracts – the best contributions are invited to go on to be written up as full papers or book chapters. While it focuses on the UK academic GIS community, we usually have people from around Europe and further afield attending. Nor are we exclusive to academics – abstracts are judged on their merits and anyone is free to attend.

Our overarching theme this year be “Global Challenges”. As is usual with GISRUK we welcome papers across the range of contemporary GIS research but we will particularly welcome papers in the following themes:
  • Crime and Place
  • Environmental Change
  • Migration and Identity
  • Intelligent Transport
  • Public Health and Epidemiology
  • Simulation and Modelling
  • London as a global city
  • The geoweb and neo-geography
  • Open GIS and Volunteered Geographic Information
More details of the call for papers, abstract format, and the submission URL can all be found on the conference website: http://gisruk2010.spatial-literacy.org/.  The closing date for abstracts is Friday 27th November 2009.
We also have a Twitter id, GISRUK2010 you can follow for updates.

Alton Towers – the best theme park in the UK?

2 10 2009

A couple of weekends ago I visited the Alton Towers theme park which is just 50 miles from our new home in Nottingham. Probably for that same reason the University of Nottingham have a number of links with Alton Towers too, not least through our new Digital Economies hub and doctoral training centre (DTC), so there was some professional interest in an otherwise family visit. My perception of Alton Towers has always been that it’s the premier theme park in the UK and it’s this I want to reflect on in this post.

My first visit to Alton Towers must have been in the mid-90s as a single male in his 20s. Thrill rides were the thing. I went back with friends after the opening of rides like Nemesis and Oblivion but then haven’t been for a while. For context, I’ve been to Disney parks in the US, Japan and France and Universal Studios and Busch Gardens in Florida. Again, mostly for the thrill rides. In the UK I went to Chessington and Thorpe Park in the 90s too.

Of course now my circumstances are different, with a wife and young family (both kids under 5). We recently went to Legoland near Windsor for a family day out (tip: spending Tesco Clubcard vouchers on this is cost effective!). After a great day out we figured that we could probably get good value from a Merlin annual pass, which gets you into Thorpe Park, Chessington, Alton Towers, the London Eye, and various other attractions. As a result, in the last few months I’ve been to all of the latter. We also went to Disney near Paris in 2008.

An interesting side note – though Merlin are the park operators they no longer own Alton Towers. The park was part of a leaseback scheme a couple of years ago. The Alton Towers park is an interesting place itself, of course, with a long history of decline leading eventually to its opening as pleasure gardens and then its eventual evolution into a theme park. It’s been through a number of owners in its guise as a theme park. (There’s a potted history on Wikipedia). One thing in particular that separates Alton Towers from many other theme parks (e.g. Chessington) is that there is lots of space, both for expansion and between the ride areas.

So, what of the park nowadays? Well the thrill rides are still there and there are new ones since I was last there (Spinball Whizzer, Air & Rita in particular). Also, like many places Alton Towers runs a parent pass scheme . You get a card listing each ride; one parent queues for a ride and gets the card stamped just before going on the ride; then the other parent can jump to the front of the queue (often in front of the fast pass line) and ride quickly. You of course have to have the kids with you to get the card. This scheme works doubly well if you buy one fast pass ticket so the first parent uses a fast pass to skip a lot of the main queue. So my wife and I were happy because we could get round the thrill rides. (I just wish that I hadn’t built up so much of a tolerance for these rides – the adrenalin and anticipation of the rides just isn’t there so much any more, though being on the rides is still fun.)

So far, so good. I think there are two issues though – one relating to having the kids along, and the other relating to the zoning and park experience.

Of course taking kids to a theme park should be a great day out for everyone (especially with the parent pass, etc.). Our experience of Alton Towers was that the kids rides are perhaps too concentrated in a couple of zones. Thorpe Park, for example, seems to have kids rides much closer to the thrill rides so everyone can be happy. Because of the larger size of the Alton Towers park, it’s actually quite a trek (especially with short under-5 legs) between the zones. Still, the rides themselves are fun (though the Charlie and The Chocolate Factory ride doesn’t really work in my opinion – it’s trying to be too macabre, doesn’t really fill the space, the animatronics and pretty poor and it doesn’t really convey the narrative of the story).

And then there’s the zones of the park. This needs some serious rethinking (tricky of course, given the rides are fixed!). The zone with Oblivion (X-Sector) feels like a half abandoned corner, Ug Land makes no sense (the Rita roller coaster is based on a drag racing theme or something and just doesn’t fit the prehistoric theme), and Storybook Land has almost nothing in it. The main entrance way, Towers Street, is a sad reflection of the main streets of places like Disney with mostly closed building hoardings rather than exciting retail outlets. All in all, I feel that Alton Towers has been resting on its laurels, relying on the thrill rides to bring people in, but in my view these parks are about the whole experience, including its weird internal logic, and not just the rides. My suspicion is that the period of ownership by Dubai International Capital, part of the Dubai sovereign wealth fund, is when the vision was lost but this is only because I suspect an investment business probably has less specific interest in the theme park business. This lack of focus on the park as a whole is also reflected in the gardens areas. The plants are reasonably well maintained and the fish thrive in the ponds but architectural features, such as the Gothic Prospect Tower seem to have been allowed to decay. Couldn’t some of the income from the rest of the park keep some of this heritage alive? Similarly, down in the gardens areas there are buildings, including what looks like an old tea room. OK, I’m middle aged and a father now (how did that happen?) – I could have killed for a decent cup of tea and some cake at a quiet spot in the gardens. It seems like a missed opportunity in the family market.

Best ride? Probably Spinball Whizzer – not the most intense ride but having the car spin round so you’re facing different directions through the ride was an exciting addition, adding a new dynamic I hadn’t experienced before.

And the ‘geo’ aspect?  Well the park’s crying out for better mapping: tailored mapping and interactive mapping are possibilities, but even the current all-in-one map could be greatly improved to help route finding through the park. Don’t put labels over the junctions! And how about virtual games in the park areas too, a form of location based activity?


24 09 2009

The AGI Soapbox has happened. Now of course I’m biased, but it seemed to go off pretty well. A room full of folks (and their ononmies), armed with geobeers and 10 presentations from souls brave or foolhardy enough to risk this format. I think it was the right decision to withhold the titles and let it unfold on the evening. We had a variety:

  1. Steven Ramage, 1Spatial – “THE LANGUAGE OF BUSINESS” – mostly serious
  2. Addy Pope, EDINA – “Go-Geo! – a geo-information discovery tool and GeoDoc – a metadata creation and management tool.  What can they do for you?” – title length probably says why this didn’t work in this format
  3. Gary Gale, Yahoo! – ““Neo this” and “paleo that”, it’s all just “Geo” – worked overall, amusing but maybe not quite enough to say on each slide?
  4. Simon Lewis, MapJuice; John Fagan, Microsoft – “15 “geoweb” innovations since AGI Geocommunity 08” – competent but probably not memorable (ask me in a month’s time for the 15 innovations!)
  5. Ian Painter, Snowflaks – “Behind every great Neogeographer is a Paleotard” – I suspect audience vote for nailing it with a funny presentation with a well delivered payload. Certainly got my vote.
  6. Andrew Larcombe, Net Dojo – “Serious (geo) play, or ‘why we need to be open to innovate'” – message fine but not punchy enough in the delivery
  7. Chris Parker, Ordnance Survey – “Geovation” – presentation was fine but peculiar scheme…
  8. Chris Osborne, ITO World – Ito! – bombed. Pitch, not enough to say on each slide. 20 secs can be a long time…
  9. Mark Bishop, Chris McCartney, Tom Probert,  PBBI – geogags! Nah.
  10. Peter Batty, Spatial Networking – Queen Vanessa! And a hat.

This is a more difficult format that it first appears. 20 secs can be longer than some appreciated. And if you don’t go with a message, you need good gags.

We had a Geobingo winner, Andrew Newman of Natural England – congratulations! Personally I was just relieved that one of the cards came through…

Thanks go to: Hayley Merrill from 1Spatial for keeping the Geobingo scorecard, getting the cards out there, giving the prize, etc.; Steven Ramage for the Geobingo idea; Nick Summers for setting up the Twitterfall (Swisscom fail, though in extremis) & videoing; Chris Holcroft for loan of the PA kit; and the speakers for sticking their necks out on this.

I hope I’ll be the one to be back with this next year! And so to geobed, before chairing the not-the-keynotes-and-not-the-geoweb SDI stream first thing tomorrow. See you at 9:45.


AGI Soapbox ready to go

22 09 2009

Everything’s as ready as it can be for the Soapbox event at the AGI GeoCommunity conference tomorrow. I have 9 presentations loaded up from the following speakers:

  1. Steven Ramage, 1Spatial
  2. Addy Pope, EDINA
  3. Gary Gale, Yahoo!
  4. Simon Lewis, MapJuice; John Fagan, Microsoft
  5. Andrew Larcombe, Net Dojo
  6. Ian Holt, Chris Parker, Ordnance Survey
  7. Chris Osborne, ITO World
  8. Mark Bishop, Chris McCartney, Tom Probert,  PBBI
  9. Peter Batty, Spatial Networking

and we’ve filled the last-minute slot too – Ian Painter and Eddie Curtis from Snowflake will also present. Each presentation will last 5 minutes, comprising a fixed structure of 15 slides, 20 seconds a piece. The presentations are quite a mix: presentations on business communication, product pitches, barbed commentaries on the geo scene, and finally straightforward geo joke telling.

I suspect that the fixed format where presenters have no control over their slides once the presentation starts will prove quite a difficult style to master. 20 seconds a slide can be a long time or a really short time if you misjudge your timings… Who will fly, who will die? We’ll see tomorrow evening from 5:30 in the bar before the AGI party!

To add some additional audience interest we will also be running a game of bingo (Geobingo, in fact, brought to you from an original idea by 1Spatial) – geojargon spotting made fun, with a prize for the winner.

If you’re at the conference, you can give live feedback to the speakers through Twitter: we’ll have a Twitterfall of comments with the #geocom tag running during the Soapbox event. If you’re not at the conference (shame on you!😉 you can get a feel for the event by following us on Twitter and we’ll try to upload at least the best presentations to YouTube.

#geocom – ready to go…

22 09 2009

Lunchtime on the day before AGI Geocommunity kicks off… We’ve spent the morning setting up (mostly stuffing conference bags and working out how to populate the GeoCommmunity Live blog). Pics here. Meanwhile Pitney Bowes and Oracle have been running user group sessions, and this afternoon there’s the chance to try geocaching or OpenStreet Mapping. Things seem to be very well sorted, not least due to Claire Huppertz’ efforts at the AGI in pulling together the programme, sponsors, hotel & catering, oh, and everything else.