This post follows on from my previous post about using OSGeo Live 6.0 (and subsequent versions) for teaching on a module called “Geospatial Information Services”. This is delivered to the University of Nottingham’s MSc in Geographical Information Science, as well as other postgrad courses at the university. I’ve presented on this topic at FOSS4G before too.
I’ve decided to add an additional entry for my experience in 2015 as a) I think the previous post was getting a bit long; b) the teaching IT environment has changed quite a bit so I’ve had to rethink the strategy; and c) the year’s solution is much easier to redistribute so I intend to link to re-usable content here.
The change in IT environment was due to the loss of our dedicated IT cluster (for reasons of space) so this teaching has had to transfer to a centrally managed cluster. Thankfully the UoN Information Services team have been very helpful in managing this transition. However it did mean that it would no longer be possible to boot from a USB memory stick, to run the OSGeo Live system as the native OS. Essentially then, to keep using OSGeo Live as the working environment for the practicals in this module, this meant going back to running it in a virtual machine.
While I’m aware that there are solutions that can be built around using VMWare products, particularly with academic site licensing, my preferred VM solution continues to be Oracle VirtualBox as it matches the open ethos of this module, in which I teach about and use the open OGC standards, we build on the open OSGeo software stack, and we use UK Government open data for the GIS analysis and web mapping. VirtualBox is open source and thus simple for students not only to install at home on their course, but also to take away and use in any environment they go on to or back to after the course. The VirtualBox host runs on Windows, Mac, Linux and other OSs.
My first thought was to use Portable VirtualBox. This is essentially a clever way to package VirtualBox so that it can run off a USB memory stick. The Portable environment is, as far as I can tell, a cunning script environment that adjusts the launch scripts for VirtualBox to take account of the shifting drive letter/mount point assignment of the memory stick, etc. It’s a neat solution: the VM host software and the VM itself could all be on the USB stick for a completely portable solution. However this runs up against a security policy on our system preventing software from running of a stick (for obvious reasons, I guess), and making a suitable hole in this policy seemed to big a challenge to even raise with Information Services.
Thankfully the IS team were happy to install the VirtualBox host as part of a special “Geography” image for the lab I’m using for this module, to support the transition from the locally managed lab. A small detail, which I wasn’t involved in having to set up, is that VirtualBox has to be truly locally installed on the machines as part of the image as it hooks into various device drivers, whereas many of our PC cluster applications run through a virtualised environment using Microsoft App-V. In this pattern, the VirtualBox host is installed on the host PC – the plan was to have the OSGeo Live VM running from a USB stick. (If you look back at the previous post, you’ll see that in the first year I ran this I had a scheme to run VirtualBox with the source OSGeo Live VM image locally installed in the PC’s C: drive but saving diffs to a SAN. This seems rather over-engineered now!)
This has turned out to be fairly straightforward. I’m using OSGeo Live 7.9 as it was the latest version at the point where I had to prepare the VM. I copied the 7.9 image from the OSGeo Live web site. As previously I made a couple of tweaks. I installed the VirtualBox “Guest Additions” in the VM – this allows the user to copy and paste between the VM and host, to mount host file systems inside the VM, and to dynamically resize the VM “screen” size by adjusting the host window size, all useful facilities when using the VM.
Installing the Guest Additions is not simple as I’ve found that one usually has to go back a long way inside the package manager system in the VM to refresh and update its contents before being able to download the Guest Additions dependencies, before finally being able to install the Guest Additions. This is a real barrier to access, and one reason that I try to solve all this on my particular distribution of OSGeo Live for my students. This web page is one place to start if you’re faced with this task. (The OSGeo Live 7.9 VM is based on Xubuntu, a variant of Ubuntu).
This year I placed all the required data inside the VM image in a directory /home/user/data/giservices. All the data in this directory is unzipped as I’ve identified any unzipping step as one where lots of virtual disk space gets consumed (by having the source ZIP files hanging around still as well as one or more unzipped copies of the data).
A thing to avoid at all costs is the VM running out of system disk space. If the VM runs out of disk space for Xubuntu, the OS will fail and once shut down will typically refuse to boot again. It’s a fiddly job to mount/open the virtual disk from the host operating system to free up space (albeit my recent experience is with having to mount a caspar file from the bootable USB system rather than the VMDK file of the VM). Such rescue jobs are best carried out in a Linux host or possibly MacOS as they are more flexible for mounting foreign virtual disks (though possibly this is because of my experience).
The USB stick preparation is much simpler than in previous years (which involved complex formatting of the stick, and cloning using CloneZilla). This year I’ve again used 16GB, USB 3 Kingston DataTraveler drives (these particular ones, as it happens). These were very reliable last year and have been so far this year too. USB3 is well worth having as it can speed up access to the system. On reflection, I’d have been better using 32GB sticks (such as this Data Traveler), for reasons I’ll explain below. Preparing the drive is simply a matter of reformatting the stick with a single ExFAT partition – this format is writable on any OS (e.g. NTFS is read-only in standard MacOS) and can cope with files larger than 4GB (such as the VMDK virtual disks). The VirtualBox virtual disk files can then simply be copied onto the USB drive. This is a straightforward exercise, first formatting all the new USB sticks, then one by one copying the contents from my hard drive to each stick. For what it’s worth, the copying was a lot quicker on a newer system – a) with an SSD hard drive, b) with a USB3 port, and I suspect c) with a better designed internal USB bus in the laptop.
If you want to try my version of OSGeo Live 7.9, with the Guest Additions installed for VirtualBox 4.3 and with the open data package for my my module, you can get it as a 5.5GB ZIP file. Download this file, unzip it on your hard drive, and copy the contents to the root of the ExFAT formatted USB stick. Note too that you can also use the contents from a hard drive – you don’t have to copy it to a stick.
Assuming that the host PC has VirtualBox installed, the VM can most simply be added to VirtualBox by double-clicking on the “OSGeo-Live-7.9.vbox” file (this may show up just as “OSGeo-Live-7.9” in Windows, if Windows is hiding the file extensions). This should open VirtualBox with the “OSGeo-Live-7.9” virtual machine automatically added and configured.
The configuration includes two auto mounts for the host C: and Z: drives (the latter is a network drive on our PC cluster). If these are not present or somehow restricted, VirtualBox will present warnings that can be ignored. These auto mounts are to provide a means to back up data out of the OSGeo Live environment. The following command can be executed in a Terminal window in the VM to mount the host C: drive at /home/user/c-drive in the Xubuntu filesystem.
sudo mount -t vboxsf -o uid=1000,gid=1000,rw C_DRIVE /home/user/c-drive
At present this is unfortunately not reliable – writing to /home/user/c-drive usually produces a “Protocol error” indicating something is not yet right in the VM configuration (though it is possible to use “touch” in the Terminal to create files in this directory).
An alternative, more reliable means to give more storage space and to allow a means to backup, is to plug an extra USB drive into the PC, and associate this USB drive (using the VirtualBox “Devices” menu for the VM) with the VM OS rather than the host. The OSGeo Live Xubuntu system will see the external USB drive as if it was attached to the VM and will auto mount it in the Xubuntu filesystem in the /media directory.
As discussed above, disk space is a concern. The VM files fit on a 16GB USB drive but there’s a potentially fatal problem that I will have to be careful to avoid. The actual VMDK virtual disk file is a dynamically allocated virtual disk with a 20GB capacity. As supplied the VMDK uses ~12GB of disk space but this could expand to 20GB if the virtual disk is filled inside the VM, and this would exceed the USB stick capacity. Unfortunately I spotted this problem after buying the sticks! The best solution is to buy 32GB sticks – at present the VMDK capacity can’t be simply reduced to less than 20GB. Theoretically one could alternatively create a new VMDK with a smaller capacity and use a tool such as “dd” inside a VM to copy the contents from the existing virtual disk to the smaller virtual disk. In practice I’m being careful with the practical exercises to avoid the virtual disk’s allocated storage growing so much that the VMDK file reaches the USB stick capacity.