Raspberry Pi Rabbit Hutch-cam

This post describes how to setup a Raspberry Pi with a Pi camera board and a (wireless) internet connection as a webcam serving a latest image (updated every 15 seconds) and a short timelapse containing the most recent hour of images in a 20 second movie (updated once per hour). The website has basic password protection. I’ve set it up to monitor a rabbit hutch, but obviously there are other potential applications. There isn’t anything very novel here – this post serves mainly to document my setup in case I ever need to set it up again, which seems likely, as we’ve just ordered a Pi Noir camera for additional night-time monitoring…


My kids got a rabbit this summer. It lives in a hutch in the back garden, placed so that we can see into the hutch easily from the kitchen window. So, the kids can easily check on the rabbit when in the kitchen, but not when they are in the lounge watching TV, and certainly not when they are out-and-about. So my Xmas Pi project was to set up Joey-cam (the rabbit is called "Joey", because kid-reasons), so that the kids can check on the rabbit from their smartphones wherever they are….

However, the hutch has closed (opaque) compartments, so in addition to a live image, the kids also wanted to be able to look at a timelapse of recent images, to be able to quickly and easily check for any sign of movement in the last hour.

The final requirement was that Joey-cam should be accessible from anywhere over the internet, but to have some very basic password protection, so that it wouldn’t be completely public.


I’m assuming a Pi (doesn’t really matter which), with a Pi camera board attached, a clean Raspbian install, and some kind of internet connection. Mine has wifi (via a small USB wifi dongle), which is very convenient given its location. It also needs to have an SSH server enabled, so that you can log into it from another machine. I’m assuming that the reader understands how to do all this already. This post is about the camera and web server setup.

Set up

All of the code associated with this post is available from this github repo.

First log in to the Pi from another machine (eg. ssh pi@ip-address, replacing ip-address appropriately) and then download this repo with:

wget https://github.com/darrenjw/blog/archive/master.zip
unzip master.zip
cd blog-master/pi-cam/

Run the camera setup with:


The script finishes by running raspi-config. If you already have the camera enabled, just exit. If not, enable it, but don’t yet reboot – you will need to reboot soon anyway. But if you haven’t set a sensible hostname yet, it’s probably worth doing that, too.

The camera scripts can be enabled by running:

crontab -e

and add the following lines to the end of the file:

@reboot /home/pi/blog-master/pi-cam/camera-script.sh 2>&1
20 * * * * /home/pi/blog-master/pi-cam/hourly.sh 2>&1

Save and exit and check with crontab -l.

Next set up the web-site by running:


Reboot with sudo reboot and log back in again after 30 seconds. Check that photos start to appear every 15 seconds or so:

ls -l ~/timelapse/

Assuming so, try pointing a web-browser at your Pi (http://ip-address/ – replace ip-address appropriately). You should get a basic page containing the latest image, and the page should update every 20 seconds. There will also be a link to the “latest movie”, but it won’t work straight away – wait a couple of hours before trying that.

If you aren’t going to open up your Pi to the world, then you should be done.

Adding password protection

If you are intending to open up your cam to the internet, it’s probably worth adding some basic password protection. The very basic protection I describe here is probably OK for something as mundane as a rabbit hutch, but if you are monitoring anything more sensitive, then you should google how to lock down your site properly.

You can find more detailed instructions here, but the tl;dr is to

sudo nano /etc/lighttpd/lighttpd.conf

and paste the following at the end of the file:

server.modules += ( "mod_auth" )

auth.debug = 2
auth.backend = "plain"
auth.backend.plain.userfile = "/home/pi/lighttpdpwd"

auth.require = ( "/" =>
"method" => "basic",
"realm" => "Hutch-cam",
"require" => "user=joey"

Replace the required username (here, “joey”), with something appropriate, and create the file /home/pi/lighttpdpwd containing a single line in the form username:password. Then restart the server with

/etc/init.d/lighttpd restart

Test it out and find diagnostic information in /var/log/lighttpd/.

Opening up

You need to open up ports on your router to allow access to the Pi from outside your home network. This is router-specific, so you may want to just google for instructions for your router. Note that I have another public-facing web server on my home network, so I map port 81 on my router to port 80 on the Pi to have a public facing webcam on port 81 of my home network address (http://my.home.address:81, where my.home.address is replaced appropriately).


You will almost certainly want to customise the very basic web page in /var/www/index.html appropriately for your application. You can also tweak various things in camera-script.sh and hourly.sh, particularly settings such as resolutions, qualities, frame-rates, etc.


Raspberry Pi 2 cluster with NAT routing


In this post I will describe how to build a 12-node cluster of Raspberry Pi 2, how to set it up and configure it with a NAT-routing, DHCP-serving head node, and some simple ways to manage the slaves. In order to minimise typing/copy-and-pasting, and to make everything as simple and automated as possible, all of the commands, scripts and config files associated with this post are available in an associated github repo, which the reader is encouraged to browse in conjunction with reading this post. All of the scripts are very short and simple (just a few lines), and could easily be entered directly at the command line. There is no hidden magic here.


To give an idea of what I’m talking about, a picture of my finished cluster is given below. Click on it to view additional detail.

A 12-node Pi 2 cluster
A 12-node Pi 2 cluster

In the foreground are 12 Pis in off-the-shelf cases, stacked in 3 columns of 4. They are connected to a 16-port switch in the background via the regular Pi ethernet ports. They are powered by three 4-port USB chargers (one per column), which are resting on top of the switch. The head node has a USB ethernet dongle, which is in turn connected to an upstream switch, providing internet connectivity. In the picture, the head node is also connected to a display via an HDMI cable (and to a wireless keyboard), though once up and running, the display and keyboard are optional.

For this cluster you need: 12 x Raspberry Pi 2, 12 x (stackable) Pi 2 case, 12 x uSD card (64GB, Class 10), 12 x short patch cable, 12 x short USB to uUSB cable, 3 x 4-port, high power USB charger, 1 x 16 port switch (12 port would do), 1 x USB ethernet dongle, 1 x HDMI display and cable, 1 x USB keyboard, 1 x long patch cable for internet uplink, 1 x 6-way mains power adaptor/extension.


I am using vanilla Raspbian as the base OS. Using an Ubuntu laptop, I flashed the 12 uSD cards one at a time with

sudo dd bs=32M if=2015-05-05-raspbian-wheezy.img of=/dev/mmcblk0

It takes between 3 and 4 minutes per card. Before assembling the cluster, I booted up each card in turn in a Pi 2 to do some very basic initial config. On first boot “raspi-config” will run. I made 4 changes to the default settings. From the main settings I selected “Expand filesystem” and set “Overclock” to “Pi2”. From the advanced settings I changed the RAM split to “16” and enabled SSH. I then finished and rebooted to re-size the card, before logging back in (pi/raspberry) and doing a “shutdown -h now” and then powering down. Again, this took between 3 and 4 minutes per card.

After assembling the cluster, put the cards in all of the Pis, but don’t power up any of the Pis. Pick a head node and attach the ethernet dongle, the internet uplink, a keyboard and display, and then boot up just this head node. Assuming that your internet uplink will respond to DHCP requests made by the dongle, the Pi should auto-detect the fact that there is internet via the dongle (eth1) and not via the internal NIC (eth0), and configure itself appropriately. Check you have connectivity by logging in and doing a ping http://www.google.com or some such. Also change the password on this node to something secure (using passwd). Make sure you have internet connectivity on the head node before proceeding (ifconfig provides useful info).

Grab my code from github and install a few required packages with

wget https://github.com/darrenjw/blog/archive/master.zip
unzip master.zip
cd blog-master/pi-cluster
sudo sh install-packages

This could take around half an hour, and the node will reboot when it is finished. Log back in and return to the script directory to continue. Configure the network with

sudo sh setup-network

This will first configure the two network interfaces correctly, then set up a DHCP server to manage the network settings of the nodes on the internal network, and finally will set up iptables correctly for NAT routing of internal traffic from the worker nodes to the internet and back, ensuring that the LAN-side nodes can all connect to the internet properly. Again, this script will reboot when it is finished (it only takes a few seconds to run).

Once the head node comes back up, log back in, check you still have internet connectivity, and then return to the script directory.

Now boot up all of the other nodes. When they boot, they will send out DHCP requests which the head node should respond to, correctly configuring all of the nodes for internet connectivity. Wait at least 2-3 minutes for all of the nodes to have a chance to boot up properly.

Now on the head node run

sh setup-cluster

Note that this does not need to run as root. This will first generate SSH keys on the head node (for passwordless SSH). Just hit return (3 times) to prompts. Once it has generated the keys, it will scan the internal network to find the other nodes. It will store a list of workers in the file "workers.txt", which will be stored in both the script directory and the home directory. It will then copy the keys to each worker node in turn. For this, you will have to accept the connection and type in the password (raspberry) for each node in turn. This should be the last time you need to enter a password for the worker nodes. Finally, it will upgrade Raspbian on the workers and the workers will all reboot when they have finished upgrading. This last stage will take a long time (up to half an hour, depending on your internet connection). Once this command has completed, wait 2-3 minutes for the workers to reboot before continuing.

Note that the script setup-cluster calls three other scripts to do its work, and these scripts can be useful on their own. The script map-network scans the network to see what nodes are available, and stores them in the file workers.txt. You should re-run this whenever nodes are added or removed from the cluster. The script copy-keys copies the SSH keys to all of the workers. You should re-run this after adding new nodes to the cluster (after running map-network). Finally, the script upgrade-workers upgrades Raspbian on all of the worker nodes and then reboots them.

Finally, there is one more script, shutdown-workers which shuts down all of the workers in anticipation of the head node being shut down and the cluster being powered down.

You may be tempted to customise the worker nodes by logging in to them individually and giving them host names, etc. It’s best to resist this temptation. The Zen of cloud computing is to treat nodes like sheep and not like ponies! Don’t name them, don’t pick favourites, don’t treat them individually, and don’t care if the odd one occasionally goes missing. You might imagine that this can make it difficult to track down hardware problems with individual nodes, but it’s usually possible to track these down quite quickly by looking at indicator lights on the nodes and the network switch when the system is under load.

That’s it – you now have your own private cloud.

Use case: a standalone Spark cluster

There are lots of things that one can do with a cluster like this, but for illustration, I will now show how to use the cluster as a standalone Apache Spark cluster. In previous posts I have described how to install Spark on a Pi 2 and how to create a small Spark cluster. It may be worth quickly reviewing those posts before proceeding. On the head node run

wget http://www.eu.apache.org/dist//spark/spark-1.4.1/spark-1.4.1-bin-hadoop2.6.tgz
tar xvfz spark-1.4.1-bin-hadoop2.6.tgz
cd spark-1.4.1-bin-hadoop2.6

to get and unpack a recent version of Spark. Then configure Spark appropriately with:

cp ~/workers.txt conf/slaves
echo '#!/usr/bin/env bash' > conf/spark-env.sh
echo "" >> conf/spark-env.sh
echo "SPARK_MASTER_IP=" >> conf/spark-env.sh
echo "SPARK_WORKER_MEMORY=512m" >> conf/spark-env.sh

Having configured Spark on the head node (which will also act as the Spark Master), we can copy it to the workers with

parallel-scp -h ~/workers.txt -r -p 100 -t 0 /home/pi/spark-1.4.1-bin-hadoop2.6 /home/pi/spark-1.4.1-bin-hadoop2.6

Note that if you subsequently change the config, you don’t need to re-copy the entire Spark distribution. Just re-copy the conf directory with

parallel-scp -h ~/workers.txt -r -p 100 -t 0 /home/pi/spark-1.4.1-bin-hadoop2.6/conf /home/pi/spark-1.4.1-bin-hadoop2.6/conf

A simple Spark session can then be run with

bin/spark-shell --master spark://

When Spark eventually starts up, you can enter the following into the Spark shell


After exiting the Spark shell (Ctrl-D), you can shut everything down with


See my other posts for additional pointers and further reading.

Some useful links/references

The main relevant link for this post is that of the associated code on GitHub:


If you are a Git person you might want to clone or fork this repo. The main other post I found useful for setting up the Pi as a NAT router was:


There was also an article on turning your Pi into a Wifi access point in Issue 11 of MagPi that I found somewhat useful.

Docker on Snappy Ubuntu Core on a Raspberry Pi 2

In a previous post I gave a quick introduction to Snappy Ubuntu Core on the Raspberry Pi 2. This was based on the very early version of Ubuntu Snappy Core that was released around the time of the Rasberry Pi 2. There were various problems with that early release that have since been fixed, so in this post I want to give a quick update on the status of Snappy on the Pi, especially in relation to Docker.

If you have an early version of Snappy for the Pi 2, you should manually download a new image and re-flash your SD card. I did not have any luck in using the Snappy upgrade mechanism (which is somewhat ironic). You can download a new image here. I used ubuntu-15.04-snappy-armhf-rpi2.img for this post.

You can check if you have an old or new version of Snappy by running snappy list. That gave an error with the old version, but should work fine with the new version. The new version also sets the system clock using the internet, so date should return the correct time for any system that is connected to the internet. As before, Snappy runs an SSH server from first boot (ubuntu/ubuntu), so can be happily run completely headless.

Once logged in the snappy guides should more-or-less work. Commonly used Snappy commands are indicated below.

snappy --help
snappy list --help
snappy info
snappy list
snappy list -a
snappy list -u
snappy search docker
sudo snappy update ubuntu-core
sudo reboot
sudo snappy install docker

There is now a Snappy framework for Docker which can be easily installed, as indicated above. If you are a Docker person, this is essentially the only Snappy framework/app you need, as you can do everything else with Docker images.

The main issue with Docker on the Pi 2 (and on ARM more generally), is that the vast majority of images on DockerHub are for x86, and therefore will not work on ARM. For the Pi, the best bet is to search for images containing the text rpi or armhf. Again, some indicative Docker commands are given below.

docker search rpi
docker info
docker images
docker pull hypriot/rpi-java
docker pull resin/rpi-raspbian
docker images
docker run -i -t hypriot/rpi-java /bin/bash
docker ps
docker ps -a

Some useful base images include resin/rpi-raspbian which is a minimal Raspbian base, and hypriot/rpi-java, which includes OpenJDK7. With these (or other) base images, it is easy to layer on top with your own Dockerfile to create your own custom images. As discussed previously on this blog, OpenJDK is very slow relative to Oracle’s JVM on ARM. I’ve not yet found a public image containing Oracle’s JVM (perhaps due to licensing issues), but it’s easy enough to roll-your-own from a minimal base image. You can follow the standard Docker User Guides for information on how to build your own images.

Getting started with Snappy Ubuntu Core on the Raspberry Pi 2

[UPDATE: Note that the version of Snappy described in this post is now obsolete – I have a new post describing a newer version which works better, and slightly differently.]

This post consists of a few notes which may be helpful for people trying to get started with Snappy Ubuntu Core on the new Raspberry Pi 2. First of all note that Ubuntu requires ARM7 so it won’t run on any model of Raspberry Pi prior to the Raspberry Pi 2 (model B), released February 2015.

The Ubuntu Core image can be downloaded from the Raspberry Pi downloads page, and written to uSD card in the usual way, just as you would a Raspbian image. From my Ubuntu laptop, I use a command like:

% sudo dd bs=1M if=pi-snappy.img of=/dev/mmcblk0

Be very careful to get the device name correct for of, as this command will completely trash the output device.

Once you have an image on a uSD card, you can insert it into your Pi 2 and boot it up as usual. If you have a keyboard and display hooked up you can log in on the console, but note that Ubuntu Core can be used headless from first boot via ssh. The default username and password are both ubuntu. I ssh in to my device with a command like ssh ubuntu@raspi08.home. Ubuntu Core uses the new containerised “snappy” system for managing packages, so “apt” doesn’t work. To get an idea of how snappy works, you can read through the snappy tour from Canonical, but note that much of this tour won’t actually work on the Pi 2 right now. Here’s a console session:

ubuntu@localhost:~$ snappy info
release: ubuntu-core/devel
ubuntu@localhost:~$ snappy versions
Part         Tag   Installed  Available  Fingerprint     Active  
ubuntu-core  edge  2          -          f442b1d8d6db3f  *              
This command needs root, please run with sudo
ubuntu@localhost:~$ snappy search docker
No matching packages found: docker

You will note that Docker isn’t currently available for Ubuntu Core on the Pi 2, though hopefully that will change soon. However, at the time of writing you may also find that the final command bombs with a certificate error. This turns out to be due to the fact that the system time is incorrect. You can verify this by running date. You can manually set it with a command like

sudo date -s "Sat Feb  7 09:57:32 GMT 2015"

where you paste in the output from running date on a system which does have the correct time. You can also automate this by instead using a command like

sudo date -s "`ssh username@linuxserver date`"

where username and linuxserver are replaced appropriately. At the time of writing there are very few Snappy packages available, but one useful package is webdm. You can search for it with snappy search webdm and install it with

sudo snappy install webdm

Running snappy info will confirm that it has installed correctly. This runs a web based package manager on port 4200, so, for example, I can connect to this from a web browser on the local network using the URL http://raspi08.home:4200/. This allows the browsing of available frameworks and apps.

There currently isn’t any app or framework for Java, but manually downloading and installing Oracle’s JDK8 for ARM works fine, and runs code at the same speed as using the JVM which ships with Raspbian. It would be very easy to package up the JDK8 as a Snappy app or framework, but I guess that there are licensing issues, so I’ll leave that to others to sort out! You can find out more about how Snappy works by reading Canonical’s snappy guides.

I quite like the Snappy system, and running Ubuntu Core on a Pi 2 is potentially a great way to learn about Cloud computing in a very cheap, simple and safe way. However, we need a few key apps and frameworks before it will become genuinely useful. Ubuntu Core certainly isn’t about to replace Raspbian as the main OS for the Raspberry Pi 2 any time soon.

Benchmarking MCMC codes on the Raspberry Pi 2


In the previous post I looked at running some MCMC codes in C and Scala on the Parallella. In that post I explained how the Parallella was significantly faster than the Raspberry Pi, and how it represented better “bang for buck” than the Raspberry Pi for computationally intensive MCMC codes. However, since that post was written, the Raspberry Pi 2 has been released. This board has a much better processor than the old Pi, and double the RAM, for the same price. This changes things, considerably. The processor is an ARM7 quad core. Each core is around twice as fast as the single core on the original Pi, and there are 4 of them. In this post I will re-run the codes from the previous post and compare against the Parallella.

Gibbs sampler in C

I’m using the new Raspbian image for the Pi 2. This includes gcc by default, but not the GSL library. This can be installed with sudo apt-get install libgsl0-dev. Then the file gibbs.c from the previous post will compile and run. On the Pi 2 this runs in around 75 seconds – very similar to the time on the Parallella, and around twice as fast as all of the previous Raspberry Pis.

Gibbs sampler in Scala

The Raspbian image ships with Oracle’s fast and efficient ARM-optimised JVM by default, so there’s no issue with installing Java at all. As usual, installing “sbt” is a simple matter of copying the launcher script (and jar) into your ~/bin directory. Then the Scala version of the Gibbs sampler can be run with a command like time sbt run > /dev/null. Again, it runs in around 4 minutes 40 seconds, just like on the Parallella. So, the ARM cores on the Parallella and the Pi 2 have very similar performance. However, the Parallella ARM chip has just two cores, whereas the Pi 2 is quad core.

Parallel Monte Carlo in Scala

Again, as for the previous post, I next ran the Monte Carlo example from this github repo. This gives output like:

$ sbt run
[info] Set current project to monte-carlo (in build file:/home/pi/src/git/statslang-scala/monte-carlo/)
[info] Running MonteCarlo 
Running with 1000000 iterations
Idiomatic vectorised solution
time: 6768.504487ms
Fast efficient (serial) tail call
time: 2473.331672ms
Parallelised version
time: 1391.2828ms

Here again we see that the single threaded versions run in a similar time to the Parallella (and around twice as fast as the old Pis), but that the parallelised version runs significantly faster on the Pi 2 than on the Parallella (due to having 4 ARM cores rather than 2).


For my test MCMC codes, the cores on the Pi 2 are around twice as fast as the single core on the old Raspberry Pis, and a similar speed to the cores on the Parallella. However, multi-threaded codes run faster still, due to there being 4 cores on the Pi 2 (versus 2 on the Parallella and one on the old Pis). Furthermore, the Pi 2 is the same price as the old Pis (which are still being sold), and around a quarter of the price of the cheapest Parallella. So for standard single and multi-threaded codes running on the ARM cores, the Pi 2 wins hands down in terms of “bang for buck”, and is sufficiently quick and cheap that it starts looking like a credible platform for building cheap clusters for compute-intensive jobs like MCMC. Now to be fair to the Parallella, really the whole point of it is that it has a multi-core Epiphany co-processor that I’ve not been using or factoring in to the discussion at all so far. That said, the Pi 2 is so much cheaper than the Parallella (not to mention, less “fragile”), that I suspect that even for codes which effectively exploit the Epiphany chip it is unlikely that the Parallella will outperform the Pi 2 in terms of “bang for buck”. Now “bang per watt” is another matter entirely, and the Parallella may well outperform the Pi 2 in that regard if efficient use can be made of the Epiphany chip. But development time costs money too, and it’s really not clear that it’s going to be easy for me to run my multi-threaded Scala codes effectively on the Epiphany chip any time soon. So the Pi 2 currently looks like a real winner from my personal perspective.

Setting up a Minecraft server on a Raspberry Pi

I’ve recently set up a Minecraft server on a Raspberry Pi. There’s lots of information on line describing how to do this, but I still had some problems, in part due to a lot of information on-line being out-of-date, in part due to the fact that running a Minecraft server is right on the limit of what a Pi is capable of, and in part due to the fact that I don’t really know much about Minecraft…

Just to be clear, this is about running a Minecraft SERVER, not the game client. The game client doesn’t work well on the Pi, as it doesn’t have enough memory. There is a special game client for the Pi, the Minecraft Raspberry Pi Edition, which is free, and programmable, but is creative mode only, and has no monsters. That is not what this post is about. You can run a server on the Pi (for free) which you can use from a standard game client (which is not free). That is what this post is about.

Useful on-line information I found useful includes the Gamepedia tutorial, this Forum thread, and this how-to-geek article. It is worth having a quick read through these before continuing.

It is important to understand that there are lots of different Minecraft servers out there, most of which are Java based, but not all. There are potential advantages to not using the standard vanilla Mojang reference server, as some servers are lighter weight, and hence could potentially run better on the Pi. However, lots of servers “out there” are not compatible with the latest (1.7.x) versions of the game client, so it’s probably best to get the vanilla server up and running first, before exploring other possibilities. Note that I’m assuming a Revision 2 Model B Pi with 512MB RAM. I don’t imagine that you will have a good experience with 256MB RAM. You should use raspi-config to allocate as little RAM as possible to the GPU, and you should overclock the Pi as much as you dare.

The Mojang server is a Java server, so you need a fast JVM installed on the Pi. The OpenJDK JVM is too slow – you need the Oracle JVM. Lot’s of info on the web refers to Oracle’s developer preview of Java 8, but that isn’t necessary now, as Oracle’s Java 7 is now a standard part of Raspbian. So just install that:

sudo apt-get update
sudo apt-get install oracle-java7-jdk

Use java -version to make sure it has worked. Then just pull the server jar, stick it into an appropriately named directory, and run the server from that directory with a command like:

java -Xms256M -Xmx384M -jar minecraft_server.1.7.2.jar nogui

The first time you run this it will take an age (possibly up to half an hour). Subsequent starts will be much quicker (15 seconds). Once it is up and running, try connecting to the address/name of the Pi from the machine you usually use for running the Minecraft game client. If the connection fails due to a protocol error, this usually means that the server is too old to cope with the latest version of the game client, so you need to find a new server. If it fails due to a timeout, then it means that your Pi isn’t running fast enough. Tweak settings. Overclock more, etc. There are various settings in the “server-properties” file that you can tweak to improve the speed at which the server runs. Disabling “nether” and dropping the view distance down (to, say, 5) seem to be particularly effective. Again, you can find out more about this by googling around. My current server-properties looks like this:

#Minecraft server properties
#Sat Nov 30 14:23:39 UTC 2013
motd=Pi server

If you’ve never set up a server before and are having trouble, it may be easier to set things up on another machine and then copy things across to the Pi when you get it working. I actually got mine up and running on a fast Ubuntu laptop first, which saved a lot of time.

I’m still running the Mojang server. It’s generally fine, but gives lots of warnings about “Not keeping up”, and riding horses doesn’t really work very well. If anyone has suggestions for another server that will work as a swap-in replacement and run better on the Pi, please do leave a comment. I’d be interested in something that runs a bit faster, and copes better with riding horses, smashing blocks, etc. But my kids will not be pleased if they have to re-build their world…