For a while I’ve been wanting to do a short video to show my workflow for writing go. Last week I had a small bug to fix which I thought would fit perfectly, so I recorded me fixing it, and put some ragtime over the top.

For those who are interested, here’s a list of my tools:

• Tmux (my tmux.conf is here)
• Vim – with no syntax highlighting – honestly, try it.
  • Vim is configured to run gofmt on save
• “t” is a script I wrote which notifys when the process is done. This is useful for long running tests
• godef and vim-godef

I’ve previously shown juju running on a raspberry pi 2, albeit deploying servers in ec2. Recently the lxd project released a snap that can be used to install lxd on ubuntu snappy, it even works on a raspberry pi 2. Around the same time lxd provider support landed in the master branch of juju. Naturally I thought it would be cool if I could use juju to spin up some units using lxd on my raspberry pi 2.

The following video shows it in action:

Obviously, it’s slow and unofficial, but it’s fun. For the curious these were the steps involved in getting this working.

• Juju was compiled using a raspbian image and Dave Cheney’s arm release of go.
• I needed to turn video memory right down and the available swap space up to 1GB, my sd card will not be happy.
• The compiled binaries were copied to my laptop where I built the juju snap and uploaded it to the raspberry pi 2 running ubuntu snappy.
• Juju wasn’t able to access the unix socket lxd was using. So lxd was configured to work remotely, and juju was using it remotely (they just happened to be running on the same host)
• Some of juju’s upstart scripts set extra limits on allowable open files and processes. upstart wouldn’t start these services as they were beyond what was allowed. I removed these from the upstart file, which isn’t something you’d want to do normally.

Not exactly production ready stuff, but playing is fun.

I like go. One of the things I like is not worrying about dependencies during deployment. You just scp your binary to your server and you’re done. Well, after you configure it to connect to the database, ah yes, we forgot about haproxy, let’s set that up as well, but after scping to a server and doing some minor setup you’re ready to go, scale it out you say? Just scp to other servers, do the same config setup……..

The solution to this problem is to use an orchestration tool like juju. Now thanks to the go binary layer orchestrating your go program with juju couldn’t be easier.

Here’s how it works. The go-binary layer looks in the top level of your charm for a go-binary.yaml file that contains the name of your binary and any arguments that should be passed to it.

1
2
3

#go-binary.yaml
binary: server
args: /etc/myserver/config.yaml

Your compiled binary should be placed in a “files” directory at the top level of your charm. Build your charm and you’re all done.

1
2
3
4
5
6
7

$ tree
.
├── files
│   ├── server
├── go-binary.yaml
├── layer.yaml
├── metadata.yaml

Under the covers the go-binary layer will add your binary to $PATH,write an upstart file for your binary and start it when the server starts.

There are examples of this in action with the http (website) relation (https://github.com/mattyw/simple-server-charm)

and with the mongodb relation (https://github.com/mattyw/simple-server-mongo-charm).

If you’d like to see these charms in action take a look at the following videos:

Jupsen is a tool that allows you to drop or delay packets between juju units.

Why is this useful?

Network problems happen, if you’re working on a distributed system you need to deal with them, this is much easier to do if you’re able to recreate these situations easily.

With jupsen you can

• Block traffic between two units in only one direction
• Block traffic between two units in only both directions
• Delay packets in one direction
• Delay packets in both directions.

This can be used to recreate some of the experiments in Kyle Kingsbury’s jupsen series (from which I have shamefully taken the name) (https://aphyr.com/tags/jepsen)

If you want to have a play around with jupsen you can get it here: (https://github.com/mattyw/jupsen)

If you want to see it in action you can take a look at this demo I gave at the recent Ubuntu Online Summit.

The power of juju, and its model driven approach to deployments is undoubtedly the relationships. Being able to specify what services you depend on and what you provide to other services is extremely powerful.

To get all of those benefits you need to write your app as a charm. With the new layering approach (http://blog.dasroot.net/2015-charming-2-point-oh.html) this has got even easier. There will be more posts about that coming soon.

But I’ve always pondered the following:

Is there a useful shortcut around writing a charm that would let you deploy your app in < 5 minutes

My idea was the remote-deployer charm (https://jujucharms.com/u/mattyw/remote-deployer/trusty/2)

It doesn’t replace writing your own charm, indeed the cases in which it’s useful are limited. (you must have a publicly available file path that you want to deploy from) The intention is to facilitate playing with juju in less time than the typical tea break.

The idea goes:

1. Deploy the charm
2. Give it something to do

1
2

juju deploy cs:~mattyw/trusty/remote-deployer-2
juju action do remote-deployer/0 deploy source=https://github.com/mattyw/compojure-app-bootstrap

The something to do is just pointing it at some remote file tree (like a github url). It downloads this file tree and looks for a file called remote-deployer. It runs this as the ubuntu user. The example above installs the clojure build tool lein and creates a hello world web app (https://github.com/mattyw/compojure-app-bootstrap)

1
2
3
4
5
6
7
8

# remote-deployer
#!/bin/sh
set -ev
sudo apt-get install -y --force-yes openjdk-7-jre-headless
sudo cp lein /usr/bin
cd $HOME && lein new compojure-app my-app
cd $HOME/my-app && nohup lein ring server &
open-port 3000

Juju fans will notice that the normal hook tools are available to you in this file.

If you’d like to suggest or help make improvements the project is here: https://github.com/mattyw/remote-deployer-charm

Take a look at in action:

I’d like to share a proof of concept project I’ve been working on that combines the worlds of ubuntu snappy and juju.

I wanted to use the flashing leds on the raspberry pi 2’s piglow to tell me the status of my cloud services. The idea being that if there’s something I need to fix the flashing leds will tell me, without me having to keep checking my phone or laptop to get updates.

This demo will use the realtime syslog analytics big data bundle as my example environment. It’s deployed in ec2 but it’s controlled and managed by juju running on my raspberry pi. I’m going to show you a working environment with happy led status. Then I’m going to cause some trouble and we’ll see the leds change to unhappy.

The idea being that if I want to know the status of my cloud services I only need to glance at my raspberry pi for a moment.

There’s still some work to be done around optimising the various led patterns. There could be a way that I could convey seriousness of the problem via the leds, or even how long there has been a problem. That’s work that can come later.

TL;DR

If your job requires thinking of any kind you should be keeping a log book/ journal to keep a note of what you are doing. If you learn to love the process of keeping the journal it will be even more useful to you.

The story

When I first started work I instinctively started keeping a logbook of all of my activities. I grabbed an A4 notebook out of the stationary cupboard and started writing, this was mostly because the rate of information coming at me was more than I could handle, so I wanted somewhere to store it that was better than my head. But it was also because I could own a nice A4 notebook without having to pay for it, and this was quite a novelty.

Then this year I discovered the pen addict podcast and (despite my left handedness) I really got into paper and pens again. For the past 3 months I’ve been keeping a log of my work. Experiments, meetings, things that need fixing, ideas, new tricks, everything to do with my job.

Keep a Logbook

This is how I make use of a logbook, it’s mostly common sense:

1. Always have it with me, even if I go for a walk.
2. If you’re working at your desk, have it within arms reach.
3. If you take a break, leave a note of where you’ve left off.
4. When you return from a break refer to your log book to help you regain context.

This is what I write:

1. Things I need to remember.
2. Things I need to do.
3. Things I want to talk about with colleagues.
4. Ideas for future projects.
5. Solutions to problems
6. Results of experiments I’ve done
7. Cool tricks I’ve discovered (editor shortcuts, programming patterns etc)

Reviewing your Logbook

An important part of any process is reviewing. Most days I will flip through recent pages of my notebooks and record anything that I think is important in Evernote or my teams kanban board. The act of reviewing notes also helps cement any new tricks I’d learned. The whole process I use is a topic for another time.

Love your tools

Something I’ve known to be true in programming for a number of years is that if you can learn to really love the tools you use then no matter how mundane your task seems to be you will love doing it, because using the tools is part of the fun. I’ve started to realise that the same is true of a number of walks of life. I’ve bought a couple of fountain pens, made my own pocket and full size midori notebooks, spent a stupid amount of money on field notes. But I also have a collection of programming related notes, ideas, insights that I keep catalogued. The process of writing all these notes is one I enjoy, in fact I look forward to it because I’ve come to love the tools I use to do it.

My tools

Here’s my current stuff (left->right, top->bottom)

1. A4 clipboard from some team building thing I did a number of years back.
2. A Field notes sized fauxdori (it also supports midori inserts).
3. A lamy safari with LH nib.
4. A Kaweco skyline with F nib.
5. Full size fauxdori. (It also support field notes)

I made both fauxdoris myself, which means the act of using them is a pleasure in itself.

Using a Raspberry Pi 2 with Ubuntu Snappy installed we’re going to write a simple snap that will flash a piglow attached to the gpio pins.

If you want to skip these steps you can download the snap itself from here

1. A simple piglow flashing binary

To make things simple we’ll use go to produce a simple binary that will flash the legs of the piglow. We’re using go because the static compilation means our snap will have no dependencies to deal with. The code itself is small. https://github.com/mattyw/flasher I modified the piglow library I was using to get a bit more feedback about piglow errors, the change is trivial to revert should you need to.

2. Create the snap

The package structure is simple

1
2
3
4
5

├── bin
│   └── flasher
└── meta
    ├── package.yaml
    └── readme.md

flasher is the binary I produced in step 1. The package.yaml just contains information about the snap:

1
2
3
4
5

name: hello-piglow
version: 1.0.0
vendor: me
binaries:
 - name: bin/flasher

Once you’ve got all this in place build it and send it over to your pi.

1
2

snappy build .
snappy-remote --url=ssh://<ip-address>:22 install <snap-name>

3. On the Pi

You need to give your snap permission to access the i2c device

1

sudo snappy hw-assign hello-piglow.sideload /dev/i2c-1

4. You’re done

1

sudo hello-piglow.flasher

If all goes well you should have a flashing piglow

In this blog post I attempt to connect monads in haskell with generics in go, avoiding flame wars as I go. If I’ve made mistakes please let me know and I’ll gladly fix them.

The Thought

I’ve just finished listening to episode 10 of the bikeshed which is largely a discussion about haskell, and well worth a listen to. However the conversation that starts at around 29:40 is what triggered this thought.

Haskell without monads

If you want to do IO in Haskell you need to use monads. But it’s possible to write a number of programs without IO(1).

According to “A history of haskell, being lazy with class” 6 years passed between the first release of haskell, and monadic IO being introduced. Why so long?

The language designers were under no pressure, they had time to think about how IO should be dealt with in the language.

The question of Go Generics

Go doesn’t support generics, and it’s often cited as a big downside or even a failing of the language. However you can still write very useful programs in go without generics(1). But here’s the thing:

The language designers are under no pressure, they have time to think about how generics should be dealt with in the language.

This is important. Just because a language doesn’t do something now, doesn’t mean it never will. If you’ve not taken a look at go previously because you’ve been swayed by the “no generics” argument I recomend taking a look at go again.

(1) An infinite number for maths fans

Alan Perlis wrote

1

A language that doesn't affect the way you think about programming, is not worth knowing.

That’s an interesting statement, it made me wonder what people think they’ve learned from certain programming langues, and how much it differs between people.

If you have a moment, I’d love to know: Survey