Tim Retout's www presence

Fri, 25 Jul 2014

London.pm's July 2014 tech meeting

Last night, I went to the London.pm tech meeting, along with a couple of colleagues from CV-Library. The talks, combined with the unusually hot weather we're having in the UK at the moment, combined with my holiday all last week, make it feel like I'm at a software conference. :)

The highlight for me was Thomas Klausner's talk about OX (and AngularJS). We bought him a drink at the pub later to pump him for information about using Bread::Board, with some success. It was worth the long, late commute back to Southampton.

All very enjoyable, and I hope they have more technical meetings soon. I'm planning to attend the London Perl Workshop later in the year.

Posted: 25 Jul 2014 08:36 | Tags: ,

Tue, 22 Jul 2014

Cowbuilder and Tor

You've installed apt-transport-tor to help prevent targeted attacks on your system. Great! Now you want to build Debian packages using cowbuilder, and you notice these are still using plain HTTP.

If you're willing to fetch the first few packages without using apt-transport-tor, this is as easy as:

  • Add 'EXTRAPACKAGES="apt-transport-tor"' to your pbuilderrc.
  • Run 'cowbuilder --update'
  • Set 'MIRRORSITE=tor+http://http.debian.net/debian' in pbuilderrc.
  • Run 'cowbuilder --update' again.

Now any future builds should fetch build-dependencies over Tor.

Unfortunately, creating a base.cow from scratch is more problematic. Neither 'debootstrap' nor 'cdebootstrap' actually rely on apt acquire methods to download files - they look at the URL scheme themselves to work out where to fetch from. I think it's a design point that they shouldn't need apt, anyway, so that you can debootstrap on non-Debian systems. I don't have a good solution beyond using some other means to route these requests over Tor.

Posted: 22 Jul 2014 22:31 | Tags: , , ,

Mon, 21 Jul 2014

apt-transport-tor 0.2.1

apt-transport-tor 0.2.1 should now be on your preferred unstable Debian mirror. It will let you download Debian packages through Tor.

New in this release: support for HTTPS over Tor, to keep up with people.debian.org. :)

I haven't mentioned it before on this blog. To get it working, you need to "apt-get install apt-transport-tor", and then use sources.list lines like so:

deb tor+http://http.debian.net/debian unstable main

Note the use of http.debian.net in order to pick a mirror near to whichever Tor exit node. Throughput is surprisingly good.

On the TODO list: reproducible builds? It would be nice to have some mirrors offer Tor hidden services, although I have yet to think about the logistics of this, such as how the load could be balanced (maybe a service like http.debian.net). I also need to look at how cowbuilder etc. can be made to play nicely with Tor. And then Debian installer support!

Posted: 21 Jul 2014 13:17 | Tags: ,

Sat, 07 Jun 2014

Day of Action

Today I attended the Don't Spy On Us campaign's Day Of Action at Shoreditch Town Hall in London. I'm not sure how much actual action there was, but the talking was interesting.

Retrospective Day of Action drinking game: drink every time you hear the phrase "If you have nothing to hide, you have nothing to fear." The spooks have a really good marketing department.

I don't write a lot on the internet any more - something I regret, actually. It can't even be because there are alternative places I am writing, because over the last couple of years I have been closing most of my social media accounts. I just share much less of myself online.

On the internet, nothing is ephemeral. Bruce Schneier says so. Choose your words carefully.

The thing about blogging is that it's so public. It's often tied to the writer's real-life identity. One of the valuable things about social media services is that they supposedly let you restrict who can read your words - the trade-off being that you must also grant access to advertisers, spies, cybercriminals...

Most memorable moments of the day:

  • Alan Rusbridger contrasting how he felt as a journalist working on the Snowden stories when in the US versus the UK - when in the UK, it's like "you know something bad is going to happen".
  • Being asked to leave a tiny meeting room, in order that I could show my ticket for the next talk, only to retake the seat where I had left my bag. Go bureaucracy!
  • Shami Chakrabarti leaving in her sunglasses like a rockstar.

In general, there were lots of famous people walking around as if they were normal. I was in the same room as Cory Doctorow, Jimmy Wales and Bruce Schneier at the same time.

Ahem, action (mostly for UK citizens):

  • Sign the petition.
  • Join the various groups who organized today (Open Rights Group, Liberty, Big Brother Watch, Privacy International, Article 19, English PEN).
  • Ask to meet your MP to talk about privacy at their next surgery - you can phone the House of Commons.

Posted: 07 Jun 2014 23:00 | Tags: ,

Sat, 15 Feb 2014

Backporting some Perl modules

I've started backporting some Perl modules to wheezy-backports - for starters, libbread-board-perl, which is now waiting in BACKPORTS-NEW.

At work I've recently been trying to automate the deployment of our platform, and was originally trying to use Carton to manage the CPAN dependencies for us. It seems like it ought to be possible to make this work using CPAN-only tools. However, in practice, I've seen two strong negatives with this approach:

  • it's a lot of work for developers to manage the entire dependency chain, and
  • it takes forever to get the environment running.

Consider, when you spin up a fresh VM, you need to build Perl from source, and then compile every CPAN module you depend on. This includes all the modules needed to run all the test suites. That's not going to be fast. All that, and you still need a solution that works with the distro's package management, because you still need to install all the build dependencies.

So, I'm trying a new approach - if someone else benefits from the packages I backport, even better.

Posted: 15 Feb 2014 22:20 | Tags: ,

Thu, 06 Feb 2014


I attended FOSDEM this year. As always, it was very busy, and the Brussels transport system was as confusing as ever. This time it was nice to accidentally bump into so many people I know from years past.

Lunar's talk on reproducible builds of Debian packages was interesting - being able to independently verify that a particular binary package was built from a particular source package is quite attractive.

Also Mailpile declared an alpha release. The bit in the talk that I didn't know already was the description of exactly how the search function works. Seeing how they integrate GPG into the contacts/compose features brought home how lacking most (all?) other mail clients are when it comes to usable encryption.

On Sunday afternoon I managed to grab a seat in the Go devroom, to hear about crazy things like how YouTube are putting a daemon called Vitess in front of MySQL to abstract away sharding (at the cost of some transaction guarantees). You would have thought Google would already have a scalable data store of some sort?

Other bits I remember: Michael Meeks talking about GPU-enabling spreadsheet formulae calculations. And hearing Wietse Venema talk about Postfix was pretty awesome.

Posted: 06 Feb 2014 08:23 | Tags:

Thu, 02 Jan 2014

OpenVPN and easy-rsa

One of those enlightenment moments that I should have had sooner: every time I have seen someone set up an OpenVPN VPN, they have generated all the certificates on the VPN server as root using easy-rsa. This is kind of strange, because you end up with an incredibly sensitive directory on the VPN server containing every private key for every client.

Another angle is whether you trust the random number generators used to create all these keys - does your hosting provider use a weak RNG?

Instead, you could set up your CA using easy-rsa on a separate machine - perhaps even air-gapped. Then private keys can be generated on each machine that wants to join the VPN, and the certificates can get signed by the CA and returned. (The easy-rsa package has been split out of the openvpn package in Debian unstable, which makes this more understandable.)

Is there a security benefit? You could argue that if your VPN server has been compromised, then you are already in trouble. But I'm thinking about a setup where I could run multiple VPN servers for redundancy, signed by the same CA - then if one server gets broken into, you could kill it without having to revoke all the client keys.

By the way, the default RSA key size used by easy-rsa is 1024 bits at the time of writing (fixed upstream: Debian bug #733905). This is simple to change, but you need to know to do it. One of the 30c3 lightning talks was about bettercrypto.org - a guide to which cryptography settings to choose for commonly used software.

Posted: 02 Jan 2014 20:56 | Tags: ,

Wed, 01 Jan 2014


So, happy new year. :)

I watched many 30c3 talks via the streams over Christmas - they were awesome. I especially enjoyed finding out (in the Tor talk) that the Internet Watch Foundation need to use Tor when checking out particularly dodgy links online, else people just serve them up pictures of kittens.

Today's fail: deciding to set up OpenVPN, then realising the OpenVZ VPS I was planning to use would not support /dev/net/tun.

I'm back at work tomorrow, preparing for the January surge of people looking for jobs. Tonight, the first Southampton Perl Mongers meeting of the year.

Posted: 01 Jan 2014 18:02 | Tags:

Mon, 02 Dec 2013

How not to parse search queries

While I remember, I have uploaded the slides from my talk about Solr and Perl at the London Perl Workshop.

This talk was inspired by having seen and contributed to at least five different sets of Solr search code at my current job, all of which (I now believe) were doing it wrong. I distilled this hard-won knowledge into a 20 minute talk, which - funny story - I actually delivered twice to work around a cock-up in the printed schedule. I don't believe any video was successfully taken, but I may be proved wrong later.

I have also uploaded the Parse::Yapp grammar mentioned in the talk.

In case you don't have time to read the slides, the right way to present Solr via Perl is to use the 'edismax' parser, and write your code a bit like this:

my $solr = WebService::Solr->new($url);
my $s = $query->param('q');

# WebService::Solr::Query objects are useful for
# 'fq' params, but avoid them for main 'q' param.
my $options = {
 fq => [WebService::Solr::Query->new(...)];

$solr->search($s, \%options);

The key thing here is not to put any complicated parsing code in between the user and Solr. Avoid Search::QueryParser at all costs.

Posted: 02 Dec 2013 22:58 | Tags: , ,


At the London Perl Workshop last Saturday, one of the lightning talks was about Questhub.io, formerly known as "play-perl.org".

It's social gamification for your task list, or something like that. Buzzword-tastic! But most importantly, there seems to be a nice community of programming types to procrastinate with you on your quests. This means I can finally get to work refuting lamby's prediction about gamification of Debian development!

Tasks are referred to as "Quests", and are pursued in themed "Realms", for that World of Warcraft feeling. For example, there's a "Perl" realm, and a "Lisp" realm, and a "Haskell" realm, but also non-programming realms like "Fitness" and "Japanese".

Of course, part of me now wants to construct a federated version which can be self-hosted. :) Another downside of questhub currently is the lack of SSL support - your session cookies are sent in plain text. I hope this changes soon.

Posted: 02 Dec 2013 21:55 | Tags: , ,

Copyright © 2007-2012 Tim Retout