I went to linux.conf.au this week, and learnt about some pretty awesome tech (as well as hearing some entertaining, inspiring, touching and terrifying talks from the likes of Paul Fenwick, Bruce Perens, Karen Sandler and Jacob Appelbaum).

So here’s a quick dump of cool projects I learnt about, with links where I could find them.

browser id:

Openid is not so great, due to:

• usability / confusion (requiring a url as an id)
• reliability (if your provider goes down, you can’t log in)
• lock in, (hard for the user to migrate providers)
• privacy (your provider knows every url you log into, every time)

Browserid serves one simple purpose: to prove you own an email. It’s distributed. The browser, (not a third-party server) is the login intermediary.

As an example: The login process to gmail generates a shortlived (in the order of hours or maybe days, I guess) cryptographically signed statement that you own that email, which your browser stores. Other sites just need to grab gmail’s public key and then they can themselves verify that the assertion you sent them proves you own (or can log in to) you@example.com.

Demo site: myfavoritebeer.org

Not awesome yet: There’s no browser or email provider support. But it’s developed by mozilla, has scaffolding in js to work already in all browsers. Certs are stored on browserid.org for now, until browsers implement native support (so they are just as bad as a server intermediary, but only as a stopgap).

mediastreams processing api:

Very cool demos, which are online (although they require a dev build of firefox).

css calc()

Computation in css. Very cool. There are implementations in IE and Firefox so far, and work on webkit is in progress. This is not just a convenience like SASS and friends - it allows for previously impossible mixing of units like 100% - 10px.

News on arbitrary metadata in git:

Apparently many people feel this is important for VCS interop (git-svn, fastimport, etc), and there are proposals for adding support (but it’s difficult to figure out how to do right in a way that won’t prohibit other useful things in the future).

openstack

Uses orchestra by ubuntu for deployments.

gerrit for code review. Use Jenkins plugins, pushes state notifications to launchpad. openstack wrote a patch submission tool: git-review. Rebases & pushes patchset to gerrit.

testing ctdb

Autocluster is a tool for testing clustered samba (and other things, presumably). Virtual kvm based clusters. Uses guestfish to manage volumes inside the kvm guests.

dbench for automated performance tests. Can describe any kind of IO workload.

Assessing the potential impact of package updates

Tools for spelunking package relationships (on debian at least): - Recursive depends, reverse depends, apt-cache, germinate.

misc projects / pointers

sozi is an inkscape plugin for chopping up vectors into slide presentations.

fossil scm is a VCS that tracks docs, wikis, bugs, etc in with the source code. Cool idea, apparently some bad implementation decisions though.

safe-rm: replaces rm and has a blacklist of folders you probably don’t really want to remove, such as /usr/. No more bumblebeeing (can’t find the link, but there was once a bug in the bumblebee uninstall script that removed all of /usr as root).

handbag makes android accessory dev easy(er)

libvirt-sandbox: new library and command-line tools for app-level sandboxing with LCX and/or KVM. Should be coming in Fedora 17.

Vsualisation: gapminder.org

freedombox: 100% free tiny personal server project. Uses all open-source & federated social software like diaspora, buddycloud

instamorph: malleable plastic that’s solid at room temperature, for making ad-hoc hooks, connectors, docks, etc. The aussie equivalent is apparently called “Polymorph”.

For unfortunate reasons, at work we have intellij module files that are:

1. checked into git (because they are useful and hard to recreate)
2. changing all the time for no good reason (because intellij is a bit like that)

For the most part, the changes are just noise because paths, versions, etc. differ slightly across dev machines. Sometimes the differences are completely meaningless (unordered elements in a file being rearranged). But sometimes we do want to check in new versions of these files, say when we add a new module.

Since they are checked in, we can’t just add them to .gitignore. But we can still mentally ignore them, with this rather ridiculous hack:

$ git status | sed -E -e 's/\x1b\[[0-9][12]m(.*\.iml)/'"`tput setaf 3`"'\1/'

This changes the normally red (or green, or blue) lines in the git status output to yellow instead when they refer to an intellij .iml file. It’s completely hacky, but it’s better than breaking the build because you forgot to commit a file amongst all the noise.

You can put this in a script you call instead of git status - writing git status takes too long anyway, I just call my script g.

You’ll also need to make sure you have enabled colour “always” in git status output, otherwise you’ll get no colours at all:

$ git config --global color.status always

If anyone has a better way of ignoring files while having them checked in, I’d love to hear it. Do other VCSs allow you to deal with this any better?

I’ve been doing some browser-side javascript lately, and getting frustrated at the mess that is browser-side modules.. So here’s a tiny library that, given a tree containing javascript files, will give you a single javascript file containing all of the modules, as well as a commonJS require method to use in browser-side code. No more accidental globals, your variables are local and you just add your module’s public interface to properties on the module-local exports object.

Of course, it also includes a tool to grab all the javascript code of your own and that of your recursive dependencies, specified in a zero install feed.

Quick hand count: who knows what String.split() does?

Most developers probably do. Python? easy. Javascript? probably. But if you’re a ruby developer, chances are close to nil. I’m not trying to imply anything about the intelligence or skill of ruby developers, it’s just that the odds are stacked against you.

So, what does String.split() do?

In the simple case, it takes a separator string. It returns an array of substrings, split on the given string. Like so:

py> "one|two|three".split("|")
["one", "two", "three"]

Simple enough. As an extension, some languages allow you to pass in a num_splits option. In python, it splits only this many times, like so:

py> "one|two|three".split("|", 1)
["one", "two|three"]

Ruby is similar, although you have to add one to the second argument (it talks about number of returned components, rather than number of splits performed).

Javascript is a bit odd, in that it will ignore the rest of the string if you limit it:

js> "one|two|three".split("|", 2)
["one", "two"]

I don’t like the javascript way, but these are all valid interpretations of split. So far. And that’s pretty much all you have to know for python and javascript. But ruby? Pull up a seat.

me: Hey node, how can I tell if my child_process.spawn() call failed before it even runs the desired command? Is there a return value? An exception? Maybe there’s an error event I can listen for?

node: Even better! Here’s an example from the documentation to do exactly that:

var spawn = require('child_process').spawn,
    child = spawn('bad_command');

child.stderr.setEncoding('utf8');
child.stderr.on('data', function (data) {
  if (/^execvp\(\)/.test(data)) {
    console.log('Failed to start child process.');
  }
});

me: Thanks, that’s… That’s just stunning.

(For extra points, this doesn’t even work if the child process inherits stderr, as my actual code does. Is this just a silly example from the documentation, or is there really nothing better?)

A few days ago I got a shiny new toy: a 3d camera from thinkgeek (I’d link to it, but it seems to have disappeared from their catalogue). I’m a massive fan of 3d photos / video, so it’s pretty cool to have a device that allows me to take stereoscopic photo pairs simultaneously (you can do it manually with a static scene, but those get boring).

Sadly (although not surprisingly), the quality is not great. The limited resolution is not really an issue given how you’re likely to view them, but the pictures come out awkwardly stretched to half the expected horizontal resolution. They are also pre-combined in a single JPEG, they are the way around for cross-eyed viewing, and the colour balance is frequently off between the two sensors (which can be really jarring).

Seeing a lot of manual photo fixing in my future, I set out to automate it. And thus stereoscoper was born, as a way to bulk-convert stereo images to other formats. Aside from the obvious geometry changes (the horizontal resolution and image placement), I also learnt all about histogram matching in order to make the colour balance consistent across stereo pairs. And in order to make animated gifs that match up nicely, there’s even an interactive mode where you can fine-tune the alignment of the image pairs.

In the wiggly-animated spirit of 3ERD (note: some images there are NSFW), here’s some fun we had in the park with my new toy:

(click to toggle each animation. It’s off by default to save your brain from having a fit ;)

(stereo)

(stereo)

(stereo)

(stereo)

Update: Click the (stereo) link under each image for a cross-eyed viewing version.