It's summer in Spain, and outside there's a dry heat, but in the hack lab, it's frigid. I'm sitting across the table from Otavio, and each of our laptops is showing the same identical screen of text, editing the same file. Slowly. The file is located in Colorado, and we're fighting serious conference lag, plus transatlantic lag. Type a few lines, wait a minute, hope it shows up, and restart the converter to see if it worked.

We're struggling with figuring out how to make it work, and pair programming across the table like this helps -- although we also struggle with each others accents sometimes, so we're eventually mostly communicating via text on the computers.

Eventually Otavio rolls off to bed. Later, everyone else in the conference goes to bed or whatever -- I don't really notice. At some point a cleaning woman comes in, and seems suprised to see me. I muster up enough Spanish to ask if this muy fio air conditioning can be turned off. A bit later, I make this commit and then go out, look at storks in early morning light, and go to bed.

r59731 | joeyh | 2009-07-24 12:30:52 -0400 (Fri, 24 Jul 2009) | 1 line

up to r15000; also skip the packages/po accient revs

That would have been 6 am local time. The log shows us back at it by 11 am local. There's so much detail in these logs, and it's hard to tell how any individual bit will turn out useful, but the value in aggregate is undeniable and that's why we struggle to retain it as technology marches on.

A few days later and the conversion is caught up to the commits documenting it. Everyone goes home. Later still, and work is done to verify the conversion. Problems are found, fixed. The worst of them involves the Danish language, and absolute evil. Tempers flare, and I have the last argument I'll ever have the pleasure of having with Frans.

At some point in there, I do rather more coding in C++ than I ever have before, or ever wanted to. Frans's objections are addressed by that, more or less. We're using a weird tool that was developed as a kind of one-off for another project, but happens to meet our needs pretty well -- but we're stuck dealing with its bugs. And its need for copious handholding and historical research. The file we started out editing grows to two thousand lines, and now covers ten years of odd bits of history.

Suddenly it's a year and a half later. I'm in a cabin in winter, Otavio is down in summer, and I'm once again dealing with lag (dialup lag this time). We run the converter again -- one last time. And a time more.. and four or five more "last times". Finally, after a longish day, it's done.

This was not a major project, just some little bits of time here and there, maybe a week's worth in total, spread out over a few years and four or five people. It hardly seems worth writing about, just another codebase converted from Subversion to Git, a bit behind most of the other ones.

Just thought I'd give you a glimpse behind the curtain.

Traditionally, Unix has operated by piping between commands ("find | grep | sort | uniq"). I wonder if we are experiencing a shift to command composition being a more common model?

I started thinking about this when I considered the following command line:

chronic mr run git annex get

Unpacking, that uses chronic, which arranges for quiet running of mr (unless it fails, then it shows the full output), which in turn runs git in every repository it knows about, and git is asked to run the git-annex subcommand, which in turn has a "get" subcommand of its own.

So, in other words, this is a cron job that syncs large files in a set of git repos, quietly. You might want to make it run niced also, which adds 3 more commands:

chronic nice ionice -c3 trickle -d100 mr run git annex get

This is the most extreme example I know of (unless you tack in a "sudo"), but there does seem to have been a gradual shift to using more composition, and more subcommands. In git in particular, two-levels of subcommands is not unknown, for example "git remote add". Subcommands, especially as implemented in git (or earlier, in arch), are kind of a special case of command composition that adds a namespace. I'm sure that a count of commands with subcommands would show it exploded in the past 10-12 years. I'll bet a count of composable commands would also show a significant increase.

Anyone have other examples of long compositions of commands?