wow!

Not every day you get an email like this:

link is the result of my work making mooix multilingual. It is twenty seven thousand lines. It assumes my VServer patch.

It allows allows multiple users to communicate with the same mooix session at the same time in different languages, without ever seeing any language but their own unless that language is being spoken (via the "say" command or similar) by another user.

This is, the two different users will be typing different commands in different languages, and seeing different languages (but hopefully the same semantic content!) in the output.

Example runs are at link and link

These two scripts (the first in English and the second in Lojban) each have their users doing exactly the same actions, in effect, but with totally different results. Note that unless each specifically asks for the other language, the English user never sees Lojban, and vice versa.

Lojban! Mooix! Incredible.

work..

First day in Charlottesville on this one month gig. Lots of interesting and very small arm hardware (pack of cards size and up), typical first day on the job brain-dump. A bit of pressure already to decide if I want this to be a perminant job, tempting in several ways but I don't know. Not even sure I can manage this 9-5 + commute thing that people seem to have so much stock in for more than a month.

Charlottesville has a nice small college town atmosphere, with the office right on an enjoyable downtown pedestrian street and some good food. Got my Greek food fix tonight for the first time in too long. Looking forward to some good bookstores too.

I got to see d-i from the perspective of using it on some custom non-standard hardware and trying to figure out how to automate it and customise it. For exampple, I'd never really seen the way d-i is kinda broken if you run a daily build and try to install testing -- using the rest of d-i from testing -- as a problem, but it is a problem if you want currentish d-i that's likely to still work when you try to use it. Also initrd preseeding is a rather killer feature that we should not have left out of the sarge d-i. It'll be good to use this experience to improve things.

what a day..

6 hour drive to Charlottesville, then I decided to take the camper off the truck today, which was a mistake. Issues with the legs tilting under the camper since I'd started off on a bit of an incline and it just got worse. So borrowed jacking eqipment, drove an half hour into town to get more, half an hour back in terror of a police car as I realized I'd left the license plate on the camper in my hurry, then raced the dark to use the jacks to support the camper so I could retract and straighten the legs. Now at 9 pm I'm just done with that 4 hour adventure and things seem ok. Yeah, this is sure a lot of fun..

ugh

Truck broke down (clutch) and I'm stuck in Manteo until at least tomorrow, basically offline too except for handy open wifis. I see Debian is releasing, makes up for some of this recent bad time.

stuff

Spent some time roofing earlier this week. I enjoy climbing around on beams and being up high, so it's fun, though also scary at times. It seemed hard to get things done quickly up there because I had to be careful all the time.

Kite seems to have crashed hard this morning. I got the onsite tech to do a remote reboot and it seems ok, but I don't know what caused the crash. I'm hoping to give kite some probably much needed physical maintenance when I'm in California.

stuff

Slept in far too late this afternoon, easy to do on grey and chilly days.

The d-i beta release that I've now been in the final stages of putting together for entirely too long was nearly 100% ready to go last weekend, only waiting on a few obscure architectures to finish building some stuff. Then secure apt entered testing and broke everything, and we had to add fixes to the fixes, and stuff like apt crashing on the alpha platform began to be potential beta blockers. Looks like this may get resolved yet, since mvo rocks, but it's been painful.

And I'm still waiting on overly delayed builds for some architectures (cough sparc).

The only thing I'm missing here at the farm now is a working oven. Otherwise it's been suprising how quickly I fell back into old modes.

sleep

Gunnar, I'm pleased to inform you that I stayed up until 11 am this morning (and slept till 2 pm), and that the day before I was up until about 8 am (and again, slept till 2:30). This is the first really bad sleep schedule I've had since Oldenburg.

I'm glad that this makes someone happy.

Oldenburg

So I'm here at Oldenburg, it's 4 in the morning, and I'm kinda waiting for some more people to go to bed. People bring all kinds of interesting hardware to Oldenburg. A sample of a few tables:

• A Hewlet Packard Advancestack J2600A, festoned with ThinLan cable, ethernet, and serial, sitting under a monitor, and on top of a VAXstaton 4000. To the side is an Atari TT030. In front of it is an old Sony Vaio notebook.
• Neat pyramid of an ancient A/B KVM switch on top of a tiny Soekris system, on top of am Epir (?), on top of an unlabled white box.
• Tiny embedded stack for an architecture I'd never heard of before and have forgotten, next to the thinnest Japanese laptop ever, next to big clunky old laptop sitting nearly on top of another little Japanese laptop, next to a case-less motherboard laid on on the table w/power supply.

Anyway, I'm waiting, because flying in meant I couldn't bring this kind of stuff, and instead I just brought a simple access point, which everyone in the room with a laptop ended up using to get online. But since I'm also busy making d-i run on the access point and install Debian at the same time it's routing that traffic, and since I destroyed its firmware a few hours ago, it's been running ok, but is useless for development until I take it down and reflash it.

Still 9 users at 4:30 am...

moving

Unfortunatly stressful couple of days here. I started moving back in at the farm, had to get the phone turned on, which is never any fun out here, and they claimed it was on though I had no dialtone. This has finally been resolved today without them having to come down the hill, which is a big relief, and maybe I can avoid driving into town tomorrow for the first time all week.

Although I'm on dialup again. Ugh. One of these days I will be able to say "never dialup again" and really mean it, but I've been thinking tht for going on 8 years now..

Also today I found out I need to replace part of the stove pipe, which could be tricky.

hmm

The perl monster program is down from 130 mb to 10 mb runtime after I removed some Data::Dumper insanity. That thing seems to copy recursive data structures internally and fails to clean them up.

hmm

I wih I could say that I occasionally do things like this just to make sure there are still plenty of users available to file more duplicate and inflated severity bug reports than we need and continue long after the fix is available.

hmm

I want a machine that will tattoo the following on a vistim's forehead:

DEHELPER IS NOT DH-MAKE

That is all, thank you.

.fi

Finally almost over the jetlag I think, 7 hours difference is actually easier than 5. But I was still tired most of the day until after supper and didn't accomplish a whole lot, although I did add laptop-detect support to tasksel and a laptop task. Then swimming, sauna, and hacking in the "sandbox" until $LATE.

Tomorrow, instead of going to Estonia, more hacking. Hard decision, that.

busy day

Spent most of today helping my Dad hang sheetrock, then this evening made some time to finish (one of) my papers for DebConf. Travel tomorrow.

busy busy

I was up until 6 am last night working on the first video game I've written since back when I used my atari. It has a AFAIK unique concept that still meshes perfectly with a number of classic games. And it uses ascii art. More on that later, if I get it to a playable state.

I spent a lot of time today looking at how d-i handles letting the user specify parameters to pass to kernel modules. This was partially broken for 2.6 kernels, and completly broken for udev. I've now cleaned it up in d-i svn to the point where I think parameters can be passed to modules in all possible cases.

Oh and liw is not running for DPL after all which totally sucks.

And I have a trip this weekend that will swamp all of this..

blah

I'm very glad I came over to Europe a day early since I spent most of the last 24 hours asleep and/or sick. Feeling better now and hoping it was only a 24 hour flu brought on by the very stressful trip.

Nichola Griffith is a teriffic author BTW, although a little too heavy for sickbed reading.

argh

My laptop's power input is broken. Not too badly yet since it seems to still get power in more positions than not, but I've been here before and I know it can go nstily downhill from here.

I think I know how I broke it, it's a dimly remembered farce from the morning before last that involved getting up to let the cat out, tripping over the laptop cord, putting the laptop back up only to trip over it again on the return trip.

There's a good chance it's the coord and if so I could just spend another $50 for third power brick and problem solved. My memory sucks and I forgot why I had to buy the second power brick, but I have a blog, so I know that it was a short in the cord before.

Now, I've been thinking about getting another laptop anyway, but it would just be a cheap used laptop off of ebay to use for d-i testing and development. The IBM thinkpads are nice, but just don't grip me and somehow this pokey laptop with its not very good outdoors screen still makes me happy, and I'd hate for this to be the bit that killed it.

I think I'll invest in a multimeter first so I can find out what the problem is.

Anna..

Based on this:

Next stop, Openmap. Openmap was in English and I got a little further with it. But the program still got stumped when I put in that geology map. I'm starting to think that maybe there's something wrong with the geology map, even though I was able to open it on its own under ArcExplorer. Oh, and Openmap made all of my xterms all funky --- they changed from filling up 2/3 of the screen and being white type on a black background to filling up 1/5 of my screen and being black type on a gray background. Ugly.

I'm pleased to award you your official linux geek diploma. It's been a while coming, I know. :-)

(PS, you might want to look and see if openmap did anything to the .Xresources file in your home directory when you installed it.)

Vancouver

Currently on my way home from a brief trip to Vancouver. It's been too long (3 years?) since I was on the west coast and I enjoyed many of the things I like about being over here. Vancouver reminds me a lot of many of the parts of the Bay Area I liked the most, with few of the disagreeable bits, and the added fun of being Canadian. I'd like to spend some more time there sometime.

I stayed in a self-proclaimed "funky" hostel downtown. Definitely on the low end and more than a few days would have been annoying. But I enjoyed meeting the folks they sent to pick me up from the airport, and it reminded me of Wm. Penn House in some ways. I suspect I enjoyed it overall rather more than some other people.

I was pretty well sucked into Debian-meeting-world for the whole trip, with the only significant side trip being maybe 45 minutes spent exploring UBC and finding the cliffs and the beach. It can be hard to explain why I travel for two days total to spend two days doing the same thing I did last time in a different corner of the world with the same people as last time (but see next blog entry). I am probably due some more time for myself on the next such trip.. but didn't I promise myself that last time? Oh well.

Oh and it's official.. my body clock has been stubbornly set on PST for all the years I've been back on the east coast. Perhaps it was before I ever set foot in this timezone, so I give up.

ugh

Last night chicken seemed very slow for command line use, but I put it down to too much load. This morning I saw all the nightly tests had failed with various missing file errors. Then I found I couldn't log in. Yep, one of the two 250 GB disks that I have lvmed together failed. SMART is whining about it now, but didn't give me any advance warning.

Most of the stuff on there was media files and not backed up, so I'm currently rsyncing hundreds of gigabytes of data that seems ok to a different machine, then I'll yank the disk. I'm not looking forward to reinstalling, or re-downloading my Debian mirror, or to living with 250 gb less disk space on my file server until I can get the disk replaced. It would probably be prudent to add a third disk and go for raid 5, but I doubt I will. At least I have all the important info, including /etc, safely in offsite svn repositories.

Also, Dani is having a really hard time and sounds completly torn up about it. I knew I shouldn't get out of bed today..

this and that

Spent a fairly long night a few days ago thinking about what is essentially a peer-to-peer moo design for mooix 2.0. I know that multi-server moos have been done plenty of times, but I've found very little prior art on truely peer-to-peer moos. Have some interesting ideas and a lot of difficult design problems to solve.

Good greif, the GPS arrived fast. I only ordered it yesterday! Also impressively small. Living in the future is fun sometimes.. Found my first cache already, a good excuse to get out in the woods. As if I needed one: It's spitting snow, nice holiday weather.

this and that

The only coding that I did today was a patch for debootstrap to let it work with the (broken) libc6.1 in testing on alpha and ia64. If that's accepted we won't need to wait on a fixed glibc entering testing, and d-i may manage to release on time.

The rest of my worktime has been spent in well, management, keeping track of other people's fixes and tracking issues that could block release (and not). That's how things will likely be until the next d-i release, mostly. I don't like it much, but it has to be done.

I've managed to get back into a novel, Crossfire, by Nancy Kress, it's nothing special so far (too much cliched exposition) but I have had trouble reading much lately for some reason (very unlike me) so I am glad to be reading that. This evening I went down to Kingsport to a choral concert. Large orchestral and choral works are not really my kind of music, but my mom was in it so I had to go, and it was enjoyable.

Soon, I will have furniture.

recent d-i hacking

Yesterday after doing the shopping, I worked on main-menu, closing several old bugs. Unfortunatly I failed to do one fix cleanly, as it would have required inverting the return values of most of the functions in main-menu. I tried to be smart about doing that, and changed their return types so static type checking would help me make sure I'd not missed changing anything. While that helped, the result still ended up buggy (what looked like deep recursion and a crash in an edge case), and I didn't have time to pinpoint where I'd went wrong, so I threw in a hack (global variable) instead. Sigh. Late last night, I also changed quite a lot of strings, which the translators have been doing a good job at unfuzzying today.

Today I started some code that will try to look at a mounted partition and return the name of the OS that was installed on it. It's easy to handle DOS, Debian, Red Hat, Mandrake, and LSB systems, but I need listings or disk images to add ones like Slackware, Windows > 3.11, etc. My hope is that this info can be used by the grub-installer as part of the upcoming os-prober udeb, to make a nice grub menu. And also possibly by partman in the partion list menu. Unfortunatly, the rest of os-prober, if it exists, is not in svn yet.

I also wrote a short guide on building d-i with a custom kernel. I'd hoped to convince a user who'd actally done this to write it, as I think that would have produced a better document. Oh well. If debian-cd was not such a bear, this would be really quite easy. To avoid the need to use debian-cd or set up a local d-i mirror with customised udebs, I added a new image type to d-i, it biulds a 7 mb iso image with a full d-i on it (making it easy to include different kernel module udebs in there), that downloads debian from the net. Kind of a miniature businesscard image. So d-i now has CD images of these sizes: 3 mb, 7 mb, 25 mb, 100 mb, full CD, full DVD. Whee!

phone

I've turned off the ringer on my phone, so if someone I know (as opposed to some phone marketer, or friend-of-previous-resident) is the person who's tried to call me all evening, you're SOL. Permanently. Yes, I know you expected me to pick up the phone rather than ignore it as it rang, and rang, and rang, and then finally turn off the ringer, but that indicates to me that you don't know me, because if you did you'd have emailed me first. All my friends and family know to email me first. So you don't know me, or you're ignoring my express wishes, and so I don't care to talk to you. I can be such a bastard sometimes, but oh well.

This reminds me, I've pretty well decided to switch to an internet phone, just to save money, make it harder to call me, and give me an excuse to end conversations early ("gotta go, this phone call is eating all my upload capacity"), and the only decision left to make really is what area code it should be in. Pity they don't offer 1-900 numbers.

new stuff

Amazingly, the 2.6.7 kernel works on my laptop, including resuming from apm suspend. This is the first 2.6 kernel that has worked well enough on my laptop for me to use it, and it's nice to finally have caught up to the world of 2.6 only nine months after its first release. Granted, if I had gotten my act together to send in a bug report to the appopriate kernel developer, it may have started working sooner.

The new version of firefox was seriously flakey at first, and to get it to work I had to throw out and redo my entire setup, which was a serious pain. Why do good graphical browsers always end up sucking more as time goes on?

I've signed up for VOIP phone service from Vonage, and depending on how well that goes, and how much my local F&F complains about having to dial long distance to reach me, I'm hoping to drop my land line.

MLP

I'm really happy to see that Mako has set up cardexchange.org for swapping those annoying supermarket loyalty cards and screwing up their databases. Something I've wanted to do for a long time.

foo

Trying to do anything d-i release related at debconf was a collossal mistake. I have not particularly enjoyed myself here or been particularly engaged because I have to crawl into a hole to get a d-i release out. This bites.

feh

I guess this week can be best characterized as me recovering from the trip without realizing how much it had taken out of me. Including a charming episode yesterday when I barely made it home from biking after getting dehydrated, developing an upset stomach and getting very weak. Only day I did enough work to be satisfied with was today, and now I'm feeling the need to get out again, but am not sure I'm recovered. Urgh.

d-i

Two main tracks today. First was fixing up debian-cd for the new d-i boot image names, and making it use the syslinux boot messages and splash screen provided by d-i. Along the way I got to do a fair bit of hairy make hacking. Second was touching up languagechooser and countrychooser.

Next I'll work on finally adding proper udeb support to debhelper, something I have put off for too long.

d-i

Feeling really good about d-i right now. We now have a boot screen gallery with yet another excellent syslinux boot screen added this evening. ths's revamped build system is easily extensible, and I have it building 3 mb mini isos that work like a netboot verison of d-i after booting from CD. Great for testing. I wonder where I could buy some phsical CD media that's only 3.2 mb in capacity?

d-i work...

How did I forget to brag about recent d-i fun / bore everyone in the world to tears? That will never do.

Let's see. One fun thing I did recently for d-i was write a web server in 56 lines of posix shell (+ netcat). I also wrote a mail program that can send email direct to outgoing nntp servers in a couple dozen lines of shell, but it was unbrearably buggy and I threw it away.

The reason for these is that it can be very useful in debugging d-i to get at the log files of a failed d-i run, but the only way d-i supported to do it before was to save them to floppy. Now it offers this nice option to start a web server and if I'm lucky I can just browse the running installer until I find the problem.

One of the little pleasures of d-i is being able to do stuff like write a web server in shell without it being a pointless exersise or Not Invented Here syndrome. (I did search for web servers in shell first, but didn't find one; all the small web servers in C are still a bit larger, and compile to bigger binaries; busybox doesn't seem to include a web server .. yet.)

Besides having to read all the http RFCs, which I've not looked at since circa 1995, and work out how to speak the simplest http varient I could (1.0) while pretty much working with all web browsers, the only really interesting bit was this, which comes after it's read the first GET line from the client:

# Reading the rest of the client's request is important; clients tend to get
# confused otherwise. Junk it all..
while read xx ; do
        test "${xx}" || break
        test "${xx}" = "" && break
done

Without this mozilla loses a random chunk of the end of the file and doesn't seem to notice or care; wget loops repeatedly.

argh

Well it's been a frustrating couple of days on most fronts.

Yesterday and early today I spent many hours trying to get a more stable hppa development environment on my a500. Since my old development environment tended to crash every 5 minutes and I needed to build a new 2.6 kernel with the things I need for nfs root, this involved a lot of pain and eventually installation of a temporary disk-based system to do the kernel compile (and dpkg -i !) on. Finally I seem to have a hppa kernel that can handle nfs root and run for several hours w/o falling over.

The point of that exersise was to update d-i to use the newest hppa kernel packages, so then I worked on that. Only to find out, that the new kernel didn't just fix the RC security holes that we're delaying the release over. No, this kernel also dropped some modules, added hundreds of other modules, changed deps between some modules, etc. It was the level of changes I'd expect to see with a new upstream kernel version, not a minor debian version change for security fixes.

[ It really sucks when you're trying to do one thing, like get a release out ASAP and have to deal with other people for whom that is not the most important thing. The Debian kernel team more and more appear to not be interested in that. See also recent thread on updating to 2.6.10 at this late date. Or is my frustration just making me crabby? ]

Unfortunatly I didn't even notice some of the hppa module changes yesterday so I had to roll two versions of the udebs, making lots of decisions that I'm not really qualified to make as a not-really-hppa-literate person, such as which modules will be in d-i at all, and bother the ftp-masters twice to get them rushed past NEW. And I'm still not done, because this new development environment is not as stable as I'd hoped, and has begun to segfault apt during builds of d-i initrds. More and more obstacles.

It's not as if hppa is even an arch I care about, beyond it apparently needing to work before we can release sarge. Sigh.

Add to this a smashed finger which is making typing painful, and a broken debian mirror that took a few hours to track down and fix, and it's not been a worthwhile couple of days.

zeroconf networking in Debian

I know essentially nothing about zeroconf networking, but I thought I'd try to get it working on my laptop, so I'll be ready to use it at the next DebConf, ad-hoc wireless network, DNS-less office , or similar situation, where I want to access a friend's computer without reciting IP addresses, or perhaps just farm out C compiles to whatever unknown machines are nearby.

The first thing I tried was the zeroconf package. This makes interfaces get assigned an ip address in the link-local 169.254.137 range, generally in addition to whatever IP address dhcp supplies. Annoyingly this hides the "real" ip address in the output of route, althogh ip addr list will show both.

Since you really only need this as a fallback for as-hoc networks that lack a DHCP server, I edited /etc/default/zeroconf and enabled the option that makes it only assign an address as a fallback if dhcp does not provide one. I don't really understand why this is not the default.

Next I installed libnss-mdns. This allows looking up hostnames in the .local domain, which will be resolved via multicast dns over the zeroconf network of local machines. To make it work /etc/nsswitch.conf needs to be edited to have this line:

hosts:          files dns mdns

I filed bugs on the [lack of documentation about that, and also on base-files to possibly get it supported by default w/o manual editing, which we'll need if we want regular users to use this.

Finally, if you want your machine to have a .local hostname in mDNS, you need a responder to broadcast it. One such is in the mdnsresponder, which unfortuatly has an unresolved license mess that has been pending resolution for too long. It also seems to be the only package in Debian to perform this task, so I used it anyway. :-( It's rather lacking in documentation and it took me a while to work out that I didn't really need to edit its config file, since it defaults to broadcasting your hostname.

Now I could ping dragon.local; if I had two hosts they should automatically network together with no configuration.

Of course to make this really useful you need a way to discover services on the network, and mdns-scan is one way to do this easily. The howl-utils can be used to look for specific services and to publish a service. But they have no useful docs in Debian, not even man pages, so see this page for some usage hints.

Of the other zeroconf stuff in Debian, daapd looked useless unless you have iTunes, and I couldn't find anything else, aside from probably gnome and kde GUI stuff.

Finally, I've been thinking about whether it would make any sense to install any of this stuff by default in Debian as part of the laptop task or some other task. Installing libnss-mdns seems like a reasonable thing to do, once it works with no configuration. mdnsresponder would also be fine to install, if it remains in Debian. zeroconf might be ok to include as well, unless its behavior of hiding real IP addresses is too annoying.

To be useful as a server on a zeroconf network, Debian should publish running services, possibly integrated with init scripts somehow, but I don't know what would be a good way to do that.

discussion

Yesterday it seemed all of d-i turned to shit.

I guess it was just one of those days, but I started off on solving one problem only to encounter new problem after new problem until I was popped so far down the problem stack that I couldn't find my way back out. So I fixed everything (or everything I could remember) and went out to the farm to be offline for 12 hours.

Anyway, I've been annoyed lately to see us really fall from the levels of stability and release-readyness we hit for RC1. Right now I can't even install Debian on some of my hardware because the must-have bleeding edge kernels we've upgraded to are buggy. (I miss you, Xu.) We're really in danger right now of people losing concentration on the release, because the release is effectively stalled.

I'm also very annoyed to have been tracking some important bug fixes (in aptitude and gnome-session) for over a month now (in one case, more than 2 full months) and still the fixes have not made it into testing. And the desktop task is broken because meta-kde was pulled from testing. So we're not able to get in key fixes or keep a basic installable desktop system in testing, but new features and bugs somehow manage to get in quite well. Not the best combination for maintaining my sanity.

I'm ready to be done with sarge, please.

yes..

I agree with Wouter Verhelst:

• The best Planet Debian style sheet is the "boxless" one which was made the default; it has the advantage of being excellent while the others are only nice.
• The recent GR should indeed be thrown out due to the adjective in the title. So should this one, whose title implies the old document is "ambiguous".
• The gimp makes everything easy.

YA desktop install

My Dad came up for the weekend so I could replace his virus-ridden windows system with Debian. I was worried about the short time allowed, since I knew I'd have to find a usable modem, and get dialup conigured as well as word processing and a gui email program. It all came together, d-i got Debian on there with dual booting with windows and a full desktop within the first hour, then I set up sylphead and did some minor tweaking and was nearly done. The next day we upgraded the memory and got an external modem. I discovered that my VOIP cannot support a modem connection well -- I just don't have enough outgoing bandwidth. Dialup over VOIP is the strangest protocol layering I've ever done.

Today I sshed into the box and set up CUPS (and hpoj) for printing. This was impressively easy, even remotely over a dialup line with no feedback available. First thing I heard was "it printed a test page".

This stuff is getting too easy..

xfs

Today I created my first xfs filesystem, and an hour later had partman supporting xfs. It was a stright copy and paste hack-job from the ext3 support, so not difficult.

Except that grub-install hangs if /boot is on an XFS filesystem. Grr. Oh well, on to partman-reiserfs!

wow

I'm only 13 minutes into April 1st (local time), and already it's dreadfully dull. Well except for all those nice Mailman reminders.

Perhaps April 1st should become a netless day for me.

work and er, work

It's a relief to have the d-i release over, and be back to regular development, although this is shadowed by the fact that we have to change something to make the next release more doable, and I don't know what yet.

Anyway, I uploaded some pending changes, updated d-i on i386 to use the 2.4.26 kernel, including the backported SATA support. I was suprised to find d-i installing the 2.6.5 kernel on me, which was pleasant, because unlike 2.6.3, it boots and works ok; but a suprise since it should have installed 2.4.x for a 2.4 install. Fixed that bug.

Hmm, I should upgrade my own laptop to 2.4.26 one of these days..

In skolelinux work, I have hopefully made it work with sarge's base-config, although I have been unable to test this so far because the skolelinux sarge installation CD I downloaded is kinda broken, and I have not yet been in town to download the new one Petter made. If it remains broken, my next bit of work in skolelinux will be clear.

I could rant a lot here about the idiocy of the whole countrychooser issue, but what's the point? Human nature.

Oh and about that so-called meme.. I'd love to participate, but my book reading software (w3m) does not mark pages.

work

Today felt productive, a nice change after vacation. I probably broke d-i or something by making it use localechooser, but hey, the release is done. And I tore through tasksel's pile of bugs and even fixed the one that annoys many people by letting standard packages be de-selected there.

Looked at packaging laptop-detect (from Ubuntu) for tasksel to use, but since the upstream tarall contains a debian directory already which I'd need to modify, I'm not sure it's worth the pain for a 37 line script. This probably should tell me something about my own (ab)use of Debian-native packages. It also seems to depend on something loading the acpi battery modules during boot if there's acpi, which I'm not very sure anything in Debian does by default. And it needs to get apm support added to it. Oh well, maybe later.

The other tasksel bug that would be really nice to fix is to add size checking, but of course this is decidedly difficult if there's more than one partition, and even checking the download size against the size of /var is hard since tasksel doesn't actually know what a task will install, and aptitude doesn't offer a way to get at the installed size figures w/o marking stuff for install.

wiki+blog = wow

Already done adding blog support to ikiwiki, execpt for some minor UI issues. So now it has its own news feed, as well as a quite nice automatic TODO tracker.

And my codewiki has a new projects feed and will probably shortly have individual rss feeds for some of the other projects listed there.

Sweet!

(ETA to migrating this blog, and my entire home page to ikiwiki: Probably less than 1 week.)

wiki/blog convergence

I wrote a wiki engine last week, maybe I should write a weblog engine next week?

Here's my own take on converging the two, which has been done before, but which some differences in ikiwiki perhaps allow to be approached slightly differently.

• ikiwiki supports pages that are subpages of others, for example, "Page/Discussion".
• Various parts of ikiwiki allow users to select a set of pages using globs. For example, "SomePage OtherPage OtherPage/* !OtherPage/Bar"
• ikiwiki renders pages statically, and can render rss statically too. This makes rss rather less expensive to serve..

First, I plan for all pages in the wiki generate a rss feed for that single page, which can be subscribed to if you want to track changes to the page alone.

Then, pages can have a blog embedded in them by adding a special tag to the page, which includes a list of globs for the pages on the wiki that comprise the blog. Something like "Blog/* !*/Discussion", to exclude discussion subpages from the blog. This makes the most recently created of those pages be inlined into the html version of the page (with a form at the bottom to make a new post, if you don't want to commit to svn by hand) ... and enables creation of a regular rss feed too.

Some nice features that just fall out of this:

• It's easy to set up subfeeds for categories. Just make blog pages for the categories, and an overall blog page that includes everything.
• All blog pages get discussion pages, without having to worry about people spamming your blog (unless they're already spamming your wiki of course).
• You can subscribe to an rss feed for any of the discussion pages if you like. Discussion pages can even be turned into little blogs of their own. Or they could even all be aggregated into a discussion blog (Containing anything matching "*/Discussion").
• Permalinks are just links to the wiki pages that are combined to form the blog.
• You can refer to any other blog posts using wikilinks; the wiki will automatically add backlinks to pages that link to a post, too.
• Since I keep my blog in plain text .mdwn files alredy, I can migrate from pybloxm to ikiwiki using pretty much just an svn mv ...

ikwiki's TODO has the details (about half way down; that page needs to be implemented as a collection of pages/blog itself).

discussion

Why I'm not on irc

I'm not on any irc channels right now because I sensed I needed a certian distance from some things, and email tends to provide that as well as additional clarity. (And ameanability to thread pattern matching of course. ;-)

It's interesting how quite a lot of things have been excerpted from irc channels in emails from various people since I left irc the other day. Almost makes me wonder what would happen if I pulled a Knuth. I can envision these boxes of email printouts arriving at my door.. nah.

I actually remain on irc for the important thing, instantaneous communication -- if you really must /msg me, I will get the message. The most likely /msg to work right now is "plz check your email".

why I'm not on #debian-tech

So, I guess this is an example of the sort of self-recusement from #debian-tech that aj is referring to:

@1130344829 <joeyh!joey@kitenet.net> aj: sigh, did you realy have to break d-i yet again right as we're trying ot release it?
@1130344847 <aj!aj-irc@azure.humbug.org.au> ?
@1130344881 <aj!aj-irc@azure.humbug.org.au> joeyh: (also, please try to avoid making things overly personal on this channel, cf the charter)
@1130344919 <joeyh!joey@kitenet.net> fine, did one have to ignore the d-i release and make a package upload with lots of changes that broke d-i internalls. Yes one did
@1130344928 <joeyh!joey@kitenet.net> for at least two values of one
@1130344967 <aj!aj-irc@azure.humbug.org.au> joeyh: please explain what you're talking about, in technical terms
@1130344989 <aj!aj-irc@azure.humbug.org.au> joeyh: if you do so, it might get fixed
@1130345337 <aj!aj-irc@azure.humbug.org.au> joeyh: hello?
@1130345448 <h01ger!~holger@bone.digitalis.org> joeyh, i guess you're referring to the debootstrap-upload of aj ?! /me wonders why an upload to unstable breaks the d-i beta, isnt that build+based on etch ?
@1130345451 <joeyh!joey@kitenet.net> sorry, busy fixing the beakage
@1130345474 <Sledge!steve@217.147.81.17> looks like a debootstrap change is causing problems in CD builds
@1130345481 <joeyh!joey@kitenet.net> well, there has not been a usable combination of apt, debootstrap, and d-i in testing for well over a week
@1130345634 <aj!aj-irc@azure.humbug.org.au> so you're saying i didn't break it just as you're trying to release?
@1130345720 <joeyh!joey@kitenet.net> no you broke it just as the fixed NMU was ready to go into testing
@1130345758 <h01ger!~holger@bone.digitalis.org> .oO( ah )
@1130345783 <aj!aj-irc@azure.humbug.org.au> sigh
@1130345792 <neuro!~rmurray@h24-80-164-222.sbm.shawcable.net> ...and we're still waiting for the technical details
@1130345813 <aj!aj-irc@azure.humbug.org.au> joeyh: second warning. this channel's for technical discussions, eg "here's what broke: ...", not accusations, eg "you broke it" "why'd you have to break it" "yes one did".
@1130345869 <aj!aj-irc@azure.humbug.org.au> joeyh: if you want, there's still time to put 0.3.1.9 into testing
* Logging finished Wed, 26 Oct 2005 12:58:19 -0400

It's worth noting that the d-i release in question was etch beta 1, which eventually released 2 weeks after the above conversation.

Probably my fault for not being a big believer in ridigly defined topics for communications channels. After all, I still filter all of my 20-some Debian mailing lists into a single folder; I'll happily talk about anything on the one irc channel (typically #debian-boot) that I nowdadys limit myself to; and so #debian-tech seemed like the obvious place to bring up the issue since aj, sledge, and other relevant people were active on it at the time.

The technical details remain as irrelivant today as they were at the time, the relevant problem being that developers who are not involved in the installer development rarely consider how their actions can affect it.

When a policy gets int the way of solving a problem, it's time to leave, that's all.

why do all the photo import programs suck?

Twice now I've tried to set up what was apparently a best of breed linux graphical photo program for less than technical family members who got a digital camera. Both times it ended up sucking.

The first time, Anna's camera needed to use gphoto2 to pull pictures out of its proprietary usb interface. Anna has been using linux for something like 8 years now and I tried her on gtkam and we found it was hard to use and crashy. I asked her to describe what would happen in her ideal world when she plugged the camera in, and she wanted it to save the photos to a directory with the current date in it, and delete them from the camera. A quick shell script later and it did just that. Problem solved.

This time, at my Dad's, the camera was some "Vivitar" bought at a discount store without consulting me. Luckily it did work with my Dad's linux box, in fact this camera, which the gphoto2 library is not familiar with, is a standard usb storage device with a "dcim/" directory that holds jpeg files of the pictures. So getting at the pictures is trivial, if you dont mind using the command line.

Now how to make this approachable for a novice user who's using gnome? The new gnome volume mamanger stuff in gnome works well, bringing up a message when the camera is plugged in, that asks if the user wants to import their photos. Then it runs a program; this defaulted to gtkam; since gtkam does not seem to support cameras that can be mounted as filesystems, this is a strange choice. I searched for other gnome photo programs and all I could find is gthumb.

I rather laboriously taught my father how to drag and drop photos from there to folders in the gnome desktop; nautilus has some annoying glitches that make this less than easy for anyone with less than perfect eyesight and eye-hand coordination to drag about big image icons. Anyway, this seemed to work, but it turns out that ghpoto does icon caching and this screwed up after pictures were deleted from the camera.

At this point I happened to set up photo importing for the same camera in windows, and to give the linux programs credit where it's due, this sucked even more, especially since we somehow seemed to have three programs all launching when the camera was plugged in and competing to manage the pictures on it. To the extent that it worked at all (I counted 30+ clicks to move 4 photos off the camera using the better of the three windows tools), I noticed a certian resemblance to the ideas behind how the gnome/gtkam/gthumb stuff worked. Hmm..

Back to the gthumb problem, I couldn't find a way to turn off this caching, so I threw up my hands and wrote the following shell script and made gnome-volume-manager run it:

!/bin/sh

Move photos from digital camera to Desktop/Photos, keeping names unique.

sec=$(cat $HOME/.movephoto-sec)
sec=$(expr $sec + 1)
for file in $(find /media/usbdisk/dcim -type f); do
        mv -f $file $HOME/Desktop/Photos/$sec.$(basename $file)
        sec=$(expr $sec + 1)
        echo $sec > $HOME/.movephoto-sec
done
nautilus ~/Desktop/Photos

Now my Dad just has to plug in his camera and a minute later his Photos folder opens up with all the new pictures moved to it and ready to be attached to emails or printed, and his camera is empty for the next set of pictures. This turns out to be just what he wanted all along.

I only have a sample size of three or four, but so far all the users whom I've been able to observe want it to work this way. As John Corbet wrote,

the best approach is to get a modern camera which implements the USB mass storage protocol. Then you can simply mount the camera as a disk, move the image files across, and be done with it. It's fast, easy, and for those who prefer not to use the mv command, setting up hotplug scripts to launch a file manager is relatively straightforward.

Perhaps all the labourious selection of photos, copying off the camera, and deleting made sense back when digital cameras were unreliable, and image transfer was slow, and disk space maybe scarce, but it seems to me that it doesn't make sense anymore. Now time and simplicity rule; get the pics off the camera and onto disk and if necessary get a file manager up so the user can deal with them in the same way they'd deal with any other files.

Now I'm tempted to generalize the two programs that I've so far written to suck images off of these two cameras to a package that can just make it all work.

why Debian sarge won't release today

A clone of the "BattleIsle 2/3" game called asc crashes on the alpha architecture. Unlike all the other other games in Debian that fail in some way on some obscure architecture (like my package of kobodeluxe), this asc crash is some some reason considered release critical and reason to delay the release, or remove the game entirely.

Some templating engine called cheetah that I've certianly never heard of before turns out to have a boneheaded security hole. There's some holdup getting the fix into sarge that I don't understand.

A second templating engine that I've never heard over either, called clearsilver (also written in python like cheetah, because TMTOWTDI, or something), fails to build from source in sarge because someone tried to write their own patch management system instead of using something standard and screwed up. A fix will reach sarge tomorrow. Hurrah.

gcompris, an "Educational games for small children," is missing a dependency on something obvious. It's taken since 19 May to get the fix for this from unstable into testing for reasons that are too boring to mention.

Ximain connector doesn't work with Microsoft Exchange 2003. I cry a single tear.

A fun new security hole was recently found in gdb, the kernel, and lots of other stuff that lets malformed ELF executables crash these programs. Of course if we'd released sarge the day before, we'd be issuing DSAs instead of waiting and hey, sarge would be released!

Simularly, security holes in a breakout clone, the GNU TLS library, X, and openmotif. All of them have patches, none of the patches have been uploaded yet.

gdm fails to work in an obscure setup on some large network. But works for everyone else on the planet.

Yet another UI library, GLUI, has a seriously messed up debian control file, due to some kind of control.in craziness and mismanagement. It's both been fixed and is planned to be removed, but neither has hit sarge yet.

Our initrd-tools package was updated quite a lot at the last minute; one of those changes intended to make it work better on ultra-obscure powerpc subarchitectures which only kids at Easter Seals have heard about, like the pegasos, but instead planted a mine that will blow up in our faces if the package is ever actually built on the powerpc architecture. DamnIfIKnow why we had to do this before sarge released.

A bunch of bugs were filed on old kernel patches that don't apply to current kernels in sarge. Likely most of these will be removed, OTOH, why worry about this now and not sometime in the past 6 months that we've been using the same old kernels in sarge? Well, because we seeme to be near to release and someone's pet peeve popped up.

Various other kernel security holes and hangs that are for some reason listed as worth delaying the release over, even though dozens of others are not. Hey, you want another kernel hang bug, I can certianly find hardware that will oblige.

hppa64. sun4m. 80386. These cutting-edge, mission-critical computer subarchitectures all have upgrade issues and it's imperative we delay the sarge release until we either make it really easy for all 1.5 users of each to upgrade, or until cosmic rays destroy all remaining instances of these machines.

samba's smbclient is high quality software that cannot cope with directories containing 27 or more entries.

logrotate fails if /tmp is mounted with the useless noexec flag.

Something called The WBXML Library crashes on (unspecified) input XML.

A circuit simulator called oregano uses the still highly in flux cario library and so has stopped working.

tex runs out of "capacity" on someone's system.

People are utter idiots when it comes to software licenses.

slapd sucks asteroids through soda straws.

syslogd is subject to a race with new kernels and glibc, which makes it hang. Which loses, badly. Thank you 2.6, thank you NPTL. Thank goodness for smarter heads than mine.

who said the tech bubble was over?

Good grief.

Who needs FAI?

Worked all day today on fixing things to allow preseeding the second stage of the installation process. My usb keydrive is now able to install Debian with a full gnome desktop (downloaded over the net), asking me only for the root password.

Total install time: half an hour
Total number of keystrokes: 8 (with a pathetic 3 character root password)

Soon I'll have it automatically changing my shell to zsh, installing all the Debian packages I can't live without, and checking out a copy of my home directory from subversion, for a completly customized joey-machine.

Where was I when ..

.. Debian woody was released? Well, falling prey to the latest meme, I had to look, since I don't remember what I was doing when woody was released. I was apparently at the farm and deep in the middle of developing mooix at the time.

.. Debian potato was released? I was at LinuxWorld San Jose taking part on insanity such as Debian release press conferences.

.. Debian slink was released? I remember sitting in my office at the house in Oakland, and working on updating the website for this one, instead of doing work for Mainsail. There was also a big thunder and lightening storm, the first I'd ever seen in the Bay Area. I described it as

It was quiet rain one minute, a strobe out the window the minute after, and WHAM! a few second later. Then every car alarm in Oakland went off.

.. Debian hamm was released? I was in Berkeley and too busy working for Mainsail to do much other than forward the debian-devel-announce message to slashdot. This was before slashdot twisted everything sent to it BTW. I also remember the IRC release party.

.. Debian bo was released? I was in Bristol and working on Mainsail and Debian, but the release seems to have escaped me.

.. Debian rex was released? I was in Bristol and had only used Debian/been a Debian developer for a few months. I did enjoy pointing out to Bruce the mistakes he made in the release annoucement. :-)

.. Debian buzz was released? I was in Bristol and was reading about this "Debian" thing for the first time on c.o.l.a.

when did I become a gnome weenie?

Somewhere along the line, I've gone from dsliking gnome, to considering it and kde as uninteresting equals, to grudging respect, to using it by default on everything except my personal work boxes (which use ion, of course).

Today I spent a couple of hours tweaking a sarge install and gnome desktop on a "new" machine, that will eventually be used my Maggie and my Mom. I only encountered one crash (or rather, failure to start) and one small bug. I found out that sound-juicer is rather nice as a CD ripper for newbies, and the gnome media platyer thing works find with reasonable sets of music (unlike my typically more unreasonable sets).

The only truely annoying thing was that this machine has a low res screen, and many gnome dialogs were cut off at the bottom. And gnome still can't seem to keep windows above the panel at the bottom of the screen all the time. Of course as an ion user, I'm not used to needing to worry about crap like moving windows around and keeping all of them visible.

But it was overall just fun to set up, try to anticipate their needs and make it easy for them to find what they want, and the result is quite attractive, and performs suprisingly well on a machine that was literally gleft on my doorstep for free.

(Also, update to my machine naming policy -- machines that are not in my house shall be named after flying creatures. Oddly all the machines I've already named that arn't at home fit this policy already. So the repurposed earthworm has become a sparrow.)

[whatever I usually put here]

I seem to be more inclined to blog when I'm doing a lot of coding, or just bored than when I am experiencing new things and having fun. So little blogging so far while I've been in Brazil, but today I managed to buckle down and get some work down, so now I want to write about it.

I'm trying a new method for managing d-i releases, the current method has been to make lots of changes, get something that pretty much worked on i386, copy everything to testing at once, test it more and on other arches and try to get in fixes for the breakage, and then release. After beta4, I relised this was not going to work with 10 arches.

The new plan is to use testing more like it is used for debs; getting individual packages into testing when they are ready. Unfortunalty d-i has some more complex interactions than Debian, and we also have less metadata and worse bug reports to go by so automated propigation is only a dream at this point. To try to manage the complexity of tracking the status of everything, I tried to come up with some criteria to use, and created this page to track the status of everything. It's still complex, but it seems more manaeagable to have it all on one page. Well, we will see how this goes.

Brazil is an experience that I'm very glad that many Debianistas will be able to share.

What timezones have you uploaded Debian packages from?

Or, are you as bored of planes as I am right now?

joey@dragon:~>grep '^ -- Joey Hess ' $(cat changelog-list) | sort +7 | \
              perl -ne 'print unless $seen{(split)[9]}++'
 -- Joey Hess <joeyh@debian.org>  Fri, 18 Oct 1996 00:07:24 -0400
 -- Joey Hess <jeh22@cornell.edu>  Mon, 9 Dec 1996 00:08:17 -0500
 -- Joey Hess <joeyh@debian.org>  Wed, 15 Apr 1998 00:03:14 -0700
 -- Joey Hess <joeyh@debian.org>  Tue, 24 Nov 1998 00:04:53 -0800
 -- Joey Hess <joeyh@debian.org>  Sun,  8 Oct 2001 15:52:04 +0200
 -- Joey Hess <joeyh@debian.org>  Wed, 31 Jul 2002 02:42:19 +0000
 -- Joey Hess <joeyh@debian.org>  Wed, 15 Oct 2003 21:21:49 -0200
 -- Joey Hess <joeyh@debian.org>  Tue, 23 Dec 2003 13:42:53 +0100
 -- Joey Hess <joeyh@debian.org>  Sun,  7 Mar 2004 08:10:44 -0900
 -- Joey Hess <joeyh@debian.org>  Sat, 15 May 2004 10:02:29 -0300
 -- Joey Hess <joeyh@debian.org>  Fri,  1 Apr 2005 07:43:45 -1000

Of these, I'm not sure what the -0200 and -0900 ones are.

BTW:

joey@dragon:~>grep '^ -- Joey Hess ' $(cat changelog-list) | wc -l
5985

Not a complete list alas.

what I got for xmas

I can't stand presents that I can't use up somehow (hate clutter) but this year I can't complain; I got: woolen clothing of various sorts, food, artwork, and bugfixes.

what a day

My god, what a day. It started out brutal, and then it just got mean.

weather rss feed at long last

Months ago I was unable to find a weather rss feed for anywhere except major cities. I suspected this was a business opportunity, and actually mailed wunderground and a couple of other sites suggesting I'd be willing to subscrie to them if they offered such a feed. On my daily visit to wunderground to check the weather today I was thrilled to see they'd added one of those (poorly designed) orange rss buttons.

I wonder if it will stay free since this provides both of their member benefits of "No Ads" and "Weather Email" without the $5/year. They've earned their $5.

ways not to help Debian release

I'm blogging about this because I'm very annoyed about it, and also because I think it points to a common set of mistakes Debian developers make.

Recently hotplug version -17 was uploaded to unstable. This version of hotplug made some changes that, apparently, break firmware loading when it's used with the stock Debian 2.6.8 kernels and udev.

At least 6 bugs were filed about this, oddly none of them were marked as Release Critical, so the release managers didn't really notice. The maintainer of hotplug and udev reassigned the bugs to the kernel, pinged the kernel team to fix their kernel, and then sat back and let hotplug -17 get into testing.

This is exactly the kind of thing that our politic DPL candidates this year are calling a "communications problem". Personally, I'd be considerably less polite in characterising this fuckup.

Please consider the overall effect of changes in your packages, especially packages which are part of standard Debian installations. Please look at the bugs you receive, and if they are actually release critical, fix their priorities. Please be on the lookout for situations where lack of action will result in release-delaying situations, and inform the release team of these problems in advance. Please, look at the big picture; your involvement with the Debian project should not stop at the borders of the packages you maintain.

wanted on a bumper sticker

walls

That didn't last long, the new computer got (literally) stuck inside a wall by the second day, which reduced the noise level to enough to not bother me during the day. I need to add some noise blocking stuff inside the wall to get it low enough for me to tolerate at night. I'm so broken when it comes to environmental noise, oh well.

Anyway, this ended up with the bits of the computer that arn't in the wall occupying parts of the bookshelf next to it, which is kinda neat. I especially like the flat panel monitor up on my bookshelf. Since I don't use the monitor for anything most of the time, I set it up to be a slowly changing slide show. Always something new and interesting from my photo library when I look over there.

I spent a long time today installing vmware on the new computer, which was so annoyingly proprietary. It's amazing what people put up with, with hard to navigate websites, cryptic installation instructions, binary patches to fix stupid bugs, and then a whole layer of license key fun on top of it. With vmware you even get the fun of upgrading your license keys independant of the software. Once installed it is a lot faster than qemu though, so I put up with it.

For some reason, I then decided to install the Windows CD that came with the computer in a vmware instance running on it, which meant I got to go through the whole crazy installer and license key rigamarole a second time. It takes more keystrokes to enter a windows license key than it takes to do a complete Debian desktop install! Amazing. And they use a font that makes Q and O and 0 and D all look like the same letter without a magnifying glass, so you might get to enter that key three or four times before you get it right.

The plan is to keep that snapshot of an installed windows system and use it to make sure d-i does appropriate things when confronted with a system running windows. Anyway, d-i runs nicely on the new vmware system, with a first stage install taking less than ten minutes. I've already found and fixed more d-i bugs today than I had in the last week or so, thanks to the improved environment.

wah

Octavia Butler has died, at 58. What a tragic loss. :-(

voting

Now the Debian project is apparently reduced to using the voting system to decide about issues like what architectures to include in the next release. If this really happens, then IMHO, it means that Debian is completly hidebound and incapable of functioning. Nothing has made me want to leave the project more in the last 3 years, or has seemed more likely to end in that than this most recent call for votes on adding amd64 to sarge. I am not interested in being part of a project that can only make technical decisions by long drawn out voting processes.

visited countries and states

Continuing a Planet Debian blog meme, the places I've been:

I've never been outdoors in Iceland, but I'll count it anyway.

Quite a few of these include airport or train visits. Of the unvisited states, Hawaii, Alaska, Maine, and Colorado are the only ones I want to visit someday.

(Updated: August 2006, added more countries. :-)

discussion

velcro

Today was the day of velcro. Forget about duct tape: Sticky backed velcro is a powerful thing.

I mostly used it to turn my equipment cabinet from a scary mess into a nice orderly space that can also hold my small working set of books, radio, and phone. Not bad for 3 cubic feet of space. Of course I cheated and accomplished this by mounting almost all of the boxen under the cabinet, and others to its ceiling.

Hmm, my linksys access point leaks all over the FM and AM bands, I can pick it up on the radio if I'm transferring a lot of data.

vanity book searches

Hmm, now with google books search even more solid vanity searches are possible. Looking up books that mention me and people I know is interesting, found several I wasn't aware of. Their OCR even catches text in screenshots, which could be amusing.

Less amusing is getting part of some GPLed code that one wrote, that somehow got published in a book, cut off partway through because they only have fair-ish use quoting rights on that book. And having to log in to be able to see even that.

vancouver

Well it's hellaciously early in the morning, and I'm off to Vancouver on the only flight that arrives there before midnight. At times I wish Bristol were on the black helicopter routes.

utf-8

I fought with xterm for a while tonight and seem to have found a utf-8 setup for xterm that is acceptable. Which is to say, it's completely undistinguishable from my old non-utf-8 setup unless I view utf-8 documents. Last time I tried this, about a year ago, it mostly worked, but I found strange display atrifacts in xterm. That seems to be resolved, the only thing I'm really lacking now is a 15 point double width font so I can display CJK and similar languages in my "large" xterms without changing to a font other than the one I'm used to. Since I don't read those languages anyway, this is not a priority.

I guess this means I will begin leaking random unnecessary globs of utf-8 into various documents and annoy anyone who hasn't switched to utf-8 yet, much as I was annoyed into making the switch. On the other hand maybe Colin won't have to clean up my changelogs after me anymore.

upstream.planet.debian.org take 2

I blogged about wanting someone to do this before; no-one did, so I finally have. Currently it's a temporary URL, since I do hope to get the hostname in the subject eventually. I also kinda hope that Mako will roll maintaining this into maintaining planet.debian.org, since it's currently just a hacked up version of that. (Yeah, I stole the CSS too. :-P)

Anyway, without further ado: http://people.debian.org/~joeyh/tmp/upstream.planet.debian.org/

Warning: Currently contains some Serenity spoilers.

Patches and additional upstreams welcomed. I think it's really fairly interesting to see the wider perspective of people who contribute to Debian upstream.

(Oh and BTW, RMS is not included because his blog has RSS syndication disabled, apparently because anyone using it would violate his copyright.)

upstream.planet.debian.org

Just an idea I had, an aggregator devoted to blogs of upstream developers of software in Debian seems like it could be an interesting complement to Planet Debian, especially due to the broad diversity of their viewpoints.

To start off with, it could contain:

• RMS (until he gets out of NM)
• Drepper
• Select members of Planet Gnome who are not already syndicated on Planet Debian
• Select members of Planet KDE likewise
• Bram Cohen (bittorrent)
• JWZ (xscreensaver)
• Alan Cox (kernel^WWelsh)
• Greg K-H (kernel)
• ESR (intercal)
• Nick Moffit (nwall)

Email other suggestions to me and I'll update this list, but someone else will probably need to take the initiative to set it up and make all the hackergoshis.

u.p.d.o update

Trying to balance out jwz just a little bit I added a semi-random selection of additional upstream authors to Planet Debian upstream, including some people from perl and gnome. It was hard to pick who from gnome to include, and it obviously can't include everyone from a large project like that.

I left out KDE because it was impossible to guess who was key to include as I'm not a user, and python because planet python is useless for finding interesting posters due to silly truncation. Suggestions? (Also looking for upstreams from smaller projects of course.)

KDS contributed a new look for the site, mostly to make it harder to confuse with p.d.o.

Also, Mako has promised to give u.p.d.o a proper domain name and setup eventually.

unread books

Interesting to read LIW on unread books:

Sherlock Holmes bought many books, but read few of them. He stored the books in stacks on his floor. I do no think that is a pleasant way to live. I prefer a state where I have read all the books in my home. In other words, I want to be Nero Wolfe, not Sherlock Holmes.

Most of the physical books in my house are books that I've read. There are perhaps two or three somewhere in my bookshelves that I've not read. It's nice to have lots of old favorites, sometimes a well known book can be a comfort. Sometimes I remmeber I've read the book, but most of the plot points and characters seem new to me on a rereading. Though this seems to be a rarer occurance now.

On the other hand, I've been condsidering getting rid of prhaps 50% of my books -- those books that I have read once or twice, and doubt I will want to read again in the next 5 or 10 years. I prefer that most of the books I read are new to me, and I have a collection of perhaps 30 unread ebooks.

The importance of having something fresh to read was borne out when a power failure made these ebooks unavailable, and I ended up rummaging through several other people's bookcases for something new to read after growing bored with my own collection of books.

The nice thing about e-books of course is that they can sit in a directory somewhere and piles of unread ones don't lean threateningly off of tables.

Reading: Mischner on Alaska, page 800-ish of 1000+, week 2.

unix tools, vidir

I'm a fan of the unix tools philosophy, but I sometimes wonder if there's much room for new tools to be added to that toolbox. New programs that fully partake of the philosophy (like rename, netcat, par, bloxsom, grepmail, and xwit) are fairly few and far between compared to all the new programs that don't. I've always wanted to come up with my own general-purpose new unix tool.

vidir doesn't quite qualify as a real unix tool, since it doesn't (normally) operate on stdio, but it's as close as I've gotten, and it does seem to fill in a nice hole in the toolbox. The idea is very simple, it just displays the files that are in a directory as lines in a text editor, then you can make any changes you like, and when you quit the editor it renames and/or deletes files to match your changes. No need to worry about quoting issues and the full ad-hoc editing power of a good text editor make cleaning up messy directories fun.

Like any good utility, there is plenty of obvious room for improvement, it would be nice if it had a --recursive switch to edit whole directory trees at a time. And it needs to have some support added for swapping pairs of filenames.

I challange others to try to make general purpose unix tools..

unix tools followup

Following up on my earlier post:

• Mark Hillebrand pointed me to some prior art on vidir, his mvedit is the same idea, with a different set of features including recursive directory lists and file name swapping, but not deletion.
• I noticed Mark also wrote a matchedit which "allows editing of regions of files that match a regexp in a single session"
• In the meantime, I've updated vidir, adding support for file swapping.
• Enrico Zini pointed out that his tagcoll is a true unix tool, that can process its own output and can probably be used for all sorts of things besides working with debtags. A good example of how to generalize an originally very specific problem.
• Colin Watson wrote about sponge, a tool no more complex than cat, that I am sure I have written in a line of perl at least twice, but never thought to save, and whose absence from the standard toolkit I've probably worked around hundreds of times.

This has only whetted my interest. Maybe the problem isn't that no-one is writing them, or that the unix toolspace is covered except for specialised tools, but that the most basic tools fall through the cracks and are never noticed by people who could benefit from them.

So, to provide some more incentive, I'd like to start a collection of new unix tools. I've worked on this kind of thing before with my collection of biff-type filters and Debian's devscripts. Sometimes just bundling up a set of programs consistently can add to their value. I think it can here too, once a critical mass of tools is assembled.

So, write more unix tools, and tell me about them so I can collect them into a package.

I just had too much fun with unicode, see filters 2.34.

ugh, web

I dislike writing CGI crap so much that it's really amusing that I'm writing a wiki.

Today I did most of the CGI parts of ikiwiki -- page editing in a web browser (with commit back via svn) and RecentChanges (pulling the logs out of svn). Still have to add logins to avoid spam, and after that I think I'll be done with the CGI side for now. It is nice to have the web-based editing interface for minor fixes to a wiki, though I've also found it very nice to be able to edit pages in vim for longer writing sessions.

I did come up with something interesting today when I wasn't doing all this CGI stuff. Since ikiwiki often needs to run with different privs than its invoker (to rebuild a wiki in a post-commit hook, or to check in a wiki page from CGI), I came up with an interesting way to securely run it setuid.

Pass ikiwiki --wrapper along with any other parameters, and it will emit a wrapper binry that is hardcoded to only run ikiwiki with those parameters. Dynamically programmed in C and compiled on the fly of course. The wrapper is designed to be completly safe to make setuid, and can be dropped into a post-commit hook, or cgi-bin, or whatever.

You can also regenerate a wrapper with ikiwiki $(./wrapper --params) --wrapper, which is useful for changing a param. So the wrappers have sort of turned into ikiwiki's equivilant of a configuration file, which it otherwise lacks.

I suppose I might as well put publish ikiwiki's documentation wiki already.

ugh

Had a little conversation on irc that was sufficiently psychologically draining that I'll not even try to do anything else today. It did help me see that I should not be maintaining debconf anymore; ITO filed.

Overfiend sighs. I always knew Joey hated the nasty kludge that was xserver-xfree86.config. I didn't realize he'd hate it so much that it would ruin a friendship.

People change. And some friendships, apparently, only work IRL.

udev weirdness

I found a little usb drive that's the same size as my nsul2, since running it from a usb key was getting a bit cramped. So now I have a system capable of being a Debian mirror that I can fit in my hand. Anyway..

Switching over to this drive was difficult, boot seemed to hang after init started, but with no console it was hard to debug. I eventually tracked it down to udev's init script. Apparently udevsynthesize somehow makes the kernel very unhappy about this usb device:

Jan 10 00:43:41 slug kernel: usb 1-1: reset high speed USB device using ehci_hcd and address 2 Jan 10 00:43:44 slug kernel: usb 1-1: device descriptor read/64, error -110 Jan 10 00:44:00 slug kernel: usb 1-1: reset high speed USB device using ehci_hcd and address 2

And goes on to refuse to do IO to it and mark it as dead. But this only happens if the drive is connected during udev startup; connect it later and it's ok.

I've stepped through the udev init script and, since I'm running 2.6.15, I was also able to step through /sbin/udevsynthesize. Sure enough, after I run this bit and wait a minute, the drive dies:

for file in $first $default $last; do echo 'add' > $file done

The variables are set earlier to every uevent in /sys/class, which includes lots of usb stuff, but from here it's black magic to me; I don't know whether it would be udev's or the kernel's bug that writing to these sys files makes the disk stop working.

Three other usb drives work fine with udev, so it's tempting to just disable udev on this one and ignore the issue.

udeb checkups

I spent too many hours today checking almost all udebs that include compiled binaries, to make sure they were built with size optimisatons (-Os and sometimes -fomit-frame-pointer).

First, I was suprised to find a few core udebs in d-i, built from our tree, were missing that. I guess it's not been emphasised enough and kinda forgotten sometimes.

Then I checked all the udebs that are not produced from the d-i tree. Many had obviously gone to a lot of work to build a size optimised version of the binary alongside the regular one for Debian, and I really appreciate that, since it can be a lot of work. For the ones that didn't, I did test builds and filed bugs with some numbers about how much size we'd save by turning on size optimisation. I hope that those bugs are heeded, if so we should drop quite a lot from d-i initrd sizes and also from the ramdisk footprint.

While I was looking at all these udebs, I noticed that many of them were still being built by hand using dpkg-dev, or with suboptimal uses of debhelper. Attention: debhelper has supported natively making udebs for a while now, just put XC-Package-Type: udeb in the udeb's stanza in debian/control and debhelper will do all the right things for you. It's pretty handy and will make your rules file cleaner.

There's still a lot of things I'd like to check the udebs for on a packaging basis sometime. Like making sure that none of them include md5sums files (I know some do), documentation (god forbid), or other cruft like preinst scripts or what-have you. BTW, debhelper will avoid most of those common mistakes for you.

Then at some point we need to go though all the dependencies and make sure udebs all express all their dependencies, including something sane for shared library dependencies. Once we figure out what "something sane" is in this context.

ucf and debconf

I found a few tuits tonight, so with Manoj's go-ahead, I changed ucf to use debconf for its prompting. This works very nicely, and will be a nice gain in UI consistency.

The only hard part was working around the variety of hacks that people have used in their postinst scripts to run ucf after starting up debconf. I note that I probably wrote less code to make ucf use debconf than the collective size of those hacks, and apparently none of the authors of those hacks ever considered that ucf might one day use debconf, or tried to provide for that happening.

While there seems to be no good way to detect the various hacks, I came up with a reasonable workaround: ucf detects if its caller uses debconf, and if so falls back to old-style prompting, unless its caller passes it a flag indicating that it has not screwed up debconf.

Sigh. One of these days, I will de-hackify debconf's confmodule code, and all these hacks on top of hacks can melt away.. Need more tuits.

ubuntu followup

Well I seem to have hit some nerves, thankfully most of them were just interested in the same issues I am.

Scott: Read more closely what I wrote -- "more developers cared about [...] what was in testing/stable, instead of just running their unstable systems". I quite agree with you; sorry I didn't make that more clear; but I had other fish to fry in that post. Unfortunatly I don't see how doing away with testing will work for Debian, that's one reason I alluded to your "smaller focus and being able to move faster".

ubuntu braindump

We had some interesting discussion on Planet Debian today. OtherJoey® wondered why Ubuntu has been able to release, while Debian 3.0r3 has not. Keybuck pointed out that stable point releases and Ubuntu releases have little in common. This does sort of beg the question of why Ubuntu has been able to release while sarge has not. There are some obvious reasons like smaller focus and being able to move faster, but is Ubuntu hurting or helping Debian overall? Well, probably both..

I haven't really written much about Ubuntu and my feelings about it and indeed my relationship to it. Mainly because I've been taking a wait and see approach on some things. Partly because I am probably the closest thing there is to a Canonical employee without actually being one, having been almost involved at the very beginning (until I made a different choice) and having spent time in real life with the whole group despite not working there. So I've not wanted to talk too much about it before. Anyway, I've not reached many firm conclusions, so much of the following is half-formed opinions that should probably be given little credance.

First, a gripe. I've been occasionally annoyed to see Debian developers (who are not employed by Canonical) installing Ubuntu instead of trying the distribution that they're supposed to be contributors to. Mostly because I feel that many problems in Debian could be better dealt with if more developers cared about how well the installation process went, and about how well the installed system worked, and what was in testing/stable, instead of just running their unstable systems and recommending using the commerical derivative de jour to others. When DD's do install Debian and get interested in that, things tend to improve. Fast. And of course I've put some work in over the past year on these issues, to the point that I'm actually happy with Debian's installer and happy with the default Debian desktop install. So it's personally annoying to me, though quite understandable, to read about Debian developers installing Ubuntu, recommending it to friends and family, etc.

Of course we hope that the additional work that Canonical has done on polishing the desktop and other things will be merged back into Debian. I don't know for sure if their patch directory is complete, but most of those have resulted in Debian bug reports with a good chance to be merged in. I've seen some of this happen first hand in the installer; we've gotten a fair number of patched from Ubuntu. I also know of a few patches in Ubuntu's installer that have not been fed back to Debian. Some of these patches look Ubuntu specific but really shouldn't be. Some of them are just buggy. Most of them are quite good.

I'd characterise Ubuntu's progress at feeding patches to Debian to be head and shoulders above any other commercial derived Debian distro I've worked with, and on a par with the best of the non-commercial CDDs (such as debian-br, debian-np, debian-edu, debian-med, etc). It's sure a lot nicer to get mail from a developer with a unified diff, rather than pulling apart binary .debs from a derived distribution who did not even tell me they modified my program, to see what they changed.

I feel like an utter nitpicker to say this, but there is a certian lack of followthrough in some cases; they could do better at repeatedly reminding some of us to apply their patches instead of sending a one-off bug report. (Though it's not as if Debian's any better at forwarding our patches upstream.) Probably they will have some incentive to do this if they re-branch from unstable for their next release and have to get all their patches applied again. Also, I'd rather see them more assertive about applying obviously correct patches in the cases where they are co-maintainers of the patched package anyway (hi, Kamion!).

One area in which they could improve is keeping us appraised of general Debian bugs that happen to be filed in their BTS. Several times recently I've learned of a bug and only later learned that it was already in their BTS. Sometimes this goes the other way too. As Ubuntu gains users, I've been noticing other information pop up on their site which has a broader bearing on Debian, for example this hardware compatability info on their wiki probably applies to d-i in general. I assume that if I had time to follow their mailing lists and irc I'd find even more valuable information. This is the kind of thing where a fork can easily end up costing Debian developers and users more time or less information. It's probably unavoidable.

I was unhappy to read of one Canonical employee and Debian developer who didn't have a Debian system anymore. He can set up a chroot, but that's really not going to be the same, and if many of the fine Debian developers hired by Canonical come to prefer using their own distro exclusively then I wonder how much incentive and interest they'll have to contribute to Debian going forward. That could be a fairly bad turn for Debian, because we strongly rely on many of the Canonical employees in their capacity as Debian developers. And it's not as if the same thing hasn't happened before with DD's who got involved in past distributions, like Storm. I suspect this particular case is not permanant and isn't due to any division from Debian but it could happen, and it's something to keep an eye on.

One of the reasons I do not work for Canonical is because I was worried at the time that a job there would not leave me enough time for d-i and Debian work. That was probably a false concern. The other reason, that I didn't want such a high pressure job, seems to have been spot on. So of course I've been interested to see whether working on Ubuntu will force some Canonical employees to cut back their work on Debian. Some of the people working for Canonical are as much my co-workers as folks working on the CDD that employs me. I work with them literally every day on a close basis and we have spent weeks this year working together in person. And if anything I've seen a lot more involvement from them in Debian than before. Although their unavailable periods seem to come in larger blocks due to certian .. craziness .. in their schedules, their overall availability seems better to me. Although I do wish Elmo could spend less time in London.

Wrapping up, I still have concerns and areas of uncertainty, and things I want to keep an eye on WRT Canonical and Debian. I have my ambivilances about this particular veritety of fork as opposed to others we've seen. I know that the Debian developers working for Canonical actually care about doing things right, and have probably some of the same concerns as I, as well as having the kind of feelings for Debian (both pro and con) that only long-term Debian developers can have. I hope they've given some of these things more thought than I have as a relative outsider, and I'll just wait and see how it goes, while mercilessly exploiting them as much as I can. ;-)

tron and udev

Thuroughly geeky evening here, I watched Tron and learned a little bit about udev.

I have to confess I've never seen Tron before somehow. Of course I knew about the light cycles, but I never knew why Edwin called his frisbee game "Tron". Heck I never even knew where that "on the other side of the screen, it all looks so easy" sound file that I used to have a boxen play at boot came from. And maybe one of these days I'll venture out onto #tron. Besides finally rounding out these appalling chinks in my geek cred, the movie was farly amusing and even with all that CGI has managed to hold up pretty well. Longer than the majority of CGI-driven films made in the last 10 years ever will.

udev was a little bit less fun, being installed inaverdently as part of a gnome install, which is a real misfeature of Debian if the gnome system in question is not one where hotpluggable devices matter, or if you do care about existing /dev symlinks, keeping the system running, and a /dev/null that's not a plain file. So I had to pause tron and read up on the thing enough to sort these problems out. My attitude toward this is still that of someone who has been once bitten (by devfs, particularly in d-i) and is now twice shy. Some other things I don't like too much about it:

• Undocumented, Debian-specific files in /etc that claim "this file does not exist".
• Docs that stress its low memory requirements while ignoring the 2.8 mb tmpfs filesystem it sets up -- most of it in /dev/.udevdb. Granted it can swap out, but this makes me worry about any future use of it in d-i. (And did I mention how we were bitten by adopting devfs?)
• Ugly, potentially insecure and FHS violating /.dev directories.
• People associated with the software who are advocating it as the one true way. (Did I mention devfs?)
• All that appalling ugliness with races and aritrary userspace based delays between loading a module and its device files becoming available that has led to so much joy with ps/2 mice and X.

Overall, it's almost as bad as people who blog about more than one thing in a single blog entry. Maybe like Tron I just need some time to asorb it all though.

trip from hell

I actually considered getting off the plane at Detroit and going back home, because even then I could tell this was gonna be the trip from hell. This was after two hours of sitting in the 100% full plane at the gate growing increasingly cramped and increasingly dreading the 8 hour flight to come. As the mechanics pumped up the landing gear a bit, then put some more fuel on the plane (for storms over the North Sea; shouldn't you always have fuel for storms?), then fiddled with the landing gear again, then took off luggage crates for passengers who gave up and went home. Which is half of why I didn't, I was not going to delay the whole flight by another half an hour. Besides, I want to go to Germany and hack, damnit. So I sat there as the flight attendants trotted out increasingly wacky explanations.

(BTW, I noticed that they name rows on the plane with my initials, which is weird cause I never noticed that before, probably due to being in my right mind generally when flying. Also the tube from Detrot's concourse C to the rest of the airport is .. trippy.)

But that was only the beginning of course, that hardly qualifies as a trip from hell. Of course I arrived in Amsterdam 10 minutes after my flight to Germany had left. I was told that every other flight today was overbooked already, and I could only go on standby. The one bright point is this let me meet Wichert in the airport again and chat a bit. Of course I didn't get on the next flight.. but this is where it gets strange.

After being passed through countless transfer desks and ticket clerks, one of them actually took a look at my remaining ticket. And it wasn't a ticket from Amsterdam to Bremen, but one from Detroit to Amsterdam. Yes, they had switched the tickets, and nobody apparently reads these things, and somehow I arrived all the way over here with no ticket to my final destination, with my luggage in limbo (she didn't think I had any for quite a while). At this point all the ticket clerks concorded that I was on a trip from hell.

At this point I gave up and convinced them to comp me a hotel and put me on a flight tomorrow that may not have passengers hanging out the cockpit windows. That will make the total roundtrip duration some 55 hours for a visit that will now be only 72 hours. Urgh.

trainwrecks

As you read this fascinating but edited-for-brevity irc log from #debian-boot, imagine yourself as a contributor to either the larger Debian project or the Debian-Installer sub-project. At what point do you begin to feel insulted?

<Chepre> can i talk to the chief of debian please. the whole project could earn some money
<minghua> Chepre: if you are serious, write email to leader@debian.org
<Chepre> thanks minghua. is there a way to get a "official" release created with all up-to-date modules, like for Intel ICH7 sATA-Controller...as long as i pay for it?
<minghua> Chepre: chances are small. the official release is a long process
<minghua> and it doesn't happen often
<minghua> the earliest date you can expect for next release is probably end of 2006
<minghua> so you would have a better chance to hire some people/company who knows debian and provide you an updated kernel
<Chepre> OMG? does debian have not enough coders or don't they have "fun" to keep it up-to-date? e.g. the intel ich7 is already very old
<Chepre> well, can i hire the debian-team?
<minghua> Chepre: well, there is no official "team", but sure, you can hire debian developers
<gravity> Chepre: The stock kernel doesn't support your device? Why not? Is the driver not in the mainline kernel tree?
<Chepre> no
<gravity> Why not?
<Chepre> its just a default Dell PowerEdge 850 with Intel ICH7 and it isn't supported
<gravity> Chepre: Well, did you send an installation report?
<Chepre> gravity: no
<gravity> Chepre: That'd be a good first step to getting your problem fixed for future releases
<minghua> Chepre: ah, then it's supported by stock kernel after all, did you say it only works for kernel > 2.6.12?
<Chepre> minghua: 2.6.8-2 doesnt support it
<minghua> Chepre: okay, so as gravity said, write an installation report so that people know your problem
<Chepre> well, we cant wait month or years for a fix
<gravity> Chepre: You can create a custom installer image with the updated kernel for your work
<Chepre> we have to install many server in the next weeks :(
<munghua> Chepre: then I'm sure if you offer enough money, some d-i people can provide an installer with a newer kernel for you
<Chepre> well, i already got an custom installer but its a bit buggy...i need a whole netinst with the same "quality" like the official releases got
<gravity> sounds you were at the right track at the beginning... though "chief of debian" is probably the wrong person to contact :-P
<gravity> You could send a mail to debian-boot I suppose, offering the work for what you need
<Chepre> well, i also need up-to-date software, not just only the installer :( so the "whole" debian-team has to stop current work and has to start creating a stable version which is newer than the current unstable
<Chepre> minghua: do you think 50k are enough to get a 3.3 as stable?
<minghua> Chepre: by "up-to-date software" you mean all the software, including desktop, web servers, and stuff?
<gravity> Chepre: You may want to check out an alternative source. HP, as I understand it, provides the kind of support you want. Progeny does as well. And Ubuntu has their own full derivative with this sort of thing
<Chepre> minghua: right, as long the software is not a beta :)
<gravity> Chepre: The whole Debian team is not the kind of project that'll do this for you
<Chepre> so the debian-team dont want to earn money?
<Chepre> gravity: i know, open source :) ... but i think the guys making a new "official" release for me whould be very happy getting up to 20.000 us$ for one month of work
<gravity> Chepre: It would violate a lot of principles and the stable release policy to do an official debian release. You could get what you need with a name other than debian though, with some of the companies I listed above
<Chepre> where is the problem? i give you the money and you give me an up-to-date release which is only for my company, not for public
<gravity> I know I'd quit the project if there was an official Debian release for cash
<minghua> and there are like what, 10,000 packages in sarge now?
<gravity> Chepre: That's what those companies are for. Debian doesn't work like that
<gravity> Chepre: If you want to talk to individual debian developers for that, you can do that too
<Chepre> hmm, ok....now i know why the d-i doesn't recognize smp or 64bit systems

Now imagine yourself as someone who is trying to install Debian on twenty thousand stock machines for work and it's not working. At what point above did you begin to feel alienated?

(Assuming that "Chepre" is not trolling,) Neither side in this exchange is particularly meaning to insult the other, both are doing what is right and obvious from their perspective. Two ships passing in the night, two trains speeding down a track, I don't know. It's a shame we can't deal better with this sort of thing. I would be truely interested to see how other free software projects deal with it differently.

The rest of this is my attempt at some analysis and is probably not very useful..

A few mistakes Chepre made:

• When trying to communicate with someone, it's helpful to try to understand their motivations. Maybe these people are not highly motivated by money. Maybe spending 1% of your budget to buy the right someone a computer exhibitng the problem would be more effective.
• If a coder asks if you're filed a formal report of your problem, and you haven't, and you don't understand the underlying problem and rebuff their attempts to get a grip on it, escalating the issue to upper management will not make them your friend.
• Trying to pre-solve your problem by setting requirements like "it can't be a beta" and "it needs to be a 3.3 release" and "it needs to be newer than the current version" will only result in the wrong problem being solved, and smart people will not want to get involved in such a process. If you plan to hire someone to solve a problem it makes more sense to give them a broad vision of the problem to be solved, some example use cases, and let them, not you, solve the problem. That's what you're hiring them for.
• Not doing basic background research. And letting it show.

A few mistakes the channel (irr-)regulars made:

• Not recognising that "it will be done at the end of 2006" is not an answer a user wants to hear, and not pointing out alternatives such as last month's beta release or today's daily build.
• Assuming that when a user asks for a release, they necessarily want what we consider a "stable release". Perhaps they want a release that has some form of support and is free of showstopper bugs; what Debian terms "stable" is not the only release that necessarily has those characteristics, depending on how it will be used. (In other words, holy penguin pee (aka "official") might be more plentiful, and more symbolic, than you think.)
• Not pointing Chepre to the Debian consultants page. (Minor, but it's possibly the best place to have mentioned.)
• Getting offended. And letting it show.

too much cvs/svn

I glanced at my inbox in mutt, and saw the C flags indicating CCed messages as svn up conflict indicators.

too mainstream (RIP brick)

I destroyed my openbrick computer today by plugging a laptop's power supply into it. Too many volts went into my poor little server, and though I did appreciate the way some unknown hardware hacker made the system buzz in morse code in this case, it was too late to save it.

I'll try to fix it later, but today I had a house to wire(less), and so I ended up buying an off the shelf wireless router. I've never configured one before; my home wireless has always been limited exclusively to linux access points. In many ways it's impressive everything this little box can do, and how slick it is, until one runs up against its shortcomings. Like not being able to assign hostnames to clients in the dns server. And not being able to check a plain text config file into svn. Everything so far has been worked around, but I have the same queasy feeling I got the last time I walked into a McDonalds and ate a Big Mac.

today's d-i hacking

Today was fairly restrained. Although I did upload linux-kernel-di three times. But really nothing much to do turned up until Manty found a problem with CD installs that made them skip configuring the network. That turned out to be a bug caused/exposed by Joshk's recent main-menu change that makes it not jump backwards to before completed menu items. Changing two numbers in d-i fixed it. It seems that in d-i, 90% of the work to fix a bug is simply in reproducing it.

Testing that change, I tested installs from floppy (in vmware), and that is much more slick than it used to be. Also tested install from a real CD (first time in ages), and I even booted using a 3 mb initrd on a keychain and then installed from CD.

We've just put the bterm unifont reload issue for Chinese and other similar languages to bed, which is kinda scary as it involves the first code change to anna since November. But this was the last item on the release checklist that does involve ports, testing, or archive work, so it's good to see it fixed.

Now we just need to get more arches ready. In some cases that involves only testing. I have thought about mailing the debian-arch lists to try to get testers, but I guess I'll try this blog entry first. If you use powerpc, ia64, sparc, m68k, hppa, mips, alpha, or s390, please try our current daily built images for your architecture from our web site, and send us an installation report. Otherwise, your arch may not be included in the next release. All of these have a nonzero chance of working, I need to know if the probability is in the 0-50%, 50-75%, or 75-99% range.

Ah yes, something interesting. Kamion was on #debian-boot today and we chatted about him making a ssh-udeb. This won't be ready in time for the upcoming release on the 15th, but when we do get it, it will be pretty neat to be able to ssh into running d-i systems! Especially for debugging those hard to reproduce problems.

to the lady with the pen on flight 6053

Hey, I don't know who you are and I can't imagine you'll ever read this random Missed Connection ad in the vast classifieds that is the internet, but the thing with your pen has been driving me crazy. I mean, I have a laptop, which is expensive and which I rely on for a lot of things (which is one of the reasons I often don't have a pen on me). That doesn't stop me from letting other people use it, even perfect strangers, as long as I'm reasonably sure they won't steal it, or break it, or read my email. Sometimes I'm nervous when someone is using my laptop but that doesn't stop me from lending it, or being glad I can help with the loan.

You, apparently, have one amazingly fancy and special pen. So much so that you can't lend it to a person sitting next to you on an airplane. I didn't ask why it's so special, but it's fun to speculate -- solid gold? Contains your mother's ashes? Built in james bond spy cam? Whatever.

Here's the thing that blows my mind. You've saddled yourself with this thing, and by doing so you've restricted yourself, you've cut yourself off from a simple friendly thing like lending a minor possession to someone. Which is really more important, owning this object, or being able to use it in all the ways I can use a 10 cent Bic?

Call me dangerously naive, but I totally don't understand this, and I can't grasp the mindset that would consider that a worthwhile tradeoff, and like I said it's been making me a little crazy from time to time trying to get inside your head.

Anyway, if I ever run into you again, I'd be happy to lend you a spare pen for strangers.

tired

I started hunting for a house to move into in earnest today. Stuff is so cheap here compared to the Bay Area that it's almost surreal.

d-i is booting to the menu on 2.6, but unable to load kernel modules. Traditionally with d-i ports, booting to the main menu is about 1/3 of the total work. Will it be the same here?

<Hegg> woo hoo! booting got farther! Now it's a blank screen! :D

I love working on d-i, because I love supporting problems like these. Or something.

I'm exhausted for some reason.

Here's how to subscribe to mailing lists with a combined total posts of 2000 or more per day, and live. It's all about pattern recognition.

Threaded mail readers such as mutt sort emails into a tree based on which emails are replies to other emails. The mail index displays this tree of messages with the author of each indicated. Since the Subject field rarely changes, mutt does not display it unless the thread's subject changes. Combining these elements of author names, thread trees, and occasional subject changes produces patterns that the brain can use to quickly identify important and useless sets of mail messages, without actually reading anything, and without resorting to complicated and fragile killfile and scoring systems.

Some patterns are very simple. For example, this is the "take it to private email" pattern. This pattern can be thread-killed on sight; if you're feeling generous, read the last two messages in the thread.

bob    +-> Foo
fred     +->
bob        +->
fred         +->
bob            +->
fred             +->
bob                +->
fred                 +->
bob                    +->
fred                     +->

This is the "think before you post" pattern.

bob    +-> Foo
bob      +->

This is the "blindingly obvious answer" pattern. If you're bored you can read any one message to get the gist. Sometimes many of the replies will be worth reading to get a broad overview of all possible responses to a given obvious question. Most times not.

bob    +-> Foo?
fred     +->
alice    +->
george   +->
nancy    +->
steven   +->
nick     +->

Here's a deceptively simple pattern, the "uninteresting message". It might be worth checking the post date to make sure that it's not just the beginning of one of the above patterns of course. And if you're feeling generous, you can read some small fraction of them.

bob    +-> Foo?

This might be a worthwhile thread. Notably because it's short, has a reasonable mix of participants, and seems to come to some kind of conclusion.

bob    +-> Foo
fred     +->
alice    | +->
bob      |   +->
george   |     +->
bob      |       +->
nancy    +->
fred       +->

And here's a transformation of that same thread featuring "too late for the party syndrome" (or a "filibuster")

bob    +-> Foo
fred     +->
alice    | +->
bill     | +->
bob      |   +->
george   |   | +->
bill     |   |   +->
bill     |   +->
nancy    +->
fred     | +->
bill     | | +->
bill     | +->
bill     +->

This is a perfect time to er, threadkill bill. Or if feeling generous, read the second or third of his many replies to see what's got him so worked up.

To see more sophisticated patterns, you need to categorise participants of the mailing list. A few people almost always manage to turn even a useless thread into something interesting, or have viewpoints that are good foils for your own. Learn to recognise the names of these interesting posters without thinking about it (I'll name them in uppercase below). Similarly, some people never contribute anything useful, and can be filed under the label of k00k. If there are more k00ks than interesting posters it's probably time to leave the list. Most people won't fit in either category. Some people change category depending on the subject.

Here's the same potentially worthwhile thread as before, but now the people matter.

bob    +-> Foo
fred     +->
k00k     | +->
bob      |   +->
george   |     +->
bob      |       +->
NANCY    +->
fred       +->

Suddenly, three messages of the thread become worth reading, and the larger k00k-tainted subthread's likelyhood of being valuable drops substantially.

Similarly, this thread can be ignored.

k00k   +-> Foo
bob    | +->
fred   |   +->
k00k   |     +->
alice  +->
k00k     +->

And this thread has a high potential for being interesting.

bob    +-> Foo
fred     +->
bob      | +->
NORRIS   |   +->
george   |     +->
bob      |       +->
NANCY    +->
fred       +->

This thread, a variation of "take it to private email", increases bob's chances of being a k00k. Or he's really right and everyone else is wrong.

alice  +-> Foo
fred     +->
bob        +->
nancy        +->
bob            +->
alice            +->
bob                +->
fred                 +->
bob                    +->
fred                     +->
bob                        +->

This thread is a hard case to call, it depends on how good the interesting poster is in deflecting k00kitude. There could be as many as three worthwhile messages in here; I'll flag them.

  k00k   +-> Foo
  bob      +->
* NANCY      +->
  k00k         +->
* bob          +->
* NANCY          +->

It might be worth reading the interesting poster here, but probably only for a laugh, and it's just as likely that they've been cruelly trolled into answering, especially if an instance of the "blindingly obvious answer" pattern is grafted onto one of the subthreads.

k00k   +-> Foo
NORRIS   +->
k00k       +->
NORRIS       +->

So far the Subject has been ignored, and generally it's more important to look at the thread structure than the subject, but there are useful exceptions. It might be worth checking out the differently named subthread here, on the other hand its close proximity to the k00k likely means its a useless tangent.

bob    +-> Foo
fred     +->
k00k     | +->
bob      |   +->
george   |     +-> Bar (was: Foo)
bob      |     | +->
fred     |     |   +->
fred     |     +->
tony     +->
fred       +->

Here is not just one thread, but a collection of threads. This pattern, typically associated with a much larger thread than shown here, is a sure sign that a thread has gone pear-shaped and unless the followup thread has a lot of interesting posters in it, there's not much hope for the lot. (k00ks also spawn disconnected threadlets; such threadlets should be ignored.)

 k00k   +-> Foo
 bob      +->
 NANCY      +->
 k00k         +->
 bob          +->
 NANCY          +->
 bob    +-> About foo..
 tony     +->
 k00k       +->
 alice  +-> That foo thing

That's enough patterns to get started with, but they really come into their own when picked out of larger threads in an overloy long ongoing discussion. A few examples can be found in this segment of 23 lines of a much longer thread.

bob     |   |             |         | | |           |   | 
k00k    |   |             |         | | |           |   | 
tony    |   |             |         | | |           |   | 
k00k    |   |             |         | | |           |   | 
tony    |   |             |         | | |           |   | 
k00k    |   |             |         | | |           |   +-
bob     |   |             |         | | |           |
k00k    |   |             |         | | |           +-> Bar
john    |   |             |         | | +-> Bar
nick    |   |             |         | +-> Bar
barney  |   |             |         | | +->
nick    |   |             |         | | | +->
nancy   |   |             |         | | |   +->
NORRIS  |   |             |         | | |     +->
nancy   |   |             |         | | |     | +->
SAM     |   |             |         | | |     +->
nick    |   |             |         | | |       +->
NORRIS  |   |             |         | | |       | +->
oliver  |   |             |         | | |       +->
john    |   |             |         | | +->
frank   |   |             |         | | | +->
john    |   |             |         | | | | +->
tom     |   |             |         | | | |   +-> Mumble
bob     |   |             |         | | | |     +->

Part of this thread is useless by two criteria; it has far too much k00k in it, and it's nested so deep that mutt doesn't bother to try to draw the thread indicators. The next bit though shows some signs of being interesting despite its fairly deep nesting. And the tangent might be worth keeping an eye on.

Those are the thread patterns that I know I know. I suspect there are many others that I also know, since I can typically read mail with the d key as fast as mutt will scroll, stop at the interesting spots, and not miss much. If you know of other patterns, send them my way.

Finally, even if you prefer to read every post, please do us all a favour, imagine your reply is already in the thread structure, and look for patterns like "take it to private email", "blindingly obvious answer", "too late for the party syndrome", and k00k influenced posts.

thoughts (less random than expected)

This time of year it's great to curl up in the evening with a feather comforter and a good text file.

I recently read all of Andy Hertzfeld's stories about the origin of the mac on folklore.org, pretty much straight through in chronological order during one long night. Only at the last story did I find out that he'd turned the website into a book.

Reading a whole book in one sitting online is not a novel experience for me, though it seems to be for many, but doing it by accident is. My enjoyment of quantities of text online does sometimes come out in my own weblog postings, which are probably long enough, if not good enough, to be their own book.

This weblog at least occasionally violates 8 of Nielsen's 10 weblog usability guidelines. The only one I think I should fix is point 5.

Another book I read online recently is Karl Fogel's Producing Open Source Software. The contrasts between how Karl suggests doing some things and how projects I'm involved in work are sometimes large (he suggests voting on technical issues for example), sometimes due to differences in technology (the debbugs bug tracker makes conversations and other email-based use of a bug tracking system work much differently than the BTSes he describes using), and sometimes pointed to things that could be improved. More often though, he seemed spot on in extrcting and highlighting best practices. For example, he does a great job in pointing out that simple, consistent naming conventions, like #404093 for a bug report and r1021 for a repository revision, are more important than they appear.

I also enjoyed reading some reviews of Katamari Damacy, which sounds like a truely unique and fun game. I'm looking forward to playing it (probably under emulation) one of these years.

this and that

My dad's spending the night here then we're going out to Anna's new property tomorrow and building a bridge and some other construction work. I look foward to the change of pace. d-i has been keeping me hammered, and I've not been getting enough sleep.

I guess this is good timing anyway, since alioth is moving to a new server tomorrow, and will be down for part of the day anyway, so no commits.

The i386 root floppy is out of space again, so I've dropped more kernel NIC modules from it to make space for more translations. This is continually annoying.

Partman seems to be in pretty good shape .. on i386.

LWN has added full article body's to one of their RSS feeds on my request. Thanks guys! So the number of websites I still have to visit daily is down to three now.

The worst build dependencies in Debian

We really have to look at things two ways to pick the packages with the worst build deps. Crazy shell helps too:

joey@dragon:~>lnpdb () {perl -ne '($f,$v)=/^(.*): (.*)/; $p=$v if $f eq "Package"; print length($v)."\t".(($_=$v && s/, //g)+0)."\t$p\t$v\n" if $f eq "Build-Depends"' /var/lib/apt/lists/*Sources}
joey@dragon:~>(echo "length pkg count pkg"; paste <(lnpdb | cut -f 1,3 | sort -rn) <(lnpdb | cut -f 2,3 | sort -rn)) | head -20 | column -t
length  pkg                  count  pkg
2181    gcc-4.0              56     vlc
2059    gcc-snapshot         56     debian-installer
1649    gst-plugins0.8       55     camorama
1545    gcc-3.4              52     openoffice.org
1143    debian-installer     51     gst-plugins0.8
1040    vlc                  47     gcc-4.0
1012    gcc-3.3              47     boot-floppies
968     boot-floppies        44     gcc-snapshot
889     openoffice.org       43     gcc-3.4
828     qt-x11-free          41     php4
761     php4                 39     php5
739     camorama             37     kimdaba
737     gnome-applets        36     pike7.6
715     php5                 35     gcc-3.3
706     muine                34     uim
680     qt4-x11              32     pike7.4
668     gtk-sharp2-unstable  29     qt4-x11
637     pike7.6              29     qt-x11-free
614     gtk-sharp            29     gnome-applets

gcc wins by virtue of insane arch lists, vlc for depending on every library on earth, but d-i really competes on variety and breadth of build deps.

the tree boats

Of course after whinging all week about needing to get up early for work, I end up a) waking up same time on Saturday and then b) getting woken up even earlier today for a long morning of canoeing on the river. We visited a very interesting area where many large wooden barges were left to rot in a backwater many years ago and are now submerged and covered in trees and other growth.

It was also pretty hot, unfortunatly that part of the river is also downstream from a power plant and periodically gets much warmer than any water not in a bathtub has any right to be.

All that wiped me out and it was a hard drive back from Richmond.

the slowly dying laptop

Yeah, my Fujitsu laptop has been dying for a year or so, it's had surgery for power, batteries, hard drive, sound, etc, it's lost every bit that could break off and still have a usable laptop (all the suede on the bottom, the lid fasteners, the pcmcia blcker, the case around the base of the lid.. the still working pcmcia eject button is my one remaining sacrificial breaky bit). It's got the marks to prove cats love it, along with all the results of throwing it in my book bag and taking off with no concern for carrying a laptop around. The screen has 4 or 5 dead pixels and recently one faint patial dead scan line. The lid is getting worryingly wobbly. The CD drive was once crushed in an airplane and then banged back into shape so it still fits in the machine and even ejects OK. I've rubbed the letters right off some of the keys on the keyboard. The speakers turned black and exuded some glue or something long ago during a hot summer, one of them is barely audible. The headphone jack broke. Oh and the laptop is also abused regularly by programs like firefox and openoffice that really didn't expect to run on a 900 mhz crusoe.

All that, I can live with. The laptop bears its marks with pride, and amazingly, people still occasionally ooh and awe at it. Although more likely from a bit of a distance these days.

The newest wrinkle is that I'm beginning to exceed the design specs for the keyboard. Apparently they just didn't design for it to be used so much. First some keys stopped working half the time on some days, yeilding the famous irc sessions where joeyh tlks without ny A's. Then the Function key's membrane stopped pushing it back up, so it always appears pressed, though still works. Now the Alt key is going the same route and the right side ofthespacebardoesn't work, leadingtoircsessions whereItalklike this. And it just feels like I can't touch type on it half the time anymore.

A new keyboard is $100 or more. I think it's time to retire this to a d-i test machine and start looking for a new latop.

I looked at the IBM X-41 again, really like it, but still can't get past the low res display and fan. Especially with some reports that the fan runs constantly in linux. I looked at all the stuff on dynamism, but wasn't impressed enough to consider it. At the moment the Fujitsu P7120 is looking mighty appealing. Hey, it's even got the suede bottom again. And no fan.. Only problem is it has a touchpad.

discussion

the release problem

In trying to manage and release d-i, I time and time again run into problems that we have in overall release management in Debian, in miniature. While we've avoided whole classes of problems by being a smaller group, and by having different takes on who can do what with packages (which has also brought with it its own unique problems), the similarities are often striking, and none have been as obvious as in the last week.

Last weekend we decided to try for a release at the end of the month, and to have a string freeze this week. If that was a real goal, the sane thing to do would be to stop making changes except when absolutely necessary, to talk to others before making changes, to avoid committing lots of unpolished strings on the day the string freeze began, and so on. But instead we've gotten a lot of commits of new code and new strings. Some people managed to do this without breaking anything or causing any trouble; most people didn't. We're now already behind schedule. I was happy with every single day's build all last week; so far this week today's build is the first one that seemed anywhere close to releasable.

Of course this is all just human nature, and part of me is glad to see an uptick in work on d-i, even if we had to set a deadline to kickstart it. Many of the changes that are being made are indeed important, and were being put off. And it can't help that it's been more than 2 months since the last release, so we have increasing pressure to make a release, and increasing pressure to get every conceivable improvement into this one.

When this happens in Debian proper, we find it very hard to not let the schedule slip again and again, as new versions of gnome and kde, new installers, and so on are put into the distribution, and as we work to find and stamp out all the resulting bugs. Our users have very specific expectations about Debian releases, and while they don't expect speed from us anymore, they do expect a certian quality, they expect as much as possible to be fully up-to-date at release time, they expect it to be supportable and supported for a long period of time. And we can't afford, or haven't the guts to try deflating those expectations for the sake of a quicker release.

I think that with d-i, we have the luxury of taking different approaches. The pressure for a release is there, so we will have a release, and soon. The quality of that release will depend on what people do between now and then, if it's worse than the last one then some lessons may be learned.

the md5 thing

Like Gunnar I've been following this md5 isue over the past couple of days. Already I've been checking things like my backup software (duplicity uses sha1 so it's ok) and revision control system (svn uses md5 so I cannot safely keep these fun new files in svn; since I keep my mail in svn and my mail already includes examples of these files, that's difficult).

I'm glad that Debian has sha1sums in our Release files along with the md5sums. That means we only have one key md5sum in the chain between a file in the archive and the Release.gpg: The one in the Packages file. Unfortunatly, this does leave us vulnerable to some things.

At the moment if I wanted to exploit this stuff to harm Debian, the best I could do would be to upload a .deb with something like the "ice" from the paper appended to it. dpkg ignores trailing garbage in an ar file. This md5sum would go into the Packages files, and the sha1 of the Packages into the Release file, which is in turn gpg signed. Then I just have to hack a mirror and replace my deb with a new version that has "fire" at the end, plus perhaps a payload. The md5sum would be unchanged and the small number of users who use apt-secure would not notice the replacement.

But for this to be really useful some program in the package, or perhaps in a different package, would need to look for the deb in the local apt archive, check for "fire" at the end, and run the payload. Which is problimatic, becase that code would be subject to audit, and once someone noticed it or noticed the junk appended to the signed deb I uploaded, it would all be traced back to me and I could kis my developership goodbye and look forward to some time in court.

So doing this rather than just modifying the mirror arbitrarily and anonymously seems like a poor approach, unless I think I can hide things sufficiently well and want to attack the apt-secure users. There are so many easier scenarios until we get apt-secure deployed that this hardly matters, and adding sha1 sums to the Packages files should be fairly easy to do.

the first step in writing a successful GUI library

Is this..

mygui_init (void) {
    if (! connect_to_display()) {
        fprintf(stderr, "Cannot connect to display. You lose.\n");
        exit(1); /* no way to recover from this horrible state */
    }
}

Well, it worked for both GTK and Qt..

the farm

Today was a gorgeous spring day, heavy rain, leaves bursting out and everything turning green so quickly, guests down the hill to see the spring wildflowers near their peak, hot enough in the afternoon to feel it, but not sap my energy, and cool enough in the evening to make a hot bath enjoyable and worth the work. I relished much of the day, and despite loving many moments of it, I'm still ok with my plan to leave the farm. A couple of people have asked why I made that decision, and it's hard to explain.

When people come out here, after being away for a year, or ten years, the most common reaction is pleasure that nothing has changed. Now sure, things have changed -- this tree has fallen down, that one has grown, a wifi antenna has sprouted up on the side of the house, and there are computers hidden around. This bridge has fallen apart, that one has been rebuilt; the barn has been remodeled into a chicken coop, the kichen floor has taken a turn for the worse, horses no longer roam in the upper field, and the pond has silted up. But the quirky essense of this place remains the same, peoples' relationships with it remain the same. There is a timeless quality that provokes a sense of nostalgia even while living here day to day.

I've lived here for three years, and they have been three of the richest, deepest, most memorable years of my life, after which the boom years in the Bay Area pale, and the time before and college are only distantly remembered shadows. Only when I get back to my childhood do the memories have the vibrancy and newness of my first summer here. And I guess that's the thing; my next two summers here, the winters, are all blurring together, life here has lost that edge of excitement and exploration, and settled down to a routine, to a cycle. That has its undeniable attractions, but I'm finding I'm still too young to give in to them, and so I've been drawn off to find something new, both by exploring abroad for increasing amounts of time, and now, by moving.

I'm savouring every last day, and the end will not be easy -- not the least because I have to lug all my stuff back up that hill! -- but after all, it's not the end at all; I know for sure that sometime I'll return after a year or two's absense, and exclaim, "Why, not a thing has changed!" That certianty is a blessing.

thanks guys

I want to thank everyone who's gone out of their way to prove that I was spot-on in an earlier entry about us seeing deadlines as an excuse to break everything the night before.

Just please next time, don't break the debian-installer builds in the process.

thanks..

I think I may voice the sentiment of half the dorm in my appreciation to the Ubuntu folks, and presumably also the .uk guys, for their musical performance this morning. You have talents that should not be confined to HEL. Or something.

This is probably the most thoroughly dead of any of the free software I've written. I can't even dredge it out of my subversion repository, although I know the code is in there somewhere.

The idea was to lean about writing a MOO and make a MOO that users didn't need to learn a new programming language to develop in. It was a really special experience to write perlmoo. I'd been wanting to write a MUD ever since I first saw an article about them back before I was on the net. I messed around trying to do it in pascal on the 286 laptop I was using back in the early 90's.

When I got online I spent a bit of time in some MUDs and MOOs, including PernMUSH and LambdaMOO, but I never really got sucked in as a user like some people did. I found it more interesting to run my own local copy of LambdaMOO and mess around with it than have limited access to a larger moo.

Finally figuring out how to make a completly dynamic world online, with multiple users able to log in and create new things and find their own reasons to be there was a great experience. And people really appreciated the aspect of being able to talk to the moo in a real programming language, so I knew I was onto something.

What killed it was the architecture, trying to run everything inside one perl process and rely on perl to enforce inter-user security. Once it became clear that the security and scalability issues weren't really surmountable with the existing design, I was able to let go, and dropped the project when I changed jobs. Although I did rave about it at a job interview, and ended up joining a mailing list run by the interviewer. On designing MUDs. Weird little world.

A learning process, and I'd never have been able to write mooix without doing perlmoo first.

Next: ten years of free software -- part 10 shoop

This was a tool to work around a problem in Debian, namely that there was no way to specify gcc flags in a way that would be sure to work for every package that was built. I didn't use it much once I was done with that problem, but a lot of other people did. I kind of wish I'd not written it now, since the problem it worked around is stil there underneath, just paperered over by this kind of tool. Oh well.

This is one of the first peices of software I gave up to let someone else maintain, and I didn't handle it very well, just dumped it. Now I see it's had several maintainers, NMUs, and no updates in three years. Oh well, live and learn.

PS: The name? It sucked. Especially when people started using it to optimise builds for more recent hardware.

Next: ten years of free software -- part 9 perlmoo

wmbattery is the only end user GUI app I have ever written for linux. It's the only program that I forked from someone else's existing program (if you don't count debhelper). And it's the only program I've done that was partly designed in Microsoft Paintbrush in Windows 3.1.

A lot more people have ended up using it than I ever expected, and the coolness factor of seeing it on someone's desktop makes me understand why end user graphical programs are so appealing to a lot of programmers.

I think it succeeded because it worked well, was intuitive to understand, looked pretty neat, and was influenced by the underappreciated "calm computing" movement.

It's lived for a suprisingly long time, for a graphics program that was after all designed to be used with just one or two specific window managers. I still have it running on my ion desktop, where its mataphore is completly out of place. But it's sunk into the background for me, and I suspect for other users, and it's not worth giving it up for something that's a better fit with the rest of the desktop.

Oh, MS Paintbrush story is that I was home for Christmas, the computer at home was running Windows and I ended up forcing my sister Anna to help me do the graphics for wmbattery a pixel at a time in Paintbrush.

Next: ten years of free software -- part 8 pentium-builder

Well, it's probably time to talk about some software that I wrote that didn't end up going much of anywhere.

I guess that ticker failed both because it was written for an absurdly small audience (people who enjoy either 1 line high xterms, or running a patched version of splitvt on a dumb terminal), and because it missed the boat on the whole syndication thing that's made this whole blog thing such a big deal.

Looking back through the changelog, I'm mostly glad that I didn't spend too much time coding it. I used it for a little while, and then I maintained it for a little while more, and then people stopped writing me about it.

Oh, and another stupid name for anyone who's keeping count..

Next: ten years of free software -- part 7 wmbattery

I started work on debhelper 8 years ago and it's still one of my best known contributions to Debian. At the time, we had a tool called debstd that was clearly meeting some important needs, and was catching on like wildfire, but also clearly sucked. I managed to come up with a design that was cleaner and more flexible, while also having an easy migration path. Today it's a rare Debian package indeed that is not built using debhelper.

A few things stand out in how I did debhelper compared to free software tools I'd written before. I posted a proposal first, and used a lot of feedback to refine the design before I got started. I set down some design fundamentals that I've not wavered from at al all through the whole life of debhelper. I explicitly targeted users of debstd, and made it easy for them to switch over. I spent a lot of time tracking the market share of debhelper versus its competitors and working on inproving it.

If there was an early period when I was just making debhelper work well enough for me, and a middle period when I was working to make it dominate its space, we must now be in a late period, where it pretty much works for everyone, it's important not to break it, and most of the pressure is to add edge-case features, and new commands for new needs.

I've managed to avoid some of the problems in this period of debhelper's life cycle by adding the concept of compatability versions -- essentially the same as library symbol versioning -- to debhelper. This lets me make big incompatable changes without breaking things that need the old behavior, at the expense of some cruft. Still, debhelper has unavoidably developed some inertia now that it's mature, and some of the dynamicsm and excirement that was one of the things that drew people to use it is gone.

There's still constant demand for more tools to be added to debhelper, and increasingly they are for things -- new programming languages, subsystems I'm not involved in, etc -- that I cannot deal with very well. For a lot of these things it's important to say "no". But some of those "no"'s may have been misplaced, since the tools get added to Debian anyway outside of debhelper. In other cases, I have let other people maintain a particular part of debhelper, which has worked pretty well.

Debhelper has had a (second or third) rewrite listed in its TODO file for years, and the fact that I've not gotten around it it yet makes me think I probably never will. It's really hard to hand off a peice of software like debhelper to someone else to maintain, but it could really use someone with some more energy to co-maintain it with me.

My initial design for debhelper was pretty good, but one problem I did not anticipate was that a seemingly simple task like "install documentation" that was broken out into a single command like dh_installdocs could end up being complex and full of special cases. debhelper's design does not make it easy to refactor something like dh_installdocs when it begins becoming too complicated to fit in a single easy to learn command.

Next: ten years of free software -- part 6 ticker

I took over maintenance of alien from Christoph Lameter in early '97. Since he started it, it has a good name and still has wider regognition in the larger linux community (outside Debian) than most other things I've done.

Of the original alien, only the concept and the name remain. Alien has pretty well managed to dominate the linux package conversion space, mostly by virtue of being the first program to do that task, doing it well enough, and since I've enjoyed making it handle all the silly edge cases.

Alien has enjoyed more major number increments than any other software I've developed, I took it on at major number 2 and it's up to 8 now. It's also the first program I wrote that was forked by someone. That fork (and the associated linux distribution) are now defunct. Alien was also the first program I used object orientation in.

Since it's been more years than I can remember since I needed to use a package that was not in deb format, I only use alien now for pulling apart srpm files. Since a small shell script can do that just as well, alien is also pretty much in maintenance mode now.

Lately I've had to do some work to support LSB packages, but I've not tried to add support for entirely new binary package formats, such as the dreaded AutoPackage. I guess it's stopped being interesting. If someone who was a good person to take over alien came forward, I'd be ready to pass it on now. Conversely I'm happy to maintain it indefinitely.

Next: ten years of free software -- part 5 debhelper

discussion

The filters package is notable for being a peice of software I didn't realize I was writing until years after I released it. At first I thought I was just collecting some old text filters that had been floating around usenet for ages. Only later did I realize that most of that stuff wasn't free enough to go into Debian, and that if I wanted these silly filters in the distribution I'd need to rewrite many of them. Now the package is evenly split between filters I've rewritten and filters that I've gotten relicensed with an acceptable license.

The other interesting blind spot I've had about filters is that until last month I didn't realize there was no reason for it to only be a Debian package. Now that I've begun to look, I've found out that the code from it has spread its way around to a variety of other places without me paying attention.

I don't write games.. well, aside from MOOs, so filters and shoop (which I'll write about when I get up to 2000) are the only software I've released that was just done for fun with no real purpose.

Next: ten years of free software -- part 4 alien

Looking back over this series, it went longer than I expected. I hope it was occasionally interesting to more than a few readers, and thanks to the few who wrote comments in email.

Also, it seems I left out a ton of free software, because well ... (Debian!) Just listing code that I own in some way or another rather misses the big point of free software. Just listing all the other projects I've contributed to, in small or large ways, would miss the point too.

The big suprise at the ten year vantage point is that the most important thing was something I entirely left out of all 21^W22 previous posts. I can't remember the name of the little text mode game that was the first piece of free software I contributed to (by writing a man page), but I do remember the buzz of getting happy mail back from its author and getting sucked in to write a few levels or something, maybe fix a couple of bugs. Scale that up, multiply the connections a thousandfold and you have my ten years of free software.

So the best part of free software for me is sitting in the hotel lobby in Atlanta at 1 AM and talking about maybe adding build deps to Debian with Manoj and other oldtimers. It's organising a table for 30 at the last minute because suddenly those dinner plans mushroomed, and passing a paper around to get a list of names, and looking at that list 5 years later and smiling. It's getting the $5 tour of Silly Valley with Bruce. It's RMS dancing in the rain (no, really). It's a proper Debian Finnish sauna. It's learning to use chopsticks with Ben. It's sleeping on the floor in Europe three dozen times in a year. It's canoeing in Canada with Stockholm. It's relaxing at Michelle's with folks for weeks before the conference. It's meeting so many amazing people for the first time, whom I've actually known well for years before.

Something I've left out of this series is the fact that for the past five years, all of my paying jobs have been to work on free software. These days I look at work as an opportunity to let someone, who cares a lot about some improvement, expose me to new things and shift my priorities a bit and get me pointed in a subtly different direction. It happened with debconf and d-i at VA, with LSB stuff thanks to HP, with security and end-user stuff at SLX, and I can feel it happening with embedded Debian at ADS.

(Incidentially, I don't know why I only work for organisations with acronym for names these days.)

The best of these priority shifts are the ones that don't stop being a priority even after the job ends. And those also seem to be the ones that lead on to new jobs. The downsides to working this way include sometimes having to pass up a neat job, or only participate in something half of the time, because to do otherwise would demand too much time that I've allocated to projects started in previous jobs.

Letting go of old projects is, at least for me, a problem that seems likely to get more serious the longer I'm involved in free software. It's a lot better than the alternatives in the prioprietary software world though.

Next: ten years of free software -- part 23 everything else

I've written plenty about debian-installer before, so just a few points:

• The name? It sucks. Being completely generic ("THE Debian Installer") is fine, but it's so long it had to get abbreviated to d-i, and we're never sure if it's ok to upper-case it to Debian-Installer, or whether or not to hyphenate it. Also, the fact that it is called the "Debian installer" makes it less visible that it's also the Ubuntu, Skolelinux, DCC, LliureX, etc installers.

  Now Anaconda, there's a good name that can get some brand recognition of its own. I shoulda picked a name like that.

• This is the only free software project I've completely left, and later returned to. The really suprising bit is that the time after my absense has been such a great run of development on the installer.

• This is the only free software project I've managed to sucessfully hand off leadership to someone else, one and a half times.

• By far more people have contributed to this project than to any other project I've started. It's got a life of its own, for sure.

• If I used the same rules I've used for picking the previous 20 entries in this series I could do ten or more within d-i itself. I think you're getting tired of reading this stuff though, so I won't.

• Somehow it keeps on seeming worthwhile to debug the tricky problems, get on the planes, read the installation reports, and encourage the good ideas. No sign of it stopping.

Next: ten years of free software -- part 22 work

This is another peice of software with a limited user-base. If you own a Motosat Datastorm mobile satelite internet dish with a D2 and/or DW6000 controller, and you run linux, then you should use satutils to get a decent programmable interface to your satellite dish. Not to mention the fun of being able to pivot $2500 of equipment around and manually hunt for satellites using your laptop's arrow keys.

If you don't, well, I might be willing to sell you one. The software, of course, is free. :-)

Next: ten years of free software -- part 21 debian-installer

I don't have a chronological list, so this might not be my second free software program, but it's an early one.

My first foray into Debian-specific tools, dpkg-repack does one simple job and does it well. The second best part of doing dpkg-repack was just learning enough about the underpinnings of Debian's package manager to work out that it was possible to do. The best part has been the ensuing 9 years of seeing random mentions of it saving someone's time or data.

It's been pretty low maintenance ever since I wrote it, with the occasional tricky issue to resolve. I still use it as frequently as anyone else, which is not very often. Just another tool it's nice to have put in the general toolkit.

Next: ten years of free software -- part 3 filters

I'd have expected that as time went on my programs would become more and more generally usable by a lot of people. But instead it sometimes seems that I started with the big generally applicable stuff like debhelper and have been filling in corners a bit since. While I have no shortage of users of my recent code, I also have a suprising number of programs like flashybrid that have naturally limited user bases.

Flashybrid has had three users that I know of over its lifetime, I suspect one user failed to get it to work, and at least one of them (me) has stopped using it. I don't mind that, since at the least, flashybrid let me do something really neat: Embed a usable Debian system on a 32 mb compact flash memory card, and let me spin up a disk every month or so to get at the rest of the system and upgrade and maintain it. This was the basis for my embedded home server for years, and it was plenty of reason to write the code, and once it was written, there was no real reason not to distribute it. Lack of users doesn't matter from that perspective.

On the other hand, I now look at flashybrid as a first try at embedding Debian in a particular way that I do think has potentially a lot of users, if the technology to do it can be worked out. The hardware is there: Sub-$100 wireless access points. These systems can run Debian if it's set up right. So I'm going to keep on with that.

Also interestingly, years after writing flashybrid I ended up working for a company that uses vaguely similar techniques to embed Debian. Which, starting Monday, it's my job to maintain and develop further. Interesting possibilities for convergence there..

Which if nothing else certainly points out that taking the extra couple of hours to take a program like flashybrid beyond a local hack, and making a proper release of it, can tend to pay off in unexpected ways.

Next: ten years of free software -- part 20 satutils

My second try at writing a general purpose perl library, and again it was a spinoff of a larger project. I wrote this for mooix, which needed a way to convert English prose into an integer, but instead of stopping with the lookup table for "one" .. "ten" that the application needed, I decided to go all the way (up to a googolplex anyway).

The good thing about doing that is that even if mooix doesn't survive, it seems likely that Lingua::EN::Words2Nums will. It still needs to be fixed to support fractions though.

Last weekned I was suprised to see Lingua::EN::Words2Nums mentioned in a recent book on advanced perl programming. I only have a link to an excerpt of the first chapter in Chinese handy though.

(Belated PS: Mooix's name didn't suck, and neither did "Lingua::EN::Words2Nums", much. Wow.)

Next: ten years of free software -- part 19 flashybrid

Mooix is my second try at writing a moo. It turned out a lot better than perlmoo. The idea in mooix was that users would be able to program the moo on the fly in any programming language and that objects would be composed of methods written in any combination of languages. Of course this would be done securely using some fun tricks. Oh, and it would have some pretty sophisticated natural language parsing that could understand sentences more complex than the pidgin English most moos accept. It was a lot of work but I accomplished all that.

The language independence appealed to lots of people who had been looking for that in a moo, and mooix had more early contributions from other developers than a lot of projects I'd worked on before. Things were really intense for a while, I remember several months where that's all I did.

I stopped working on the project ater a few years of doing lots of work, since I had to get back involved in the Debian installer. Like with perlmoo I dropped the moo for something a bit more serious. I did promise that I would return eventually, a promise I have so far not kept..

Looking back at it, mooix was hobbled by the security system, which ended up being so secure it was hard to do many useful things and also (along with the parser) slowed it down quite a lot. Security problems with the linux kernel also became more of an issue than I'd expected. Another weak area was the object system, it seems that designing such a large object oriented system is a bit beyond me.

I'm pretty sure that I won't return to working on mooix in its current form. And while MOOs in general are still an interesting area for me, I'm more interested now in doing similar things in a more distributed fashion, which is a much harder problem to solve. Maybe one of these days.

Next: ten years of free software -- part 18 linguaenwords2nums

apt-src was an attempt to clean up some nasty corners of how Debian works with source packages, particularly kernel related stuff. I ran into a suprising amount of resistance to doing anything differently or exploring new solutions, and decided it was better for me to just ignore that part of Debian for the forseeable future. Disappointing but I can't win em all.

Next: ten years of free software -- part 17 mooix

Maintaining tasksel is annoying when I have to justify the choices it makes to other technical people who are annoyed at not having full control over this part of the installation process. It is conversely quite satisfying when I see actual users use it and be able to make choices at the level they're capable of working at, rather than just be faced with some unconfigured system that they're expected to turn into something useable on their own.

Aside from those annoying issues, these days it's a simple perl program written in a nice functional style, which has become quite extensible, so I'm pretty happy with that. I think it's the only program I've ever taken from C to perl; I tend to go the other way.

Next: ten years of free software -- part 15 sleepd

sleepd is another one of those tools written to fill a need I had at the time, which I no longer have. Has enough users to make it worthwhile to maintain it, but I'd gladly pass it on to anyone who still needs it.

Next: ten years of free software -- part 16 apt-src

The first hints of d-i began here. I took a nasty little shell script that used to be part of the boot-floppies and tried to make it into something a) not nasty and b) somewhat user friendly. A big improvement for its time, but it's now been lapped by d-i and is showing its age, and I hope to get rid of it soon.

One of the things that strikes me about base-config is that unlike pretty much all the software I've discussed before, I don't have any proprietary feelings about it. It's just a peice of software we need, which I happened to write.

Next: ten years of free software -- part 15 sleepd

I think that debmirror is possibly the only price of software that I wrote for my own use, and that got packaged up for Debian by someone else without me even asking. It was pretty popular even before it became part of Debian, since it met a need that no other tool was able to fill at the time. But I was never interested in meeting other people's needs with it, just my own, so I wrote it, made it work for me, and abandoned it to the Debian maintainers.

The funny thing is that I still use it today, and occasionally chafe at some things I don't like about it, some my fault and some added by later maintainers, but I'd not think about trying to change it. I think a lot of us treat a lot of software like that, although not often software that we wrote ourselves.

Next: ten years of free software -- part 14 base-config

This was an ill-fated perl slang widget library, which I wrote to use to make a better frontend for debconf. I think it was my first widget library. It wasn't a bad little library, but actually getting debconf to use it effectively turned out to be too hard and the GUI not good enough, and so I dropped the debconf frontend, and so had no reason to maintain Term::Stool anymore, and dropped it too.

I won't even try to talk about the name of this peice of software.

Next: ten years of free software -- part 13 debmirror

Debconf is the only peice of software I've written that is the first software to implement a spec. It could be the fault of debconf's spec, which I didn't write, but doing this seems to be a fairly sucky process, basically you're taking the result of an abstract committee process and trying to turn it into something useful, and you get to find all the bits that were not thought through. This is a unique problem to first implementations; later implementations can just ignore the spec.

Of course debconf is also one of the more important peices of software I've written, and it also laid a foundation that was used by a lot of other work I did later, including d-i, so the pain was worth it.

Come to think of it, in d-i I took the approach of writing a spec and mostly letting other people work out all the tricky bits it didn't anticipate, but that's another story..

As to the code, it was my second foray into OO perl, after perlmoo, and it kinda shows. The shell code parts used to be the kind of shell a brain scarred by working on shoop will produce too, though they've since been sanified.

It seems like in the end it's going to be harder to transition everything over to the second implementation of the spec, cdebconf, than it was to write debconf in the first place. Still, I want to do it, since we have to have cdebconf for d-i and there's no point in mostly the same people maintaining two parallell implementations. Although some parts of cdebconf don't match the power of debconf, I doubt many people will notice.

Next: ten years of free software -- part 12 termstool

ten years of free software -- part 10 (shoop)

Adding object oriented programming to posix shell is the kind of mind game that programmers play from time to time to keep themselves sharp. It takes some doing to work out how to abuse and triply nest the eval command to make something like this possible, and then you take care to implement it in an absurdly elegant way. Then you brag about it for days, and possibly years.

Some other likeminded madmen get into the act and things verge toward satire as whole schools of thought are recapitulated in the growing monstrosity. It becomes somewhat notorious.

You loose interest and don't stress about what will happen to it, but later versions never seem to quite have that original spark that was in the early code, that you keep to look over on a blue moon.

Eventually you realise that you actually learned some important things along the way, and are astounded to find yourself with some later serious projects that owe ideas to that fevered dream.

Five years later, sourceforge is still sending you spam moderation requests from a mailing list about it you never created, and from which you cannot find out how to remove yourself.

Ah shoop, the most fun I ever had with #!/bin/sh.

Next: ten years of free software -- part 11 debconf

talks accepted

Wow, the two talks which I foolishly sumitted for DebConf5 were both accepted. Luckily at least one of them (d-i) will be presented by a team. But I will at least have to write the paper for the "securing testing" talk, and of course give the talks. My only previous talk at DebConf (Toronto) was about debconf[1], years back, and that was pretty much my first public speaking experience. Well, we'll see if I retained anything from Jeff's tutorial in Porto Alegre.

Even with my talks in it, this year's DebConf lineup looks great. I find it hard to eliminate even two talks that I'd rather sleep in through or spend time sightseeing in Helsinki instead of attending.

[1] I sometimes worry that writing things like that sentence in my blog is unduely confusing to non-native-Debian speakers. Yes, "DebConf" and "debconf" are two unrelated things, a conference and a program I wrote before the conferences started, respectively. Part of Debian's culture involves setting up these little verbal traps for the uninitiated.

svnhome update

Update to my svnhome article:

I've changed how I handle my dotfiles a bit. I wrote a program dircombine that "merges" two or more subversion directories into a single common directory (just by adding/removing symlinks as appropriate). So with this I can easily split my dotfiles out into modules and inclue some modules on only some machines.

Now my home directory in svn has no actual files in it; all the dotfiles are moved to these modules, and other directories are also pulled in via externals. This makes it easy to maintain more variants of the home directory without needing to merge dotfiles between them like I had to before.

Another unexpected benefit is that since all the dotfiles that are managed by svn are symlinks, any that arn't symlinks are clearly not managed by svn, and so are easy to clean up as cruft or add to svn.

Only real downsides of this approach are needing to move dotfiles to the appropriate subdirectory before adding them to svn, and svn commit not recursing into dotfile subdirectories.

Oh, and it gives me an excuse to have a directory named "home-etc".

discussion

A long time ago I wrote an article on keeping $HOME in cvs. Now I've finally updated it for subversion and it's published at OnLamp.com. I also have a local copy that I may update over time.

svk

A week or so ago I spent several days trying out svk. svk layers distributed version control over subversion's centralised revision control model.

The good:

• It's very easy to learn svk if you know svn (and thus if you know cvs), just a few added and a very few changed commands. Oddly, bazaar-ng seems to be converging on the same UI from an arch heritage; even the new commands behave similarly.
• There are no more .svn directories in working copies; in fact working copies have no extraneous files at all, which is marvelous.
• Even working with local branches of centralised repositories has been made nearly as easy as I can imagine it could be; it's possible to commit a change locally and merge it to upstream in a single, simple command if you want to. Fully disconnected development with subsequent updates or merging to/from upstream works well too.
• It can apparently be made, with some difficulty, to mirror cvs repositories as well, which would be great for dealing with those laggard projects, like debian-cd and the debian website, that still use cvs.
• There's a nice responsive community around it; the lead developer is easy to get in touch with.

• There's no support for svn:externals. Since I use externals a lot, this makes checking out my entire home directory using svk quite tricky. I found a semi-solution using symlinks.
• A few commands are subtly different from subversion. For example, svk annotate does not display history from the central repository by default, only from the local branch, unless you add the -X switch. This can be confusing.
• If you want all the history of a repository, you have to mirror it, and this takes just as much space as hosting a full copy of the repository. For me that's something like 3 gb of mirrored svn repositories. This more than offsets the space saved by no .svn directories. Of course you can get away with only mirroring the last few revisions of trunk and maybe save a lot of space, but that is not really an option for me since I need the full history.
• The process of mirroring a repository is slow. It has to replay every revision into the local svn repository. When you have 50 thousand revisions, that takes a while. I'd be interested to know if other distributed revisions control systems have gotten around this; one test tla checkout I tried seemed to suggest they've not.
• The svk command is also a bit slow to start up due to using a slew of perl libraries.

• I managed to find one repository that svk cannot mirror at all; it crashes.
• Mirroring another repository made svk consume over a gigabyte of memory and OOM. I even OOMed svk once with a svk annotate -X command. In general it seems that it needs a lot more work done to scale to moderatly large repositories such as some of the ones I work in.

svk is not out of beta yet, and I hope that eventually some of my problems with it will be solved. So far I cannot seriously consider using it for managing my whole home directory on my laptop; I can see using it for some individual, smaller projects, if I need the disconnected operation.

When I switched to subversion, I doubted that it would be the final word in revision control systems for me. I hoped that as better/different systems came along they'd follow the lead of subversion (and others) and provide ways to import data from the revision control systems they supplanted. svk has both confirmed that feeling, and made me worry, since I've begun to realize that perhaps importing past change history is not enough; the newer systems have to be scalable enough to deal with my own, constantly growing, repositories.

suprise

I worked on debhelper today. I fixed an embarrassing lot of bugs, including finally applying kmuto's glibc symlink patch (which seems to pass the regression tests, at least). I've not worked on debhelper mch lately, no time.

sunrise

This year I've experienced something like 300 late nights, and only about 5 sunrises. I was jetlagged, up too early, or otherwise screwed up for most of those sunrises. I love the deep quiet of the night, but sometimes I hope that I eventually turn into the kind of person for whom a sunrise is normal and not something that signals that I'm up a few hours too late. Balancing those numbers out would be good.

For now though, the rare days I'm up before sunrise are unique moments to be treasured. Like Monday, when I was camping out at the farm, and the heater went out, and blankets were insufficient, and I just had to get up early (-ish, sun doesn't come up until 9 in the valley this time of year), and stumble down to the big house for hot cocoa to warm up. And got to see the frost turn to clouds as the morning sun crept across the yard. Such a morning makes me feel so intensely alive.

summer

Yes Sesse, I have been known to enjoy summer (when I'm not busy merging slides at the last minute with my co-speakers).

Though winter is nice too, and it seems to have finally gotten here.

stuff

I moved my cat to town yesterday. Remembering past moves, I'm impressed at how quickly he's accepted his new environment. Probably quicker than I, to tell the truth.

I had a nice productive session on Tuesday and finally got the debian-edu sarge CD build system into a state I fully understand, and that is fully checked into various cvs repositories. I even have my local mirror set up, and can build CDs from it, which will cut my test cycle for those images in half or more.

d-i has not been going so nicely; I've not had time to test it at all lately until today, and today I tried new images built from the udebs in sarge, and found numerous problems with pcmcia and elsewhere, some of them due to version skew. I'm currently very dispirited about how the udeb propigation to sarge has been working out, it seems its been a lot of work and complexity for no gain.

Random thought for the day: It's weird to hang out on irc with someone for years, and know them by their unique nickname, which you've never heard before, and then stumble accross the peice of fiction that that nickname comes from. Both the suprise of finding out that the nickname was not made up, and the way that nickname has come to mean exclusively that person, and suddenly doesn't anymore. I should ask more people where they got their irc nicks.

stress and comfort food^Wreading

I've been spending a lot of time at the farm latetly to ground and recharge. So much that I had planned to not go out there today. But after the past few hours, which have been one stressful thing after another as I try to get ready to upload the last debian-installer build for the release, I am ready for some unstressful time offline again, for the third time this week.

Anyway, this is one of my favorite times of year, with it getting chilly enough at night to appreciate some blankets, and contrastingly warm enough in the days to enjoy that. Not much of a fall harvest this year, since I didn't plant a garden, and Anna's was an early summer garden, but we still have some delicious squashes and sweet potatos.

After a long dry spell this summer when I didn't read many books at all, and was leaning toward nonfiction in what little I did read, I've gotten back into regular reading over the past month. Before my trip to Germany, I was enjoying some very hard SF, with an emphasis on ultra-strange worlds. So I had fun with Stephen Baxter's Raft, set in a universe with different gravity, and with Christopher Priest's The Inverted World, set on a fractal (except for the annoying conclusion). I also read the latest Ringworld book, Ringworld's Children, which was forgettable, and returned again to my old favorite, the flawed and generally overlooked The Integral Trees. Why didn't they use kites in that book? I also read a number of other hard SF books outside this theme, particularly others by Stephen Baxter.

Now I seem to have switched gears with the onset of colder weather, and am for some reason enjoying rather fluffy fantasy. A newish Pratchett book was quite enjoyable; he's really rebounded in his latest novels. I even read and enjoyed a Lackey book somehow. And I'm very happy to have discovered the novels and short fiction of Nina K Hoffman. Airborn was a particularly fun, marginally "young adult" romp.

stress

Doing the d-i release, plus GR idiocy which I should just ignore, plus working on moving, plus the upcoming Brazil trip all has me very stressed right now. I got out and took a long walk today which helped, but then this evening I discovered d-i's 2.6 support had bit-rotted in just half a week, and had to dive in and fix it at the last minute.

strange pairs of statements

According to Mark Shuttleworth:

The DCC distro doesn't use the Debian kernel, and it modifies key pieces of the infrastructure like the linking system and core system libraries. So it's not really Debian at heart.

According to the DCC 3.0 release notes:

5 [packages] are the LSB 3.0 compatibility environment, which adds LSB 3.0 compliance in such a way that the sarge glibc and pam packages don't need to be modified

It took me a long time to figure out how to read these two statements in a way that wasn't thuroughly contradictory.

The DCC dropps in ld-lsb.so linker in for LSB binaries and uses that to make them link to some modified libraries somewhere outside the normal link paths, but that's more like the distro having two hearts (one of which is used only to circulate blood through the very limited set of available LSB binaries) than not being Debian at heart.

Mark again:

There is now discussion of modifying many, many more packages, for example to use the Ubuntu X.org packages rather than the Sarge XFree86 packages.

DCC again:

25 [packages] are a backport of X.org from etch.

So if they're using Debian's X.org already, why would they be planning to go to Ubuntu's?

Mark:

The DCC distro doesn't use the Debian kernel, and it modifies key pieces of the infrastructure like the linking system and core system libraries. So it's not really Debian at heart.

Mark again:

The DCC kernel and Ubuntu kernel will be very similar if not identical in future DCC releases, and I expect that collaboration will spread to other parts of the system such as X, ACPI etc.

If you think about it, this is in a sense the strangest pairing of statements of all...

stove, eye & pc

The new stovepipe seems to work great, although something wants to burn off of it at "roaring blaze" setting. I'm disappointed that Anna's decided to avoid a wood stove. Yes, they're dangerous and dirty and a lot of work, but the warmth and general niceness of a fire and wood make up for it.

Reflexes are amazing things. I can't remember seeing the branch coming up past my glasses to my eye, but apparently it got closed in time, since I didn't put it out when carrying brush from the line cut yesterday. It's been a bit weepy since, and after waking up with it nearly shut, I empathise with all those poor kittens with gummed up eyes.

Oh yeah, I bought a used amd64 shuttle XPC on ebay. The idea is that it will hopefully be fast enough to run vmware on for some serious d-i development work, while also doing things like playing music and being a file server. And quiet enough to be in the same room with.

stop wasting my time with gmail

I completly fail to see what the fuss is about 1 gigabyte of probably overcommitted disk space accessible only via propriatary software and over no reasonable protocol. Especially amoung people who should realize just how few pennies a gigabyte of disk for your very own costs these days.

A market is a wonderful thing, and I am pleased to see that Ebay has determined the fair market value of a gmail invite to be less than 1 dollar; pretty close to the price for a similar capacity hunk of spinning metal.

By posting invites to a mailing list with a few thousand subscribers, a kind hearted gmail user probably manages to waste dozens of man hours to distribute a commodity that's worth far less than that time. There are more efficient ways to do it. Use a website to distribute them, come up with some peer-to-peer gmail invite distribution thing, or sell them on ebay. My time stops being wasted now.

+    /^Subject:.*gmail.*invi/ || \
     /^Subject: *Out of Office AutoReply: /)
 to ${MAILBOX}junkmail

Am I really the only one to whom a gigabyte of mail storage seems .. small? I delete 90% of my mail and my archives still top 10 gb, and use over 100 gb total in their various mirrorings and backups. Waiting for terramail, over and out.

stock kernels

This is a happy day, I'm finally using 2.6(.10) on my laptop and think the jump from 2.4 might finally stick this time. And I'm even using the stock Debian kernel there, which means I now admin only 4 or 5 machines with hand-built kernels, all either due to needing NFS root support or lacking physical access to upgrade boxen to the Debian kernels.

Even longrun stuff is working on my laptop, and although the debian kernel is not built specifically with crusoe optimisations, it's not bad, and is decidedly faster than 2.4 with optimisations.

Lately I can barely keep up with the security holes fixed in the Debian kernels, let alone rolling them into custom kernels of my own, so stock kernels are a very good thing.

Update: Of course I spoke too soon, what I had thought was a hang on apm suspend if usb storage was mounted seemed to be a random hang. Back to 2.4 again..

state of testing's security

Now that we're in the freeze, I can kind of get a taste of what it must be like to work on fixing security holes in stable, and can try to compare this with my half a year of working on the security of testing.

We have the testing-proposed-updates queue, which allows security fixes to be built directly against testing, so currently the main thing everyone has thought blocks packages from testing -- getting dependencies of security fixes into testing -- is not a problem (well, mostly; not all security fixes go through t-p-u; but if one doesn't and is blocked by something, we can do it through t-p-u instead).

To counterbalance this, we're in a freeze, all changes should be minimal to fix a bug or security hole, and they have to be reviewed by a member of the release team. I've been doing a lot of reviewing of security fixes, as well as a lot of work tracking down and applying backports of security fixes for holes that are fixed in new upstream releases that we cannot accept into testing due to their other changes.

Watching the list of security issues in testing has been interesting. Since the freeze began, it's been near a high water mark for holes fixed in unstable but not testing.

A lot of this is due to a few massive security hole sources such as ethereal and the linux kernel, for which it's been difficult to get security fixes backported, autobuilt on all arches (t-p-u still has some autobuilder issues apparently), and into testing. Factoring those out halves the number of holes. If we were not in a freeze, most of these holes would already be fixed in testing.

The number of completly unfixed holes has been holding stready from before the freeze, or possibly going up a bit. I think this number is more correlated to the severity of security holes, and the frequency of incoming holes than anything else, and the freeze has not seemed to change the way the numbers are moving for this one.

It is suprising that release critical bug fixing efforts have not dropped it more, but many of the unfixed holes are actually minor security holes that are (wrongly or rightly) not considered release critical. Then too, the efforts of many of us who would be working on these holes has been redirected to the backporting and patch reviewing activities I mentioned.

I probably don't have enough experience and information to say for sure, but my gut feeling is that it's quite a bit harder to manage security fixes for a frozen release than it is for regular testing. The ease of getting most security fixes into testing, even if we have to push in significant upstream changes to do it, seems to outweigh the issues with unfullfilled dependencies blocking propigation of security fixes to testing.

With that said, the t-p-u (or testing-security) queue is a very good thing and after sarge is released I really hope we can still have one to use for pushing in targeted security fixes when there are dependency issues. Best of both worlds..

Anyway, I hope that the stable security team can find some time to work on security issues in testing now that it's frozen, as they promised they'd be able to do. We can certianly use their expertise with backporting and the other kinds of problems that come from doing security support for a frozen distribution.

(If you found this at all interesting, you might enjoy my talk at the next DebConf. If I ever find the time to work on it.)

spring

It was hot so I went out to the back forty and climbed the waterfalls[1] this afternoon. Lots of early spring wildflowers are out, and the falls are unusually slick with heavy moss. It was not hot enough to go swimming, but I got fairly wet.

d-i stuff proceeds..

[1] These waterfals are not vertical, they're more like waterslides made of minerals that are dissvolved into the water while it's underground and then deposited as it flows down the hill. One of the neater features of the farm.

speaking of old books

I've been having pretty good luck finding old SF&F books that I never thought to read before via these belated reviews. So far most of even the two star books bave been well worth the read, and it's pointed me at a few series (like the "Amber" books), whose ugly tails ends put me off a while ago, but that actually had some very good books at the beginning.

I still haven't quite dared to try to find a copy of Gormenghast though.

sparc

Yay, I got a sparc! Now I am able to run automated d-i tests on three architectures. Not bad for someone who had only ever had i386 boxes until 2 months ago. Looks like it won't be too hard to add support for automating the sparc installs, most of the work I've done for automating ia64 installs over serial consoles will apply. The ultra 5's in the middle of its first Debian install now.

Actually so far the only hard thing has been getting a serial hookup that works. I ended up building an i386 out of spare parts, installed Debian on it this evening and am using it as part of my conserver network, since none of my existing servers had a spare 25 pin serial port. Well it also let me put the sparc in the room I had wanted to put it in.

south holston dam

This evening I headed out to South Holston Dam to hit a couple of caches. It's weird, I grew up only 15 miles from there and I've certianly used the lake enough and even, I think, once hiked along the shore over to the dam, but somehow I've never drove up behind the dam, or over the top of it. This 285 ft. high TVA dam is quite an impressive site. Nice to find something new so near by.

snow

I lost the beard yesterday, foolishly thinking winter might be over.

Today would have been a blah of a day, except for the weather. I was idly catching up on email, nothing better to do on a grey rainy day, when rain suprisingly began to turn to huge wet three inch flakes of snow. Snow-pattys as it were. That livened up the afternoon some, and it's still coming down, quite a large storm for this part of the planet.

The bamboo is knocked flat on the ground, and so are a number of dead tree limbs, from this wet and heavy snow. So, apparently, are my power lines, as the power went out about an hour ago. So we've pulled out the lanterns and built up the fire, and put the laptops into low power mode, and are having a good old-timey winter night.

snail

Just replaced the old laptop that I was using as a dialup box and squid cache with my second nslu2, aptly named snail. The laptop suffered dreadfully from serial overruns; the nslu2 with its USB serial seems to pump the packets a lot faster, plus I can run it in a no moving parts configuration (if I turn off the squid cache). And it's tiny, of course..

Smoosh!

Been a while since a band has made me this happy. Just what I needed right now. Check 'em out.

include <standard_smoosh_ageist_comment.h>

small world

I have to confess I've been a bit of a media junkie this week, like many people I'm rather eager to find out who will win the election tomorrow. In my search for a good source of data I eventually found electoral-vote, which is an impressive site both in its data and analysis, and also in its very good, non-flashy web design (love that map). It appealed to me strangely.

Who ran that site was a mystery, although he obviously was a unix guy. Until this morning, when he "came out of the closet" and revealed himself to be .. Andrew Tanenbaum!

Hmm, back in the day, minix also held a strange appeal to me.

slug

More nice hardware running Debian. With rwhitby's help I've gotten Debian (little endian arm) installed on a nslu2 (donated by the nsul2-linux group, retails about $100). Tiny little box with USB, lots of nice applications.

For those who are counting, that makes 10 computers running linux (7 Debian + 3 openwrt; 3 x86 + 3 mipsel + 3 arm + 1 armeb) here at the oh so rustic and old-timey farm.

I hope d-i will support the nslu2 soon..

sleepd 1.3.0

Recently I put a new drive in my laptop, and unlike the last one (but like the one before), the 2.6 kernel cannot handle apm sleep with this drive. So I had to, horrors, switch to using acpi and software suspend and hibernate. Which, suprisingly, is working ok.

But it was missing a little piece, something to put the machine to sleep if the battery got too low, and so I remembered this sleepd program that I wrote years ago and have been tying to ignore ever since despite people who persisted in using it for some strange reason. Apprently there's still nothing else to handle this job.

So now I'm releasing version 1.3.0, with a bunch of patches and improvements.

sleep schedule

My sleep schedule is dreadfully messed up. I need to find some hard physical labour to do.

sleep log

Saturday: 6 pm - 11:30 pm (in Germany but all times EST)
Sunday: 1:30 am - 4 pm
Monday: 2 am - 7 am
Tuesday: 3 am - 3 pm

My sleep schedule isn't a schedule. Time zone changes don't help, but this is rediculous.

sleep

Well. Night before last I could not get my brain to shutdown, though I tried between 3:30 and 5. Then it got light out, and what was the point? By then I had come up with a plan for how to fix linux-kernel-di, so between 7 and 11 or so, I was ripping the package to bits and reorganizing it. Finally managed to get some sleep between 11 and 3, then I got up and, feeling very tired and out of it at first, cooked for an hour and a half. That was a pretty weird day, but I got out of it with only very minor problems in the reoganised kernel-wedge, and a minor (painless) burn on one finger. While the experiment was thus generally successful, I think I will try to avoid major coding while high on sleep deprivation in the future.

Today I got a good 12 hours sleep but have still been lethargic, thanks to the excesses of yesternight. I made some minor improvements on d-i and possibly fixed the pcmcia-cs installation bug.

On tbm's recommendation, I've read a bit of Greg Egan, and discovered that I had read one of his movels (Diaspora) before. Not too happy with the characters in Permutation City, though the science, metaphysics, and philosophy is fun.

Response to d-i beta 3 has been good, and I'm pleased by how few problems most people have with it, and with how many entirely successful installation reports we've been getting (including one on m68k!). Installation reports are beginning to morph into reports on the second-stage install, which is a good thing. We need to pick the pace back up a bit and get the beta 3 errata fixed soon though.

So far the only really serious new issue that has come to light is that CD mounting in the second stagte is thuroughly b0rked -- with netinst CDs you don't test that. So soon we're gonna have to start testing with full CDs first. Ugh.

Tomorrow: Some hard physical labor for a change.

slang 2

Did some upgrading to slang 2 in my packages today. It was fairly painless, except in pdmenu, which got a bit too close to slang's internals in its shadow generation code to survive the upgrade.

Pdmenu used to use SLSMG_BUILD_CHAR to construct a raw character, after using SLSMG_EXTRACT_CHAR to pull the character that needed to be shadowed from the screen. But slang 2's UTF-8 support means that SLSMG_EXTRACT_CHAR doesn't always work, and SLSMG_BUILD_CHAR is removed entirely. Since SLsmg_Char_Type is now a structure, I hoped I could just call SLsmg_char_at, set its color to black and SLsmg_write_raw the character back to the screen, but this doesn't work right for line drawing characters. I put in a quick workaround of just drawing a pure black character for a shadow, instead of the nice black background with dim character that pdmenu was able to display before. Probably there's an easy way that I'm missing.

(Thanks to JED and since this is getting passed around a bit, the answer is SLsmg_set_color_in_region)

On the very much plus side, I recompiled aalib with slang 2 and it just worked. Including supporting binaries built for the old slang 1 aalib; and even binaries built against the new aalib work with the old one. So messy incompatable updates of the aalib/sdl library chain seem to have been avoided.

Upgrading Term::Slang to slang2 was interesting. Thought I'd forgotten all that perl XS stuff.

six not sixteen -- followup to Edd Dumbill

Edd, the typical number of questions asked by the installation of Debian's desktop task is 6, not the "well over sixteen" that you reported. They are:

1. Select X server driver. (Generally has a reasonable default.)
2. Attempt mouse device autodetection?
3. Attempt monitor autodetection? (Basically useless and has the wrong default.)
4. Is your monitor a LCD device? (I don't know why this cannot be autodetected either.)
5. Please choose a method for selecting your monitor characteristics. (Note that the best thing to choose here is "Medium", not the default, "Advanced". If you choose advanced here you'd be prompted for two sync ranges for a total of 7 questions, IIRC.)
6. Please select your monitor's best video mode.

The fact that you saw so many more from fontconfig, cdrecord, kpilot, and so on suggests to me that you were not using debconf in the default high priority mode, perhaps because of a problem during the first stage install. If you'd care to file a proper installation report with logs, I could tell you more.

Belive it or not, we've actually done a lot of work on this, and prodded maintainers a lot to cut down the question count, which used to stand at around 25. Yes, it might be nice if ubuntu could push some X config improvements into Debian, especially since one of X's co-maintainers is employed there. But I have yet to see any fix from the X maintainers or ubuntu for the most egrarious X config bug of all, bug #264792.

Note that I'm posting this as a blog entry in response to yours rather than as a private email, because the last time I emailed you (20040804214234.GA9204@kitenet.net) about negative comments you blogged about the debian installer, requesting more info, you never took the time to reply.

simple scripts and the men who love them

A few weeks ago, I wrote a very simple script to play any of my music files that have a given set of words in their filename or directory. I've started to use it exclusively instead of the (poorly designed) xmms file selector, since I can generally select any given CD as quickly as I can think with this thing. Especially given use of abbreviations, shell history, aliases, all the nice things you get with a command line utility.

The other nice thing about this is the serendipity factor, if I'm not looking for a given album, my random search criteria often turn up very interesting playlists with songs I forgot I had. Of course, this is really all wrong; it should be searching real metadata of the files, but the metadata in my music collection is middling to poor, while the directory and filename structure is good to excellent.

Anyway, it's been a while since I was so happy with 44 lines of perl written in 15 minutes.

seriously annoyed

Let's just say that I am a smidgen tired of uploads that have not been tested one ioata.

serial cables make joey crazy

I hate serial ports. I wish they'd just go away, like they have from the back of $CURRENT_LAPTOP. Unfortunatly nearly all my d-i test automation work ends up involving serial consoles and so I've had to mess with them more in the past few months than I have in many years.

Today a nice pa-risc machine arrived which will be the next one I automate. Unfortunlty its console is serial, and I am getting pretty maxed out on serial ports on my servers. I tried setting up a chain of db25 to null modem to gender changer to db9 to the hppa, but this failed to work for some unknown reason and would have been temporary anyway.

So I had to set up an old digi portserver 16 for it instead. This involved hand wiring rj45 to db6 converters, and I still haven't gotten it quite right, since ethernet patch cables do not work, but if I use a phone cable (4 wires less), I get at least a basically usable connection. Thanks to William for suggesting I try a phone cable. I suppose this means I got the 4 missing lines wrong somehow, even though I followed digi's documentation. Their "altpin" stuff is confusing.

Anyway, after I got that set up and fought with the digiserver command line for a few hours, and got it set up, I finally have a login to the new box's management console. That was too hard, ethernet has spoiled me for simple cables that just work, with nice features like connection status and autosensing.

I won't be suprised if automating the whole rest of the debian auto-install is quicker than getting the serial console hooked up.

serenity

I had to go into town again after all (necessities like groceries and wifi gear) so I took in a movie.

Serenity is ok but it seemed to downplay a lot of the spahgetti western riffs that I enjoyed in many episodes of Firefly. And I was disappointed that the reavers were explained so easily. And the more I learn about River the less intriguing she becomes, really. And of course big budget hollywood cgi+volence is feh.

On the plus side: Barefoot++; they weren't afraid to kill characters off or do some character development; the wacky little corners remained off-key; and if the series manages to continue things will have changed in interesting ways.

seeking wiki

For a new project, I'm looking for wiki software with the following characteristics:

• Supports SubPages and relative linking from a page to its subpages and between subpages of a page works like regular wiki linking with no special syntax. MoinMoin supports subpages but to link to them without giving the whole path is ugly: "['/SubPage']" I want to just be able to link to SubPage from page FooBar and get /FooBar/SubPage if it exists, and /SubPage otherwise.

• Doesn't require special markup to make words that are not WikiWords, but that exist as pages, show up as links in other pages. Probably limiting this to capitalised words, and supporting turning off the feature for a blacklist of words in case of trouble.

• Can subscribe to changes for sets of pages or the whole wiki by rss.

• Can subscribe to be informed of new pages by rss.

• Uses some standard plain text markup language, like markdown or restructured text, and not some obscure thing specific to that wiki. Adding special syntax for wiki links is ok, and removing features like markdown's support for arbitrary inline html is ok.

• Keeps full history of changes to the wiki using a standard version control system like subversion. Ideally it would be possible to browse past states of the wiki at any point in time and even edit them to create branches, but UI for that is not a requirement.

• Allows direct commits to the wiki without using a stupid web form, using a standard version control system, like subversion.

• Supports images.

• Supports WikiName based logins and can limit edits to logged in users (although anonymous svn commits would be ok at least until wiki spammers learn how to spam via svn).

• Addendum: Cannot be written in PHP, thanks anyway.

At one point I had a half written wiki that supported a cvs backend (this was before svn) and supported browsing historical versions, but it got bogged down in trying to use html as the standard markup language, which diverted too much energy to writing code to sanitise html, and stalled out before it was usable. Writing one again doesn't seem wise, there are already too many. So I'm looking for pointers to anything that meets at least some of these requirements.

Update: Subwiki uses subversion but is developing slowly and may be non-DFSG free. There's discussion about making TWiki support a svn backend, but it appears not to yet; it would probably meet all my other needs then some. GW uses subversion, but seems to be a java based academic research project. trac uses subversion, though it's not just a wiki and it might not use it for the wiki.

Rather than tight subversion bindings, I'd be happy enough with a wiki that used flat text files and was flexibe enough so I could store them in svn. I'd even be happy to use viewsvn as the history browser for the wiki. Small and minimal, building blocks, like blosxom blogs, is good.

security followup

Norbert: It seems to me that you're now comparing how well sarge's security updates are doing with those of stable and ubuntu. I've never found such comparisons worth the electrons they were printed on. Sarge's security support can certianly be improved (this statement holds true for all other security support too). Help wanted.

IMHO to an extent the important thing is that there be a committement. For example, ubuntu has made a commitment to doing security updates; I don't give much heed to the current emptiness of their Packages file compared to that committment. If they had no commitment to security updates and more updates in their Packages file, would you be happier?

secure testing ramping up

There's finally some serious progress in the Debian testing security project beyond what we've been able to do up till DebConf5. All those saunas paid off sinxe Aba and Zobel finished this Friday setting up a whole parallell Debian infrastructure for secure-testing. Archive server, katie installation, buildds, the whole lot.

Since then I've been working with the rest of the team on setting up our process and framework for doing security advisories for testing, and running 9 advisories so far through the process to work out the kinks.

It's not at the point where I'm going to make a big announcement for it and get tons of users beating on it quite yet, but if you know where to look you can find the apt repositories and mailing list and everything else.

Very exciting for this last piece to finally be done.

secure apt docs

Here is my attempt at some reasonably complete docs on how secure apt works, after observing the rampant confusion about this on debian-user lately.

secure apt

Since secure apt is finally being rolled out in Debian unstable, I've set up all my apt repositories to be signed. Actually, they were since 2003, sorta, but it wasn't documented. If you use the uqm.debian.net repository, or if you use one of my personal kitenet.net apt repositories, there is a key.gpg file in each. This file contains the public key used to sign these apt repositories, and so there's a trust path of sorts to it, I've also included a signature of that file using my well-known gpg key in the key.gpg.sig file.

That's a start. Maybe we will develop better or more standard / automated ways to distribute third-party archive keys as time goes on.

Note that I cannot just sign the archive key with my main gpg key, because this archive key does not meet the minimum standards I require to sign anyone's key. The private key is stored on a server, and has no passphrase. It can really only ensure that info in the apt repo is not altered before it gets you you; it does not protect against compromise of the server.

It would be possible to make the key more secure and better trusted, or even use my well trusted key, but that would involve much more inconvenience and work each time I update my repository. I settled on this as a compromise, and it's implemented with this in my .mini-dinstall.conf:

release_signscript = ~/bin/sign-release-file

And this sign-release-file script:

#!/bin/sh
set -e

KEYID=joey+archive@kitenet.net

rm -f Release.gpg.tmp
echo | /usr/bin/gpg -a --no-tty --passphrase-fd=0 --batch \
    --default-key "$KEYID"  --detach-sign -o Release.gpg.tmp "$1"
mv Release.gpg.tmp Release.gpg
exit 0

This too could stand to be easier to set up now that every Debian user and their dog will soon be needing to do it.

secFUD

Zdnet has published an article entitled "Security an ongoing problem for Debian". Once again this article makes various quotes from Martin Schulze's blog. It also links to I page I recently created listing possibly security problems in stable based on the issue tracking done by the testing security team, and incorrectly attributes that page to Martin Schulze.

I'm now incredibly frustrated, since Zdnet is using the information on my page to bolster the false conclusion that Debian stable is less secure than it was last month.

First of all, any security comparison that involves blindly counting security holes is innately crap. That is the kind of comparison that yeilds conclusions such as "windows is more secure than linux". I'm fairly closely familiar with the current set of security holes listed on my page as affecting stable, and many of them are not even exploitable. Some of them are in code that is not shipped in any Debian package. And so on. Comparing data that you don't understand is not a good way to reach an informed conclusion.

The really frustrating bit is that subtly Zdnet leads its readers into thinking that -- just because my page lists so many possible security holes in Debian 3.1 -- things are somehow much worse than they were a month ago when it cites our "ongoing security problems" as beginning. And they do this without even having access to the data about how many holes my page would have displayed a month ago. For what it's worth, when I run the numbers, and correct for some of the worst sources of innaccuracies (filtering out some 900 fixed holes in the process), I find 385 potential unfixed security holes in Debian 3.0. Somewhat more than the 101 listed for Debian 3.1.

Why? Well, recall that Debian 3.0 is a linux distribution that was released in 2002, and that many packages in that distribution (cf mozilla, kernel) had numerous known unfixed security holes, that couldn't be fixed due to the massive job involved in backporting those fixes. Add all the little minor security holes that tend to be skipped over in favour of the remote root of the day, and the figure of several hundred unfixed holes in 3.0 begins to make sense. It seems likely to me that Debian 3.1 is more secure than that.

Which isn't to say that I'm very satisfied with the security of Debian stable, or with the frustration that must be driving Joey to continually blogging about his troubles. But drawing the kind of inferences Zdnet has drawn is just fear-mongering.

seating

Last couple of long flights I've been on, I've found myself looking around the cabin and trying to imagine what it might look like if designed by people lacking our particular set of hangups and blinders (or by aliens). Looking around at all that wasted space in luggage racks above the center seats that could be, say, coffin hotel bays. Or put the passenger cabin on the bottom of the plane, with windows in the floor -- nice view! Or do away with the seats, just stand up hanging onto straps for takeoff/landing and carve out one's own territory on the floor while in the air (in-flight entertainment!). Or, and this is perhaps strangest of all, stop trying to pack a maximum amount of passengers onto the plane and then only booking half a flight 30% of the time.

Sure, I come up with silly ideas after sitting in the same cramped seat for 6 hours, but let's face it, aside from changes in first class and some incremental improvements/unimprovements, airline seating has not changed much at all for decades. And I refuse to believe we've found the only or the best solution.

In the meantime, seatguru is a nice resource for trying to find a seat in coach that doesn't suck.

search for sand and sky

That sums up the last two days, as I've headed out toward the coast in the Carolinas' piedmont. Little Pee Dee State Park, Croatan National Forest. The ground has gotten sandy but I've had less luck with the sky between rain and trees, which has made both travel and satalite locks hard.

Now a brief night back online to get some work done before I'm hop on a ferry to Ocracoke and am offline for the rest of the beach, where there will be big, beautiful sky but no power. :-/ Ah well..

screenshot

When I realized that I had more d-i install windows open than I had screen space, I moved them all into my clutter frame, shrank the fonts, and then couldn't help taking this screenshot. Clockwise from upper left the running automatic installs are: hppa 2.6 install, i386 install with text frontend, ia64 booting an installed system, sparc install. Not shown: s390. Next up: probably mips.

screensaver

"The screensaver you gave us is wonderful. We spent Sat. nite listening to Prarie Home Companion and watching it." -- Dad

Shamelessly republished from an old private email due to conversation at DebConf5.

SCMs are the $EDITOR of the naughties

Took me far too long to realize this. svn vs baz paintball anyone?

SCC

For what it's worth, here's why I think we need the SCC^WVancouver proposal, or something similar. In a nutshell, it all comes down to what's best for most of our users.

It's looking more and more likely to me, based partly on straightforward extrapotation and partly on gut feelings, that in the next ten years or so, we have one of the following two choices about what Debian makes available to our users:

1. Possibly two releases for all architectures. Two. In ten years.
2. Some appropriate number of releases (say, 4) on a few architectures, or with some other significant limiting change.

After it's three years old, Debian stable is barely usable on new hardware, can barely function as a server on the internet, is not attractive as a desktop, and is barely maintainable for security. If a later release is delayed even more, stable will be useless for most purposes.

So if we go down road #1, at some point we will need to find a way to make either testing or unstable a viable alternative for more of our users. I've actually already been working on doing this, through things like organising the testing security team, and by making periodic "releases" from testing on the pretext of beta testing d-i.

I observe many of our users already switching to those releases; I see increasing interest in the testing security team, and more people working in it; I see derived distributions like Ubuntu doing essentially the same thing and users switching to it in droves; I see my employer's distribution (debian-edu) giving up on stable and planning to just base releases on testing too.

We're farther down road #1 than you think.

So given this choice, I have to pick #2. Why? Well, because I don't think that any amount of work that we do to try to make #1 palatable to our users will end up satisfying all of them. Some of our users need stable releases. If we can only give them useful stable releases on a few architectures, that still wins over no useful stable releases at all.

Also, unlike apparently 90% of everyone I've seen comment on this, I actually think that leaving some arches up to the porters will allow them to make usable releases of some sort. Maybe not all arches, but enough to swing things even more toward #2 in my estimation.

I doubt that SCC is the only way to get us down path #2, but it's so far the only thing that has even a glimmer of hope that we'll avoid the trap of

1.

sauna

First non-Finnish sauna today. Was electric and too dry and not hot enough, otherwise no complaints..

satutils

Spent most of last night researching/hacking the proprietary bits of my satellite internet setup, and today writing some libraries and scripts to allow them to be used in a more unix-like way. The result is satutils, and although I have some planned additions -- especially the gpsd interface -- it's already much nicer than the built in web-based controls.

I do wonder if more than 10 people will ever use it. More than 1? But who knows, maybe there are masses of Motosat enabled linux users out there. If so, this is the package for you..

sandbox

I think that driving a bobcat is the most fun you can have for $20/hour of someone else's money. And I can't help but flash back to old anime when I am the robot monster in my mobile suit. Tired now, pics sometime..

running around like crazy

It's a hectic couple of days, starting the process of moving is hard after being anchored in place for 3 years. And this move is planned to not be complete for nearly 2 more months! Now I have a weekend trip to my dad's. When I get back it will be the string freeze in d-i, maybe I will have a little spare time during that, and then the push for the next release.

I learned on Wednesday that my laptop's CD drive can survive being covered in water for hours. This is a nice ability, but not one I want to try again.

I haven't talked about what I'm reading lately, mostly because I haven't been much. I went back and read some very early Mercedes Lackey, which is very strange to revisit. Now I'm reading various things by Norman Spinrad, who I also haven't read in a long long time, but who I enjoy much more -- he has a way with words. I tried a few of the Hugo nominees, but I could not get into them, probably due to lack of concentration.

rss2email packaged

rss2email is in NEW. This has been a case study of why contacting upstream authors is a very good thing when packaging software for Debian. If I'd not written upstream for a minor copyright clarification, he'd probably not have released rss2email 2.0 -- or at least not yet, or I'd never have known, since the web site moved. And it fixes several bugs that have plagued me all week as I tried out the original version. Also, upstream was "honored" to have his program in Debian, and is being very responsive, just released 2.1 with the changes I had to make to fit this thing into Debian and more fixes. From dormant to busy, it's nice to see.

rss2email

Now I have rss feeding into email using rss2email. I looked at toursst, but it didn't convert html to text, so hacking rss2email was an easier option. Also, rss2email sends directly to a SMTP server, while toursst writes to a maildir.

Hmm, I suppose I must have learned at least some python by now, since I had no real trouble fixing rss2email to use a dotfile for configuration instead of coding values in the script, and similar things necessary for a debian package. Not suprising, I learned C, shell, and even I suppose perl (deep in the mists of time) in much the same way. Guess this means I'll have to go back and fill in the holes eventually by reading some documentation though.

Anyway, the Debian package will be available soon if I get a copyright clarification on a few dozen lines of code.

root passwords

So, we only today realized that nobody told me the root password for newly installed systems here at work when I started. I was logging into them as root using the root password I always use for sacfificial test installs, which happened to be the same one the guys here use.

Hmm.

roan

A bit more on Roan Mountain state park. This Tenessee state park is only 2000 acres, but it feels much larger when you're down in the foothills at 2600 feet, looking up at The Roan 2500 feet above. Of course it's nestled inside the Chreokee National Forest, so the undeveloped land really does extend much further.

Hiking around today I kept feeling like I was in California, I'm not sure if this is because of all the rhodadendrons, which are a deeper shade of green, like is common out there, or the chilly wheather, or the lack of undergrowth (spring is coming slower up here), or something about the unusual rockforms I encountered.

I also enjoyed seeing some deer, and crossing a myriad of little streams, each unique. Usually when I get here I immediately head up to the top of the mountain to see the view and experience the balds. I probably won't get up there until Friday this time, and it's interesting to take in the rest of the park for a change.

ressurrecting chicken

In retrospect and in the best traditions of unix sysadmins, calling a server chicken is just asking for trouble. But it's back up now, reinstalled in just 20 minutes downtime thanks to d-i, and sporting a nice new raid5+lvm setup for the important bits (still in degraded mode waiting on the replacement drive though).

Restoring all the data will take a bit longer, since I have to download a lot of it, including the Debian mirror, and new full duplicity backups from kite. Still, I hope to have the automated test lab back up and running within a couple of days.

replacing stuff with busybox

Martin-Éric Racine: You asked about embedding Debian with busybox.

Would anybody happen to have more details about exactly which packages Busybox can replace while still providing just enough functionality to boot a Debian-based system up to an X display manager?

This is in general an impossible question to answer since any given Debian package may or may not work with any given utility from busybox. What we do at work to shoehorn a X-capable embedded Debian-based arm system into 32 mb or so of flash, is to install our set of packages, then tweak things until busybox diverts nearly every utility it supports. Then we snapshot the files used for a shutdown, boot, and the specific use of the system we want to support, and copy just those files over to our image, which ranges from 4 to 32 mb.

For the busybox related parts of this, we use a program I wrote called busybox-links to set busybox up to divert everything cleanly and (semi-)reversably using dpkg-divert (and in some cases update-alternatives).

busybox-links also tweaks a number of init scripts to work with busybox, stuff like editing /etc/init.d/hwclockfirst.sh to run hwclock without the --noadjfile parameter, which busybox lacks, and a dozen or so other such changes. Happily busybox readlink now supports -f, so the main set of things that had to be patches doen't anymore. This set of tweaks likely won't be sufficient for your system, but you can add more in a like vein.

busybox-links is contained in our ads-additions package. As for reducing the tweaked Debian system down to just the files needed to run, we use the adsrootbuilder, which is still quite specific to our systems, but which I hope to eventually make less so. Both these packages can be downloaded from our apt repository:

deb-src http://www.applieddata.net/developers/linux/debian ads main contrib non-free

Of course, that's just one of many ways to do it. Part of the problem of embedding Debian right now is that there are as many ways to do it as there are people doing it, and no agreement on the right way, and so little progress in some areas.

In the future, I for one would like to see a broad agreement in Debian that making init scripts and the like compatable with busybox is worth doing if it doesn't make them significantly slower or uglier. Then I could send all these tweaks in as patches, which would be a better use of my time than maintaining them going forward, and would make Debian an overall more flexible and more useful system for embedding.

(PS, RE what debootstrap installs; it's based on whatever is > standard priority. If you have improvements, might I suggest talking to its maintainer, not blogging?)

releases and such

Until I had a fun time playing frisbee this evening I was quite frustrated at the autobuilders, which have been not building the final build of d-i for our release all week. The system they use for prioritising builds is broken for packages like debian-installer, and so rather than moving up in the queue it's just been sitting there while torrents of last minute uploads of packages come in before it. We've done manual builds for some arches, which is annoying because it sometimes wastes the autobuilder's time later, and because it has the potential to introduce the human element, and thus, mistakes. Which we don't need for a proable release candidate build. Anyway, the timing has been truely bad and the more time we lose now, the less time we'll have to fix whatever issues are found in this release, before Debian itself releases. But maybe libtiff is more important, I can't say. Hmm, I'm still annoyed despite the frisbee, I see.

It's interesting to think back to past Debian releases. I was only mildly involved in the woody release really; my Debian involvement was at an ebb that year. The potato release was a reasonably stressful time, though I don't remember exactly why. Before that there was the slink-and-a-half release, which Sean and I put out ourselves and doesn't really count, and then the first Debian release that I remember being involved in, slink. At the time the website had to be gone over with a big s// to change release names and such, and I participated in that, a little amazed that it had to be done that way. For several years before that, I was a Debian developer, and yet releases just happened on their own, apparently. That was nice.

For this Debian release, I'm helping with d-i; with CDs; with the install manual; I've been looking over what will need to change in the website and hopefully we'll have one switch to throw this time despite all the changes required for d-i; and I hope to help the security team out some. And then there's debian-edu. Luckily I dodged the release assistant bullet, so I can avoid paying attention to many of the testing propigation issues. Colin and Steve are doing a fine job over there, but I can see how AJ could get just a little bit tired of the mass of developers that don't look outside their own packages, and always feel that their fix is the most important thing.

I wish that we had a clear release checklist for Debian that detailed everything that goes on in last weeks, days, and hours before a release, and what depends on what else. I'm sure it would be fascinating. But I doubt we've ever done things the same way twice.

I will be able to go to the last week of the Cabal^HHHnonical conference in Oxford. I do not have any release plans for that week; if d-i is not released by then, someone else will get to do it.. I expect that will be a fun and good work session; I just need to figure out a way to escape the Mao players.

release team meeting

I was in Vancouver for a meeting of the Debian release team plus James and Ryan, which was dreamed up by Andreas and me when we were in Oslo as a way to try to use some of the resources of debian-edu to improve persistent problems in Debian.

I'm pretty happy with what we managed to decide and accomplish. Once we were all sitting down around a table it seemed we'd all been thinking about the same problems, and when solutions were proposed they seemed to be the same ones everyone had been fumbling toward on their own. We seemed to quickly arrive at solutions to problems; we worked through an agenda we'd all expected would take two days in just the first day. I'm not going to talk about specifics until things are finalised and announced, but some of the results are sure to be controversial. Others will be enthusiastically welcomed by most everyone.

release team flambé

I've tried to start this blog entry about three times before, and just gave up, because when I try to describe all the things that are so badly wrong in the vast flaming thread that followed Steve's annoucement of the SCC proposal. Happily it turns out that others have kinda said it for me.

Patty Langasek

These temper tantrums have, as expected, clouded the perceptions of what could potentially be otherwise intelligent human beings. Apparently, people have forgotten how to read.

The number of people who are still posting flames to The Thread based on inferences they drew by skimming the head message is unbelivable.

I'm just.... amazed.... at how stupid the geek^Wnerd community is. Honestly. I doubt even half of them read the actual proposal, and of those who *did*, I doubt even half of them bothered to think beyond "OH NOES TEH SKY IS FALLING ON THE INTERNETS". Between conspiracy theories proposed by "Debian Developers" in the Slashdot comments (OH NOES THEY MET WITHOUT ME. CABAL! CABAL!), the cries to switch to NetBSD and the cries to switch to Gentoo, I just can't help but scratch my head.

There were a few comments I wanted to reply to, but chose not to because, well, that would be silly of me. There's no getting through to any of them and I know that. It doesn't matter what the truth is, there will always be a better "cabal" or whiner story to offset reason.

And those who actually work to bring about some form of productivity or accomplishment will always be in the crosshairs of those who prefer to stomp feet and demand they be given equal say without equal amounts of work.

Daniel

If the knee-jerk reaction results in causing pain to those close to our important teams; can we actually expect 100% that those members won't be tempted to throw in the towel in order to protect the ones that they love? For goodness sake people -- get a grip.

Anyway, I could go on, but I have had an epiphany. I'm pretty sure that my overall experience with Debian will turn out to have been divided into at least three distinct stages. Stage one began with my initial exposure and beginning to learn about Debian, and ended when I stopped just making the changes I wanted and looked outside my own packages at the larger project (think packaging games, writing debhelper, alien, etc). Stage two mostly involved me doing larger and more ambitious things that required getting the whole project to change (debconf, policy, starting d-i, maybe testing-security, etc). Stage three will be when I've given up on that and am just coasting along on loose ends before my exit. In retrospect, stage 3 may have began about 1.5 years ago, though if it has, I still seem to have about 5 years worth of loose ends in front of me.

reentry

It was good to be (mostly) offline for a few days, even despite the 200 mile drive today I came back with more energy than I left with.

BTS traffic goes in strange cycles. Days will go by with no bugs, and then I'll get a whole load of reports all at once. Problems will lie unreported for years, and then two people will report them the same week. Today I received a slew of "can you still reproduce this / please send me more info" messages, and several bug reports whose submitters closed them as user error as soon as I replied to the bugs.

I wonder where sid/Contents-i386.gz wandered off to?

T minus 10 days for d-i beta4, and still s390 and hppa are not in a releasable state.

redoing my blog

After the last blog entry I got some feedback which (politely) suggested I stop whining and just use existing pyblosxom features. I didn't know that pyblosxom supports static, incremental blog rendering. And Tollef claimed to fix his pymarkdown plugin, although the updated version doesn't seem to do anything with my .mdwn files.

Anyway, I ended up spending far too much time today redoing how my blog is set up. I'll continue on into the gritty details, but anyone syndicating my blog should note that the new preferred url for the rss feed is http://kitenet.net/~joey/blog/index.rss, although the old url will keep working for now too.

static rendering

First I tried out pyblosxom's static page rendering. To make it work I had to reorganise my blog a bit, but it was worth it. I ended up with a simple Makefile, which runs the pyblosxom.cgi direct from the Debian package. No more need to copy and munge pyblosxom.cgi -- hurrah!

I did have to work around a pyblosxom static rendering annoyance. Actually a RSS annoyance: the absolute urls thing. Since I run a local copy of my blog on my laptop, and one on my server, I can't just hardcode a base_url like pyblosxom wants to for static rendering. One of the tricks in the Makefile and associated config.py is the use of the BASE_URL environment variable, which is based on the build machine's hostname.

I also had to set up some rewrite engine stuff, and make some other small changes to try to keep old links working as much as possible. I know of one place where I slightly failed, but it mostly seems ok.

With these changes and a hefty dose of moving everything around, I ended up with this source tree for my blog, in which I can just svn up && make to build my static pages in a second or two.

There are still some kinks. Notably, if I delete a blog entry, the static version stays around. And even modifications to blog entries will not be caught by the incremental run, since I also use the pyfilenamemtime plugin. I'm not sure what to do about that problem yet.

markdown

With the static rendering bit taken care of, I next tackled using markdown. Unfortunatly, I could not get Tollef's pymarkdown plugin to work properly.

After fighting with it for an hour, I sat down and in 15 minutes had hacked together a generic pyblosxom formatter which can easily be changed to use any command as the filter program.

You'd not want to use this if your pyblosxom blog were a CGI script, but once the leap to static html has been made, pyloxsom becomes just a big old compiler, and shelling out to an external perl program once for each new blog entry is not a big deal.

(That, by the way, is my most "significant" python program to date, and I found myself googling for stupid stuff like "python string concacentation" and "python backticks equivilant". One of these days, I may actually try to learn python instead of muddling around in it..)

Everything else

Hmm well, I've descibed 90% of my weblog setup now, I might as well finish up with the rest of it so it's fully documented. I use a few other plugins:

• filtersvn is a hack someone wrote for me that makes pyblosxom ignore .svn directories.
• I use the stock pyarchives.py to get archive links, etc.

• pyfilenamemtime takes the ugly datestamp at the end of my blog entries and uses that instead of the file mtime.

  The reason I use this is because I keep my blog entries in svn, and I don't want any svn thing that changes all the mtimes to mess up the order of items in my blog. I'd really like to get rid of it, and it's not really needed most of the time, but it does come in handy when I nuke my home directory and check it back out of svn.

For writing entries I use a little blog script, that lets me edit a new entry in a text editor, then works out its filename, checks it into svn, and regenerates the weblog.

I think that's the lot. I'm obviously foolish for not just using livejournal, but all this work does give me a few minor benefits that seem worth it to me, at least some of the time.

recent d-i hacking

Last night was, in retrospect, a fine night, but at the time it sucked. I was about ready to go to bed when a tester came into #debian-boot and reported that his /etc/fstab lacked a /proc entry. It turned out that my changes earlier to partconf-mkfstab had broken partman's /target/etc/fstab generation. Actually, it turned out that said generation was all a nasty hack around the misdesign in partconf-mkfstab. When I corrected that misdesign, the hack broke. So between 2 and 3 am I cursed and fixed this up properly. The nice thing is that the new fstab generation code lines up the columns in the fstab prettily.

Then as I was hoping to finally go to bed, I discovered that base-installer was broken, for reasons to silly to go into here. I fixed that and finally got to bed by 4:30 am.

Today was kinda more of the same. I found a particularly gross bug in netcfg, whose dpkg client killing code failed if the pid of the process was < 1000 or > 9999. Ugh! Then another netcfg bug causing it to write (null) to /etc/resolv.conf.

I've just finished up splitting lvmcfg so it plays better with partman, and I've done a successful lvm install. For once d-i is doing something that I couldn't do by hand (I don't know the lvm tools), and it feels good.

I've not had time to do anything but d-i for days and days, and I am looking forward to the 16th when this will all be over. Of course I'm not the only one working hard on d-i; Anton did many partman improvements today; translators are working hard (entire new Welsh translation added today!); Denis rewrote the cdebconf text frontend for S/390, etc. It's been a very busy day judging by the commits.

Now off to test today's CD images, which should be the first to properly support XFS, since the 2.4.25 kernel deb is finally in testing.

Getting excited already about DebConf4 in Brazil. If you're attending DebConf4, would you be more interested in a talk on d-i (I could go on for hours, see above), or a mostly non-technical talk about speeding up Debian development? Mail me.

re: udev, the usual scapegoat

MD, the only problem with the "not udev's fault" theory is that the kernel in question is not very modular: usb-storge is built in, and udev manage doesn't load any modules at startup..

I don't know yet if it's an udev issue or a kernel bug triggered by udev. That's why my original entry said "I don't know whether it would be udev's or the kernel's bug". Doesn't sound like a scapegoat to me.

re: ubuntu

Norbert seems to miss a few of the things I said in my post about Ubuntu, including the fact that I can understand why DDs would use Ubuntu despite it annoying me. Perhaps being annoying^HHHed is my natural state; your post didn't help.

Oddly, the reason he gives for using Ubuntu, a release and security support, is not one of the reasons I'd have considered. You see, we're released sarge too, several times, with each release being better than the last and the most recent (rc 1) being quite good. Perhaps most don't consider d-i betas to be true Debian releases, but I do. As to the security support, please consider sarge to be security supported from here through its release; these things are being tracked, and fixes are being expidited, and currently of every DSA since woody was released, only 5 are not in sarge yet (bear in mind that there have been 7 DSAs this week alone).

There's a good possibility that some of your old preconceptions about security support for testing will need to be reexamined after sarge is released too, BTW.

Re: Pledge To Killfile Andrew Suffield

Mako, I think I understand what you're trying to do, but I don't feel comfortable supporting it because of the way it's singling out one individual and the poisonous behavior that's already engendering on debian-project.

I don't use killfiles myself, but when I find someone being a git/idiot/repititious on a mailing list I begin deleting more and more of their posts (and the replies to their posts) unread, and ignoring them more and more. Conversely, other people generate posts of such uniform high value that I will stop to read anything they write even if it's in the midst of a thread I have written off as useless.

I think that a system that tracked such behavior and generated a list of people whom others were not finding useful paricipants (and possibly the converse list too), might be a more neutral way of accomplishing what you are trying to do here.

There, I've reduced it to a technical problem, now we'll get somewhere. ;-)

RCS addiction

This week has proven to me how addictive it is to use a revision control system. Of course I learned this by trying to go cold turkey (at work).

The project at work was explicitly not being kept in revision control, for reasons I won't go into, and there were two of us working on it. And it involved changing lots of files and copying them around to different places to run them on different systems and stuff like that. At first we were working on different parts, but as we converged on the same problem areas and began trading off on working on some parts things just got messy and stressful.

Do I have the most recent version of that file? How could I update this other file on the web server? Should I send a diff with my changes so you can integrate them in with yours? By email or scp? Could you take a look at this change here on my laptop, sorry the screen's so small and hard to read. Didn't the version of this we had before lunch work better than this new one? On and on..

After finishing that and moving on to a different thing that is in revision control, I was struck by how all this chatter across the office went away and things just worked and we had a common foundation and a shared history to work from (even though I had to go back and read the history, being the newbie). The kind of foundation that can make communicating about the actual issues very quick and very obscure to anyone who lacks that context.

So you switched to pax last September because you needed to munge the symlinks that way?

Even more, revision control added a background communications channel that hadn't been there. So make a change only you expect to care about and maybe an hour later your co-worker says, out of nowhere "nice fix to the dist script". Or alternatively, you see another quiet commit tagged "fixed dist script breakage". Of course that's the way things normally go in all the projects I work on; indeed some of them like the Debian testing security team even manage to do almost all our communication over that back-channel.

Aj was blogging earlier about code comments and how the revision history of a line of code in an is increasingly more useful than whatever outdated comment happens to be above it. Not only can you see who put that line in, and how it used to read, but what they were thinking at the time. AOL to that.

Incidentially, this week I committed the 1000th change to tasksel, and the 30 thousandth change to the Debian installer. And both were just as good for me as my first hit of CVS, thankyou.

rc3

Things are getting mildly crazy again, as d-i rc3 nears. Predictably, some people tried to get last minute changes in and risk delaying everything else by doing so. People don't learn.. On Wednesday the udebs for rc3 will be pushed into testing, certian rc2 install media will break, and the pressure will really be on.

I'm either getting burnt out or I've done that enough times that I just refuse to really stress out over it anymore though.

rand(topic**2)

So, a new rule. If you blog about something related to installing with the debian-installer, you must also file an installation report. Otherwise, I will mock you. ;-) Just assume the d-i team is getting 1 beer per successful installation report (corporate sponsors are welcome).

Today was spent messing with boot-loaders and getting a beta3 update released. Only 2.5 arches to go.. The ports status page is showing a lot of yellow and green.

Gratifying thing a day or so ago: a user came in with a pcmcia cdrom, and while d-i didn't support it in the published images, I was able to build him a 3 mb iso image that had the necessary stuff to see it. All with a couple lines of changes in the udebs lists.

I would like to drop the broken debian-installer-demo package, unfortunatly this otherwise useless package is a sentinel package, used to let the autobuilders know they need to autobuild d-i. That process also builds the installation images, but the autobuilders don't track BYHAND tarballs. The only idea I have for a sentinal package to replace it would be a debian-installer-manual. Problem is, while the manual is arch specific and is indeed built already, it would really be more useful to include manuals for all arches in such a deb. I may compromise and only include the current arch, anyway. It seems better than adding a debian-installer-hello-world. The only other idea I've had is a debian-installer-undo package but installing windows or macos from inside Debian is not something I know how to do.

I tried to get my tickets to Brazil for debconf4, but the credit card is not cooperating. Will have to call the bank tomorrow.

rand($topic)

Today turned out to be the d-i subversion flag day, so I spent most of the day moving stuff around, updating docs and links, and cursing alioth's substandard support for subversion. Despite some fixes, the repo is still getting wedged periodically.

I've been really, insanely busy lately, spent most of Friday demolishing a house, spent yesterday doing something less exciting that I can't remember but certianly ate all my time. So I've discovered the dark side to the excellent rss2email:

---Mutt: =.blogs [Msgs:245/331 New:117 Inc:7 565K]---(from/date-received)-(all)--

Yes, this stuff can stack up just as bad as reglar email, and yo can't just stop hitting the web page for a few days.

Today was also the end of the non-free general resolution voting period, and it did even worse than I'd predicted. In a sense this is a good thing; the non-free advocates have a definite mandate to point to, for perhaps a few years until the situation is perceived to be significantly changed again. Does this mean all the threads about non-free will end? Hah.

random whatevers

Back in the Bay Area, I used to know Seth David Schoen as a smart, articulate guy with a cool beard. Nowadays, he's blowing me away with his eloquence and insights, as exemplified in his history of the DeCSS haiku (which he wrote).

Snow this morning, so I finally got a good hike in the snowy woods. I've also stabalised the problem with my laptop's power cord, so this has been a good day.

Strange stuff in d-i today; it turned out that libm was bloating up to 15 kb on the root floppy because wireless-tools was added to the net_drivers floppy and the library reduction took that into account. Rebuilding the wireless-tools-udeb to use its own internal math code instead of libm's not only reduced libm back to a 1k stub library, but it shrank the iwconfig binary some 12k. Puzzling, but I'm not complaining. Except that glibc is a bloated pig, and we really should move to uclibc eventually, before this floppy fills up for good.

llrd is indeed nice, although I'm not sure if exporting you entropy pool size to the world by default is a good thing.

random tip: redirecting alsa to usb sound card

This took a bit too much work to figure out so here it is. If you're like me and had to buy a usb sound card (nearly as sensible a name as a cable modem) since the laptop's onboard headphone jack broke, and would like to have that card used by default when it's plugged in, but otherwise have builtin card be used, create an /etc/udev/rules.d/00_local.rules containing:

# Default to using additional (USB) sound cards when they are available.
KERNEL=="pcmC[D0-9cp]*", ACTION=="add", PROGRAM="/bin/sh -c 'K=%k; K=$${K#pcmC}; K=$${K%%D*}; echo defaults.ctl.card $$K > /etc/asound.conf; echo defaults.pcm.card $$K >>/etc/asound.conf'"
KERNEL=="pcmC[D0-9cp]*", ACTION=="remove", PROGRAM="/bin/sh -c 'echo defaults.ctl.card 0 > /etc/asound.conf; echo defaults.pcm.card 0 >>/etc/asound.conf'"

Only programs started after the sound card is plugged in will use it of course, and this might not work if you have it plugged in while booting.

random thought: musicians in movies

One of my little quirks is that I really dislike knowing who a movie star is. If I recognise someone from a past movie, that pretty much ruins my suspension of disbelief. So I try really hard to not find out anything about movie stars. Luckily a little makeup and some good acting can fool me often, or I'd enjoy a lot fewer mainstream movies.

(Data point: I recognisd only two of the stars in Oceans Eleven, and I cannot recall their names.)

OTOH, there's something I really like about seeing non-actors who I know cameo in a movie. Mostly musicians, but also authors, directors, scientists, athletes, etc. Even better is when I know them only in passing and don't know who that was until the credits. There's often more depth, hints of a more real personality, and I just focus on these people and start trying to figure out what's up with that character.

Notable examples are Blues Brothers, many Jackie Chan movies, and Because of Winn-Dixie.

random thought

I wonder how many of the people who constantly whine about Debian removing non-free documentation have ever thanked the authors of any of the many free docs in Debian their work?

random stream of conciousness

GPG signatures keep rolling in from the DebConf4 keysigning. Pity I somehow goofed saving the file of keys, so I lost my notes about whether I'd verified quite a few, and was only able to sign the keys about 10% of the attendees. I've already uploaded the few keys I was able to sign to the keyservers.

I don't check email address validity when I sign a key, and it seems that the new trend is to do so (to some degree, I can think of lots of holes in the techniques people are using), which means I've been getting lots of gpg encrypted messages whose subject purports them to be signed keys. Anyone monitoring my email is probably having some unexpected fun trying to figure out what all this encrypted traffic really is; I don't normally get much encrypted mail. I suppose that increasing the amount in use is only a good thing -- that's part of why I gpg sign all my outgoing mail after all.

I had a nice bike ride yesterday up the trail from the park in toward town. I've not biked for ages, and I rarely see many bikers around town here, so it was nice to see how much traffic that trail gets. Also rather suprising to hear three different languages in small town Bristol on that trip. It's a very well done green way, too.

I unpacked a bit from Brazil yesterday, and did laundry, and other things to prepare for this week's trip to Ocracoke. I guess the rest of the stuff that's in my bag can just stay there for the trip. I've become very bad about unpacking after trips, in fact I have still not entirely unpacked from my trip to Canada for DebConf2, which was 2 years ago. I discovered that old suitcase when I was looking for a pair of shoes -- and it still has some other stuff in it too. That's the worst, but when I packed for Brazil I had to unpack some stuff from Germany. I am just no good at dealing with objects, having them in a bag shoved away somewhere is easier.

That reminds me, I've done some moving in at the new house, which involved taking some boxes out of storage that'd been there since I moved east from California. This included a lot of stuff I'd forgotten I owned, and some things I've been missing for years. About 50% of it was thrown out though. I have very little compunctions about getting rid of belongings now, there's only a small, shrinking set of things I have attachments to.

Alternating between living in two different houses this week has been interesting, and not very stressful. I was worried that it would be a lot of trouble; glad it's not been so. I'm happy with where I am WRT work, for the first time in a couple of weeks.

random idea re new queue

I had an idea about a different way to handle the new queue, to take some of the pressure off the ftp-masters and ensure that if no ftp-masters are available, some new queue processing is still possible. The basic idea is to allow developers to send in signed .changes files as vouchers that they have examined a package in the new queue and believe it is suitable for Debian.

For this to work, packages could be uploaded to an alternate, public new queue. Of course you have to take care of any possible crypo export and copyright issues when doing so, so some packages may not be eligible for upload there. But this lets developers examine packages in the public new queue, if their maintainer uploaded them there. An obvious package to upload there would be a package that is not really new, just split into a new binary package.

A developer would then download a package from the public new queue, check it and make sure they belive it's suitable for Debian, re-sign its .changes file (and rename it), and upload the new changes file. Call this advocating a package. The specific checks the developer performs is up to that developer; they're putting thier reputation on the line that the package is suitable. We would probably develop a standard checklist. Once a package got the requisite number of signed .changes files, it could automatically be accepted from the new queue w/o ftp-master action.

I expect there would be problems, possibly involving updating the overrides files that I am not aware of, since I'm not a ftp-master. We'd have to work those out. There might also be problems with developers abusing this priviledge, so the ftp-masters would probably need the ability to limit the number of packages a maintainer could advocate in a given time period. The ftp-masters might be more comfortable with only giving this ability to some smaller subset (but still in the dozens) of developers at first, though I'd hope it would in the end be something most developers could do.

random blah

In town today in a hurry to do my errands, I made a mistake. Ate a burger at a fast food place. So I felt bloated and nasty until I got home and washed it entirely off my palate with a supper of black bean soup, jalapeños and sour cream. If only I had the willpower to be a vegetarian full-time.

It took over 12 hours, but I've finished the dump/reload for the new version of subversion. And today I committed revision 10000 to my main subversion repository. Wow!

The new power supply arrived, and my, it's tiny. Same design flaws as the old one though.

I'm afraid that the main thing Orkut makes me want to do is get out and meet new people who are not on Orkut. No offense intended.

After giving up on a novel by Benford 1/3 through after it ignored lightspeed delays between Earth and Venus (there went my suspension of disbelief), I've been reading Clement, and the current book has its interesting points, but he seems to have left out the first two expository chapters or something. 50's sci-fi -- only good in small doses.

random bits and peices

The taint of netscape lingers on. Mozilla firebird has appalling DWIdM behavior when resolving urls that consist of only a non qualified domain name, when the web server is down. I have not been so enraged by a peice of software in recent memory. My laptop narrowly missed going through the car's windsheild (warparking). I expect that the verisign sitefinder people are busy trying to hire the intreped mozilla developers. Mozilla is purged from my laptop, and I'm using w3m again. Won't last.

Yesterday something strange with debconf. I had been unable to reproduce bug #226963 for a long time, and its submitter was also having trouble. Then suddenly the same bug popped up again (filed against xfree86), and I was able to reproduce it with ease, and fixed it. I suspect that whiptail's behavior changed in the corner case of exactly what it returned when nothing was input into a string input box. But why did it pop up briefly weeks ago, and disappear until today? Odd, but not important enough to investigate in depth.

In d-i land, encouraging progress with the m68k port, but otherwise we've been stalled too long with busybox tar issues. Lately I've had a hard time mustering any helpfulness to stupid user questions on irc and the mailing list. Not sure why, just a mood I guess. Maybe d-i needs a FAQ.

I noticed that fceu ultra moved to main because of the efp ROM, which is free software and in main. Unfortunatly, nestra hangs when running this ROM, so I am leaving it in contrib. I did mail its author and file a bug report though. So much for getting rid of one of my two packages in contrib so easily.

Yesterday and today I've been working with Per Olofsson on a very odd series of sponsored NMUs, of pcmcia, including builds of the pcmcia kernel modules (which I hope nobody still uses). It was potentially very complex, but Per managed to make it impressively easy.

random

Debian mail is being slow again, which doesn't help in trying to coordinate the d-i team working on the release. I did the "final" initrd upload yesterday, but it turns out to FTBFS on m68k and hppa, I know of very rough fixes but am still waiting for the maintainers for the respective architectures to get back to me with better fixes.

I looked over some of our oldest installation reports, closed some, and sent out an enourmous ping to 350+ of the rest, asking their submitters to please try again with a modern debian installer. I hope this will generate some useful data. Since we seem to be falling hopelessly behind on processing installation reports, with a backlog of over 700 now, I will probably just eyeball and close any reports that I don't hear back from the submitters on.

(Erm, that won't help master's slow mail problem, will it..)

It's probably time to revisit the usefullness of installation reports. It's rare for a report now to be about more than one problem, as opposed to early ones that were typically about what worked at all. So it would be better to get them reported to the right package in the first place.

I was hiking around in Steele Creek last evening and ended up on the ridge, near dark having taken the wrong fork in the path and about half an hour out of my way. Barely made it home before full dark. Still it was a good hike.

I have finally seen Eternal Sunset of the Spotless Mind, and rather enjoyed it, although it may turn out to be too sappy to reward watching it again. I also finally saw Dogma, and was suprised find it not offensive (not the religion, but the tendancy Kevin Smith has for lowbrow jokes) and enjoyable, and I quite liked the God character at the end.

A few days ago, I got not one, but two independant requests for me to write nearly the same article. From different people working for two loosely affilitated publications. Strange. With luck both articles will happen.

I have a crazy bochs+screen+expect setup that lets me test an entire d-i install in bochs with one command. It takes about 3 hours, but the plan is to run it overnight and add the result to my automated d-i test page. Still needs some work though; I got it to run from cron, but if it's run from cron it cannot mount the CD. Strange.

quod libet

Earlier I dismissed the quod libet music player since it needed 20% of my laptop's cpu to play music. The new version has been improved, it only needs a still exorbitant 13%, but at home I use my shuttle PC for music now, so CPU usage doesn't matter. And I think quod libet is worth the slightly lowered battery life on the road too.

There's no end to the features -- and plugins full in all kinds of interesting edges (onscreen display, audioscrobbler, alarm clock) -- and while it's still under development with a rough spot here and there, mostly around the plugins, it mostly just works.

I'm especially impressed at how powerful the tag editing is in quod libet. It makes it so easy to clean up the missing and broken metadata in even a large sound library like mine, and the effects are so immediate and nice as things get sorted into the right place, albums ordered as missing track info is added, album covers displayed, that it's addictive just to putter around in there.

Some other features that IMHO make it worth using include the ramdom album plugin, which lets it pick a new album at random when it finishes playing the current one (my collection is too varied and big for random track play to work, but this, mostly, does), the gorgeous album list, and the queue, an odd little feature that lets you insert any songs you like into a play queue, to play after the currently playing song. Which solves a lot situations I've been in before.

And the little details are all there too, little things like right-clicking on the play button to turn off the music player after the current song finishes; and automatically and quickly finding new music on startup; and properly supporting multiple artist albums.

And that just scratches the surface of things I've found to like over a day.. there's a whole regexp/tag based search engine in there, a music prefernces learning system with weighted playback, internet radio, playlists and more. And it's actively developed, and kinda oddly marketed squarely at Debian users.

Buh-bye rhythmbox..

quality Sony hardware randomness

BIOS Upgrade
* This BIOS firmware will resolve the following symptom:
1. Machine self-power on when the body is being touched, without pressing the power-on switch.

programmer job security

Іf І wеrе rеаⅼⅼу ⅰɳtеrеѕtеⅾ ⅰɳ ϳօЬ ѕеϲυrⅰtу rіgҺt ɳоw, І’ԁ ⅾrоρ ρеrⅼ fⅼаt аηԁ Ьеⅽоⅿе а υɳіⅽоⅾе е×реrt. Іt’ѕ оЬ⌄іօυѕⅼУ ϳυѕt tօо ԁаⅿɳ ϲօⅿρⅼеⅹ, ⅼооkѕ ⅼⅰkе wе’ⅼⅼ Ье ⅾеаⅼіηg wіtҺ ⅰѕѕυеѕ ⅼⅰkе tҺіѕ fоr аgеѕ.

prayer rug

Today I received snail spam containing this "prayer rug". Here are some details on the scam, and scans of the complete mailing. Between the Jesus (with eye-opening action!), amazing underlined and ZIPPIED letter, survey right out of the latest blog meme, and people with innapropriatly big hair, this is a joy to possess, has to be a collector's item, and I'm tempted to go for the big bucks with it on Ebay.

Why can't email spam be this much fun?

power back on

Power failures are much more fun and exciting if they last only a few hours, or at most a day. Not three whole days in the middle of winter. I did a lot of reading (Michener, Jo Walton, and several Heinlein juvies) and learned to cook on a wood stove. The woods were gorgeous with the trees bowed down by all the ice that also knocked down my power lines. Today I was sufficiently tired of reading (and had so little unread material) that I did my tax return instead. Then the power came back on, so I didn't have to deal with another night of it.

I'm quite tired of local security holes in the linux kernel. I hope this recent spate is the result of an audit, and that we will see an end to them soon, and that it's not instead the start of a neverending stream as a rich new vein of problems is uncovered. Wishful thinking?

I'm a hell of a lot more stressed out now than I was for the past three days..

power

My power comes down the hill on a half-mile long spur line that ends here. Saturday night I heard a loud crack and the lights went out. The next morning I saw one less line crossing over the valley to the pole out back -- the downed line had snapped right off the pole for no apprent reason, and fallen into the garden.

I was impressed that they were out looking for the break even before I called (it only took out my power). Of course they hadn't a clue how to get down here until I found them up on the road. The guy looked it over and said he'd get a team out early today. I spent the night in town.

It was fixed by noon today, but they made a godawful mess doing it. They chopped out all the scrub and larger trees all along the line in, leaving them in an impassable mess under the lines along with lots of garbage. They even cut down one of the old christmas trees planted under the line on the valley floor -- which had to be 30 feet short of the line running above it. The sheer hack-jobbishness of the resulting wound makes me sick.

pirate

Here's a usage of the word "pirate" that was completly new to me:

But there's one thing very curious and unique about Doctorow that goes against all publishing wisdom: he pirates himself, with, apparently, the full consent of his publishers who would clearly like people buying his books instead of downloading them free.

(Markup mine.)

So piracy is no longer only boarding and plundering a ship on the high seas, nor is it selling cheap copies of CDs at a flea market, or even sharing copyrighted content on a peer-to-peer network. No, piracy now includes publishing one's own personally copyrighted work on the internet for free, if it's also available on dead trees. Inspired.

Perhaps what we're seeing here is the beginning of the end of any remaining negative connotation of the word "pirate".

Personally, I'm proud to be a pirate.

pictures..

From Anna.

(movie)

pics

I finally found the time to go through lots of other people's pictures and collect some for my photo galleries, of the wonderful time in Brazil with folks at Michelle's, and later at Debconf 4. Thanks to everyone who brings a camera to these events, I treasure the memories you record. Just to even the score, I'll bring a camera to the next one, in Oldenburg.

Phone problems resolved (I hope)

Finally a visit from the nice lineman, who replaced the box, and I'm seeing blazing 33.2 connections (that's baud people) when my sister is not hogging the line.

Phoney day

Unfortunatly I spent a large portion of today waiting for a phone repair man who never showed up. This is the sixth time a repairman called for the trouble on my line has neglected to actually visit the site of the problem (this guy seems to have gotten lost); it's all massively incompetant, and a collassal pain. Thank you Sprint for the dozens of lost hours, and the raised blood pressure. I am eagerly awating some future day when I drop POTS service completly.

Last night I researched alernatives. I don't think cable reaches out here, really, but I sent them a ping. DSL is right out while the telephone line sound resembles a CB radio on a crouded channel. Besides, the CO is probably 5 miles away. Interestingly, satellite providers seem to support linux these days, but on the other hand at least one of them is rumored to be going under, and there are tons of horror stories of it ending up slower than dialup, and of obscene usage caps and similar problems. There is no cell phone reception here due to topographic issues. The nearest WISP is at least 15 miles away, and I have never picked them up closer than 10 miles from here. Hm, not many options. OTOH, I regularly see maximum speeds of 700 bps down (up has no real problems), so I have to do something.

Chainsawing is not as satisfying as using an axe as a way to work off excess aggression after another bout with the telco, but at least I got half of the big logs sawed into smaller logs. Dinner was a reasonable stir fry. Most of the snow has melted off and I guess it will be a warm night, mid 30's (F), so I'm not bothering with a fire after all that chainsawing. Sigh, I had meant to go out hiking in the snowy woods while it lasted, but beta2 and the telco have eaten all my time lately.

pho comes to Bristol

I'd not realised how much I was missing a good bowl of pho. Bristol's a small town, where dining options tend toward places like the Blue Circle, the Corner Dog House, and Kay's. Oh, we have a good Chinese place and a decent Mexican, but not much else. And so it's suprising and a big deal that a Vietnamese restaurant opened here a few weeks ago, at least to anyone who's appreciated bun or pho before. Last week was my first time there and we ended up chatting to the owner and I suggested to her that they should put pho on the menu. Today I was back and their special was pho. Nice! Perhaps it's the cooler weather but I really enjoyed it. They need to work some on their peanut sauce though. Now if we could only get an Ethiopian place in town..

discussion

PhatNoise Music Manager

Why do prioprietary pieces of software often have names like the above? (Other examples: Macromedia Flash, Chipmunk Scripts Guestbook, Mojo Mail, PeopleSoft PeopleTools, Golden FTP Server Pro, Invision Power Board)

A walk though the CVE lists will turn up hundreds of hilarious examples of similar names. It's easy to guess whether a given security hole in that list affects Debian based just on the form of the software name even if one has never heard of the program before.

Even free software projects that seem to partake of a proprietary mindset show it in the names: Mozilla FireFox, OpenOffice, Mozilla Thunderbird. So different than w3m, vim, mutt. A whole pile of history, cultural, and, I imagine, linguistic difference.

Even if at first saddled with such a name, many free software projects seem to evolve in the other direction: Debian Installer -> d-i, X Window System -> xorg.

As someone who can't name software, I find this strangely fascinating.

personal wishlist 5087

Given some random English text like this:

Go to foo.com, and scroll down to the bottom and click on the link for "special offers", then pick FrobNaz from the list, and click on the third link down on the next page.

.. I want an URL.

personal wishlist 5086

I want a shell that does not actally chdir until it forks to run a command. Ideally this mode should be a toggle. Perhaps I work with crazy loopback / cronned mounts / umounts too much.

personal wishlist 5085

<email> or FOAF information in RSS feeds. Especially, but not only from planets. Hitting 'r' should just work, damnit.

personal wishlist 5084

I need a mail filter that downloads urls in emails, and if they are html, or plain text, attaches the first N kilobytes of them to the email (up to M kilobytes total per email, signatures excluded). It doesn't work to nag everyone to try and remember that some of us read mail while offline, and that we'd prefer that you include actual information that the email is about in the email. Besides, N may vary from person to person or time to time.

personal wishlist #5083

Are we all losing our short-term memories, or is it a fact of human nature that after typing in an email, we assume it's done, and just don't think about that attachment we promised to include with the email? I see it more and more often. .. Er, and I do it more and more often. I'm tempted to collect some data to see if this plagues mutt users more than users of other MTAs; we have to remember about the attachment all the way through to the end of the email, through saving it and existing our text editor, and only then tell mutt about it.

Anyway, so mutt needs a hook that runs before a mail is sent, with a nice little script that guesses when we promised an attachment, and left it out. Or, there needs to be a way to include an attachment on the fly, as soon as you type those fated words. I can think of several ways to do that, all hacks. OR, sendmail needs to finally grow that promised AI, and block at SMTP time emails that fail on their promises of attachments. Yeah!

Personal wishlist #5082

There should be a standard tag used to link to printable versions of web pages. Failing this, there should be some very good heuristics in the browser. No, I don't care about printing, but I do click on those little "print this page" links at every oportunity, same as I'll click on a link for a PDA or phone version of a web page when possible. Such pages tend to be much closer to the way the web should be: Devoid of stupid background colors, pointless graphics, javascript, and <blink>advertisements</blink>, and without of large sidebars. The web page sidebar is the late nineties and early naughties equivilant of the fifties tail fins on a car.

And "print this page" pages never have the dreaded "next page" link at the bottom. If I've decided to view your page, I want the whole thing, to drag off to my room and gnaw on and digest at my leisure offline; I don't want it doled out in little tidbits with a five minute delay in between each while I reposition the laptop for optimal wireless receptivity and download the next bit over a saturated phone line. So all I want is a way to view "print this page" pages by default.

Brr, it's cold. Not the most productive day ever either; I spent 2 hours fixing the remaining bugs in netcfg, but it seems that should have taken less time. I've very glad I got the new wireless card now, so I can use the net in a heated building. And I'm sorry to miss LinuxWorld this year, I had hoped to make it for at least one day and see several people again.

peacock

Coming down the hill this afternoon was like stepping into five or fifteen years ago, the peacock was out of his pen and on display on the freshly mowed lawn. Everything that's happened since vanished for a while. The berries are near their peak and were gathered by the handful, and given away. Soon it'll be time for the evening thunderstorm.

This week here has been like that, stepping back into life here like I'd never left, wondering how I ever left. We're leaving the peacock out, the foolish young bird is trying to intimidate the dogs as they lie on the lawn, and I wonder how this moment can ever last.

pavlovian RC bug response

I have a very loud proliant server which powers itself on and installs Debian 30 times a day, starting at 4 am. I can't really hear it when I'm in my bedroom, or at least I've learned not to be woken up by it. If all the installs go well, (or fail immediatly and miserably), they're done by 11 am or so, when I get up.

If, like today, there is a release critical bug of some kind, it waits 30 minutes between installs, and retries them. This can easily keep it loudly running until 6 pm.

After a few months of this, I have an innate need to file release critical bugs when I hear a loud computer during daylight hours. I also find it very satisfying to file them.

patch followup

Followup on my previous review of Ubuntu's patches to my packages:

• xbl

  Ubuntu adds a build dep on libxext-dev. Possibly valid, since the configure script emits a Makefile that somewhat unnecessarily links with -lXext, which could be considered an explcit use of libxext-dev (which is a dep of libx11-dev so gets pulled in automatically anyway).

• xjewel

  Ubuntu adds a build dep on bdftopcf, which is not a separate package in Debian.

• big-cursor

  X fonts are in a different place in Ubuntu so it's patched for that.

• bsdgames

  Still mangled to use unncessary and obfuscatory patching systems (dpatch), still adds a bunch of acronyms many of which do not belong in wtf as I blogged about before.

• debconf

  Reduced to just branding changes.

• debhelper

  A bit hard to tell since they've forked off from 5.0.7 and diff against 5.0.1 for some reason, yeilding a huge useless diff. Most of their actual changes seem the same as before; ignoring potential errors from scrollkeeper, X path changes. The thing I objected to the most, the striptranslations hack, has thankfully been removed.

• xaos

  They reverted out all the undocumented upstream changes I complained of before. The new diff removes the libvga linkage and adds an icon.

  Of course, there has been a bug in the Debian BTS for a while asking for an icon; if one was actually sent to me I would probably add it immediatly.

• rbscrobbler

  Removed from Debian. Still in Ubuntu with the same ugly diff as before and me still listed as maintainer.

• base-config

  I spent about a month removing this from Debian, which did nice things to the previously impossible Ubuntu diff.

• xgalaga

  Seems their previous patch has been reverted.

So things have improved for my packages, at least after I made a stink about it and some people pretty clearly made it a priority to fix it. I doubt that this can be extrapolated to any other set of packages though.

Total useful stuff gleaned this time: One minor build-depends fix, one icon.
Total estimated time to subit those to Debian BTS: approximatly 2 minutes.
Total time it took me to extract them from the diffs: about an hour.

past few days

To busy to blog, really. Yesterday's talk on testing security seemed to go well and generate a lot of interest. Afterwards was dinner with Intel. Today we took the ferry to Suomenlinna and did a touristy things at the old fort there and picnicked.

partman

Today I took my first close look at partman, and found lots of nits to pick. I have a patch that fixes some of them, and I must say, I found partman a weird mix of fairly intense shell code that was still somehow easy to navigate and read.

Also I've finally released a new build of the d-i images and updated linux-kernel-di to include the 2.4.25 kernel (244 udebs). Will see about transitioning d-i to 2.4.25 next, and then we'll finally get the xfs support which so many users want for some reason.

Somehow Jello got me interested in playing more old emulated games, curse him. Blew a fair bit of time on that in the past few days, nothing excessive though.

Some interesting things are happening which I am not at a liberty to discuss ...

overdose

On Saturday we had exceedingly large amounts of family in town for the holidays, including all four of my sisters. It was also Anna's birthday, and the day of the solstice party (a bit early this year). Unfortunatly I'd gotten 4 hours or so of sleep the night before, so I had limited energy to deal with houseguests and 12 family members all in one place. I ended up bailing on the solstice party which is a shame, but was the right decision.

Somehow I ended up in the right frame of mind for mildly scary C code so I ended up rereading all of anna (the program), and refactoring most of it to be easier to follow, as well as removing some large appendixes that are no longer used in current d-i. Now it's in a state where I understand it well again (except for peices of two functions that hook deep into the inscrutable libd-i), and I'm ready to begin some further development on it.

Probably I'll add the "anna install udeb" idea to it; this has the potential to lower d-i's memory reqirements by letting later-running parts of the installer load up support udebs only if they need them, or on demand, after swap is turned on, instead of loading up masses of udebs all at once as it does now. Doing this robustly and with a good UI could be interesting.

It snowed a little today, which always a big deal here in the South. I went out to find The Rooster and froze my face in the icy wind off the lake on the half hour walk back. Also got to hang out with a subset of the family, which was saner than everyone en banc.

out

It seems like every day by the time I'm ready to stop working and go outside and get some exersize, it's dark out. Thought about going to the Y and swimming or something, but I finally decided to just go out before doing much work today and hiked around for several hours in three different parks, doing some more geocaching. Doubled my number of finds and enjoyed more unseasonable waether that let me do things like wade through a creek and walk around barefoot -- in January!

Oh well, now off to dinner, and to work.

oslo

I'm wrapping up my trip to Oslo for the skolelinux developer's meeting. This has been a short but good trip; KLM/Northwest managed to redeem themsevles from the last, disasterous trip; I got some of my snow ration for the winter and got to see what Norway is like in winter, at least in the big city; I had a good time with old and new friends.

And the actual purpose of the trip has also gone well, I think; some useful work was accomplished and we had some good discussions about future plans. I finally met my other co-workers at SLX, and I got to see debian-edu in action in a very nice classroom setup. Sometimes it's been easy for me to loose sight of what we're working on in debian-edu, since most of my work is infrastructural and has little to do with the actual classroom, so getting grounded like this helps, and will certianly let me do a better job. It was also good to get confirmation that the work I have managed to do on this job has been appreciated.

So I've really enjoyed this trip and I especially like returning to this place, which I now feel I know much better than after the DebConf here. Oh yes, I also got out and about some on the last day of the trip, and managed to get slightly lost and go geocaching in the snow at a nice bird sanctuary with a lake. The conference environment can be such a cocoon -- we even had kind locals to lend us a place to stay and others who drove us to the school most days, and so getting out on my own was good at the end, and I need to remember to budget time to do this before or after every conference.

Oh also, a word about sleep.. I know that I have a low point in my natural circadian cycle around 5 or 6 pm, and then it peaks back up around 11 or midnight, with me often going strong until 3 or 4 am if I don't watch myself. Apparently my body has finally managed to fit this into the CET timezone, so when I'm over here, I manage to get to bed by 11 pm CET (6 pm EDT), sleep until 5 or 6 am (midnight EDT) and then am up for the day. That's much better than sleeping during the other low point of my cycle that I normally use at home, especially when you need to be up by 9 am and not, say, 4 pm, due to being in a coference with teachers and other morning people. Only real problem with it is that I only get about half the sleep I need, but it works for short trips at least. If I had arrived a day earlier I could have been completly over the jetlag by the beginning of the conference.

Speaking of sleep, I'm following a weblog of a sleep researcher, which amoung other things has some interesing thoughts on how society treats sleep. In particular, see this post. And after mentioning that, I have to link to this post.

os probing

Joshk and I got to work on os-prober today, and d-i will now detect existing DOS, Windows, freeDOS, hurd, and linux installs, and properly set up at least grub chain loading for the DOS ones. Will download a Hurd image tonight and get it booting too. Properly booting other linux distros will take more work. Actually, all I've been able to test so far is freeDOS; installing any of the others is too much work.

organising notes

I've finally realized I need a better way to manage random notes and info on my computer than what I've been using -- a mixture of some permenant text files holding info such as phone numbers that are kept in revision control, and random notes jotted down in files with random names that are often lost and deleted. I pretty much realised this last night on the train when I found that a) I'd deleted the file (probaly named "foo") containing the note I took that morning telling which line to switch to to get "home" and the stop b) I'd also lost home's phone number c) my gps's waypoint for home was inexplicably 200 miles west, somewhere in Sweden (WTF?) d) I had several mails sitting in my inbox just so I could refer to them for info such as my departing flight. I streched my memory a bit and did get back home, but my memory for random peices of information is not getting better (I've not been able to remember my last 3 home phone numbers), and this mess had to end.

So I looked at available tools for managing this kind of stuff. I rejected the more rigid and graphical PIMs and seem to have settled on a little curses program called "hnb", which is a kind of outline editor. With some tewaking the UI is no longer an angry fruit salad[1], and is almost bearable, though it has some rough spots. But it's easy to quickly add notes, it's very flexible for storing any longer-term informaton in a nice hierachical layout that can be quickly navigated and searched, and it saves the data as xml which I can check into subversion for safe-keeping. I've also started using it for recording what I've done on my job, for my weekly reports, as well as todo lists. It would be nice if it had support for encrypting the data with gdb (a patch in the the BTS), and stored some metadata, such as last modified date for notes. Overall a good tool that's worth the learning curve.

[1] I wish I could file bugs on Debian packages just saying they had this disease and be able to expect to get them fixed.

oops

It's been a rather long time since I've written any object oriented programs, and also long since I did a lot of perl programming. Today I found myself writing hundreds of lines of perl classes to automate an annoying (and reasonably complex) problem. Took some time to get back on the chainsaw powered bicycle that is OO perl, but I have survived the ride once again.

Unfortunatly once I fed the program a reasonable data set, it began to use over 150 mb of memory. Oops. Guess I'll have to consider this one a rough draft and do some scalability work later since I had really planned to run it on ten times this much data.

ooops

Yesterday I made a silly mistake, and now I am stuck in South America w/o a GPG key. Not the end of the world, but sure to be annoying in the next couple of weeks.

Anyway, Brazil continues to be fun, and I'm somehow managing to keep my head above water on coordingating the d-i release too. I came up with a schedule for the next release yesterday (and also did some very tedious skolelinux CD building), and today I've been in delegation mode, finding lots of tasks and people to do them. So losing the key is not too bad, it just means I have to delegate all package uploads too, which could be a good thing.

Pity that the installer is completly, hopelessly broken in unstable, but at least the version in testing is, hopefully, usable, and I have the tools to keep a handle on the situation.

ontology of bug reporters

Benjamin Drieu wrote a great list of different types of bug reporters, all of which I've seen in the wild. I do need to make an addition to the list:

The drive-by patcher

This variety of patcher sends patches without sufficient explanation of the bugs they fix. Often easily recognised because the patches will frequently not be unified diffs. The patches will often fix whatever problem they had, while introducing lots of bugs for everyone else, so beware.

By the way my favorite bug report of today was verbal, and went something like "It stays dark when I turn on the computer.. I tried to do something to the whatchamacallit, but it doesn't work". This bug report was ignored. The followup, and I quote: "hi--not sure, except that I decided to press the Power button on the TV screen, which seemed to do the trick.."

(Hi Mom!)

one step forward and a dozen steps back

Still very undecided about blogging.

Today in my blog mailbox, I have seen one person wondering about adding killfiling to his RSS aggregator. Another describing how he writes some notes before opening a web browser to write a new post, in case the browser crashes or he forgets what he was going to say before he gets to the submit page. Another announcing google's service which lets you search blog posts. And another doing the 2005 equivilant of double sigging (in html, natch).

I guess that reimplementing solved problems like email filtering, proper text editing, and kibo-grep is as good a way as any other to use our time, but it all seems a bit pointless to me.

One keystroke to Debian

This rocks. A big improvement from our previous low, but I expect to see it drop to zero over the weekend (update 11 Sep -- done). This is only the first stage install of course; base-config still needs hand-holding (update 13 Sep -- not anymore!).

I'm sure the FAI folk are laughing at me, but I think it's exciting..

on data hoarding

This post, (via Ian M) meshed with some things I've been mulling on for a while about how we treat different types of data today. The relevant quote from that post is this:

I'll bet that we will look back of this era of quasi-networking and wince, "How did we ever live that way?" And the idea of wanting to carry all of your content with you will seem both old-fashioned and rather ridiculous.

I was recently very happy to acquire a large quantity of data files (about 15 thousand), each of which would take me hours to days to do anything with. And I have a few hundred CDs and DVDs with a terabytes or so of achived data on them. There is probably more stuff here than I would ever be able to look at in a regular lifetime if that were all I ever did. I'm adding stuff to the collection regularly.

All of this is data that is not freely available; some of it is private stuff like email archives, some more is music, and every other sort of digital media known to man. Except oddly, software, which is almost nonexistant from the hoard except for a few bits that slipped in by accident.

The chance that I'll ever look at any single one of these files is probably on the order of one in a thousand, although it's certianly possible that a computer might look at them all and search out some that I will be more likely to look at.

Even though I know that I will probably never use most of this stuff, I hold onto it as archives mostly because it's easier to do that, than to decide if I really need to keep it. Or to get it back if I mistakenly discard it. That basic tradeoff, plus the fact that it's very easy these days to aquire large quatities of data and hold onto it cheaply, are the main factors behind this hoarding.

What I've described might seem extreme, but it's not very uncommon and most of the people reading this probably do it too, to some extent. But the really interesting thing to me is that if this data were the only data I was able to use, I'd be very unhappy and bored and unable to accomplish much. The interesting thing is all the other data that I don't bother hoarding, because it seems so likely that it will be easy to get it if I need it.

Some examples of this data -- and there are untold terabytes of it out there -- include: the complete source and binaries for a dozen operating systems; the complete revision control history of a thousand free software projects; tons of archived web sites, concerts, and movies in the internet archive; every usenet post ever made; every mailing list post I've ever read (or ignored); dozens of bug tracking systems with millions of bug reports; probably not very good scans of any artwork of any significance; more ephemeral blog posts than even boing boing can link to, etc.

I don't hoard that data. The tradeoffs dont make hoarding it worthwhile; it's easier to just hope that most of it will always be available, and use my limited resources to keep the small parts of it that I'm responsible for backed up and available to others. Which tends to feel like a better use of time and resources than hoarding data, by the way.

I suspect that there is more of this sort of data out there that's important to me, than there is for the average person right now. After all, most people don't find old bugs in bug tracking systems and the history of software's modification in revision control systems interesting, and I understand there might even be a few to whom usenet archives from the 80's arn't fascinating. And I know that the amount of data out there that's important to me has gone up over time.

So the big question to me is, will I begin to find my hoarded data is useless, will it ever stop being important, and will I stop adding to the hoard?

Some technological advances, like faster bandwidth, could tilt the balance that makes some things worth hoarding now. But at least for me, data still seems to be worth hoarding if the cost of accessing it is more than the cost of keeping it on media, or if its future availability can't be guaranteed. There are a lot of services that don't meet these criteria, things like itunes, and bittorrent, and netflix. These things won't stop hoarding; they'll just make hoarding easier, despite themselves.

If it were still the 80's I'd be hoarding nearly every freely available thing I listed 5 paragraphs above. Some of it no longer needs to be hoarded thanks to technological change, some of it due to new services, but it seems to me that the really important change that tilts the balance from hoarding data to not is a growth in trust, and a change in what kinds of data are valuable.

I trust the Internet Archive. I (sorta kinda) trust Google. And so I trust them to keep their content available, and so I don't try to hoard it. More and more I find freely licensed data, whether it's Free software, CC licensed media, or whatever, to be more valuable than proprietary data. And because it's freely licensed it can be archived online for everyone, by everyone, with network effects, and no need for hoarding.

on antivirus spam

Adam Kessel writes:

Does anyone have a rational explanation? Even better, can someone educate the software developers and the people who purchase their software to end this scourge of false "virus detected" emails?

Those messages are a form of unsolicited commerical email. Developers of antivirus software don't need to be educated; as you note they have no excuse for not knowing that viruses forge From headers. They need to be treated as spammers.

The continued commercial success of antivirual software depends on a steady stream of viruses, on a continued low-level awareness of the problem amoung computer users, and on a complacent consumer mindset that accepts this problem as the way things are, doesn't bother to learn about ways to avoid the virus problem, and instead plonks down money for antiviral software and continual upgrades.

So manufactursers of antivirial software have no incentive to educate the public about the systematic problems that make viruses possible. And they have everything to gain by sending out unsolicited emails plugging their products, especially if the mail can provide such compelling evidence that their product actually works, and that the user needs it. I'll bet the response rate from this stuff is ten times better than from your typical spam run, and the antivirus producers get paid to produce the software that does it, and get free use of their customers bandwidth to send the ads, to boot.

It's a vicious cycle, which the news media also feeds off of. While the better news sources do (occasionally) mention that a virus affects only windows machines, the majority of news media do not. And I've never seen a story in the media about the virual problems that suggested finding a better computer platform that could avoid them entirely. Why should the news media bother to educate its audience, when windows viruses are good for a news story every week or two?

This is how inneffective the news media are at explaining viruses: Every time there is a significant windows virus or worm outbreak, my mother fires up her linux system, logs into my server, and sends me an email warning me about the impending doom and destruction. I'm not making this up.

Anyway, I don't know what to do about the news media, except for ignoring them, but I know what to do about these antivirus companies. Treat them as spammers, and treat their customers the same as you would the owner of an open relay. We need a RBL for computers known to be harboring antivirus software that sends spam messages, and support in spam filters for blocking their mails. We need to find other ways to pressure antivirus companies to shape up.

Oldenburg

This is my last morning in Oldenburg, I'm up obscenely early hacking on quik-installer on an oldworld powerpc machine that Gaudenz left with a failing d-i run when he went to bed around 5 am, just as I was getting up. I cannot explain my sleep schedule here (I'm been waking up at midnight EST), but the Oldenburg developer's meeting is like that, old hardware, wacky sleep schedules, and an informal, relaxed, and very productive atmosphere. The trip has certianly been worthwhile despite the airline's worst.

I wrote before about how much more work seems to get done at Oldenburg than at other meetings, and this year has just borne that out again to me. Rather than focusing on any one problem, I've been floating between a dozen different areas of d-i work, mostly working with the people who are themselves expert or very interested in that area, so I've worked with Anton on partman stuff, with Colin on powerpc (floppies), With fjp on countrychooser and the manual, with Jeff on build system stuff, with markos on localis^Hzation-config and termwrap. In many cases just being someone to bounce ideas off of. My main work by myself has not been on large problems, just things like fixing the size of the i386 floppies, coordinating a release planning meeting, and fiddling with root on software RAID. So yeah, these meetings really work out well, there are more people working on d-i here than at the last DebConf, and on such a variety of things.

I also like the continuity of having the meeting in the same place every year, of going to the same restaurants. I enjoy traveling around to new places attending other meetings, but not feeling obligasted to go out and see the sights, not needing to learn where things are, these cut down on the stress and along with the excellent management by otherJoey[TM] (who's been doing this for 10 years after all), make everything go very smoothly. I hope I'll be back in Oldenburg next year.

ohio linuxfest

I'm kicking myself right now for not thinking of going to the Ohio linuxfest. After all, Cincinatti is one of the few places that is one plane hop from there. I was in Cincinatti just a couple of weeks ago, although since I was there because my proper flight to Atlanta got canceled and I had to take the long way around to Atlanta, I was more worried about catching my flight to Germany.

Anyway, Cincinatti is more than an airport that I occasionally pass through. I need to try to remember that for next year. Especially since I never seem to do anything linux related in the US these days for some reason.

October reading

Lately I finished C.J. Cherryh's Forge of Heaven. I was happily suprised that this book, second in a series, stood well alone. It's Cherryh at her best, written in the style that's all her own. Parts of the setting feel slightly recycled from Downbelow Station and other old novels in the Merchanter universe, but it's well enough different, with such an immense timeline behind it that it stays interesting. The nanotech/genetics stuff didn't really grip me, but other parts made me flash back instead to the best of Kim Stanley Robinson: Pure geology-porn. Yum!

Now I'll certianly have to go back and check out Hammerfall, as soon as I can extricate myself from the clutches of Neal's latest opus, which I'm 50% through. The System of the World is better than the previous book in that series, because things are actually happening, but not as good as the first and none of it has been as fun as Cryptonomicon. Perhaps if the Enoch Root thing actually interested me, I'd get more from these books. As it is I read them for the odd little facts, the humor, and because I'm well sucked into Neal's fan-base by now.

note to self (or, e vs a)

If you mount a microdrive on a pcmcia adapter and somehow the prtition #5 that you thought it had is not there, but partition #3 is, and contains a "joey" directory, rm -rf ing the directory before looking in it and checking what you really mounted is probably a bad idea. Especially when the laptop's hard disk chatters a lot during the rm.

On the other hand, svn up after fscking your home paritition and seeing it pull all the files out of cache is pretty fine.

note

I'd like to note for the record that many of the things I say in this blog are worded too subtly for you to understand. Yes, I mean you.

not me

Since people are now randomly messaging me on irc about it and stuff, I just want to note for the record that this mp3 is not an interview with me, it's an interview with Jaldhar H. Vyas.

not grokking cross-site scripting

I wonder if I'm the only one left on the planet who finds it hard to take cross-site scripting attacks seriously. To put it another way, I've just introduced a cross-site scripting hole into every website with user-determined content out there, as follows:

1. YouDontWantThisML is a new markup language that extends html with new square bracket tags. A YouDontWantThisML compliant browser will thus interpret [p] as <p> and so on. This is a fairly simple modification to most web browsers, so I won't post patches.
2. As an extra bonus feature, if it sees a [YouDontWantThisML] tag, a YouDontWantThisML compliant browser will pop up a game of asteroids and not let you continue browsing until you've shot the flying saucer. This "captive asteroids" feature is sure to make users demand YouDontWantThisML support in droves. And pong is due to be supported in YouDontWantThisML version 2.0!
3. Unfortunatly if your web browser is YouDontWantThisML compliant, I've just realized that you're vulnerable to CSS exploits such as this one, which can be inserted on any web site out there with user-defined content (such as whichever one you're reading this on).

   [script]
   blah blah evil code here
   [/script]
   

   Another example might be using eg, viewcvs with an url like this one:

   http://hostname/viewcvs.cgi/[script]alert("BOO"+document.cookie)[/script]
   

Fixing this new set of cross-site scripting holes in nearly every website out there might take a while, but don't look at me, it's not my fault for inventing YouDontWantThisML, and it's certianly not the fault of any browser that added YouDontWantThisML support.

noooo

Clint: I can't believe they're removing the religion from Pullman's Dark Materials in a movie adaption. But then, I can't believe there's gonna be a movie either since I had assumed the religious elements relegated this most excellent series to a niche market, at least in the US. This may come of living in the bible belt.

I've only read the first book, and am saving the latter two for very special occasions.

PS: Did they take the religion out of the Narnia movie? Just curious.

noisy morning

Very bad noises from my laptop hard drive, which has had three failed sectors ignored for two years (which only ate some mp3 I never listen to..). Also, the peacock is apparently lonely and set up right outside my window yelling all morning. Urk.

noise line

I'm out of practice and I didn't get it fully justified, and the end is a bit weak. Still, this has some cute techniques hidden inside.

\pe\
\rl\
 -e'
TOP:
$l=
<>;;
$c.=
$l
if$t
;$t=
$t||
$l=~
m{^e
x}x;
eof(
)?1:
goto
TOP;
eval
$c;
####
warn
four
(J).
char
(E).
per.
line
(H).
####
"\n"
'< \
$0 \
2>&1
exit
for(
keys
%::)
{$s=
$_;
y/
n-z
a-m
/
a-m
n-z
/;
eval
q{*{
$_}=
*{$s
}if!
de}.
fine
.q{d
&{$_
}}}
sub
zbz{
anot
}sub
yvar
{l.f
().
cker
}sub
w(){
" "}
sub
pune
{mom
().
her.
j()}
sub
sbhe
{0||
just
.j()
}sub
s{j(
).ha
}END

Copy to file, set +x and run.

no, no, and no

.. or random replies to other blog posts.

Francesco P. Lovergine: Contrary to what you say, your mails for proftpd were not "parked" in the testing security team mailing lit. They were ignored because we were already tracking the issue at the time. All known security fixes for proftpd have already entered testing. Our vulnerability tracking information is publically available and well-documented.

Also, if you neglect to use the BTS for security issues, that is your own problem, the resulting insecurity of your packages is your fault and I have no sympathy for you.

Scott James Remnant: Merging a bunch of translation updates that were developed somewhere else into d-i "right before a release" would result in an unusable release for at least the languages so merged and quite possibly would generally break it. d-i has constraints like needing to fit all the translations on particular sized media. It's also fairly hard for translators to get a working translation of d-i blind without testing it, and testing it as a third party is hard. This may not be true of other software such as dpkg, but it's manifestly true of d-i.

The l10n-sync stuff is unfortunatly noisy, but distributed revision control, while admittedly an awefully big and keen hammer, is not a solution.

Also, the idea of losing a bunch of useful historical data when landing a large patch from a co-developer is one of the things about some distributed RCS systems that gives me the screaming horrors.

Night out

Well I had a nice night out at the farm, lit a fire and cooked soup and sausages on the wood stove. Got in an argument with a barbed wire fence (ouch). Enjoyed the shaded places where frost is on the groud until February and saw the winter sky and moon. Didn't stay out long enough, and now I'm in town and stressed out.

If you're trying to accomplish something, a little optimism is a useful thing. Keep your realism out of my optmistic way please. If you don't think Debian's going to release for another six months: be aware that I consider you part of the problem.

next debconf

Only 5 months until the next DebConf, in Oaxtepec MX this time. As I read tonight the very well done Final Report for DebConf5, and remember that excellent conference I am already looking forward to the next one.

I've been thinking about driving down from Virginis to Oaxtepec instead of flying in. It's a long drive, but maybe I could stop at a few DD's houses and/or travel down with some. In fact, I hope we get a lot of American DDs who have not been able to make recent DebConfs now that it's so relatively near to home.

So, time to think about any talks I'd like to give there. The absolute best talk at DebConf5, the one I would most like to emulate, was Enrico's polygen extravaganza. The key things were that a) Dr. Zini rules b) it was a simple talk introducing us to something fun c) large quantities of running code were written on the projector during the talk.

I might try to emulate the parts that I can (without taking Enrico-emulation medication) to some degree in one talk, if I can just find the right subject. Since I hate writing papers for talks, I also got an idea for a way to use a talk time slot that will be impossible to write a paper for before time. Gotta hurry, since it turns out talk submission closes amazingly soon: On the 6th!

news

I'll be spending at least the next month in Charlottesville VA working for ADS on Debian ARM stuff. This may turn into a permanent job.

new year's resolution?

I'm --> <-- this close to resolving to never, ever work on any new free software project that keeps any generated files in revision control. And wishing I didn't have to qualify that with the "new".

new system

I got the cheapest computer available at $LOCAL_CHAIN_COMPUTER_STORE today, and it works fine with d-i (installation report filed, of couse). 2 hours from unpacking the box to typing this blog entry in X on a system with my standard config, most of that time was spent learning how ps/2 mouses work in 2.6, working out just how crappy $RAMDOM_MONITOR_THAT_SOMEONE_LEFT_IN_MY_HOUSE is, and downloading debs and svn checkouts. I threw my regular host-naming scheme, such as it is, away for this one. This system is the first new computer I've bought in 2 years, and the first new desktop computer I've bought since approximatly 1998. It will serve as a temporary server for the house, and workstation, until the real server arrives, then it will be a workstation and d-i test system.

new stuff

"New Packages" in aptitude had been piling up since October. I finally looked through it and tried out some new stuff. duplicity kicks butt and will be great for offsite backups. 1/4 through the diveintopython book. And inkscape is fun and trippy, it's really brought out the artist in me. As for games, airstrike has the potential to be fun when finished. Nice animations.

The power jack on my laptop has developed a disturbing habit of brifly flickering the power when jiggled. Not too bad yet, but I cringe, remembering the vaio power connector from hell. Sigh.

New River gorge

I remember when I was a kid the trip up to my Grandma's in West Virginia seemed really long. I'd bring along books, games, pencil and paper and usually ended up trying to amuse myself by counting the number of cars vs the number of trucks that we passed or something like that. The only interesting bits were passing the very occasional city, or the more frequent large mining operation. Oh and the tollbooths, and the three tunnels under the mountians. And barges on the Ohio river. But to a kid, most of the trip seemed to consist of the boring bits inbetween, and I tried hard to fill them with some kind of value.

I guess I'm still doing that because on a recent drive back from Charlotte, I found myself tooling up I-77 with my laptop on, hooked up to the car's speakers and talking to me, sniffing for some hidden interests besides the winding highway and occasional pretty view. Which is to say for, wireless networks and geocaches.

I've made that trip up 77 many times, and every time I've driven over the New River gorge and blamed the bridge and the speed from giving me only a one second glimpse of the river. This time I learned that there was a cache somewhere approximatly under that bridge, and so I turned off a few miles before and headed up the road through a little town, being suprised to find some wireless networks on the way, and ended up at a converted railroad trestle, now a horse path, with a really nice view of the river and all around.

I never did find the cache, because of some boring software issues that had me off by hundreds of feet from its real location, but that's ok, I'm just glad I finally stopped to see the New River, and happy that I turned that trip into something of more value than just a journey to a destination.

new job

Well I've finally gotten a job again [6 mb divx mov thing], starting in a few weeks I will be working for Skolelinux. This will involve a lot of Debian and d-i work, and I'm really looking foward to it. I'll also be moving soonish, though I don't know where to (probably not Norway though.. maybe to my Dad's or something).

new disk

I put a new, 80 gigabyte disk in my laptop. The old disk has had two bad sectors for over two years now, plus it was chronically full and slow, and noisy. The new disk makes the laptop feel a lot faster and is quite silent.

I've not decided yet if I want to use some of the oodles of disk space to put a Debian mirror on my laptop.

It does seem that I will be upgrading this laptop until it uttery falls to peices, instead of buying a new one.

network explosion

The other day I found myself coming down the hill in classic burden on head pose right out of National Geographic, except the burden was a small box containing three computers. Some things are timeless and others change so fast..

I'm now up to 6 computers on my home network here at the farm. Although most people would probably only recognise two of them as computers of any sort, and one of the two obvious ones is fairly well hidden. Two each of i386, mipsel, and arm.

I need each and every one of the six to be able to do my job(s). Scary.

natural tunnel

Well, a fairly stressful and nearly disastrous little trip over to Natural Tunnel today. I hadn't meant to go so far but I missed the road to the lake and didn't feel like turning around. The cat was not amused at an extra hour of driving, and I was a bit worried about the annoying hydralic clutch on the truck, which may be acting up again, but I got here.

Unfortunatly somewhere on that very winding road to Gate City a lot of stuff tumbled around in back. The worst of it involved a drenched pan of used kitty litter. And I'll stop there. Anyway, no harm done, and I enjoyed a bike ride and hike down to the tunnel, and got some serious exercise all the way uphill coming back.

The tunnel has a freight train line going through it, and years ago we'd hike down there and balance our way accross the trestle over the stream, then walk alongside the tracks and through the tunnel. It was magical and just a little bit scary, and when a train did go through, it was LOUD. But really it was a lot less dangerous than my exploits reading books while walking along the tracks on the way to school, and sometimes crossing a temporarily stopped train. We were aware that the tunnel could be dangerous, and we were careful.

I really see what's become of the tunnel today as symtomatic of a lot of changes in America in the past 15 years. They added a chairlift! They put in walkways at the bottom, a separate pedestrian bridge, and a viewing area. And lots of signs threatening visitors not to go out of bounds and actually enter the tunnel. All very safe, and boring, and enough to make me feel like a cow going down a ramp to slaughter.

I can't say whether I illegally tresspassed inside this great natural tunnel or not this time...

national wireless

Inspired by an article on using bluetooth with T-Mobile for nationwide wireless internet access, and having thought about doing some road-tripping, I was all set to go sign up, until I found that T-Mobile doesn't really cover most of the southeastern US. Drat.

It seems that Verison does cover where I live, as well as most of the area covered by T-mobile. They have two networks, a slowish one and a new fast one in major cities. It may be possible to do the bluetooth cell phone trick with them too, but I'm more attracted to the pcmcia cards that connect to their network because a) no extra battery or device needed b) I hate phones c) laptop doesn't have bluetooth anyway. The Air Prime 5220 works with linux, indeed I'm having flashbacks to the way Ricochet worked with linux, same pseuso-serial interface. Only problem is my current laptop doesn't do cardbus either, IIRC, so won't work with this card. The Sierra Wireless Aircard 555D works similarly and appears to not be cardbus.

Update: The lower cost route is apparently to get a phone and a usb cable, and a rate plan that provides free weekend and evening minutes. Data access is apparently charged the same as voice access with such a plan. Instructions for one phone which will work with linux.

Any advice in this area from someone who has nationwide internet access working in the US is appreciated.

naming scheme

I badly need to find a new naming scheme for my rapidly growing home network. At the moment the important hosts in the network have random names that were chosen on the spur of the moment (chicken, paperweight) or are remnants from naming schemes of past networks (dragon). A few systems are named or aliased functionally (mirror, phone). But I also have a lot of systems that are only used for testing, and so my network has been sprouting hosts named after the most prominent word on the peice of hardware: wavelan, aptiva, pavilion, ia64. But this will fail as soon as I get two pieces of similar hardware and besides it's booring.

I've never been any good at naming schemes though. There was a great series of blog entries aggregated on Planet Debian about this a while ago. It's too hard to find old weblog posts, but eventually I dug up LIW's summary

Since two of my machines, and two of the hardest to rename, were already named after animals, I've decided to use animal names for the naming scheme. To keep it from being too generic, I'll use only mythical creatures for laptops, and test machines will be named after invertebrates. Happily, I have few test machines that are also laptops. I decided to leave the functional names alone; it's too much fun to "nmap phone".

The final list in the same order as before is now chicken, kraken, dragon, (mirror, phone), limpet, jellyfish, earthworm, elephant. I even added a block of 16 pre-assigned names for new machines on the test network so I needn't worry about it for a while.

My debian-installer test methods

Since people who are working on d-i often wonder what's a good way to test changes, I thought I'd write up some of the techniques I use for d-i testing. This won't be a comprehensive list or a howto, just a brain dump of the kind that some occasionally find useful.

There's really only one key peice of advice and that is, do not use debian-cd to build a custom CD image just to test your change to d-i. Even if you know how to do it, even if you have a setup that will build working netinst images, it's just not worth the pain of messing with debian-cd, and with ISO images. There are so many better ways to test a change to d-i, that building a custom CD image should be your absolute last resort, and only done if you need to test something specific to the debian-cd package, or if you need to test a specific type of iso image.

My favorite way to test a change to d-i right now is to build a udeb with the change; copy it to installer/build/localudebs; make rebuild_netboot; and untar dest/netboot/netboot.tar.gz into /var/lib/tftpboot. Then netboot a test laptop. This takes less than 5 minutes to boot up into d-i for each test. It of course requires a machine that can netboot.

One step I left out is that if the udeb in question is not one that is on the normal netboot initrd, then I have to echo 'udeb-name *' > pkg-lists/netboot/local. This ensures that the udeb and all its dependencies are put on the initrd. Otherwise it would not go on, and d-i would load an unmodified version from the network.

When I'm downstairs and do not have a ethernet wire to netboot from (and can't from wireless, on my hardware), I use my second favorite method. This is to build a hd-media image instead, with my udeb on it, and write it to a usb keychain. Often I only copy over the dest/hd-media/initrd.gz to a keychain that already has a recent version of the installer on it, as this is much quicker on slow usb 1.0 controllers.

If you can't netboot or usb boot, the next best thing is to use a miniiso. One is built in dest/netboot/mini.iso as part of the netboot build. This is a small iso that has only the d-i kernel and initrd on it, once it's booted it's the same as if you'd netbooted d-i. Since it's such a small iso (3-4 mb), it will build and burn very fast.

If you have to boot from CD and the machine has no or a slow network connection, an alternative is to edit config/i386.cfg, uncomment the "monolithic" on the first line, and make rebuild_monolithic. The resulting ISO in dest/monolithic/mini.iso will have nearly every udeb on it, and you'll only need to download the debian base system from the network.

If the network is very slow, loop mount a debian netinst iso on a nearby machine with a web server, and hey presto, you have a partial debian mirror that is enough to install base from.

The above cover almost every method I use of testing changes to the installer. One other trick I use is avoiding rebooting to test every change, if only small changes are needed, I'll make them on files in the running d-i, test them until they work, and then make the same changes in my source tree (or finish the install and copy the changes over with scp if they're larger).

One neat trick I've seen is if you're testing d-i on the same machine you develop on (with care), you can just mount your development partition from inside d-i, and chroot into it. Now you have a real d-i test environment, and a development environment side-by-side.

It's not quite ready yet, but I'm already finding d-i preseeding to be a useful complement to testing. Why press enter over and over when you can automate all the parts of the install except for those you're working on?

Finally, I often find it's quicker to use vmware or qemu to test d-i CD images, if the thing I'm testing is not too far into the install. Though these are slower, the time gained by not needing to write installation media often makes up for the slow speed. And of course I use nothing but virtual machines to test floppies..

One thing not to do when testing a change to d-i is to try to run it in a regular debian system. Maybe this is ok for initial testing during development of certian things, but you'll often run into busyboxisms, missing commands, and other problems and it's rarely worth the bother.

To make any given change to d-i, I often boot my test machines a dozen times, and build new initrds with 3 or 4 different versions of the thing I'm changing. I may repeat this for five or ten changes a day, which is a lot of rebooting and testing. Without the tricks described above, I doubt I could manage more than one test cycle every hour, for less than 10 test cycles a day, not enough to do a lot of development. So a final peice of advice is do everything you can to shorten that test cycle, be it faster hardware, or whatever, because your time really does matter.

music stuff

It's strange that the one thing that makes me think I should upgrade my laptop are music player programs. Not web browsing (a tad slow, but just fine thankyou), or compiles (no problem), or even fullscreen mpeg video (pegs cpu but works fine), but simple mp3 and ogg players. Oh, they work, mostly, but all of the new ones seem to use 10% of my cpu to just sit there and play a music file.

I've mentioned my troubles getting rhythmbox to work well with my crusoe CPU before; changing the decoder backend made it use 1/3 the CPU power and be usable in power save mode. A few days ago I tried Quod Libet, which looks like a great player and some serious competition for rhythmbox, but uses 20% of my CPU! I edited the constantly updating scrollbar out of the code, which reduced it to 15%. Still not acceptable, sadly, not if I want everything else to work reasonably fast and the batteries to last longer than 4 hours.

The other thing I checked out the other day is this Audioscrobbler music-tracking service; while I have it running you can see what I'm listening to. I packaged rbscrobbler which lets it work with rhythmbox.

moving

Lots more progress on moving over the past three days. Friday I packed a lot of stuff and tore down my lab. Saturday I returned loaned furniture, and what a nasty day to do it, cold and intermittent rain. I also drove the Trivette's old yellow truck for the first time. (John Goerzen has nothing on this truck.) That was interesting.

Today I packed 8 computers in my car and William and I drove down (in snow!) to Brian's. Brian has a server room in his basement with raised floor and the whole 9 yards -- including a space age floor to ceiling control center -- and he gave me a whole rack to play with. With William and Brian's help I racked up the whole d-i test lab, which ate most of a full height rack (and also 6 hours of work). With 7 architectures and no two machines alike, it's not the prettiest rack I've ever seen, but it's certianly an interesting one.

Keeping compulsive notes about everything in the lab paid off in rebuilding it, although of course I found the things I forgot to document. If you have dia, here's the complete spec for power, serial, and ethernet: machines.dia

Unfortunatly it seems that the lvm+raid I'm using on the server had issues, apparently it started last night when I test booted it and continued today, and fsck found more and different problems with the /home filesystem than I've ever seen before. So now I'm working on fixing that, as well as re-IPing the whole test lab, which has already wasted several hours of the time it took to install everything.

Anyway, major thanks are due to Brian for hosting this!

moreutils update

I'm still getting a fair bit of feeback on moreutils. The latest release (0.3; still in Incoming) includes a new C implementation of sponge, and the README lists seven additional tools that I am considering or am in the process of adding (mime, ifcfg, and, not, z, tmp, and add).

That's only a subset of things that have been suggested to me lately, seems that deciding what to include and what not will be the challange with moreutils (well, that and getting a license statement on everything, something tools of this magnatude often lack). So I'm documenting them in the README for now to see which ones people want.

Also, Chung-chieh Shan pointed me at h4sh which lets you speak Haskell to your shell. Craziness!

moreutils

Following up on my other posts about new unix tools, I've uploaded the moreutils package to Incoming and alioth. Currently it includes isutf8, sponge, vidir, ts (timestamps stdout) and vipe (discussed below). Hopefully this collection will grow to include other useful unix tools no-one thought to write 30 years ago.

vipe is my new personal best at writing a generic unix tool. It allows you to edit the contents of a pipe in a text editor. For example:

find foo -type f | vipe | xargs rm

Sorta handy as a way to make ad-hoc changes to data in a pipeline, or even just to examine it.

More base-config and d-i hacking

Today I woke up and realized that the choices lists for tzsetup for US and CA and BR (and AU probably) were really sucky. Canada is just not big enough to warrent 21 time zone choices! So after some work it now has customised questions for those countries. Bit more hacking on base-config to make it not display menu items that will do nothing if run, and it's looking pretty good overall. This evening I did a test install of next week's d-i images (featuring all the cvs versions of everything, updated kernel, etc), and that is also looking pretty fine. Looks like we may have hashed out a new dest/ organisation too, on irc. Lots of other stuff bubbling away in d-i-land. Now for new linux-kernel-di for the third day in a row, poor elmo:

linux-kernel-di_0.26_source+i386+alpha+ia64+m68k+mips+mipsel+powerpc.changes

money

I've followed AJ's experiment in funding free software development with his AJ Market with interest and have thrown money in the pot once or twice for some of the projects I care about. It's an interesting approach.

I'm too lazy to do anything like that, or rather, I'm already paid sufficiently to work on free software that doing a joeyh market would too much resemble work. However, I did decide to try something I've seen done occasionally, since I also wonder how that works out -- I added a paypal donations button to the website for one piece of free software. Just as a test, to see if it annoys anyone, and if anyone bothers clicking on it.

I chose alien since it's well known, regularly used by a large number of people and even appears on the first page of otherwise quite amusing results on a google search for "alien". (I suppose that adding google ads to the page would generate more money, but in a probably less interesting way.)

Unlike AJ, I won't promise that clicking on the link will motivate me to work on anything. In fact, the most likely use of any money I get from this will be to pass the buck along to the AJ market or something similar.

MLP

Biella has an excellent writeup of DebConf4 -- with pictures. I couldn't begin to approach capturing the essence of the conference like that, and her special perspective just adds more to it. So some MLP for her writeup. Any family or friends who's wondered what the heck goes on at those DebConf things: now you know.

mindless consumption: TV

In the past several months I've been watching some TV shows to see if my problem with the medium is general or just due to things like advertisements and TV schedules. Since I don't have a TV and am obviously a law-abiding individual, this probably involves DVDs somehow. What I've found out is that a) I actually do enjoy some TV shows, b) I do better if I can watch an entire series or at least a whole season at a time, which lets me really get into it, and be done with it (or spend some time thinking over it and savoring it) nearly as fast as if I were reading a book, and c) I don't see any reason to own a TV still.

I started out by watching the first 3 seasons of the original Star Trek over about 2 weeks. This was by turns painful and amusing. It was great when I got into the couple of classic episodes that I remembered seeing before. I lost interest by the third season though. It's weird to be able to watch the Trekkies movie and understand almost all the references. What I actually most enjoyed most was letting the individual plots roll off me and watching the other things, from the sets and FX to the characters to the trekkieness of it all, develop over time. Which set the stage nicely for the next bit of TV I gorged on.

Though I actually owned a TV when Babylon 5 was on the air, I only saw a few episodes and found it impossible to get into, so I watched some Deep Space 9 instead back then. This time I watched b5 it all the way through, all 5 seasons. That was fun, I really enjoyed being able to appeciate the much vaunted b5 story arcs without having to wait weeks in between each show and forget things.

I switched over to anime for a while and watched Cowboy Bebop and a couple other less memorable shows.

Then I may have watched some TV show that is only now begining to air in the US, though this seems hard for a law abiding person to do without a time machine or an airline ticket. Anyway, if I did, I found it to be a cut above any SF I'd seen on TV before, and was even willing to wait a few weeks for some episodes to come out. (Like the next one, which I want to see, already!)

I took a detour from SF again to revisit Scrapheap Challenge (aka Junkyard Wars), which is the only show I missed when I got rid of my TV. I was not suprised, but was still disappointed to verify that it was all downhill after the first few excellent years of the show.

Finally, spurred on by interesting stuff on the dpkg changelog, I watched Firefly, which I'd heard of before vagely and had seemed too silly a concept to work. To my suprise, it did, and I really enjoyed its brief run (including, rarely for me, the comedy and even accents) and have to consider myself a fan of this show even if I'm not (quite) one of any of the other stuff mentioned above. But I'm possibly the only fan who doesn't mind that it got canceled in the middle, because based on that short run I'm prefectly comfortable imagining how everything turned out. Having a few more years of it made would have been nice, but it would have just been icing. I'm happy to have the cake.

One other interesting thing I've found out is that I have a really strong memory for TV shows I have seen. Just like the way I can revisit a place I've been years ago and know just where everyting is, I picked right out the two episodes of b5 that I'd seen sometime before. And once I've seen a TV show once, I can almost never stand to see it again. Why I'm like this when I'll happily reread favotire books many times and sometimes accidentally reread non-favorites a couple of times, I'm not sure.

At this point I think I'm pretty much over gorging on TV. I might have missed a few worthwhile shows from the past 5 or so years, but I'm getting pretty tired of wading through the formualic bits (opening sequence, credits, plot) to get to the interesting bits (character development, storylines, show development, exposition). So it's back to books for me for a while.

mind the gap

Today I was struck by how much of a gap there is between two parts of my family and (local) friends. It's almost a generational gap, but not really. It makes it hard for half of us to communicate much with the other half, except for a few individuals who grudgingly span the gap. It's the division between phone users and email users.

On the one side are mostly the older folks, but enough younger ones too, who use the phone without qualms, but see email as a chore that should be put off. Or who, horrors, don't have email. On the other side are those like me and Anna who'd rather go to the dentist than use a phone, and who either enjoy email or at least appreciate that it lets us respond asynchronously.

In my circles whole little forwarding networks have sprung up to deal with this. Luckily we don't seem to have any people who prefer to only use instant messanging/IRC, although I hear that a few degrees from me there are some. So the current routing for a given message is pretty easy to figure out -- it's never more than two hops to anyone.

It's nice that I can receive voicemail via email now: This has actually resulted in me keeping up with my voicemail, and it means that I myself have begun to do some bridging, but only in one direction. It's of limited use though, since I think neither side likes leaving voice mail much. It will be interesting to see what technology, if anything, bridges this gap in the future..

microblog

Cali - BART. More Muir Redwoods. Neat Spanish houses. Feb flowers. Jen, Swati, cats. Fog n haze. Gleaming shops.

No dinginess, no teeming masses, no urban blah. Choosing what to perceive.

metal roofs and rain showers

The combination of metal roofs and spring thunderstorms is amoung my favorite things on the planet. This actually explains a lot, but I don't want to go into the details right now.

meetings

Gotta love working on free software, when the meetings are likely to be on weekends. Today I had to get up early for a d-i meeting, then had an hour for lunch and fixing debootstrap (still one issue blocks it from working with d-i again), before the second, Debian release team meeting, which ran until 5 pm+. I abandoned that one toward the end for company and general sanity, and so missed an important bit.

making a smaller busybox for the d-i boot floppy

I tried today to build busybox-floppy-udeb in some various ways to reduce the size of the boot floppy enough that a 2.6 kernel might fit on it. I thought it was interesting as a kind of one hour survey of the options for this kind of thing.

Linking busybox statically against glibc drops the total size by more than 120k, even when compared to the reduced size glibc that mklibs generates.

Amusingly, uclibc.so is ~20k larger than the mklibs-reduced glibc.

busybox FTBFS with dietlibc, didn't try to fix.

newlib is missing some functions used by this build of busybox (stat64, lstat64, mainly), Didn't try to hack this to get it to link, although the size of newlib (18k) makes it seem pretty promising if it would.

klibc is not in Debian yet, so didn't try it.

Since the winner so far was just linking statically with glibc, I tried linking statically with uclibc too. That was the smallest of all, 411k.

Unfortunatly even after all this the 2.6 i386 boot floppy for d-i ends up being 380k too big...

Why this blog won't last; investigating alternatives

I have thouht about setting up a blog for some time, but have been leery of doing so. It seems to have the potential to become an enormous time-sink, there are various aspects of hard-core blogging that seem very pointless to me, and the tech behind it (including RSS) feels shakey to me. But Planet Debian rather pushed me over the edge, so I'll give it a try before passing judgement.

This weblog is using pyblosxom (as I type this I add "iab pybloxsom pyblosxom" to .vimrc in the other terminal). Individual entries are written on my laptop, then svn added and svn committed, currently by hand. I already have a cron job on my server to svn up my home directory every hour. Before I wrote a single entry, I have to say I thought about this for a while, and I'm quite sure that manually svn adding the file, making sure my laptop is in range and that the network is up, and committing it, is likely to exceed my threshold for busywork very quickly. Which is one reason why this weblog is not likely to last, at least in its current form.

I try to automate everything. With one command, I can automatically lintian check a debian package, gpg sign it, use svn or cvs to commit my changes, tag the release, upload the package to the apt repository on my laptop, (which uploads it to the apt repository on my server, which uploads it to Incoming), update any web site associated with the package with its current version and changelog, possibly upload it to additional repositories as well (CPAN, another apt repo, etc), and possibly mail an announce list to let non-Debian users know about it. I go to these lengths, because if I don't automate it, it won't get done..

So I doubt that I will last one day without growing a "weblog" script that puts up an editor, comes up with a filename based on the title of the entry, html-izes it, svn adds it, and commits it if I am online. That's the obvious quick unix hack.

I think that the longer term hack, and possibly the proper approach, involve email. Ideally, I should just be able to mail an entry to my blog. I have looked around, and this is not a new idea. Hep is a possibly over-specified and -designed thing that can apparently gate email to blog and back. Unfortunatly, it has no security and wants to run as a daemon, and includes its own mail server, web server, and POP server, and is configured over the web. No thanks! Blosxthis is a quick hack that can be fed emails from procmail, does minimal security checking, and dumps them in a directory for use by (py)blosxom. Unfortunatly, it relys on a plaintext password instead of the dead-obvious idea of checking the mail's existing gpg signature (all mail is gpg signed, right?), and does not use headers or htmlise (xmlise?) the mail. I'm seeking something more appealing than either of these.

This guy had the same idea yesterday, and here is an old post from the author of pyblosxom describing something that sounds just about perfect to me. So I'm not the first person to think of this, and there seems to be plenty of activity in this area without me, so I will probably put off doing any work on it as long as possible.

This is getting long, so I will skip going into my opinions about more general weblogs vs email topics until another time.

LWN security comparison

So LWN did this article (subscriber only) and somehow I found myself with tons of mails about Debian and security in my inbox and hours spent on irc too today. Almost as if I were the other Joey.

First of all, AJ's post has one innaccuracy -- LWN did not look at bug #327210, but at bug #320017. Understandalbe confusion, perhaps vim's modeline support will be free of security holes one of these years. (Turns out I got it wrong not AJ --joey) Oh and also, the apache thing had a DSA today and is fixed in unstable, with testing hopefully getting the fix tonight. Though you won't see that in the bts.

I wish I had the energy to add a row to LWN's table to see how the Debian testing security team has managed with respect to days to get a fix in. I know that we have issued advisories for clamav, evolution, pcre, and vim, while fetchmail, proftpd, and apache have been or are in the process of reaching testing through more usual means, and the php holes are fixed for some things and not for others, as they affect lots of different packages that duplicate code.

Really, it's probably too soon to add us to the table, after all the fact that we are doing advisories is not even officially announced yet, and we were only able to start doing advisories after some of these holes were discovered, and are still in catchup mode.

I'm glad that LWN didn't choose to add kernel holes kernel to their table. I do think that like every security comparison I've seen so far, this comparison is flawed in significant ways, beginning with the criteria used to select vulnerabilities and going on from there. But it is a useful comparison as these things go.

I'm also glad that I took the time to read chapter 6 of Biella's thesis, since it clarified for me why these things seem to come up from time to time in the project as a seeming crisis and how that's not entirely a bad thing. Looking forward to the other chapers.

lowmem

It seems silly to call between 32 and 48 mb of memory "lowmem", but so it is in current d-i. Today I got the minium memory requirements back down to 32 mb, from 44 mb. I added a lowmem udeb which provides a whole low memory infrastructure, with various hooks. This should help someone who really cares about low memory support to get things down to 24 mb or so, I think. I still belive that 16 mb installs should be possible, but it would require some significant work on d-i.

low point / high point

This has been a crazy and frenzied week.

The low point was certianly Tuesday when I needed to move some bookshelves to storage to (mostly) finish my move and started up the truck and it .. didn't. Turns out something blew a fuse in the camper, and due to bad wiring this ended up draining the camper's battery and the truck's. Jumping a v8 deisel engine with a little compact car is .. interesting.

High point was today, when we got the satellite dish to raise and lock onto satmex5 and pull in email. Tempered by the annoying number of proprietary computer products needed to make that work, but I'll manage to replace at least one of firmwares of the two linux-running devices. Now if I can only get the power system straightened out.

loose ends

I Drove up to Richmod with Maggie for thanksgiving. I noticed that I have a ton of old mail about minor projects (wmbattery, procmeter3, alien, bsdgames) that I have been neglecting for a while, and I went through and dealt with a lot of that. One mail had been in my inbox since June!

login:

This screenshot is what happens when I decide to try to log into as many machines as I can all at once. Stats:

• 41 successful logins
• 11 failures
• 3 ways to spell "Joey Hess" (and "grue")
• 1 previosly unknown hardware failure (argh)
• 2 other machines unavailable due to that hardware failure
• 6 machines powered on remotely for login attempts
• 1 remote hard power cycle needed
• 5 ssh host keys changed
• 1 machine unavailable due to firewall
• 1 account I didn't know I had until reading a motd
• 4 machines unavailable due to being turned off
• 2 machines unavailable due to me not wanting to get up and plug them in
• 1 machine unavailable due to broken netboot image
• 6 first logins ever
• 51 linux boxen
• 51 debian boxen
• 1 solaris box
• 2 boxen smaller than my hand
• 52 hunks of metal and silicon
• an unknown number of countries
• 3 continents
• 1 hour wasted

loggy days

I've not been concentrating well or doing much the last couple of days. Yesterday I was outright lethargic and did nothing all day but catch up on 1.5 years of Pratchett books. Today I was at least able to stomach using the computer, and did a few things, like putting a provisional syslinux boot logo in d-i, and setting up some d-i backup stuff, and talking some build system changes over with ths. A nice long walk helped bring up my energy some, but I've still not managed to focus down and get to coding today.

I have started grappling again with an annoying d-i issue though. It's the problem of what distrubtion d-i should default to installing. Currently, d-i uses suites, not codenames, and will install testing by default. We need to use suites (testing, unstable) in the UI for several good reasons, but this does present a problem, since eventually we want d-i to install stable. The trick is how to do this without having a flag day when we rebuild all the images to default to using stable. One way to do it is to make the images default to using stable already, and require testers choose testing (possibly via a boot flag).

Another approach, and the one I am currently leaning toward, is to make d-i know the codename of Debian it prefers to install, and check the mirror to see if that version is available. If so, make the suite it corresponds to the default in the question. So d-i would know that it is currently targeted at installing sarge, and would look at sarge's Release file, find that sarge is testing (or, eventually, stable), and make that suite the default.

linux tools for geocaching

Hmm, I seem to have turned Mike Beattie on to geocaching. I've been meaning for a while to describe my setup for geocaching using linux, so here goes.

First you need a tool to download cache data. I've found two packages, the one I settled on, and which currently has a deb in Incoming, is geotoad. This lets me query for caches of any type nearby to whereever I am. It can be used interactively, as well as cronned, and it does smart caching to avoid hitting the website too much, so I have a daily cron job that updates a html file and a gpx format dump of all caches within 50 miles of my last recorded locaton. Of course that's put into svn and updated along with the rest of my homedir, so I have it available on my laptop for reference. Note that some uses of geotoad may violate the TOS of the geocaching.com especially if you're not a paying user; caveat emptor.

(I might write later about my opinions of the plusses and minus of having a centralized, for-profit site for geocaching, as well as possible FOAF-like alternatives or enhancements, but that's a different article.)

Anyway, once you've got the cache data, you might want to load it up into the gps or otherwise transform it into any of the seemingly hundreds of different formats. Here gpsbabel is king. Due to a bug in the linux driver for my usb to serial converter, I currently have to use a real serial port (on my desktop machine) to upload cache info to my gps, which can be annoying.

Getting to the cache often involves road navigation, and for this I use gpsdrive. This can display caches (coverted to its own format via gpsbabel), as well as road maps, and with gpsd can put you on the map. The only downside is that the maps themselves are not vector format, but are just images downloaded from online map services. There are some handy scripts that you can use to download very high detail maps of everywhere within (say) 500 miles of your current location, though some uses of these scripts may violate the TOS of varios websites (again..).

I've also recently began to play with using roadmap, which uses vector maps which are freely available based on data from the US Census Bureau. A debian package has been ITPed but not uploaded yet. The great thing about it is you can do address lookups, and it can read out street names as you drive around and has basic support for routes. Still lacking is support for giving on the fly directions to your destination. And its waypoint support is not ideal for geocaching since it's limited to points along roads.

Once I get near to the cache I typically put the laptop away and do the hike with just my gps. Unless water or very strenuous activity is involved I do take the laptop along in my hackpack so I can decrypt hints and make notes. I've written about using hnb for notes before. One final tool that I'm rather amused to be using on a regular basis again now that I'm geocaching is rot13.

linux on seatbacks

KLM/Northwest flights have a seatback screen, which does help while away the long flights once the laptop battery is dead, but which I've always felt was pretty badly designed, starting with the control device which suffers from modern video game console buttonitus, with 23 buttons, two of them for changing channels (which do nothing at all normally), and one of them nicely positioned to accidentially ring for a steward if you press it while taking the control device out. Then the LCD screen itself cannot be tilted to a usable viewing angle if you're tall and the seat in front is reclined..

But the worst thing is that just after takeoff, when the system comes on and 800 bored passengers try to use it at once. In the past year, they've taken to announcing on the PA that it's normal for there to be "some delay", but that's an understatement, I've seen it take 5 minutes to respond at all to a button press. As pasengers loose interest, or manage to get a movie playing, As soon as I noticed that I assumed there was a server or servers somewhere under the hood, that handled user input, and was getting overloaded. Anyway, just another glitchy and poorly designed computer system, which unlike most such I'm forced to actually use if I run out of anything else to do in the airplane.

Anyway, if you're stuck on one of these flights, here's something to try as soon as the system comes on, while it's still in overloaded mode. I was able to reproduce it twice, but perhaps I was only lucky, or things were more broken than usual and my actions had nothing to do with the result -- I don't really know. I was "typing" ten keystrokes ahead on this laggy system, trying to get to the map display, and I accidentually started a movie. I then pressed stop, but the system was so lagged it just paused the movie and did not go back to the GUI. Normally the controller sports an "interactive" light. This light went out, and I got a channel display in its place. I flicked though dead channels, down to 1.

At this point the program seems to have crashed, as I was left staring at tux the penguin above a linux boot screen that ended with a "press enter to login". I was so suprised I recoiled from the thing, the idea that this poorly designed and broken system could be a linux machine was a shock. They must have the GUI set to respawn, since it quickly switched away and slowly started the user interface back up, which took 5 minutes. I assume normally this load happens during takeoff.

After that I did essentially the same sort of actions again, and reproduced it again, and this time read most of the boot messages, this seems to be a small embedded system using busybox, and not very well put together (some boot errors). After this second go, most people had movies playing, and the system's load went down, and I cannot reproduce it anymore. I'd be interested to know if someone else can.

Link with a fun related story.

Update: At least one other DD has seen this too, under the same circumstances.

Update 2: Micah Anderson saw a similar problem on a SwissAir flight running a slightly different system and took pictures.

linux laptop in 24 hours or bust

Anna bought a laptop today and gave me 24 hours before she goes home to install linux on it and get it all set up. Ho boy.

The install went fairly well except for an annoying kernel oops on a sound driver; but it took less than an hour to get a functional desktop, and I even had her do the half of the install that did not involve working around kernel oopses. Not bad really.

Then we had to rsync everything over from her desktop, install a few laptop bits that d-i missed as well as all the apps she likes, tweak X, install a few peices of priorietary software (including ndiswrapper for wireless, ugh), etc. Finally tonight I set up the network she'll use between the desktop and the laptop, configured her synaptics touchpad better, and got alsa working.

The only thing missing is hibernation (well, and the modem); since software suspend isn't even in the 2.6.8 debian kernel, I didn't bother; and I imagine I may end up spending as much time as I did on the whole rest of the install on getting that working during her next visit.

BTW, my only gnome exposure is when I'm setting it up for various family members, so I am late in noting that gnome 2.8 is rather a nice improvement over the last one, and it's really nice that it got into sarge.

linking

It's weird when completly reasonable and thoughtful people disagree over a technical issue. For example, I see absolutely no difference between linking to a web page and linking to an image[1]. And yet I know many reasonable people who consider the former to be fine (and probably consider linking policies that prohibit it as close the censorship, and damaging to the web), and yet consider the latter to be some form of theft.

This is expecially strange since most of these people would not mind if I made a link to a tarball of code hosted on their website, even if it contained some images.

Apparently part of the concern is someone using an image on one's server in a way one didn't intend, perhaps taking advantage of its existence to save the bandwidth of hosting their own copy, or passing it off as their own work, or not giving credit for it. But when I put a web page on the internet, I welcome any and all forms of linking to it. If someone chooses to use the existence of that page in a way I didn't anticipate, then I have tools, such as copyright, or taking the page down, or modifying it, to let me deal with that. If someone takes advantage of me hosting the page to drive traffic to their ad-driven web site, and pounds on my server more than I expected (hi slashdot), I welcome the traffic, and deal with it. If I didn't want the world to look at it, I would not put it on a globally accessible web server. If I can deal with these issues for a chunk of bytes that represent text in html, I don't see why I cannot use the same techniques for a different set of bytes that represent an image, or why I should consider it to be any different.

In fact, I'd much rather that people linked directly to images and html on my website, rather than copying them, since in my experience when a high-profile copy is made of a document I've written, it can be annoying not to be able to update it.

Anyway, there must be a concern I'm missing, or a different set of constraints that make my outlook on this different from other reasonable people's, and I'd appreciate being told what it is.

[1] Yes, images are often loaded inline as part of another page, and yet an innovative browser might well inline webpages that were linked to as well, in some interesting way. Other browsers do not inline images, because they're text based browsers, but still let you see the image as a separate step.

life update

Half a week left on the month of getting up early and going into the office. I'm suprised to find myself not much minding the getting up at 8, although I blew it this weekend and returned to my typical sleep(-till-eleven) cycle.

After a lot of uncertainty, I am going to Oldenburg once again this year; flying out next Sunday and driving over from Amsterdam with fjp.

I also probably have a job again. "Probably" since it's signed off on but I'm not formally hired yet. I'm finally going to get to do some work on embedded systems, while still using Debian, so lots of interesting challenges and work to do.

I had to make a tough call on this one and opt for working part time instead of full time, since full time would have meant having to drop at least one of my long-term projects.

I'm being drawn back inexoriably to the farm. The thought of another freezing and/or smokey winter, sunset at 4 and no running water till you break the ice in the creak seems somehow appealing -- or maybe it's just a need to be back in the stillness of a winter night with all the stars out and it freezing cold and the outhouse waiting way down thar. Anyway, looks like I'll be able to stay again, if I dare.

I want to build a sauna.

life it too short to write blog titles, but long enough to blog

Wow, all of my sisters have blogs now. The parental figures have resisted the urge to adopt this newfangled thing though. Including the Jerry/Tomoko parentals, though all of us addicted to Kai hope that will change.

We've had a minor family get together this weekend at the campground here, with 10 people (if you count kinda adopted family of the Cobacks). It's been good, I even got to hike on the top of Roan Mtn. today; lots of AT thru hikers up there this time of year. And of course the views were spectacular. And me without geocaches loaded onto my GPS, which I seem to be getting a bit dependent on.

Hey, I'm looking for a sticker of some sort to advertise "free mobile wireless access point within 150 feet of this point". I'm suprised the signal gets that far, and despite all the fun of satellite internet FAP, I'm still willing to share.

library

Am I silly for thinking the best part of the new Bristol Library is the posh reading room in the rotunda with views out to downtown and its high speed wifi? I've not even checked out the stacks yet tho, let alone the upper level, so my opinion may change.

Also nice that on this, its opening day, it's ten times busier than the old library used to be, with people wandering all around, which is pretty cool.

let's break everything!

ths has committed his revamped d-i build system. It'll break a lot of stuff, but the nice new image names, and much improved configuration will be worth it. Go ths!

lessons learned

d-i beta 3 is released for 6 architectures. Unfortunatly we had to drop powerpc at the last minute, and there is plenty of errata for many of the other arches. i386 seems pretty good. The thing to take away from this is that people sometimes just won't respond to repeated requests for testing until you make a "release", no matter how bad it is.

leaving Vitoria

Tomorrow we'll leave Vitoria for DebConf4. I'm very glad I got to come here and spend some extra time in Brazil, and really experience living here and not only attending a conference. Shrimp and coconuts on the the beach, the packed neighnorhood square last Sunday night, the views of Virotia from the hill, a bad sunburn, visiting a capoeira class, the hammock on the roof, the snack shop that transformed into a bar somehow, cards with the family; I've stored up a lot of memories.

d-i progresses, and even works, depite what seems like the best efforts of at least one person to repeatedly break it over and over. I don't know what to do about that, but the less said about it in public the better. Just more stress I don't need; worry about not having enough testing lined up during DebConf is enough.

laundry farce

I was doing my laundry today, in town since quite a lot of large items had piled up. I grabbed a handful of change from my book bag, and just happened to look at the first quarter before I put it in the machine. It was a norwegian 20 kroner peice. Oops. The next coin I fumbled out was some variety of eurocent. I'm suprised I didn't come up with a hounduran 50 centesimo next, or maybe some canadian change, or a subway token or something. The ashtray in my car is full of various nationalities of pennies. I know change is one place where I can't keep any semblance of order or control, but this is ridiculous. I have seen machines that purport to be for people like me; you pour bags of coins you could never use in the top and get a few bills out the bottom. Tempting.

lately..

What have I been up to lately? Well let's see, I learned today that coming between a man and his Bob Dylan is not wise. Yesterday I rewired the server room to use a new switch which will hopefully fix all my network problems, and I realized that at 9 machines (one virtual) I'm hitting the point where sysadmin work is beginning to take too much time, so I may need to ease up on the rate I'm adding new test boxes to the network. I've been moaning at the debian kernel guys too much and thinking vaguely about YA mooix redesign, and trying to enjoy the somewhat unseasonable weather, and consuming too much science fiction and anime. I think that's all.

late

I wish I could do nothing all day and not feel vaguely guilty. Got up insanely late -- like 5 hours later than usual, missing two meals late, and went to see Lord of the Rings. I'm glad I finally saw that; it could use an edit (and less in your face CGI) but it had some very nice parts (like following beacon fires in chain across the mountains). But now I have 2200 unread Debian mails, ergh. Oh well, another day..

last few days

Saturday was a strange day. I gave up trying to get to sleep the night before, at around 5 am, and had one of my rare but often very productive early morning hacking sessions, in which I tore apart debian-edu's task packages and made it use tasksel. Around 10 am I was finally feeling tired, so I slept until about 3, and then went out and did as much physical activity as I could to reset my close so it wouldn't happen the next night. Swimming at the lake was also refreshing. For a wonder, it worked. Yesterday I went out to the farm for a day and night.

Today was the end of the d-i string freeze, so dozens of uploads, and I bumped most version numbers in it to 1.0. More success reports on s390, so we will be able to include that in the next release, and we're finally done porting to all the sarge architectures. Wow. The release has slipped a couple of days, but it largely on track. Steve posted general freeze plans for Debian (I'm too lazy to link), and d-i's release should fit into that perfectly.

Today was also network redo day here at my house, since all the peices finally arrived. I'm very happy to be back to having a Debian box connected directly to the internet and not gated through a mildly broken appliance anymore. I deployed my 12 dbm omni, and covered the whole neighborhood and several acres of the park behind it with open wifi goodness. I managed to stuff the wired network back into two rooms (it had expanded accross three rooms and 2 floors, and nasty wires were everywhere). Now I have a genuine 32 mb system to test d-i "lowmem" installs on too. The only casulty is netbooting the installer on my test laptop downstairs, since there is no wired network there. I tried to set up the proprietary AP to do wireless bridging via WDS, but though my server saw the link, I never got the AP to talk to it. Anyway, the network is much better than before.

I'm glad that the Shuttleworth group / SSDS / Warthogs / no-name-yet guys finally have an official name. Even if it was chosen by committee. ;-)

laptop musings

Choosing my laptop two years ago was a no-brainer, I just went to OLS and saw the laptop that half the kernel hackers were using. My Fujitsu lifebook has proven to be a really nice laptop, with excellent design and construction and it's held up well over the years. But I've given it quite a beating, and it's beginning to show its age, with all the little rubber and moleskin bits on the bottom eroded away, cat scratches on the lid, a hard drive with three bad sectors (I know where they are), and worse, two dying batteries. I used to get 6+ hours, and would use it off wall power all day long. Now I get 45 minutes.

It's also not as fast as it once was, barely usable for viewing video, and some newer apps are annoyingly slow. So I've been mulling over whether to spend money on fixing it up for another year or find a new sub-notebook.

One of the hottest new laptops in this class amoung linux users seems to be the IBM X40. It does seem to be a solid and well done laptop, but I've just never been turned on by thinkpads for some reason. When I saw it in person it seemed a tad larger than the laptop size I prefer. It also apparently has a 1024x768 screen, a resolution I no longer find usable.

I browsed over the laptops at Dynamism and didn't see anything that really excited me there either (enough to pay those prices anyway), and it's not clear that newer fujitsu's are as nice as mine, so I decided to avoid getting a new laptop for now and just ordered replacements for both failing batteries. Much cheaper, I can live w/o the CPU upgrade, and the only thing I'll really miss the the possibility of a laptop with a brighter display that could be used in more direct sunlight.

Maybe I should use some of my laptop budget to attend the next OLS and see what the kernel hackers are using next year..

kite offline (until you read this)

Kite (my coloed server) has been offline now for a few days. Only the second major downtime since I put it in the colo 2 years ago, this is the first the be due to my own mistake. I was actually trying to make the system more robust when I set up a serial terminal, and turned that on in grub and rebooted, but unfortunatly grub is not happy with my serial port, and the machine won't come up now.

The onsite staff helped me figure that out, but drew the line at fiddling with the software on the machine. The coop's staff and members have so far been very slow to requests to get someone onsite and do the fairly simple fix to grub, which leaves me, along with several family members, and a number of organisations, without our server. It's been painful, I've found just how much a single point of failure kite is and I definitly need to do something about that, probably by colocating a backup system somewhere else -- maybe somewhere a bit closer to me.

Of course by the time you read this, it'll be back online or the situation will be resolved in some other way.

just in range

      Link Quality=_2_/70  Signal level=-93 dBm  Noise level=-96 dBm

Why is it that nearly out of range wireless links make me happy? Yes, leaves are wet tonight.

joy of bugs

Filing a good bug report can be a lot of work, but it can be a lot of fun too. Sometimes it's just a matter of taking some data and whittling it down to a short and simple test case for a bug that many people have seen before, but never quite tracked down. This can be suprisingly fun, especially when the bug is in a programming language.

Or it might just be filing a simple clear problem description and finding out that no, you're not the only one to see that strange problem, but nobody else had gotten around to reporting it, and then watching the developer go off to fix it.

But the really fun ones to me partake more of debugging, when everything is fumbling in total murk and darkness one minute, and then a sudden flash of insight lays the bug bare before me. This is perhaps one of the top ten things about really working with a computer that I wish I could teach to people who haven't got it.

I've spanned the spectrum today, with significant bug reports in each of the categories above. Now if I could only find the energy to figure out how to file a good bug report on the 2.6 kernel, for not waking up my laptop's drive on resume from APM sleep.

isolated hardware downpours

The shuttle xpc that I decided to get as a media server / d-i test machine instead of buying a second used laptop arrived today. Just after I'd gotten that Asus router that seemed to die in Oldenburg back to working.

I didn't really spend much time choosing it on ebay from amoung the other used shuttle machines on there. For example, I had thought this was a new enough AMD chipset to be an amd64, but it's not, it's a k7. This probably shows how out of touch I am with current hardware. Since I didn't buy based on wanting an amd64 particularly, I'm pretty happy with it just the same.

Especially since it already has a 4 in one memory stick reader that includes a compact flash reader, so I can stop messing around with compact flash on pcmcia for work and just set up boot disks for my arm machines using the shuttle pc on my desk. And it can also serve as the terminal server for my arm boards.

I think it'll run vmware fast enough to do some good d-i development, and it can also boot from USB, although not using d-i (yet). And netboot, so if vmware feels pokey and I can do without the music that's the only other thing it is likely to be doing, I can reboot it for d-i tests too.

Reasonably quiet too, although it'll probably be turned off at night.

Anyway, I christen thee.. dodo.

Is your hardware clock set to GMT?

Somewhat apropo to my previous post, I happen to have spent a good part of today reworking how Debian configures the clock settings at install time. This is a part of the installer that goes way back to simple origins, was never really designed properly or easy to use, and accreated code and bugs until it was un-maintainable.

Some of the worse code it accreated had to do with the single most confusing prompt ever in the Debian install process, which I think many users read as follows:

Is your mumble clock mumble mumble HUH-WUZZAT!?

This is thanks to the idiocy of the PC (and other architectures) hardware clock not including a field for the timezone, so originally it had to be manually set back and forth at those "spring forward" and "fall back" times that I loath so much.

Then the operating system (ie, Windows) got involved, and would manually set the clock back and forth, which worked unless the computer was turned off. As it almost always was at 2 am when this change is supposed to happen. They tried to compensate for that by changing it when the system booted up on the morning after. I don't want to go into all the pain and bugs that led to. If you can't imagine some of them, you've not been using computers long, and I envy you.

Then Linux got into the act and decided that the clock could store the time in a sensible timezone that didn't do these daylight savings shenanigans (namely, in GMT). Linux could just convert the internally used timezone to the user's timezone when asked for the time, and the user could be automagically moved between timezones (ie, from EST to EDT, and no I don't know how that's done) and the clock never needed to be reset.

This actually works well, and the resulting automatic tracking of daylight savings changes is part of the reason we want Linux to run on our phones, our TVs, PDAs, airplanes, microwaves, cars, and even on our wristwatches. All of which have been done except perhaps the microwaves.

Problem is that we still want to use Linux on our computers too, and some of us also want to use Windows on the same computer. And Windows expects the clock to be set to the local time, and wants to try to change it twice a year, and so on. So to make Linux get along with Windows in this situation (to some extent), we had to ask the user that horrible, horrible question:

Is your hardware clock set to GMT? (If it is, Windows will hate you.)

The alternative being to do as many other Linux systems did and just default to one or the other, which is at best sub-optimal and at worst leads to a world of pain on the next daylight savings day.

But Debian's installer has gotten pretty sophisticated and some folks worked out that it could pretty easily guess at answers to that question that will be right most of the time, so once the work I've done today gets released, it's very unlikely you'll see the question anymore when installing Debian. Or I'll get a zillion bug reports for messing it up. Or both.

Anyway, this is one of the reasons I hate and despise daylights savings time. It's eaten days of my life, and they weren't only spent moving hands on clocks like all the days of your lifes that will be wasted by it in the end; they were spent dealing with complex stuff like this, when all I wanted to do was to help people install Debian.

ion3

I spent some time this evening switching from ion2 to ion3. It was painful, because there's no coherent documentation for ion3 yet, and even incoherent docs are somewhat scarce, since there's no longer a wiki. I found myself reading the source more than once. Still, it was worth it. I eventually recreated my old ion2 config, and then played some with the new features.

I've never been very enthused with ion3's new scratchpad, or the other stuff for overlapping windows, because unlike some ion users, I don't have a nagging hankering for overlapping windows. Likewise the statusbar thing is just reinventing the wheel, and the dock is better (now usable for me), but boring.

So I'm very happy to see unique new concepts in window manager design are still popping up in ion3 despite those additions. I'm also starting to see why some recent ion innovations have been described with videos, it's kinda hard to describe a floating split or how the PaneWS works in words.

The floating split is especially useful; now my main desktop can reconfigure itself dynamically in some handy ways as I move from frame to frame. I've got a log monitor barely visible down in a bottom corner of the screen, just a few lines there so that I might notice some strange message out of the corner of my eye (see calm computing). Move the mouse down to the bottom of the screen (or otherwise move focus down; basically, "look down"), and it expands to let me see many more lines of logs. No clicking, window dragging, or any of that nonsense needed either, it just there when I need it, and then it goes away.

A similar trick lets me switch between my large main web browser / coding / email frame and a collection of small frames holding irc sessions and stuff that is hidden at the bottom of it. Works much better than the old nested workspace approach.

ion and lua

I've been using the ion window manager for half a year or so, and it's mostly sunk into the background, like a command prompt it's nothing special, just something that's there, that always does what I want, and without which I would be crippled.

It started for me when I noticed a request for a Debian developer to sponsor a new package that I'd never heard of. For some reason I did, probably because I had nothing better to do at the time. I've been sponsoring ion for Per ever since (hope he becomes a DD soon!). While I sponsored it, and played with using it, I had been using the same WindowMaker setup (including the same background!) for going on 5 years. I'd tried to escape by way of ratpoison, gnome, larswm, and blackbox, but I always returned to WindowMaker after a few minutes. I was set in my ways. Finally, after sponsoring the package for 5 months or more, I took the plunge into using ion full time, and have not looked back.

It didn't take very long to get a setup that worked for me, and I didn't learn ion very deeply while doing so. My basic ion configuration has not changed much since, though I upgraded to ion2, rewrote my config files to ease maintenance, bagan using the new dock, etc. Perhaps three days of learning and getting adjusted to ion, and adjusting it to me, followed by months of ignoring it.

Lately I've been getting more into fiddling with my ion configuration. Tonight I played with nesting workspaces in frames, and came up with a fix for an annoying problem I've had: I tend to have lots of windows I only want to glance at occasionally, doing downloads, on irc channels, or doing other long-running tasks, and the tabs for these would clutter up my frames until it was hard to find the important stuff that I was working on. My solution was a little nested WFrameWS named "clutter". In the screenshot I have clutter attached to one of my small work frames, while I'm reading mail in the main large frame. All the cruft I'm doing -- downloads, waiting for responses on various unimportant irc channels, etc, is in this one frame. Ion even does an excellent job of resizing it, so if I move it to a larger enclosing frame, I get lots more details. And with a keystroke I can banish the whole lot off-screen and get back to quietly coding.

Which brings me to lua.. I've written before that I've been having a hard time lately getting motivated to learn new programming languages. For example, I've toyed with python, but its solution space is so close to perl's for me, and I'm so experienced in perl, that there has not been enough payoff to learn it. It seems that by way of ion though, I may just have found a reason to learn another language. Ion is extensively tweakable in lua, and I've absorbed some lua basics just by editing my config files. I can think of so many interesting things to do with lua on top of ion, and the language seems nice enough that I think I do want to learn it now, in detail.

Anyway, it's just a window manager, it has its warts, it's not for everyone. I'm very glad it's the window manager for me.

inside the WR850G

I have a WR850G wireless router, which I no longer use much for reasons explained before. I knew this ran linux, but only today did I find out how to run shell commands.

It's a small mips box running 2.4.20, busybox, zebra, pppd, dnsmasq. Of course I did not receive a written offer from Motorola for the source to this GPLed software.

wget is available and works, but I've not succeeded in running any programs yet. I'm not sure if this is a problem with the busybox on it not running non-builtins, or a problem with my binaries. I guess if I really wanted to hack my router though, I should have bought a linksys..

importing old advogato blog entries

Long before I got my own blog, in fact back before I knew what I blog was, used advogato for an online diary. Today I finally took the time to import that data.

Advogato could make this easier if they had a way to get a full RSS feed of a blog. Perhaps they do and I didn't look hard enough, I didn't find anything on google either. As it is, I had to write a quick web spider/converter. Seemed to work ok, although some relative links to pages on advogato are broken. Oh well, that's a project for another day. This is why I prefer to control my own data even for things like a weblog.

(Update: Sigh, I forgot about advogato's xml-rpc interface, which would be the clean and close to right way to do it. Too late now.)

Anyhow, all the old entries can be found in my blog, starting here.

ikiwiki

Thanks to everyone who contacted me with wiki suggestions.

After looking at a lot of wikis, and finding none that really met my requirements (svnwiki and zwiki probably came closest), I decided to write my own. Currently it's some 300 lines of code that implements everything I need in a wiki for my project right now.

A few minor features are lacking (online page editing, search, RecentChanges, themeability). Actually, I've turned the traditional wiki concept inside out, and instead of a big engine that stores pages, history, converts a custom markup to html, etc, etc, my wiki just runs on a set of files, formats them to html using a standard formatter like markdown, handles WikiLinks, and writes out static html for the web server. It's a wiki compiler.

To let other people edit the wiki, just set up a subversion (etc) repository, let people commit to it, and run the compiler in a post-commit hook. Of course, it takes care to only rebuild pages when something has changed that makes rebuilding necessary, so a typical commit will take less than a second to compile and go live.

I'll probably add all the missing features as I need them, and might eventually release it. BTW, I'm calling it "ikiwiki".

If you don't grok it, don't bash it.

Hi Colin!

foreach my $header in (%headers) { print WFILE "$header: $headers{$header}\r\n") }

No, no, no, back to perl beginners class with you! For one thing, that has three separate bugs in it. For another:

print WFILE "$: $headers{$}\r\n" foreach keys %headers;

I can hear all the pythoners moaning about the implicit $_, but this is in fact the idiomatic perl method of writing this common experession. Oddly enough, it's not only shorter than the only way to do it in python, but it has less punctuation than the python version, by six whole characters, and it eliminates the headers[header] thing that mashes together two too similar variables.

Also, Colin's original perl version and python version had exactly the same number of punctuation characters (17), though he claimed more for the perl version. That is not the first time I've seen such a mistake in a pro-python rant. It does not inspire confidence.

Now, while I'm speaking of python, and confidence, I was very disturbed two days ago to discover that python, for all its vaunted strict type checking, does not find the bug in this program:

did_bar=0;

if bar():
        did_ber=1
        
if did_bar:
        print "did_bar set"

Even pychecker will not find the bug. Maybe python programmers do not make these kinds of typos, but I do, and in "production software", the above mistake can be a beast. After I found this I threw diveintopython at the (figurative) wall, and abandoned python for the nth+1 time.

if it's not broken ... ?

I have a little script that displays a summary of when I last uploaded each Debian package I maintain, as well as the Standards-Version each package is at. After stable releases or once or twice a year, I like to go through and upload all my packages again, often on some pretext of policy changing, but really just to see what breaks.

So I did that today for a few of them, mostly some old games that I still maintain for no particular reason. Unfortunatly I also took the time to look at the security profiles of these games, which were sgid games. I've felt for some time that keeping sgid games in Debian is only inviting security problems, since most users play games on single user systems nowadays, and the global scores files we get with a sgid bit are just not worth it. Except for nethack, and maybe things like bsdgames.

Anyway, this led me to looking at the code, and so I ended up finding a couple of security holes in xemeraldia and xgalaga. These have been fixed, and these games no longer have global high score files in Debian, but now that I'm two for two with checking this, I'm almost scared to look at the rest of my games.

Moral of the story: Sometimes touching something that's not broken turns out to be worth it; and also, if you have a setgid game that was written before 2003^W6ish, or any setuid game, please fix it so it does not need any special permissions to run, because finding these security holes really sucks.

ia64 d-i automation

I've been playing with the ia64 mainly on weekends, since it's not very work-related. Today I switched it over so the permanent, development install is netbooted and uses nfs root. This lets me use d-i without worrying about accidentially destroying the system I use for development. Then I set up a serial console, using conserver. Since the ia64 always netboots now, switching between the development system and d-i testing only requires changing a symlink on the tftp server and signaling the ia64 to reboot by sending some escape codes down the serial cable.

So with that all properly automated down to a single command, I have a remotely accessible ia64 d-i test system that's as easy to use as vmware (and really somewhat faster, though annoyingly displaying at 9600 baud). Next steps will be to really begin automating it by preseeding debian installs and examining the install logs for success/failure..

Anyway, if d-i developers would like access to this ia64 test system, I can probably set something up.

I hate writing titles

I spent part of today in town at the internet cafe and errands, and the rest was routine d-i work. I'm still working on partman, now getting it to offer autopartitioning first to simplify the install some.

Reading lately has been Haldeman, Barnes and Michner. Unfortunatly, The_Hemmingway_Hoax wanders down the tired trap of incomprehensible time travel paradox. Although the reversed time sequences are some of the better I've read. Candle is not my favorite Barnes, but better than some. Better than the ones that involve time travel paradox, come to think of it. Meanwhile, Alaska is a strange mix of stright historical account and fiction, and I have trouble telling where the dividing line is. It's also insanely long.

I'm reading Alaska on paper, and I had a weird moment where I thought, "Hmm, a map might be nice right about here, it's a pity this book is a plain-text ebook". And then looked at the physical book in front of me. If this tells me anything, it's that I've succeeded in moving to ebooks.

Since switching to emailed RSS feeds for 75% of the web sites I used to visit, I spend a lot less time on keeping up, and I'm looking for more useful feeds. I've added the BBC, cannot find a weather or news feed for any city within 300 miles though. There's probably a market here.

Oh and I may actually pay slashdot money if they put the full article texts in their currently useless RSS feed. Maybe. I know I would pay LWN for this, but they never answered my mail about it.

how to flash HP iLO firmware without linux drivers or windows

If you're like me and have a HP Proliant system with iLO and need to upgrade the iLO firmware, but have neither windows nor an installation of one of HP's modified kernels with the special iLO drivers, nor anything else except w3m and sed, here's a way to update the firmware without the pain of first installing one of those things.

1. Pretend that you have RedHat or Suse and go to the firmware download page for that distribution, and get what they call the "Online ROM Flash Component for Linux - HP Integrated Lights-Out". This is a self extracting shell script.
2. sed 's/rm -rf/#rm -rf/' < CP005072.scexe > foo
3. sh -x foo
4. Notice where it tried and failed to run flash_ilo from in the above, and cd there: cd /tmp/sctmpdirnnnn
5. Make sure the ilo164.bin looks like a valid firmware to you and upload it to iLO using the web interface.

How to file a bug report on the Debian installer (for Debian developers)

It seems that very few people who are not Debian installer developers can work out which package to file a bug report on for installation. Since this is gotten wrong so often, and always ends up wasting the time of one or more developers to categorise the bug report, I hope at least Debian developers and more experienced users can take the time to read the guidelines below before filing bug reports.

• Please do use the d-i tag for bug reports, if the bug has to do with the debian installer or at least affects standard Debian installs.
• d-i is not the old boot-floppies installer. Please do not file a bug on boot-floppies unless you installed a distribution prior to sarge.
• Do not file bug reports on the install or installation pesudo-packages. We would like these pseudo-packages to be removed but have not convinced the BTS admins to do so. In the meantime your bug will likely rot there until one of the (infreqeunt) triages of these pseudo-packages.
• If you have a general report about an installation, with multiple problems, and/or you can't be bothered to figure out what the right package is, then file an installation report, against the installation-reports pseudo-package, using the template, and filling it out completly. Note that incomplete templates may be be ignored. Given the time it takes to fill it out correctly, your time may actually be better spent in reading on and figuring out the proper place to file the bug. And only a few overworked people read and process these, so we tend to use them mostly to spot frequently reported issues. While your report will probably be skimmed quickly by a couple of people, there are about 900 before it in the queue for more detailed processing. If you would like to help, see here.
• If you would just like to report a success, perhaps with general comments, feel free to file an installation report. We love these, and will be happy to close it immediatly while incrementing our internal it_seems_to_work counters.
• If the bug is a kernel bug, then file the bug report on the apropriate kernel package. Filing bugs on the kernel pseudo-package is a valid choice, but will probably lead to a less effective response than filing it on the actual problimatic kernel source package. d-i uses the same kernels that are in the Debian archive as debs, so this applies even to kernel bugs that break the installer.
• If the problem occurs during base system installation, you should be able to read the logs and work out what deb is breaking debootstrap and file a bug on it or on debootstrap.
• If you've gotten to here, I hope you can take the time to work out the appropriate udeb on which to file your bug. I expect you'd have no difficulty with working out the buggy deb if the problem occurred as part of the second stage of installation -- after the base installation and reboot into the new Debian system.

  You generally don't file a bug on general; instead you track the error message or other problem back to a specific package and file the bug there. Well, the same applies if the problem ocurred as part of the first stage of installation. Unless the problem is specific to the installer's initrd or other media, or is too general to fit anywhere else, debian-installer is generally the wrong place to file the bug, and bugs incorrectly filed there have to occasionally be triaged to the right packages.

  If the bug is some other problem in the installer itself, then look at the logs, which are marked with the names of the udebs. Or run the installer in expert mode to get a good idea of which menu item (== udeb) causes the problem. It might help to familiarise yourself with the set of udebs that make up the installer, perhaps by looking though the d-i Packages file. In general the names are quite obvious; things like choose-mirror, base-installer, grub-installer and partman (for partitioning). If you have a debconf message and want to trace it back to the udeb that contains that message, check out the pot file for the entire installer. If you have a file on the d-i initrd that you have determined is at fault, you'll have to use some guesswork or look through our svn archive, since there is no Contents file for the installer (yet).

It might take a few minutes to learn which udeb to file a bug report on, but by doing so you save the d-i developer's time and begin to learn about what components make up the installer. For a Debian developer, it's not far from here to making contributions and bugfixes of your own..

how the other 90% lives

Just got off the phone with my Dad who has a windows machine that I guess is chock full of viruses. He described the mouse pointer being freezy and jumpy, which is pretty amazing since AFAIK windows' #1 priority is to keep it moving smoothly. Internet connections stall or die for minutes at a time. The real clincher is that outgoing traffic is significantly higher then incoming traffic. And some odd credit card bills have been happening lately.

Of course we'll be replacing this mess with linux, but I'm just amazed at how badly he's apparently being affected. I assume it must be quite a collection of viruses and other parasites since no single successful parasite should hurt its host that badly.

how I voted on non-free and why

I have not been participating or reading enormous and flamey threads about non-free on the Debian mailing lists. If the point of those threads is to change anyone's mind on the issue, then they have failed. The point of this post is not to change anyone's mind either, just to record part of the content of my own.

I voted in favour of asuffield's proposal to amend the social contract and drop non-free from the Debian archive, even though I do not believe that a vote is the best way to effect change in Debian. I believe this change will happen if people step forward to do it, regardless of the outcome of the general resolution, which I expect will probably fail.

I support removing non-free because I believe that:

a) In sarge, it will not be useful to the majority of our users. First because the majority of our users no longer need software from non-free to productively use the Debian system. But also secondly because regardless of the outcome of this GR, the sarge installation will not ask about including non-free in sources.list, and only users well-versed in using Debian will notice, care, or modify sources.list manually to include it. If they do so modify it then they can just as easily add supplimental apt sources outside of Debian. And, experience suggests that many of these users will. So the outcome of this GR is irrelevant to most of our users.

b) The continued divisive and hurtful discussion of this issue is harmful to Debian, as is our continued estrangement from the FSF over distributing non-free software. The only way to stop the issue from harming Debian is to settle it once and for all, and the only effective way to do that is to drop non-free. For unfortunatly, if the GR does not pass, and non-free remains in the archive, the discussion will at best die down only to resurface again in a few years. Therefore, even if I didn't belive in a), I would still belive it was in Debian's best interests to drop non-free.

Note that if I am wrong about point a), then we can expect to see many complaints from our users when sarge is released. Current indications based on the reports of sarge installation beta testers are that we will not see such complaints. Most users do not notice that non-free software in unavailable on their installs of sarge.

Unfortunatly, I do believe that this resulution will probably fail. It may receive a majority of the votes, but will probably not attain its required 3:1 majority. If that happens I hope that effort to split non-free out to a separate project will still continue. If it does I will support it by removing the packages I maintain from non-free and moving them to the new archive.

hotplug

Today I got a digital camera working under linux for the first time over usb. This was really pretty easy, what was not so easy was debugging a hotplug problem that ran the script I'd set up twice when the camera was plugged in. What is it about applications like pcmcia and hotplug that call for vast, evil, undocumented shell scripts in /etc? Have these people never heard of the unix tools philospohy? Argh!

Support for hotplugged devices is an area where Debian could lead and excel in integration, but things like the README.Debian of libgphoto2-2 are depressing:

These scripts are intented to be used with a front-end application. libgphoto2 is not a front-end application. I will close all bug reports asking for an automatic installation of one of these scripts during the installation of the libgphoto2 packages (same thing for print-usb-usermap).

The scripts in question are mostly badly designed and broken hacks, but there seems to be little desire to find a right way to handle these things, and make Debian do it. Instead we're happy with packaging broken hacks in examples/ and refusing to support them. Meanwhile, other distributions do fairly useful things by default when usb devices are plugged in, and get good reviews.

hosed

It's been a long day, too long to remember what I started out doing this morning, or type. So why am I typing then? Oh well. I think we're going with partman.

Oh yeah, I remember somewhere in there amoungst all the furious typing and coding, playing several games of lost cities out on the porch with Mags, and it was insanely warm, so we were even barefoot. Like summer.. Sweet.

hmm

After a couple of days of travel with occasional automotive near-disasters, it's been almost relaxing to work today on tearing down Dad's sunroom. Exhausting though.

Hit of bandwidth

I spent this afternoon at the coffee shop getting a nice little hit of bandwidth in lieu of other stimulants. It's nice to have my d-i daily cd images, bugs, mail, apt all synced up on the laptop for a change.

My cron jobs have been nagging me for a while about some new versions of some of the perl modules used by grepmail, so I got those updated. Also released a new linux-kernel-di, adding both 2.4.24 kernel for i386, and vorlon's xfs kernel images. This may have the record for most binary packages from one source now:

joey@kite:~/src/debian-installer/kernel/linux-kernel-di>grep ^Package: debian/control |wc -l
248

Yoiks, I'm glad that's all auto-generated. ftp-master must hate me. It should reduce after we can drop the 2.4.22 kernel for i386.

I just RFA'd slrn. I've maintained slrn since 1996. But I rarely use it now; it's easier to do everything via email, and so I have leafnode gate the few newsgroups I read to mailing lists, and subscribe to those. I've been dropping rather a lot of packages lately to make time for d-i, but slrn is a biggie. I hope it finds a good home.

Oh yeah; I held off two whole days before writing my script: svn://kitenet.net/joey/trunk/bin/blog

high-voltage fun

Since I can't find a remote managed power strip for less than $300, I made my own with some x-10 wall socket modules and old x-10 firecracker stuff from circa 1999, mounted in a wooden case. Unlike that time when I was ten, I didn't blow any breakers, nor did I electrocute my bed.

Now I can individually control computers in my d-i test rack that need hard power control, like the alpha, powerpc, and sparc, as well as auto power off the whole test setup, including the switch, console server, and all test machines. I expect to finish adding alpha to my setup tonight, then I'll be automatically test installing d-i on 7 arches every night.

I guess this will pay for itself in a couple of months on lower power bills, too.

hick roots

Turns out one of my great-great-great-grandmothers was a Hatfield. You McCoy's watch out.

Far too much time was spent today fighting with an ancient laptop, OS, and printer, ending in abject failure. Family tech support is such a beast, I swear to never do it again for family members not running linux, but I've sworn that one before.

I stumbled across John Goerzen's blog, which is not on Planet Debian, and from there to his Rail Passenger site. There's a certian amount of propaganda there, but also many pleasant reminders of my own month long round-the-country trip by rail. I'd like to do that again, sometime but perhaps with some time in sleepers and a cut through Canada.

On the d-i front, we have t-p-u set up for d-i (thanks, James!), and so unstable is unfrozen. Lots of uploads of pending stuff, so things are set to become more interesting tomorrow.

heh

halloween release?

The other day I got really frustrated with the d-i release treadmill, as previously documented in this blog here and here. I had to go out to the farm and spend some time offline, which worked pretty well, at least I was no longer snapping at people today for their honest mistakes (and mine).

I was slowly downloading some fun halloween songs when I finally realized that the d-i release is scheduled for effectively Halloween (or a few days before if we're lucky). I don't think I'll put any ghosts and goblins in the installer, but maybe we can do something fun with this.

It's a pity that Debian itself seems nowhere near releasing sarge, blocked by security infrastructure (much as the woody release was blocked for a long time) and by the regular RC bug count. I feel that we've lost momentum again. Keeping that momentum going, keeping the whole project focused on a release is probably one of the hardest jobs of our release managers.

Hacking timezones and stuff

Fun little coding session tonight, I made tzsetup use d-i's country information to nearly automate the selection of the timezone. Now installers can just hit enter most of the time, or choose from a short list, instead of fumbling through the badly organised zoneinfo layout. Of course this is waiting on the countrychooser from Christian P. to become really useful, but it vindicates splitting country from language selection in my eyes -- by getting accurate country information up front, d-i can have much better defaults for stuff like time zone selection, and the install becomes simpler overall.

Anyhow, it was fun to survey all the weird time zones out there. If anyone knows why it's significant today that some parts of Canada "did not observe DST 1966-1971", lemme know. Now if only Kentucky and Indiana would get their acts together and stop ruining the US timezone list for the rest of us.. (Don't even get me started on the evils of DST.) But it's astonishing how much code we waste on our antiquated, baroque timekeeping system.

The rest of the day was random bug fixing, and Mom's 61st b-day.

hack hack

Very nice debian-cd team meeting today. If the rewrite happens I think it will solve all the problems with debian-cd as well as several other issues. The fuse stuff Steve is working on will also add some amazing features for working with Debian CD images. And Phil has a great idea for an errata package.

Besides that, I think we've solved the very long-standing base-config charmap reset bug, and I've also started to add support to tasksel to let it guess whether the desktop task should be selected by default or not.

guilty pleasures

Well, I can't post this blog entry without admitting to one guilty pleasure of mine, which is watching the Veronica Mars tv show. I feel guilty about it because it's such a stereotypical LA high school type TV program. Cliques, nerds, fancy cars, girl detectives, it's hard to admit to watching it, and yet we also have some good characters and mysteries, and some great narratives, and what seems (to this old fart) to be a pretty accurate depiction of how kids these days use technology from cell phones to the internet. So I enjoy the show.

Anyway, I was pleasently suprised by a bit of dialog in yesterday's episode that started out like this, and went on to a spot-on bit of Linux vs. OS X advocacy:

Wait, how can you even have an opinion on Ubuntu if you haven't tried it? 2.6 kernel, live CD, they even had Gnome 2.0 the day warty warthog came out!

I won't claim Ubuntu as a guilty pleasure of mine; in fact I have some concerns along the lines of what Ian Murdock couragiously raised (and has been quite unfairly flamed over). All the same, it's Debian based, it uses my installer, I'd recommend it to any high school kid who's into tech.

So, big props to all the Ubuntu guys for breaking out into the mainstream like this!

great day

This is the kind of day that makes winter bearable. Thank goodness for high pressure fronts, it turned out sunny and almost warm. I couldn't help but go barefoot a bit, though I didn't get in my spring sunbath quite yet. I did get out and did some more geocaching, including three separate cemetaries, then a few hours of biking.

great blue heron

This winter between weather I preferred not to be out in, the need to be out in it for necessary chores like hauling water and firewood, family obligatons, and heaps of work on d-i, I mostly lost the habit of getting out and hiking around in the woods every day. That's been changing some as the weather has improved lately, and as cabin fever threatened to set in.

I followed a great blue heron home today, from down at the far end of the meadow, up the creek past the waterfall and the pond. I stayed well back, but the poor bird must not have been happy to have this human following it. They're one of my favorite birds, with many personal associations, and it was nearly magical for me.

GR vs consensus

This recent mess with the Debian social contract being amended, and this then turning out to have consequences that few expected, is making me even less enamoured of the General Resolution process than I was before. It seems that the GR process is a very poor replacement for the system of rough consensus (and dare I say, running code) that it replaced. It is slow, unweildly, and does not encourage us to listen to each other and reach useful compromises and consensus.

It used to be that the pattern was that an issue would come up, we would discuss it, argue over it, and eventually either drop it, making no changes, or work toward a consensus because there was no other way to deal with it. If the issue had to be dealt with quickly, the DPL would step in. This process worked for such core decisions as a drafting of the DFSG and Social Contract.

Now when even a midly controversial issue comes up, the words "GR" are soon uttered, and the entire process collapses into a flurry of proposals, seconds, and possibly an eventual vote. This encourages polarisation into two or more camps and campagning, and does not encourage listening to those who disagree with you. The end result is a worse decision than we would achieve under the old process, which is reached after less thought and less weighing of alternate opinions.

Propoents of the constitutional government and GR process used as their main agrument for it that the Debian project was growing too large to support a system of consensus; as it is hard to reach a consensus in a large group. But a large group of developers did not vote on the most recent GR. It is possible to forge a consensus amoung a small group such as the 200 people that voted on it; we've done so before.

I'm not keen on proposing that we throw out the GR process, because that would mean .. another GR. But we should attempt to make it a course of last, and not first resort. If only 200 people are interested in editorial modifications to the Social Contract, then they should be able to work as a subgroup to form a consensus on that, and such a consensus should have considerable weight.

In the meantime, I find myself increasingly drawn to the small subprojects in Debian where consensus still lives.

gorgeous day

February is off to a great start, it felt like spring today with highs in the upper 50's. Of course everyone who could dropped by the farm on this first decent day of the year, so there was company all day.

We got another half cord of wood in too, which will be nice if the rest of the month does not live up to today.

googlewacking

I'm not suprised that alien ranks in the top ten results for alien in google. But it is suprising to find a picture from that page in the first page of the corresponding google image search. Especially since they didn't even choose the picture of an alien, but the picture of a UFO.

But not as weird as, last night, realizing that I'd not read anything by Andre Norton in ages, googling her, and seeing the sad news at the top of the page that she's died only hours before.

googlebar meme

Ah, a blog meme I cannot resist. The contents of my google search history, in apparently ascibetical order:

• DC license plate
• WR850G GPL
• WR850G busybox
• WR850G copyright
• WR850G shell
• bbc radio stream
• bush children
• daily show
• diebold ohio quote
• effect of second hand marajuana smoke on infants
• export env var in python
• flash
• freematrix
• fuckinggoogleit
• gcc priority standard
• kqed
• mediamatters
• micro telnetd
• mitre can-
• motorola homenet wireless router default password
• tesla debunk
• votewatch
• xchat register

Left out a few near dups/misses.

good introductory game programming langage/library?

Today I helped Ben install Debian and watched him begin to learn basic unix commands. It's been so long since I learned unix, it's nice to recapture that fun learning process again. And reassuring to see a teenager drawn immediatly to the command line: as soon as the flashy keen gnome desktop poped up on his computer he wanted to know how to get to the terminal he had been playing with on Anna's.

So Ben wants to learn some programming, and he's mostly interested in doing graphics stuff, although not entirely. And he has some simple programs he wants to write, so he's primed and ready to learn. I think basic 2d graphics is enough, but I don't know what programming language or library to recommend to him, since I last did graphics on, approximatly, the Atari 130 XE. An equivilant of which would be fine, I think he'd be happy to start with simple sprites, point and line drawing, etc.

So far the best fit I've found seems to be pygame. OTOH, Ben has been learning just a little perl already. But I'm pretty much open to anything, as long as it's free software. If I recommend non-free software to him it would probably be some macromedia thing since I've seen people experimenting around with graphics in that in exactly the right way.

Any recommendations?

gmail spooler

I saw in Boing Boing that someone has developed an "automatic gmail invites spooler".

Unfortunatly it doesn't seem to keep a queue of unfullfilled requests, so it won't deal well with being out of invites. That would seem to be an easy enhanceent to add, the alternative is to hit reload occasionally or launch an evil wget command. I was able to get an invite within 10 minutes, which I fed back into the service since I don't plan to use gmail.

Since I last ranted about gmail invites stupidity, my maildrop filter has caught about 100 mails, including some big threads on lists like nanog, arguing pro and con about this form of spam. At a guess, these arguments are happening on a great many mailing lists. I hope this website will either help reduce the amount of offtopic gmail traffic on mailing lists, or convince google that it's time to stop with the invite business already. Google, you've wasted enough of the techincal community's time and bandwidth. Shame!

giving s/390 a try

Note: This is an old blog entry. It was turned into some more up-to-date documentation that is part of Debian's hercules package, in /usr/share/doc/hercules/giving_s_390_a_try.html

It turns out to be suprisingly easy to install debian s/390 in hercules using d-i. At least I didn't know anything about s390 and I got it to work. And it's suprisingly fast at most things. Quick howto:

1. Make sure you're running a 2.4 kernel, hercules does not work with 2.6 (#241064).
2. Install hercules.
3. See README.Debian about virtual networking perms. I added a tun group, put myself in that group, made /usr/bin/hercifc suid and executable only by that group, and made /dev/net/tun writable by the group. Also, be sure to echo 1 > /proc/sys/net/ipv4/ip_forward and make sure hercules will be able to get to the internet.
4. Download installer-s390/images/current/generic/* from unstable or testing.
5. Make a hercules.cnf in the same directory:

   CPUSERIAL   002623
   CPUMODEL    3090
   MAINSIZE    64
   CODEPAGE    default
   XPNDSIZE    0
   #CNSLPORT   3270
   #HTTPPORT   8081 noauth userid password
   #HTTPROOT   /usr/share/hercules/
   NUMCPU      1
   LOADPARM    0120....
   OSTAILOR    LINUX
   PANRATE     SLOW
   ARCHMODE    ESA/390
   
   000C    3505    kernel.debian parmfile.debian initrd.debian autopad eof
   0120    3390    dasd0
   0A00    3088    CTCI /dev/net/tun 1500 192.168.42.1 192.168.42.2 255.255.255.0
   0A01    3088    CTCI /dev/net/tun 1500 192.168.42.1 192.168.42.2 255.255.255.0
   

6. Create the dasd (disk) image. This will make a 900 mb one: dasdinit dasd0 3390-1 root
7. Start hercules. Type "ipl c" to boot d-i from the emulated card deck. (Old-skool! I'd pay good money to see d-i boot from a REAL card deck ;-)
8. The answers to the first 6 questions about networking are ".1", ".1", ".1", ".2", ".192.168.42.1", ".192.168.42.2". Note the dots in front pass the line on to d-i. Note that you can use ". " if it asks you to press enter.
9. Continue with rest of networking setup and ssh setup. Key generation will take a while.
10. After ssh setup is complete you can ssh to installer@192.168.42.1 and continue the installation in a nicer UI, which from here on is the same as any d-i run.
11. Except for partitioning. You get to use a command line fdisk program, and before that you may encounter a loop at disk initialization (bug #281407).
12. To reboot, type "stop" at the hercules Command prompt, and then "ipl 120" to boot from the dasd.
13. ssh back into the installed system and it'll run base-config for you, rest of the install is same old same old.
14. Now save this image and use it for s390 development work.. and now that you have a s390 development system and know how to use the debian installer, why not contribute some time to making the d-i s390 installer a little more polished..

I need to get an ipaq, just so I can have a s/390 in my pocket.

gettin better

Anna was by with her laptop. During the hurried install of Debian on it after she bought it, I put off setting up some hardware. So, in less than an hour today, I got the camera (including programming a hotplug plugin to automatically download all pics), scanner, and printer all working. Wow.

Now if we could figure out why sound works, but it can't beep. And maybe get it to suspend to disk/ram, although Anna doesn't really care about that.

geocaching bug

Somehow I avoided it until now but I've been bitten by the geocaching bug. I'm suprised at how many caches there are around here, including at least three in the park behind my house. I've ordered a Garmin GPS as my xmas present to myself, and hope to have some fun with folks tracking these down.

It was just barely snowing as I hiked up the hill from the farm today. Seemed to stop as soon as I got out of the valley and into the car though.

gearing up

The 2.6 support is now available on the d-i netinst CDs, and tomorrow's images will even install 2.6 onto the system. Besides working on that today, I checked to make sure that d-i can still install in lowmem mode on systems with 32 mb of memory), tried to get it autobuilding again on the many arches that broke recently (sigh), put together a release issues and check list, worked around partman/cdebconf's issue with indentation in the partition list, etc. Gearing up toward the next release.

Unfortunatly, we have a real situation with discover in testing, that may hold up the release if it's not resolved, and also a file conflict between glibc and initscripts that's breaking installs on ia64 and alpha at the moment. Also, the newest 2.6 kernel in testing is 2.6.3, which does not run well on my hardware. (Not that 2.6 generally runs well on my hardware. Why am I wasting my time with it, again?) It would be interesting if we got the installer releasable before testing was itself releasable, but I hope not.

Reading apocalpyse 12, which, 10% in, seems more obvious and less wild than earlier masterpieces such as the regex one.

fun with debian-cd

Not something I often have, like most everyone who's used it, I generally dislike working with debian-cd. Last night I must have been in a strange frame of mind -- or perhaps really the hard work had been done in setting up my local mirror, getting a good beefy machine to run it on, modifying debian-cd so it could build just the first CD, and getting fully acqcuainted with it. But just a few hours of fiddling, and no more than two coasters and a dozen builds, I finally produced a full size sarge CD that I'm quite happy with.

The official sarge CD #1 has so far been approximatly useless; it's not worked as well as the netinsts, and when you're done you'd get the choice to install a mail server, or a file server. No useful tasks, no desktop, it seemed we couldn't fit anything on there. Last night I realised that all the task information in debian-cd was out of date, so I regenerated it, and made lots of little tweaks. The new CD has every task in it, and does a good job of setting up the desktop, and contains pretty much everything else I'd hope for on the main Debian CD for sarge. This weekend's official build should be the same, and I think we'll finally be able to include a full CD in a d-i release.

firsts

A day of firsts..

First year of blogging -- I started this weblog 1 year ago, though I'd previously used advogato w/o realizing that was really a blog. I should import those entries. I've posted a nice round total of 200 entries in the past year, comprising about 42000 words, and I have to wonder if I shouldn't have tried to write a novel instead..

First tire changing -- Well I've only been driving for 3 years and am not into cars to I've always let the pros do it before, what can I say. Only hard bit is placing the jack.

First car accident. Luckily it only involved going over a curb and blowing the abovementioned tire. I do feel pretty stupid though for doing that. :-(

first day back

So I spent a week at the beach with family. It will shortly blur into the approximatly half a year total I've spent at that same beach-side campground over the past 25 years of my life, and is nothing to write about here. Except that I feel a bit guilty about doing that so soon after coming back from Brazil, and so soon after starting work. Skolelinux has not gotten enough of my time.

But hey, I'm pretty relaxed, and have some better perspectives on some things, and nothing went badly wrong in my absense. So I've been catching up on mail and writing some rather long replies that I'd not have thought to write a week ago.

I may be an incurable sci-fi romantic, but I can't believe the crap I've been hearing on the news instead of any followup on the really significant thing that happened while I was away entertaining the seagulls, mosquitoes, and sand flies -- the SpaceShipOne flight. Pity I missed the coverage of that while the story was still on the radar, I can't wait for the next one.

first cache

Put together my first geocache today. A lot of caches around here have very low value junk in them mostly, and this cache is an attempt to be a bit different and more interesting.

Today was a lot better than yesterday; the kernel update wait is finally over and I can do lots of d-i stuff, and my finger stopped hurting.

finally

Today all the pieces are in place, and I sent out the official announcement of the beginning of security support for testing, with some caveats. Hopefully soon it will not be unwise to use testing in an untrusted environment.

fighting for rhythmbox

Having noticed that rhythmbox is a pretty nice music player, I though I'd try it on my laptop. Unfortunatly, this was a disaster, thanks to Transmeta(TM) Crusoe(TM) Longrun(TM). Apparently gstreamer gets really confused when the cpu suddenly starts running much slower, so even though it only needs 6% or so of my quite adequate TM5800, if longrun jacks the processor speed up a bit, and then back down, gstreamer decides that a 300 mhz cpu is too slow for it and spews staticy garbage out the speaker. Oops.

Pegging the CPU at anywhere between 800 and 867 mhz and not letting it move around makes gstreamer work fine -- at the expense of my battery. The beautiful thing is that this means that running perl -e '1 while 1' makes this media player play better. Although longrun -s 100 100 is a better idea. Possibly running rhythmbox in the full gnome environment would generate enough CPU activity to make this bug much more rare than it is on my lean and mean ion machine.

This is not an unheard of problem with transmeta CPUs (does it affect others?), but it's never caused media player problems for me before, only affecting some games that calculate timing loops at start, before the CPU has ramped up to full speed, or so on. I don't know if gstreamer breaks because it for some reason needs 3x the cpu as xmms to play the same mp3, or because it has some broken buffering.

Filing a bug on this kind of thing is hard and often yeilds sympathy but little hope of a real fix. Happily, rhythmbox supports a second audio backend, libxine. I've never had any sound problems with xine, and it didn't let me down here either -- and it uses 1/3 the CPU too. Hopefully a version of rhythmbox supporting use of xine can be added to Debian eventually.

Anyway, that's how I spent another several insomniac hours..

fighting computer-induced ADD

I'm looking for tips to fight computer-induced ADD. When I am not online, I rarely have this problem, and when I'm not using a computer, I never do. But take right now, when I am:

• copying a new d-i build to keychain for testing
• listening to music
• reading a page about the mars rovers
• helping two different guys on irc
• chatting on a different irc channel
• downloading something (I forget what)
• reading and responding to email
• blogging

By the time I finished writing this list, the first item had changed to "testing the d-i" and "debugging a problem in hw-detect". Earlier today, I began to boot up d-i and could not remember what I was supposed to be testing this time around, since four new things had intervened in my attention since I started the build. And I know that I do my best work when I'm completly focused down on one problem.

It's insanity, living life in little slices like this. Please help!

discussion

fiction

Some of the fiction I've been enjoying lately:

The Golden Compass by Philip Pullman. Another of those books that's esily (mis)categorised as "young adult", while really being worthwhile to readers of many ages. I keep wanting to compare this to the Harrey Potter books, but I won't, because that would devalue it. Highly recommended. (This means to you, Anna.) I started to dive right into the sequel but decided to save it for a rainy day when I could really enjoy it.

Very Bad Deaths by Spider Robinson. Not my favorite work, it all seemed fairly predictable to someone who's read every single book Spider wrote before this. Of course there's a reason I keep going back and reading everything of his I can get my hands on, so I'm not really complaining. By an interesting cooincidence, it's set in Vancouver, which I visited the next week.

Trailer Park Boys is a Canadian tv show that was mentioned to me by either Luca or neuro, I forget which. Everything mindless entertainment with cliched characters can ever hope to be. Also disterbingly realistic and a ton of fun.

Coalescent by Stephen Baxter. Fairly interesting social SF with a nice broad historical scope. Pretty flawed though.

Little Faces, a short story by Vonda McIntyre. It's been too long since I've had the pleasure of reading something new by Vonda McIntyre and this was quite an interesting one especially due to its lack of any exposition. Enjoyed working things out; this three-species symbiosis is the kind of thing McIntyre writes about very convincingly.

Junk that's not worth mentioning by John Ringo. Annoyed I wasted my time.

The Anita Blake series by Laurell Hamilton. I would be much happier with this if the nonstop action slowed down for more than a page at a time. Still, some interesting takes on some overused bits of its genre. Apparently this series goes quite downhill later on so I'm planning my exit now..

Nausicaa of the Valley of the Wind directed by Hayao Miyazaki. I think this rounds out my collection of Hayao Miyazaki's work. While pretty early and reflecting a lot of things he did better later, it's still a moving tale and I couldn't help but like it.

Goodness, that's less than a month's worth, I wonder how I get any work done.

.fi

Waking up at 3 am and hiking around for several hours waiting on anyone else to wake up, I got to see an hour or so of very slow sunrise over the bay here at the university in Finland. Then the sun began moving faster. I love long sunrises and sets and that was a very nice morning.

I spent some hours today getting Debian installed on ~30 loaner laptops for the conference. Would have gone faster if we'd had more than 6 power cables and network cables, since two or three people is too many to babysit automated installs. Anyway, d-i worked pretty well.

Hmm, I forgot and left the mail server turned off all last night. Sorry to those whose email was delayed.

feh

After all of the serial console work described above, I don't even need the serial interface, now that I've used it to configure the IP address of the network console.

Automating the debian install proceeds..

family planet

Enough of my family are keeping their own weblogs that I set up a "planet" aggregator for them.

I didn't really use the planet software for this, because it doesn't seem to be packaged for Debian, and because rawdog is, and seems well documented and easy to setup and customize, and has a nice backronym. It would also be a nice way to generate my own blog, if I had a good rss editor to use to construct raw rss files. Hmm.

fall color

The leaves have not been very colorful at home this year, tending toward the browns, so it was very plesant to see all the bright colors driving down off the plateau and around Mount Airy. Really spectacular.

endorphins

Another very full weekend in what's shaping up to be quite a series of get-out-and-do-something weekends. At first the plan was to get to play with a real live bulldozer and help Anna put in a ford in the creek at her new place, but when the bulldozer sadly didn't show I ended up going back to Anna's and spending a nice evening there. Then, back home and thinking I was done this afternoon, I found myself up helping Ken and Janese unload a huge truckload of appliances at their new house up the hill and then had a nice after party on the deck.

And I still have a bulldozer to look forward to next weekend..

ending the tyranny of unix permissions

In Debian, we work on developing a unix-style operating system, and when the project was getting started, the obvious way to deal with delegating powers to various people was naturaly to use standard unix permmissions on our master server. So the list administrators were the only people who could write to the list configs and archives, the BTS admins were the only ones who could write to the BTS, the archive admins were the only ones who could manipulate the archive, the build daemon admins were the only ones who could modify various bits of that stuff, etc.

As time went on, these services were split off to seprate machines and in some cases access to the machine running a service was limited to the maintainers of that service, again in a way that would be familiar to unix admins everywhere. But as time went on and things like revision control systems became available, Debian also began to make some choices that didn't involve unix permission or host based security.

The best example of this is the Debian bug tracking system, which allows anyone to manipulate any bug status in any way without any authentication at all. It works because the BTS keeps a record of all changes, and informs interested parties about changes, so vandalism or accidents can be reverted. Aside from spam, vandalism in the BTS is such a rare problem that I cannot think of an example. Thus the Debin BTS partakes somewhat of the WikiNature, although it predates the wiki by a long time.

The archive itself only uses unix permissions at its core, while the upper layers do not partake of them: Any maintainer can upload any package and superscede data in the archive automatically, although actual unix write permissions are very limited. The archive administrators use archives of old versions of packages, automated checks, and forced manual review of certian classes of packages (such as NEW) to limit this so that Developers don't make messes. Social norms are used to avoid other problems.

The Debian website is in an interesting period in its history. It's been maintained in a way that mostly avoids unix permissions for a long time, by keeping it in CVS and building it automatically. However, for reasons that I suspect have never been adequately explored, not all Debian developers can commit, you have to be in the webwml group. Though gaining access to the webwml groupo is trivial for any Debian developer. But the most interesting thing is that Debian now also has an official wiki, which is open to editing by all, and that increasingly it is used for at least the more fluid and development-based website stuff. For example, the debian-installer home page is a thin stub giving the essential information and links, and mostly linking to the wiki for the rest.

The debian-installer project itself occupys another point on the scale between locked down unix permissions and free modification by all. People have to get an alioth account and demonstrate basic competance to get debian-installer write access, but don't need to be Debian developers or spend more than an hour or so to get access. Once you have access, you can change anything in svn (and everything is in svn) we don't limit write access to parts of the project. We avoid malicious commits through patch review.

I could give many more examples of subsystems in Debian that exist at different point in the spectrum between locked down unix permissions and a wiki. There seems to be a definite pull toward moving away from unix permissions, once ways can be found to do so that are secure or that allow bad changes to be reverted (and blame properly assigned). Cases of moving in the other direction are rare (one case of this is the further locking down of the Debian archive server and BTS server after the server compromise last year).

Anyway, the point of this is that, if you survey the parts of dealing with the project where Debian developers feel most helpless and unempowered, the parts that are over and over again the subject of heated discussions and complaints, you will find that those are the parts of the project where unix permissions still hold sway. This can range from simple cases such as a cron job that only one person can look at and modify[1], to various data files that could perhaps be kept in svn, but aren't, all the way through to stuff like the Debian keyring. I would love to see a full list developed, although many of the things that remain are obscure little corners like certian blacklists in the BTS, bits of the buildd infrastructure that only a half dozen people know about, etc.

The challenge, then is to find ways to open things up to everyone, without throwing security out the window. It has to be done a piece at time, and some of the pieces that are left are the ones most resistant to this change. I hope that people can keep this progression away from unix permissions in mind while working on Debian infrastructure.

(If I ever ran for Debian Project Leader (which I won't), this would be one of the areas that I would be sure to investigate basing my platform on. Speaking of which, it's nearly election season again, isn't it?)

[1] The best solution I have found to this common problem so far is simply to keep the cron job in svn with everything else, and if someone modifies it, review it and update it via semi-automated means (some simple command). There are probably better ways to do it.

egoboo

Joey Hess sucks -- what a fun thing to find on google.

(Update: If the link's still broken, use the Internet Wayback Machine).

In other news, I have successfully put all my crontabs under revision control, in my subversion repository. I would post a link the the 15 minute hack that did this, but, you see, I wrote a 15 minute hack called "debmirror" 4 years ago, and got flamed for it mercileslly last month, because it was not sufficiently commented...

Oh what the hey: svn://kitenet.net/joey/trunk/home-full/.cron + svn://kitenet.net/joey/trunk/bin/loadcron

drowing alive in bugs

Time for another pass through my recent bug reports, to fix those that have fallen through the cracks. This resulted in a fairly large set of changes for debconf (~ 3000 lines), and several fixes in base-config and debhelper. Plus quite a lot of bug triage on the debian-installer pseudo-pseudo-package.

Now there's only 220 d-i bugs, 163 bugs in my packages, and 185 installation reports to go. Heh.

Possible new d-i boot logo:
This is one of several that we're considering; imho it works better with syslinux than any of the others so far. volkany got this too me less than 24 hours after I sent him an unsolicited plea for a logo.

This morning I tried half-heartedly to implement my new configuration scheme for d-i, but found out quickly that it would require much more work than expected in main-menu, and gave up on it. Problem is that main-menu expects the menu items to be static.

drinking the 2.6 kool-aide

There is a certian attitude some people I know have about the 2.4 kernel that I find really annoying. It's basically that it's obsolete, unsupported, not worth bothering with, should be dropped immediately, etc. And that's a reasonable position until you look at the sheer mass of machines out there that have not been able to upgrade to 2.6 yet, and the many places where all the new stuff (initramfs generators come to mind and so might udev) surrounding 2.6 is simply not ready for users. And the total lack of care that many of these same people have about providing any kind of reasonable upgrade path for 2.6.

Well, after trying each new 2.6 kernel on my fujitsu laptop as they came out over the years from 2.6.2 on, I've finally found one that seems usable on it as a replacement for 2.4. It's just passed the main smoke test of being able to suspend and resume while playing music and downloading a file without missing a beat (using apm; the apm vs acpi thing is much like the 2.4 vs 2.6 thing I mentioned above, except acpi is unlikly to work for anyone). I'm not sure it's the kernel itself that started working; my main problem before was disk access hanging on resume, and I've recently replacd my hard drive, and perhaps 2.6 only had problem with my old model of drive.

So anyway, it works for me, therefore it is perfect, so I can now without any qualms enter the ranks of the 2.4 snobs.

DPL election

For several weeks I've felt that this will be the most interesting DPL election ever. Now that I've read the platforms I'm still confident of that. I'm glad that we have several people running whom I can happily vote for.

FWIW, I was approached to be part of "team skud", but I am not confident enough of my leadership abilities (hah!) or my personal situation in the next year to take that on.

DPL 2

<tbm> got a minute. I'm wondering about your latest blog
<joeyh> the one I committed 2 minutes ago or the pervious one?
<tbm> the dpl one
<joeyh> ok, sure
<tbm> what does it mean to focus on technical things? I focused on organizational things, but mostly in a way so that other people can do their technical work without having to worry.
<joeyh> I basically think that part of the whole dpl-as-figurehead problem is that we have stopped trusting our DPLs to lead the project toard large technical changes to Debian (software)
<tbm> that's right, but I think the majority doesn't want that anyway. Like Ted T'so, who said he wanted a "strong leader" but when I asked him to lead the kernel team he said he'd prefer to maintain a kernel in non-free than a crippled one in main.
<tbm> funny asking for a strong leader but then ignorning the leader
<tbm> but yeah, it seems we disagree about the role of the dpl; I see it more as organizational/secretarial so other people can do their technical work
<joeyh> right, which is why I'm focusing on it to the exclusion of all the other stuff the DPL should do, to try to counterbalance that majority as best I can
<tbm> like getting you a laptop for d-i for example
<tbm> okay
<joeyh> look at Shuttleworth. He's provided strong technical directions to Ubunutu. Occasionally incorrect, arguably often iffy, but it's done a lot for them
<tbm> oh, I see. Right. What I think Debian needs is someone to say which way to go when one of our discussions/flamewars don't end in consensus, which they recently never do
<tbm> but somehow I doubt most DDs would care what the dpl said anyway.
<joeyh> we need a DPL to make decisions like "Debian will ship a Gnome desktop by defualt in the sarge release", and who can get the whole project behind that. Otherwise it leaves people like me having to work around the projet's inability to make the decision under the radar
<tbm> they got so used to "quiet" dpls
<tbm> yeah, I agree with you on this.
<tbm> okay, thanks for clarifying
<joeyh> do you mind if I post this on my blog?
<tbm> no

DPL..

Since I find the DPL position increasingly uninteresting except as a position of technical leadership, I will probably use the following simple metric (which I think of as the "degrees from Bruce Perens" metric) to rank my choices for Debian Project Leader in this year's elections:

1. Order the candidates based on the perceived solidity of actual technical changes they propose to make to Debian.
2. Rank "further discussion" above the first candidate whom I could not bear to see as DPL.

I won't bother listing the result, it's pretty predictable. But I will here excerpt all the at least vaguely technical content from the candidate's platforms:

• Jeroen

  "Debian needs to focus on technical excellence with free software"

  "I will encourage more use of the official wiki"

• Ari / Zeke

  "I propose that all software in Debian be relicensed"

  "Also, we should remove all pictures taken by cameras with non-free firmware from the Debian Project."

  "I'd like to introduce a new Debian GNU/Plan 9 port"

• Steve

  (Steve wrote mikmod. Amazed I never connected those two dots before.)

  "more documentation on best-practice packaging and testing methods"

  "considering the impact of changes on the release schedule"

• Aj

  "The most important thing that I think would benefit Debian is increasing its tempo."

  "And sometimes doing it fast helps you to do it right"

  "getting updates accepted into the archive more frequently than once a day"

  "having frequent beta releases of etch/testing that we can legitimately call a release (benefiting from the ongoing work of the installer and testing-security teams)"

  "having reliably quick resolution of RC bugs in unstable"

  "automated testing efforts"

• Andreas

  "having more frequent and regular releases."

  "We have a good, realistic timetable for the release of Etch, and this should be made a priority,"

  "Becoming More Purpose-Driven"

• Johnathan

  "When projects like Ubuntu and Progeny come along and pay people to do the boring parts of the work, we should be ecstatic."

  "a life-size statue on the grounds of Rensselaer Polytechnic Institute"

• Bill

  "Be strict in what you send, liberal in what you accept."

  "One of the problem of Debian is that too few developers consider the distribution globally rather than a collection of packages."

dpatch/dbs/etc/etc/etc/etc considered harmful

Followup to last post, transcript of #debian-boot.

<Caesar> joeyh: Edumacate me on the evils of dpatch
* Kamion drops all patch systems down a deep well, there to stay rotting forever with any luck
<Caesar> How come?
<Caesar> I honestly though dpatch was pretty neat, but I'm obviously not seeing part of the picture
<Kamion> horrible obfuscation when you're just trying to unpack the damn source package
<Kamion> $ debian/rules patch
<Kamion> $ debian/rules patched-source
<Kamion> $ debian/rules apply-patches
<Kamion> $ debian/rules unpack
<Kamion> $ debian/rules extract
<Kamion> $ debian/rules source.make
<vorlon> Caesar: take a look at a dpatch package using debdiff sometime
<Kamion> $ make -f debian/sys-build.mk source.make
<Kamion> # aaaaaaaaaaaaaaargh
<vorlon> Kamion: where's debian/rules setup? :)
<Kamion> vorlon: knew I'd missed at least one
<Kamion> all the above are variants I've seen, with the possible exception that I might have misspelled the last one
<Caesar> So you anti-dpatchers prefer to see all the changes just rolled up in the diff.gz?
<Kamion> yep
<Kamion> use a revision control system
<joeyh> huh, my techniques is debian/rules build && ctrl-c :-P
<Caesar> Fair enough.
<Kamion> joeyh: that's my last resort
<Caesar> I need to get onto this whole revision control system bandwagon...
<joeyh> but that's only the fun of unpacking, you then have the fun of trying to patch a broken patch to put in a securityu fix or so
<joeyh> so then you get to edit patches or patch patches or some such insanity
<Kamion> bonus points for putting directions to your revision control system into debian/README.build or something so people can inspect it
<Kamion> pitti used to put patches in debian/patches/ that patched files in the debian/ directory, until we re-educated him with a stick
<vorlon> haha
<Kamion> if there were just one patch system, then it would be OK to learn its toolset for editing patches (there's usually something that lets you work on a patched tree rather than on the patch file)
<Kamion> but no, there are about seventeen patch systems with lots of people's pet variations
<Caesar> That's what I was liking about dpatch
<Caesar> It had a semi-decent toolset
<Kamion> it's all very well for people using dpatch for all their packages, but for people who have to edit packages all over the place, they don't have the luxury of just one system
* Caesar feels dirty for having made numerous QA uploads of orphaned
* packages and added dpatch support
<joeyh> well, that's basically a "please don't adopt this" upload
<joeyh> or a "here, waste an hour if you do adopt this" at least
<Caesar> Cringe
<Caesar> Here I was thinking it made the patch more atomic...
<Caesar> (which was my reasoning behind using dpatch)
<Caesar> i.e. made it easy to throw away individual patches if they weren't required any more
<Kamion> the approach used by the perl package is somewhat better
* Caesar goes and looks
<Kamion> it stores patches applied to the source package in debian/patches/, but ships the .diff.gz with them all applied
<Kamion> you get duplication, but you also get to unpack the source with the standard tools and see what gets built, and in a worst case as an NMUer you can just change the source tree directly and let the maintainer sort it out later
<Caesar> I guess NMUing is a more of a big deal in Ubuntu because you don't have specific maintainers for specific packages
<Kamion> well, (a) that's sort of sometimes true, (b) our beloved universe maintainers all seem to be dpatch/similar fanatics so ...
<Kamion> some battles I have given up fighting
<joeyh> pity that arch thing didn't work out

hth

downtime

So I got home from Europe, didn't feel too tired, went to bed around midnight. Woke up early morning to let the cat out. Woke up late evening to that ugly feeling that I didn't know if it was am or pm. Then up until 7 am, now I seem to be back to "normal".

don't forget the MOOs

(This is a copy of my response to the LWN article Toward a free metaverse, which won't be freely available itself for another week or so.)

There's an unconcious bias in this article toward 3d graphical worlds, and I think it's important to realise that that's not a necessary component of a virtual world, although it's certianly a component that will appeal to a lot of people and that fits in nicely with some current ways of working with computers -- ie, sitting in front of expensive screens weilding a mouse. Long ago we had MOOs, MUDs, etc, and they were text based. Just words, you know. And nearly all of them were free software. It might well be that text, or speech, or the like ends up being a better interface to virtual worlds of the future, especially if they are tightly integrated and layered on top of the real world.

And I'm increasingly convinced that if these worlds don't integrate into the real world, if the ideas that have been bubbling up out of ADVENTURE and the MUCKs and the MMORPGs and such don't spread out and become something larger and more inclusive than their own little pretend world, then in the end that whole area has been nothing but pointless games. Which I hope it hasn't, but perhaps I'm getting old..

Speaking of freedom, there are levels and layers, and it's possible to build the free on top of the non-free. After all, I'm typing this on a device that is present in the er, First Life, which is not particularly free in its implementation; it has non-free firmware and BIOS, and is composed of atoms of some substances that are less easily available than air, and which follow some annoyingly inflexible physical laws, and it needs a constant input of metered electricity to run. But on top of all that is some free code that gives me a measure of freedom.

Similarly, it would be possible to run free code in Second Life, and given how rare it is overall for such services to support coding at all, let alone explcitly give a program (or object's) creator the freedom to make it Free, that Second Life does so seems encouraging to me.

If we get a distributed standardized free online world by 2010, I'll be happy indeed.

(Disclaimer: I write text based MOOS, and have never used Second Life, though I chatted with someone there on the phone today.)

disconnected followup

Hey Matthew, I wrote /usr/bin/bts because I use my laptop excactly like you do (except s/liferea/rss2email/; s/baz/some svk/).

Oddly though, I still haven't gotten my blogging system to let me queue up blog posts while offline. It's designed to theoretically allow it, but I never seem to find the time to make it work.

So the real point of this post, besides making sure who know who should collect on your kind offer, is to have something to test as I frob my scripts until I can push it out to my blog after I turn the wifi back on.

devscripts

I'm now a member of the development team for devscripts. I realized that Julian could use some help and was pleasently suprised a) that there was a development team already and b) that I was quickly added to it. Being able to commit to a package is such a nice motivation for doing work, and so I've spent a fun bit of time working on different parts of it. I even had to learn about perl formats for one bit, although I'd never intended to venture into that dark musty corner of the language.

I've grown quite a pile of my own persoanal scripts for debian development, and one thing I hope to do is integrate them into devscripts where I can, without adding any non-generally useful stuff. The first two programs I've added, which were already in a queue to be added before, are svnpath and debcommit.

svnpath rather deserves a blog entry of its own, and I still nurture hopes of getting the basic idea into subversion one day. It's a very simple program in the Unix Tools philosophy that makes a lot of things about using svn much easier to do, with much less typing.

debcommit supports the "modify debian/changelog then commit" form of development. It saves having to document a change twice by recycling items from debian/changelog as commit messages. There's an alternative way to do it, "commit changes and generate debian/changelog at release time" which I tried to make work, but which turned out to not work very well for me. Apparently not for many other people either, since all the co-maintained packages I'm involved in, even periphilly (d-i, debian kernel, debian-cd, devscripts), use the former method. So I hope debcommit will be useful for a lot of people. Now I can finally stop giving out svn uris to people who need it and they can get a maintained version in devscripts. (Oh yeah, did I mention it supports both cvs and svn, and can tag releases too?)

Speaking of changelogs, over the past several years as team maintenance of packages in Debian has become more popular, we've had to deal with a new problem: How to represent changes by multiple package maintainers in a single Debian changelog entry. Also, how to indicate whether or not a given changelog entry is done, or if it's still unreleased and accepting new changes. Since the Debian changelog format was not designed with this in mind there are, unfortunatly, almost as many ways to do it as there are co-maintained packages. This is a problem if one wants to write a program to automatically process and/or generate them.

It almost seems silly to devote as much space to this as I'm about to, but here goes ...

The d-i Format is the one we've used in d-i for several years:

package (version) UNRELEASED; urgency=low

  * Joey Hess
    - something
    - something else
  * Adam Di Carlo
    - yet another change
  * Updated translatons:
    - Arabic (ar.po) by Ossama M. Khayat
    - Czech (cs.po) by Miroslav Kure
    - German (de.po) by Dennis Stampfer

-- Colin Watson <cjwatson@debian.org>  Thu,  5 May 2005 16:22:49 +0100

The UNRELEASED is the best method we've found to mark it as not yet released. I used to use "(UNRE)" in the version field but was convinced otherwise by the team. The "updated translations" bit is a compromise, rather than listing each translation in its own author block, we save space and make them easily ignorable by grouping them together. This part is automatically generated in d-i, and we tend to have a lot of translation updates. The problems with this format are that it's not easily machine differentiable from single-maintainer formats, and that the double indenting wastes space and even breaks some naive formatters. Also it doesn't work with dch. And if only one person ends up making a change, you have to reformat a lot of stuff to go back to a traditional changelog format.

The Parens Format (not to be confused with the Perens format, which you will see at the end of a few very old changelogs such as analog's) is one I've seen here and there:

package (version) unstable; urgency=low

  * A change (Closes: #nnnnn) (Simon Horman)
  * Another change (dann frazier)

 -- 

Leaving off the release line entirely until release is a nice idea, but it makes dpkg-parsechangelog fail and so the package cannot be built this way. The maintainer names in parens are ok but stack up annoyingly with the bug closing, and are not machine parseable, and if only one maintainer ends up making any changes in a release, you have to remove all the redundant maintainer names.

My Attempt at Changing the Actual Debian Changelog Format to really support multi-maintainer changelogs is this:

package (version) UNRELEASED; urgency=low

  * something else
  * something

-- Joey Hess <joeyh@debian.org>  Sat,  2 Jul 2005 18:28:55 -0400

  * yet another change

-- Colin Watson <cjwatson@debian.org>  Thu,  5 May 2005 16:22:49 +0100

This is not a valid format, but it could be supported. The problems with it are that there's no way to indicate that a maintainer made no actual changes, but did the release, which all the other formats allow; and that it rather encourages new changes be added to the top, instead of the more usual bottom.

Now, we could develop an entirely new format that has all desired properties. Perhaps someone would find the XML Changelog Format nice;

<!some grotty xml dtd here>
<package name="foo">
    <version="1.0-1">
        <change by="&joeyh;">
        was changing something but forgot what while writing this entry
        </change>

        <change by="&other;">
        some other change
        </change>

        <release date="Sat,  2 Jul 2005 18:28:55 -0400" by="&joeyh;" />
    </version>
<... />
<package>

dpkg-genchanges has pluggable changelog parsing backends, but it would break a lot of other tools. Oh and it sucks. If you like it, try Gentoo's.

The Dash Format, or something like this, was spotted the other day:

package (version) UNRELEASED; urgency=low

  -- Maintainer Name

  * something
  * something else

  -- Other Maintaer

  * yet another change

-- Maintainer Name <maint@debian.org>  Thu,  5 May 2005 16:22:49 +0100

This has basically the same plusses and minuses and the next one. And it is nice the way the sig-like "-- " is repeated, but somehow I don't like it as much as this next one. Maybe because it looks odd to put the sig first, or because of the extra newlines.

The Braced Maintainer format is one that I've only seen recently:

package (version) UNRELEASED; urgency=low

  [ Joey Hess ]
  * something
  * something else

  [ Adam Di Carlo ]
  * yet another change

-- Colin Watson <cjwatson@debian.org>  Thu,  5 May 2005 16:22:49 +0100

I like this one because it uses normal indentation and the maintainer names are machine parseable while still being easily readable. It's the best of the lot. I hope that Debian begins to settle on this, or a format with similar benefits.

So I did some work on making dch support Braced Maintainer format better. It now recognises it and will automatically add the maintainer names to disambiguate changes made by multiple maintainers, so a traditional changelog will morph into a multi-maintainer one as required. I also added an option to make it add and understand the UNRELEASED distribution, so dch can decide whether to append or start a new entry. These changes should pay back the typing it took to write them, and this blog entry, in less than a year's use. ;-)

Hmm, the problem with hacking on either devscripts or debhelper is that after a while it seems that you're making changes which only have the result of making it easier to hack on devscripts or debhelper. Which means it's time to stop.

demos

So while I've been swapping CDs and running through Debian installs on a variety of (slightly dated) hardware tonight I've been staying entertained by watching side two of Mind Candy, a DVD of video captures of old PC demos. Rather a nice idea to use video capture, since many of these demos can't be run anymore on modern machines.

It's hard to do the DVD justice in a single blog entry but here's my personal take on it. As a non-European who wasn't too in touch back in the early 90's, I missed out on the whole demo scene thing, but did see one or two. The only one I really remember is, predictably, Second Reality, which at the time was very impressive and I remember watching it several times with friends. Looking back on it now, I have to admit it seems less impressive. Instead I did quite enjoy another slightly later demo on this DVD, 303 by Acme.

But at this point most of the stuff in old demos like these is nothing one hasn't seen before, and I can only take so much electronic music at once, so I was glad to switch over to the commentary track. I really enjoyed that because of all the interesting historical notes and some very intense commentary by some of the creators of the demos.

The best thing I've taken away from this is just a general impression of what it was like back then for a bunch of folks in a group, probably in Europe, to get together somewhere for a couple of days and stay up all night working together on a project together, and hopefully end the meeting with something special. Which when you get right down to it, sounds kinda familiar..

Debian-Installer beta 4 released for 9 architectures

The Debian-Installer team announces the fourth beta release of the Debian sarge installer. Notable improvements in this release include:

• Support for the arm, hppa, and mipsel architectures, bringing the number of supported architectures to 9. (The mips architecture is not yet ready for this beta, it will be released in a few days.)
• Experimental support for the 2.6 kernel, on i386 only. The 2.4 kernel remains the default and recommended kernel for most hardware.
• Detection of existing operating systems. The following operating systems can be detected and will be added to the boot menu of the installed system: Windows, Mac OS, Linux, GNU Hurd, DOS
• Translated into 35 languages, including new translations to Indonesian, Basque, Galician, Romanian and Welsh.
• Many bug fixes and user interface improvements.
• Fixes for most of the listed errata for beta 3 of the installer.
• A new boot logo by Alessandro Polverini and Andrea Mottola.

We're confident that this is our best release ever, so give it a try, install Debian today, and don't forget to file an installation report so we can continue to make the installer even better. Links to bootable images and documentation are on our web site.

(Whew!)

Debian Weekly News

It seems that Joey has gotten into the habit of using a post from my blog to pad out Debian Weekly News if he needs some filler, and if I managed to write something interesting that week. This probably means that I should try to redirect some of that energy to participating in DWN again, but I probably won't, since the impulse to scribble things down here is something that just comes to me when I have something specific I need to get out of my head.

Working on DWN took a much different kind of energy, and I'm immensly greatful for Joey for taking it over, (probably confusing a lot of readers who may not have noticed that the second half of the editor's name changed), and putting out what I'm astonished to see is now 139 issues of DWN (I stopped after 110). Joey, if you ever get overloaded with Debian beyond your immense capacity (touch wood), I hope you drop DWN first because your other work on Debian is critical, and I hope you are lucky to find a good a replacement as I did.

Debian on the linksys wrt54gs

I installed OpenWRT on the linksys access point that I picked out especially so I could run linux on it (worth the extra $20). OpenWRT is a nice, small system, and it's already made the access point a lot nicer.

But I thought I'd go one step further and put Debian on it. Of course even the "s" model has only a little flash memory, not enough to store Debian. So I installed and set up nfs client stuff on the access point, and a server on my laptop.

root@fly:~# df
Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/root                 7296      4224      3072  58% /
dragon:/srv/nfs/fly    4806904   4207520    355200  92% /debian

Getting Debian debootstrapped was tricky, since debootstrap uses some POSIX shell things (like printf) that are not in OpenWRT, and also has a binary that it needs to run, which wouldn't, since OpenWRT uses uclibc. That also ruled out cdebootstrap, and in the end I tarred up a mipsel chroot from my cobalt raq and used that. Which works nicely:

root@fly:/debian# chroot . su - joey
joey@fly:~>uname -a
Linux fly 2.4.30 #1 Sat Apr 23 18:13:56 EDT 2005 mips GNU/Linux

I'm not sure where to go from here. Ideally I want to be able to run arbitrary programs from Debian on the box even when my laptop is not around to provide the NFS root. Whether to accomplish that with filesystem trickery, by getting rid of OpenWRT and using flashybrid, or just getting an environment on the Debian side that can build uclibc binaries, or what, I don't know.

In the short term, I should probably start by building a working .ipkg of cdebootstrap. Until then, if anyone wants a small Debian sarge mipsel tarball that can work on this box, I have one handy.

Debian founder not allowed on Planet Debian?

Ian Murdock wonders why he was removed from Planet Debian. Since I've appreciated seeing him blog, even though I don't entirely grok what Progeny is trying to do with their componentised linux, I had a look at the cvs log:

revision 1.19
date: 2005/04/29 22:47:33;  author: keybuk;  state: Exp;  lines: +32 -2
remove imurdock, from many complaints

And this in the config:

# Removed, 2005-04-30
# -- please provide a feed without Progeny advertising content
#[http://ianmurdock.com/index-full.xml]
#name = Ian Murdock

It's weird that Ian apparently did not get an email about this before being summarily yanked from the aggregator. IIRC he does not have access to read the above file, so I hope this helps him figure out how to get back on.

I also hope that talking about one's work in a given derived Debian distribution for one post out of 15 is not so annyoing to the Planet Debian readers that it is really reason to be removed from the aggregator. After all, Ian is not the only one to talk about work on derived Debian distros. And I do read Ian's posts as discussing his work; he's just working on stuff at Progeny on a high and a broad and also a marketing level.

(Oh, and BTW, Ian -- you're soo wrong about daylight savings time. That's the one thing that ever made me want to live in Indiana. Besides, zoneinfo in sarge will be broken..)

debhelper/cdbs/dpkg/yada/yada

In response to Colin/Gergely:

If dpkg had had something cdbs-ish or debhelper-ish built into it from the beginning, we would have neither cdbs or debhelper, but probably a poor imitation, like the crap that is in rpm. The world would be a poorer place.

Of course there might be a few less grossly bloated debian/rules files out there, but I'd not count on it. And we'd not have had debstd or yada, so it would not be entirely a bad thing.

Colin's comments about how interconnected and, dare I say, fragile cdbs is are not reassuring. I remember similar problems with the system I wrote before debhelper, which used one common debian/rules and a short debian/config for all my packages. That's one of the reasons I ditched it for debhelper. Hope you have better luck with cdbs.

debhelper / autoconf grudge match

Two things recently have made me think about using debhelper to generate a debian/rules that does not need debhelper to build a package.

First, I've been following Helen Faulkner's account of her progress through the new maintainer process, where she was asked to debianize a package without debhelper. I understand the reason to ask a new maintainer that.

But I put myself in the new maintainer's place (suppose I was going through NM again), and I wonder if I would bother to do it by hand, or might instead just run the build once with debhelper in verbose mode and paste the commands it decided to run into the rules file. This could almost be automated, and would almost work. It would be very easy to detect, but I suspect at least the first new NMs to try it to get past that requirement might get by on points for out of the box thinking.

Secondly, Keybuck blogged about wanting to use debhelper in dpkg, but not wanting to add more build-dependencies.

It struck me today that one possible elegant solution to this problem would be to exactly what we already do for Autoconf, Automake and Libtool.

Now before there was debhelper, I used a system close to that for maintaining some (30 or so) packages. I quickly found that updating all my packages each time I made some change to the system was too much work, and I so I threw it away and wrote debhelper. I was happy to find that I could then maintain 90 or so packages instead of the previous 30.

So over the years I've compared doing it the debhelper way, where you build-depend on many small programs with a well-defined, static interface (and a way to change the interface via debhelper compatability levels), with the autoconf way where you essentially compile a build system rarely, but have to do significant work sometimes to update to a new version.

It's hard to compare them exactly because the domains of debhelper and autoconf are so different, but there are similarities like the tools needing to deal with requirements that change outside the domain of the package. For debhelper, think Debian policy changes, and for autoconf think new operating systems or architectures.

And I think we see the same sorts of problems with autoconf and automake in Debian as I saw when I was maintaining my rules file in a similar way. We often have to modify packages just to get a new autoconf feature, such as proper support for an architecture. We sometimes find packages whose existing automakage has fallen out of date and need to spend a lot of effort on making it work with a current version of automake. Oh yeah: and we get nasty enormous diff.gz's sometimes. Debhelper avoids these problems.

My conclusion: Autoconf has some very good reasons to work the way it does, but most of these reasons do not apply to modern unix systems, which already include recent version(s) of autoconf. On these systems the extra build dependency of doing things the debhelper way does not matter, because autoconf is easily available.

That is the main reason why I sometimes use autoconf like debhelper, calling it everytime as part of the package build process. Unfortunatly unlike debhelper it does not have a very well-defined or unchanging interface (and automake is even worse), so sometimes this introduces more work than using autoconf the old fasioned way. At this point it's anyone's guess which use of autoconf in debian results in less work in the long run, and reasonable people (including, I think, Keybuck) think I'm bonkers to use it this way.

Anyway, the all of the above is why I myself have little interest in providing a way for debhelper to be "compiled" into the debian/rules, and why I have even less interest in supporting the general mess that would result if such a tool existed and were used widely in debian. But at the same time I think it's a kind of fun idea and if someone wants to implement it, and use it in their NM application or in the packaging of dpkg, and does a good job while managing to discourage other people from using it unnecessarily, I'd probably accept the patch.

Escpecially since most of the obvious problems I can think of that you'll encounter while doing so are real bugs in debhelper that need to be fixed anyway.

debhelper

The new debhelper udeb support is reminding me of the early days of debhelper, when rules file after rules file was converted to something much simpler, and the experience of doing so feed back into debhelper development to make it better an better. I have so far only converted a half a dozen, but the urge is there to change over the whole d-i tree.

One thing that I have realized as a result of this is that I've perhaps gotten too conservative about what I let into debhelper. I rejected the original request for the udeb support; I often let new utilities sit in the BTS for up to years, waiting for some small problem in their design to be resolved (dh_installinfo), or just because I am reluctant to let them in. Partly that's a good thing, since I am still dealing with some mistakes made in the early days (dh_md5sums, dh_du, dh_installmanpages). Partly it's a bad thing because this has resulted in quite a lot of dh_* programs being relegated to packages outside of debhelper (dh_installxmlcatalogs).

Partly this is a result of debhelper being a mature project now. There is a lot of pressure not to break backwards compatability, and not to break stuff (too badly/often). So it's not suprising to see a certian shift of interest/development to younger projects, like cdbs.

Anyhow, it's a tightrope to walk, and I've made some good decisions too. The debian/compat stuff allows me to retain backwards compatability while still exploring better incompatable behaviors. Outsourcing most of the maintenance of parts of debhelper (dh_python, dh_perl) to those who are best equipped to maintain them, while keeping them in the package has also helped.

Going forward I will try to be more open to new ideas and new utilities in debhelper, and I have a glimmer of a possibly correct way to handle the long-promised OO rewrite, if I can ever find the time for it.

debconf5

I find myself unable to write any kind of wrapup or overview of my time in Finland for DebConf5. It was awesome, it went on so long that I became used to the weirdness of having everyone right there in person, and living on campus again, just like I became used to the midnight sun.

And that's really all I can say, so I'll just end with a few teriffic resources from the conference:

• annotated group photo -- finally I can figure out who everyone is for sure, even the ones like me who didn't wear badges and assumed people knew who they were.
• talk video archive -- days of near-professional-quality video of all the talks. So I didn't miss AJ's talk on debbugs, or the very informative derivers panel, or any of the gigabyte of other talks I have queued to watch sometime. And eventually I may even be able to watch my own talk (and my shared talk) and learn something from it.
• lots of photos from DebConf5 -- why I don't bother buying a digital camera of my own. With more still to come.
• And hey, LWN wrote an article on my talk!

Can't wait until next May!

PS, try as it might, the weather here in the South hasn't quite managed to recreate the atmosphere of a proper Finnish sauna, and I already miss them dreadfully.

DebConf lightning talks

One of the talk slots at DebConf this year will be used for a series of lightning 5-minute talks on different subjects. I finally have more than enough talks to fill up the 45 minute time slot (thanks Jeroen!), but still only three speakers, so the schedule will look something like this:

• (T+0) 2 minutes: Joey, introduction
• (T+3) 2 minutes: Jeroen, How to pronounce Jeroen van Wolffelaar, and other names
• (T+5) 5 minutes: Jeroen, Doing efficient package searches on packages.d.o
• (T+11) 5 minutes: Joey, Debian in 4 MB or less
• (T+17) 5 minutes: Damog, WNPP: Automatizing the unautomatizable
• (T+23) 5 minutes: Jeroen, Actively discovering bugs/issues with packages
• (T+29) 5 minutes: Joey, Significant Choices
• (T+35) 5 minutes: Jeroen, Tracking MIA developers
• (T+41) 4 minutes: Joey, DebConf debconf (cdebconf?)
• (T+45) end!

These choices are a bit arbitrary, Jeroen could also talk about "Working of the PTS", or "Datamining on Debian packages metadata". I could also talk about "Rembering Emeritus Developers". We have more than enough proposals, but not enough people giving them. Me and Jeroen will both be wiped out at the end of this tag-team marathon. Help!

If you're not me or Jeroen, please consider submitting your own lightning talk idea, or take over one from the list, like "Debian wish list" or "Rembering Emeritus Developers". It will be much more interesting to hear from a lot of different people, and it's easy to do.

The deadline for last submissions is April 15th.

This advertisement paid for by the DebConf sauna campaign.

daylight savings time idiocy

Just found out about this, which will presumably keep Debian swamped with users with systems that need updating for the new DST for many months, if Congress does pass it:

Daylight saving time is going to be extended by four weeks to shorten the
winter, lengthen the summer and save energy.

If only they'd redefine pi again while they were at it, we could save even more energy not writing all those decimal places down.

"Kids across the nation will soon rejoice with the extended daylight on
Halloween night that will allow for an additional hour of trick or
treating," Upton said.

Yeah, I'll bet the kida in Alaska love putting on their Harry Potter robes and going outside into the nice spooky .. 24 hour sunlight! Did you know they observe daylight savings time in Alaska too? Crazy.

Personally, I'm with Click and Clack on this one.

d-i: working

Today for the first time d-i can be used to install etch. CDs are still broken though, and installs of sid are still not working, since sid's apt needs gpg to work, and gpg is not yet included in debootstrapped systems. Maybe this will be fixed later today.

d-i work!

I'm at the Roan Mountian campground this week, and have been taking advantage of the peace and quiet and the resumption of d-i development to work on d-i.

I've fiddled with lots of parts of it today, but the big news is that I made d-i able to build images for the 2.6 kernel using initramfs. Besides simplfying the boot process nicely, initramfs allows appending additional images to the main boot image, and the kernel assembles them all into one filesystem in memory. This is great because it points to a way to support non-free kernel modules and firmware, which has been split out of the Debian kernels entirely in 2.6.11 -- ship the images separatly and let the user cat them onto their image if they need them.

There's still some details to work out, but I think it's all doable, and there will be benefits beyond supporting non-free stuff.

d-i stall

We're at an interesting point in d-i development now. With rc3, the installer for sarge seems to be done, the number of new problems found in rc3 has been quite small and nobody expects another d-i release will be done before sarge is released. The installer is effectively frozen. Still, there's always the chance that we will need to make a new upload of part of it to fix some unknown, critical problem. Since testing-proposed-updates is still not ready, we have to plan for such uploads still going in to testing through unstable.

And that has put a damper on d-i development going forward, since using unstable is the only way we have to work on new stuff, and we can't yet just upload anything to it and possibly break things. Before rc3, this was sometimes frustrating, but at least led to us concentrating our effort on polishing the installer, manual, etc for sarge. Now it's really feeling like a problem.

We have a ton of things we plan to improve after sarge in d-i; we're branched in svn and ready to go; some development is being done, but we can't pull everything together and produce a working unstable version of the installer. So I find myself worrying about things like lost time, momentum, and what the catch-up period will look like when we finally begin merging things back together again.

The last time I had the pre-release freeze blues, it affected base-config and debconf and debhelper before the release of Debian 3.0. I ended up going away for a year and concentrating most of my effort on non-debian projects then. It seems that the drive to work on d-i is stronger though, so eventually, something will give way. Either t-p-u will finally get set up, or the pressure to diverge unstable from sarge will become strong enough to override the reasons to avoid doing so. Indeed, I think that latter is already beginning to happen.

If we don't get a t-p-u in time, then the two worst cases seem to be that d-i stalls out entirely as we're waiting for sarge to release and it takes a long time afterwards to get things moving again, or that d-i development resumes before sarge release, and sarge ends up being delayed further by side effects of that development. These are not pleasant shoals to navigate between.

d-i SCC ready already?

I'm stunned to see that sid's d-i appears to be ready for the SCC archive reorganisation already after some work we did today. The Debian mirror list now includes machine readable data about which architectures each mirror supports, and all the programs that use that list take that into account. Somehow I thought it would take more than a day to do this, it's great be be out of the freeze and back to full speed d-i development.

First to benefit from this should be the amd64 port, which should be able to install soon without manual mirror entry or customised versions of the installer.

The other suprising thing is that SCC seems to already be more a reality than was thought, since it seems Debian's main Bulgarian and Slovenian mirrors, as well as all of our Belarussian, Colombian, Israili, Indian, Moroccan and Ukrainian mirrors, already carry only a subset of architectures. Indeed, 30% of all our mirrors don't include all release architectures.

d-i review

While this article suffers somewhat from the "new distro release; let's review the installation and stop there" disease, it does include one of the best reviews of d-i I've seen in print. Only major factual mistake is they miss the availability of the 2.6 kernel.

Waaay back in winter 2000 I met Adam Di Carlo, then the leader of the boot-floppies team, at a wild after hours Linuxworld party in some club in New York City. Those were heady times, and giddy with all that was going on, I told Adam that I'd like to lead the installer team for the next release; I thought it was time to rewrite the crufty and much maligned Debian installer. Adam was surprisingly accepting of the idea. Little did I know..

A new installer for the next Debian release didn't happen (the boot-floppies stuck around through the releases Debian 2.2 and 3.0), but here I am 4 years later, on the eve of the unveiling of the first debian-installer release candidate. Luckily, this project was never about coming in on time and under budget, but about doing something right. And though I can see the barely apparent flaws when I look at our work, I think in the whole, we have.

I began to design the new installer in the summer of 2000, between June and August. I really feel that if I hadn't stepped up to do it, someone else would have soon thereafter. It was time. Some of the first bits of code were also written in this period, and notable early contributors were my old college pal Randolph Chung, who wrote udpkg (and later with Anthony Towns, cdebconf), David Whedon who did the first work on hardware autodetection, James Troup who (eventually) added udeb support to the archive, and Wichert Akkerman, who modified dpkg-dev to work with udebs. And probably a half-dozen others whom I'm forgetting. With this and some other groundwork laid, and a reasonably complete design (with some huge gaping holes in it) done by mid September, it was just a simple matter of coding up each component of the installer. In theory.

By December of 2000, the first d-i demo image was released. All it could do was load some bits of itself, and it didn't even run chrooted. By January of 2001, d-i was booting from a floppy (I found this part of the development very painful: real floppies!), and downloading the rest of itself over the network, but not doing much else of interest.

By this point we were six months into the project, which really seems like enough time to write a complete working linux installer, but with less than a dozen people working on the installer, and the need to code large chunks of it from scratch, we were making slow (if steady) progress. At least the groundwork we laid then has proven quite firm. Unfortunately at this point we were nowhere near having an installer suitable for a Debian release, and the next release was (we thought) going to happen soon. So in a very hard decision, we put d-i on the back burner, and Adam came back and dusted off the boot-floppies for one more go.

Over the lifetime of the d-i project, there have been a total of 18,837 distinct commits to its CVS (and later, subversion) repository. This is a plot of number of commits per month. The effects of bringing back the boot floppies are clearly visible in the decline that began in mid-2001. The spike of renewed development beginning in July 2002 exactly coincides with the release of Debian 3.0.

Looking back, there was too much emphasis on avoiding working on d-i to avoid stealing development time from the boot-floppies, and this was a mistake. In fact, several of us d-i contributors had little to do with the boot floppies, and found other projects to work on in this time period. (I wrote 22 thousand lines of code for another project; half the size of d-i.) In hindsight, we could have kept working on d-i without slowing down the boot-floppies development significantly, and d-i would be a year more mature today, and you'd probably be complaining that etch was not released yet and that sarge was looking old and crufty right now.

Some work was done during the slowdown, mostly by those who needed something other than the boot floppies for their own purposes. Raphael Hertzog did probably the first successful Debian install in using a system based on the installer (but with many local enhancements and changes) and much of his work was folded back into d-i. Not much later, Tollef Fog Heen and other developers at Skolelinux poured a lot of effort into d-i to get it into shape for their Debian-based distribution.

Around then I passed the d-i leadership to Tollef, and took a break. I don't know much about development between then and July 2003, as I was not much active in it, but it seemed to be going slowly, and was becoming one of the main blockers for sarge's release. There was even talk about digging up the boot-floppies, stitching its head back on and reanimating it for one more release, but that seemed .. distasteful.

That July, I attended DebConf3 in Oslo, met many of the current d-i developers, and was astounded to see that we had a working installer. Sure, it had lots of problems and warts; it crashed a lot; it only installed on i386 -- but it worked! I witnessed my first successful d-i install at DebConf3, a full three years after beginning the project. I was hooked.

And now development began to explode. By September 2003, installs from CD, floppy, and the network were working, USB keychain support was on its way, and we had a working powerpc port and starts on other ports. At this time I began leading the project again, and the number of d-i developers mushroomed.

While it doesn't represent all of our contributors, 142 people have committed to the d-i tree during its lifetime. Here's a plot of how many joined the project by month. The really big surges are when we moved to being hosted by alioth, and when translation of the installer began in earnest. 55 of our committers are not Debian developers, and may of them are translators.

The past year has featured milestone after milestone, and it seems that d-i just keeps getting better with each beta release. We've had our share of problems and slowdowns for various reasons, but the team has pulled together and really gotten things done. It's now typical, and not unusual, for someone to install Debian and tell us things just worked great. We're ported to 11 architectures besides i386, and lots of fun features like OS detection, LVM, and even pcmcia have been added to the installer this year. At least three Debian based distributions are in the process of releases using d-i as their installer.

Was it worth it? There were probably quicker and easier ways for the project to get a new installer for Debian. PGI, Anaconda and other installers exist for Debian now, alongside d-i. And while we used many pieces from elsewhere, d-i's design required a lot of code that is not normally part of an installer -- things like its own internal package manager. After all, d-i is not just an installer, it's a miniature linux distribution.

According to sloccount, the boot-floppies totals only 30 thousand lines of code, about 7 person-years of work, which is close to accurate as it was developed between 1994 and 2001 by a team of several part-timers. Sloccount says d-i is 51 thousand lines, or 12 person years (1.5 of them mine?). (For reference, Anaconda is ~62 thousand lines.) Of course these metrics don't begin to cover the enormous amount of testing, data collection, etc that it takes to do a linux installer.

So we surely could have avoided a lot of work if we'd based d-i on an existing installer, but one thing people don't take into account when they make this critisim is that d-i is itself based on several core things that Debian had spent years developing before we began to write d-i: The Debian package system and tools, debconf, debootstrap, the Debian development process -- all of these are key underpinnings of the Debian installer that have shaped it in unique ways and made its internal workings much different from a traditional linux installer.

Now d-i is four years old (counting its initial 9 month gestation), and while that's old for much software, it's in a sense quite young for an installer. There's so much hardware that d-i has not yet run on, and the tens of thousands of users who have tested it are only the beginning. After sarge releases, our next challenge as a team will be to decide where to go from here, and take advantage of the excellent base we have in d-i to do interesting and useful new things.

For me working on d-i has been a continual challenge and a delight. I've learned a ton, I've traveled to far-off countries and met unique and interesting people. I've installed Debian thousands of times. Aside from Debian itself this is the largest development project I've ever been a part of. I'm very glad that I was foolish enough to start writing a new Debian installer and that so many people believed in the idea and made it happen.

d-i rc3 released

It's done!

(Disclaimer: If above link is a 404, it may not be.)

Doing a release like this is stressful partly because of all the waiting. I've found an excellent way to deal with this: If I can do some nice demolition work with a crowbar in between waiting for people to become available and finalize things, waiting for images to build, transfer, and web sites to update, then I am much happier. So I happily helped my dad tear down his old back porch this afternoon. Just don't make me stop before the waiting is done, please.

Even though we've done these releases several times in the past year, and even though I've learned the hard way to automate it as much as possible and that manually cutting corners to save time generally doesn't, there is always some new challange or odd little bit that has to be fixed up by hand, and often this leads to mistakes. Sometimes we get a cascading failure mode, such as:

1. The scheduled weekly build of the CDs is not usable for the release, for $REASON_I_HAVE_HAPPILY_FORGOTTEN.
2. Build machine is badly overloaded, so move some of the CD builds to another machine.
3. Now the new machine updates the cd image archive, and oops, the update program does not expect this and deletes everything the other machine had uploaded.
4. Now a stressed admin who is up too late runs another rsync to fix this. But oops, it's reversed and he deletes all the images off the first machine too.
5. Thank goodness for multi-hundred-terabyte tape libraries. Pity they involve even more waiting..

Anyway, it's done, and exactly on schedule too.

d-i rc3 progress

Well, yesterday it looked like everything was finished building and seemed to be working (except perhaps the full CD desktop task), so I posted to let people know it was time to do final checking before we call this d-i rc3. This morning I did see some expected reports of new problems (there are always new problems, especially when the installer depends on various bits of debian that still arn't frozen), happily so far they seem to be either fairly minor, or have been quickly fixed.

I'm still gearing up to do my own testing of the rc3 images. Since this means sitting in front of lots of different test machines, burning CDs, moving keyboards and monitors around and doing non-automated testing for a change, I expect it will take several days to work through everything I want to test.

Oh yeah, the initrd-tools megaraid2 bug, while not a new bug, is such a glaring hole in the current installer that I hate to release with it. I wish I had access to a machine with a megaraid2 in it so I could figure out how to fix initrd-tools. It seems likely that it just needs to look at some additional files in proc to work out that it needs to include the megaraid2 module.

d-i rc2

Did the release dance today. The only real problem was that I discovered a problem with usb keyboard and the 2.6 kernel on ia64, that only affects autobuilt images such as those used by rc2. Oh well, I threw it in the errata since it wasn't worth redoing the release build for this bug. So after a lot of hurry up and wait, and months of work rc2 is out.

It kinda feels anticlimactic, and we're not being swarmed with users (at least yet, I only see 15 or 20 downloads at a time), and I think that's partly because this is the first release that hasn't fixed really serious problems in a past release or added key missing features. The installer has been sufficient for a while. Of course that's the point of rc2, and from here I hope we don't have to make a big deal out of stable d-i releases. We might rev the installer for minor problems like kernel security holes, but doing so won't be a problem or even very noticible. Soon we'll open up the tree for etch d-i development, which should bring some excitement back into things.

d-i random bits

There's something about this logfile that really tickles a geeky part of my soul. Yes, that is a complete install, from the BIOS boot messages on to the root login prompt. Run via cron.

It's interesting to watch userlinux release d-i based installation CDs, they're leaning toward the smallest possible isos and so are releasing with netboot mini.iso's and businesscard CDs. Yet another derived distro using d-i before sarge is released..

d-i pre-rc2

There's a built-in delay in my blog post propigating over svn to the server, and another delay before they hit Planet Debian, so about the time you read this, d-i pre-rc2 will be released, with lots of fun new features.

This release took me only one day to put together; of course I cut a lot more corners than I would for a true release candidate. I hope it turns out ok.

In only a week, we'll start the release process for rc2 itself. Could take as long as 3 weeks.

d-i playing nice

Now after I install with d-i, I get a nice grub boot menu that includes my newly installed system, plus freedos, plus menu items for other versions of linux installed on the machine. We're working on adding complete support for lilo, and I think yaboot (and macos) too. Already it's very nice.

Have any linux distributions done this before for anything but windows? If so, I'd still like to see if I can find some useful code in them. If not, well, the code is now available, and not at all Debian specific.

d-i mushroom

joey:~/src/debian-installer>cvs diff -N | wc -l      
21906

This started off as a simple project to add better messages to the prebaseconfig progress bar. Something like 2 lines of code changed and many templates added was my initial estimate. Of course many thousands of lines are just po file changes, but it has still became a much larger change than I expected, since various things had to be restructured to make the progress bar work better, including moving utilities from rootskel to di-utils, and reorganisation and code cleanup in di-utils. Not the first time a simple d-i change has mushroomed on me.

I think I will test this for a few more hours before committing it..

d-i in c't

I'm following this at one remove through a German language barrier, but I understand that the big and important German computer magazine c't has released an issue containing a Debian sarge CD with d-i. I've had a chance to read a machine tranlated version of the accompnying article, and look at the CD. Aba has some more first-hand information

The CD uses d-i preseeding to automate installs. Looks to me like you have to boot it with "auto" to get that, the default is a normal install. Good decision. Other mods include their own partman recipe for server partitioning, their own kernel (including kernel udebs!), and a "ctsrvcfg" that configures packages specially. So it's really on the verge between being Debian and a custom sub-distribution.

Kinda neat that rolling such a thing is (relatively) easy enough to do that a magazine has the resources to do it; though on the other hand I'd be happy to have stock d-i media on there too. It will be interesting to see if we hear from many new users because of this.

d-i hurd craziness

simulantaneous hurd and debian install

I knew that second usb keychain would come in handy for something..

d-i fun

wget http://people.debian.org/~bubulle/babelbox.tar.bz2 -O - | tar jxO babelbox/welcome.tar.bz2 | tar jx; for f in welcome/*.wav; do echo $f; bplay $f;done

Thanks to Christian Perrier for collecting these and of course for all the contributors in so many languages.

Also, here is the first "installation report" I've seen for d-i on the Mac Mini.

d-i: bleh

Today's breakage:

• anna got screwed up. My fault, bleh. Fixed.
• choose-mirror still wanted to install stable. Fixed.
• localechooser defaults the country to Australia for all English speakers.
• Language list display is a bit corrupted with new slang2.
• CDs still broken, but I know why, it's because they build against etch's debootstap-udeb, which doesn't yet know how to debootstrap etch.

Glad to see dpkg and busybox fixed, and with luck d-i will be working for etch by tomorrow.

d-i: another day..

Wow, we're past debootstrap. Etch installs are working from netboot. Installs of unstable still fail; the installer-breaking-transition-o-the-day is secure apt. CDs still broken, looks like still debootstrap or contents issues there.

Phil Hands has put together a great hack, a way to install Xen using d-i. He didn't modify d-i at all, just used preseeding. He's also doing some other stuff with preseeding that's pretty cool, and might turn into some kind of centralised community preseeding resource. Nice, this is the kind of thing I wanted to see happen with d-i preseeding. Check it out.

d-i (and svn)

Knowing 1.0 was due out on Monday, I spent quite a while today trying to do a test import of d-i's CVS repo into subverson. Unfortunatly I ran into a svnadmin load bug, which prevents both cvs2svn and refinecvs from working unless I disable tags and branches. Worse, I'm sure it was not fixed in 1.0. Oh well.

Frank Lichtenheld has done an exellent job with lintian support for udebs. It seems more complete than the linda support. It was great to go over his list and see what's wrong with our udebs. There were about as many m5sums files and conffiles incorrectly in udebs as I had expected, but the rest of it does not seem as bad as I'd thought. We can probably fix it in a few days.

Joshua Kwan's wireless configurator support in netcfg is promising, I gave that a try today too.

d-i: after sarge

The period just after a stable Debian release has always been an interesting time, but never before have I been trying to keep the Debian installer working during a flood of post-release changes. It's been frustrating.

Suprisingly, the worst breakage so far has been in debootstrap, which had some major changes just after sarge released. I hope these will actually make ongoing dependency change issues less likely to break d-i, but in the short term fixing the various things broken by these changes has kept d-i from working for weeks. I've had to NMU debootstrap twice, though the second NMU is for what's really a bug in busybox.

A bug in dpkg also breaks debootstrap; this has oddly been tagged pending for two weeks without an upload.

The ongoing slang 2 transition has helped keep things interesting, as more and more things are rebuilt with slang2. This tends to make debotstrap work even less well than it did before, and the installer's build process also has to be updated on a nearly daily basis. Alastair has been doing a good job at coordinating this transition though, and it's not kept d-i from working for any long length of time. Finally today it looks like d-i is (mostly) working with a fully slang2 build.

The CD builders were tired after the sarge release, so the first ever etch CDs were just built today. They don't work yet, probably thanks to yet more debootstrap breakage).

There was also a small libiw transition, and probably several other things that I've thankfully already forgotten about.

One of the first things the d-i team wants to do is switch d-i to using the 2.6.11 (or .12 if packaged) kernel, but there's no point on rolling that out when the install dies half way through, so it's been put off due to the debootstrap and other breakage.

We have managed to make some good progress on d-i despite it not quite working to install etch yet. The rescue mode that didn't make the cut for sarge is rolled in, adding one of our main outstanding missing features. We've switched over to languagechooser, updated the manual for etch, fixed quite a lot of bugs, fixed a large number of places that hardcoded "sarge", and have generally managed to fix many annoying bits that were put off until after sarge.

At the moment though, my main concern is that d-i be able to install etch again, and that it keep working until after DebConf. If it does, we should have a productive development session before DebConf, and work on lots of neat stuff, but if it doesn't we'll have to continue with the frustration of patching up after unstable uploads that break the installer.

d-i & svn take two

Happily I was wrong last night, and subversion 1.0's cvs2svn was able to import the d-i cvs repository, with only a little bit of manual rcs file editing. I let that run overnight (and got to bad around 4 am), and this morning I was able to get the test repository up and working, and added a branch that has everything moved around in a more logical layout. So far the only big problem is a nasty one involving the d-i manual, which encodes cvs revision numbers as part of its translation process.

I had somehow missed Colin's post to d-d-a about release schedules, although he had run it by me beforehand, so it was no big suprise. But this meant I had to summarise d-i status and what we need to do in the next 3 weeks, so another long email was written. We have knocked down a lot of todo items since beta 2: half of the errata, grub, graphical boot screens, wireless configurator, base-config simplified prompting, better image names, countrychooser. I was suprised at the end of that list how long it was. Still tons of stuff to do though.

Now I'm wiped out after two late nights with not enough sleep. Bed.

d-i

Response from my d-i help wanted posting has been better than expected, especially after it was in DWN and now LWN. Notably, there have been quite a few contributed splash screens, or reports of people working on them. And Bastian Kleineidam added udeb support to linda. There is also clearly a lot of interest in d-i for the sparc, so I hope that that team pulls it off in time. (And I'm sorry that I forgot y'all were working on it!) There has even been some S/390 activity. And some, but not enough, work on categorising installation reports. We still need to get more Debian developers involved in d-i.

It seems that d-i's problems with the default distribution to install are sorted out, after a long night of coding. Today I wrapped that up, uploaded a lot of other pending d-i stuff, including the scary new version of base-config, and worked with waldi on getting base-installer and apt-install to call apt in a safe way. That was an annoying problem, we have /cdrom mounted outside the chroot, but apt wants to use it inside the chroot. The old method was a grody hack that exposed libc mismatches between d-i and the installed system; the new approach uses a bind mount, which will cause trouble for 2.2. I doubt there is a really good fix. Hmm, maybe a small web server written in shell ... nah.

I battered my head against the d-i's main-menu/anna/retreivers wall a bit more today as well. I put in a fairly decent workaround to the overall problem in cdrom-detect, so SCSI cdroms should be usable again w/ a driver floppy, but the root floppy still has this annoying situation if you want to use it to load d-i from a cdrom -- after every step, it goes back to network setup, even if you have no network. Aside from splitting it into two floppies, I still don't know what to do about that. Perhaps this problem will solve itself that way; the current root floppy is nearly full.

"created using debconf"

Please do not put text in conffiles claiming that they were "created using debconf", as I am quite tired of having debconf be blamed for this kind of policy-ignorant foolishness. Please blame sh, sed, ed, readline, slang, whiptail, the unix kernel api, perl, python or perhaps your own broken postinst script. Thank you.

CPAN Purity tester

Here's a fairly sweet perl hack that I just wrote. Test how much of a the guts of a program comes from sweet, delicious CPAN, and how much is nasty perl code you wrote.

joey@dragon:~> cpan-purity cpan-purity        
usage: cpan-purity [--verbose] [--mine=regexp] -- program args
** CPAN Prurity test results: 98.08% pure CPAN code.
** 7107 lines in 24 files were from CPAN, and 139 lines in 1 files werenot.

joey@dragon:~> cpan-purity -- perl -e 'use Coy; die "whups, gotta go.."'

        -----
        Two rabbits walk. A 
        pair of lovers beside a pool. 
        A trout in the dam.
        -----

                Homer Simpson's commentary...

                whups, gotta go.. 

                        (Sayings of -e: line 1.)
** Looks like that was a perl one-liner..
** CPAN Prurity test results: 99.98% pure CPAN code.
** 6028 lines in 16 files were from CPAN, and 1 lines in 0 files were not.

joey@dragon:~> cpan-purity --mine=debconf dpkg-preconfigure
dpkg-preconfigure: must specify some debs to preconfigure
** CPAN Prurity test results: 70.49% pure CPAN code.
** 6745 lines in 41 files were from CPAN, and 2823 lines in 20 files were not.

joey@dragon:~> cpan-purity ikiwiki
Forcing /home/joey/src/ikiwiki/ikiwiki to run without taint checking..
usage: ikiwiki [options] source templates dest
** CPAN Prurity test results: 91.59% pure CPAN code.
** 12273 lines in 27 files were from CPAN, and 1126 lines in 1 files were not.

After all, writing more perl code is bad, but CPAN rules, right? So they tell me.

That was a lot of fun to figure out how to write, pity it's too special purpose for moreutils. :-)

Oh and by the way, the extra 600 lines shown in ikiwiki, up from the 300 lines it was two days ago include nearly everything you'd expect in a modern wiki, from templating to user login and registration, etc. It's nearing 0.1 state.

countrychooser hacking

Tonight some hacking on countrychooser. Lots of cleanups, and some very nasty hacks to make it group the large country list into regions. Something feels a bit wrong about substituting stuff into po files on the fly, and then search and replacing with non-breaking spaces in the generated templates file. But hey, it works.

Also I found out that debconf seems to eat those non-breaking spaces when loading templates. cdebconf does not. Probably perl utf-8 DWIM nonsense.

It seems that many changes to d-i nowadays, such as the one above, involve more time spent working on i18n issues than on the actual code. I guess that's ok, but I'd rather work on the actual code. I'm glad we have people like Denis Barbier who take up the slack of folks like me on i18n issues.

I developed a very scary theory this morning about the X bug that has been plaguing my laptop since this fall. This is the weirdest bug I think I've ever seen, and it looks like it may be a bug in the transmeta code morpher. Possibly; I hope not.

copyright

Could the folks I know who arn't involved in software stuff, but produce work that's copyrightable please read this and think about whether you want your work to go down that road, and what you can do to prevent it.

I assume that any people I know who are involved in software stuff are already aware of the issue and have had to spend enough time on copyright issues to get through the first year of law school. I know that I have.

I really hope that people in a hundred years time can laugh at all the absurdities of what we're doing to ourselves with copyright today.

confidental to Anna etc

To suppliment the dumpster diving, you can buy that item on ebay, in bulk even.

One problem however is that your departure, as well as destination point, can't be in the boonies.

conference flu, round two

The worst thing about coming down sick on the way home is I don't realise that I am. After all, the plane flight begins with an auspicious rainbow seen from the air over London. If I feel hot and then too cold, it's just the over-agressive air conditioning of this Altanta-crewed flight starting in. A bit of a sore throat is just leftovers from waking up too early in the morning. I manage to doze off for a few hours, and see it as just a blessed relief from the monotony, rather than the very unlikely event it is (can't sleep on planes). And the parched thirst when I wake up, that's normal in a dehydrating airplane cabin. I arrive in Atlanta, and somehow fit in with the herd better than usual, receive barely a glance from the passport guy, as I croak out some inane greeting -- and realize that my voice is truely going. But the usual mad scramble through ATL, to the furthest, deepest corner, where they hide the flights to TRI, is quite routine. When I finally get home, I still don't realize how sick I am, that headache and this overpowering tiredness is surely just because I've been up since 2 am, local time. I mostly miss Dani's visit, crash out into bed, and finally realize I've got it when I wake up in the really early morning, and the real symptoms of this thing (which I'll spare you) begin. Looking back, with the fever and everything, I was not entirely in my right mind for most of that trip home.

color

The testing security team has finally gotten around to doing some ranking of severities of security holes, and so the list of holes in testing now sports some colors. There's a pleasing amount of white and not too much pale red and very little deep red, which I will not try to interpret much more, because what types of holes get what colors is still a bit provisional.

codewiki

Taking an idea from AJ, I've set up my own codewiki. I'll use this to collect info about software projects I'm working on, and since I'm lazy, it will also serve as the main web site for many of them.

I've already moved many stub web sites for software over to there, and now moreutils has a web site there, and I finally have a place to document some of the useful tools in my ~/bin.

Of course it's powered by ikiwiki, which is nearly ready for a release.

cereal

Jeff, if the Metacity window manager is Cheerios, and I agree that others do resemble Marshmallow Froot Loops (possibly sprinkeld with E), then ion must be a subcutaneous nuclear power cell.

I'll take that over a rocket engine on my cereal box any day.

cd writer insanity

I've burnt at least 20 coasters today, as I try to figure out why my CD writer cannot burn any isos larger than about 128 mb anymore without producing discs that the kernel complains have media errors and refuses to mount or read.

Since I need to be able to burn CDs for work, getting this fixed has an annoying urgency. Worse, I've changed a lot of stuff on my system since I last burnt a full-sized iso last week, and so I have far too many variables to juggle. Excess heat from the new drive I added just below the CD drive? New version of cdrecord? New drive on the same ide controller? New ide cable? Just hit a bad patch on my stack-o-blanks? Damaged the CD drive while removing it earlier? Computer doesn't like rainy weather? Or maybe a crazy ISO that just won't burn while others will. At this point I can eliminate half of those and am going crazy working on the others. And thinking about just buying a new CD writer.

Aargh, I hate CDs. They're so completly the modern floppy, except they manage to suck even more.

Call for lightning talks for DebConf 6

My proposal to use one of the talk slots at DebConf 6 for a series of lightning talks has been accepted. Now I need to find some people (besides me) to give some of the 7 or more talks, each under 5 minutes long, that will fit into the time slot that I hope will be the most intense 45 minutes at DebConf.

If you're interested, I've set up an amusing page in the wiki where you can propose a lightning talk.

busy day

I had to work today since I'll be away for Thanksgiving. I ended up juggling several tasks all day, which never leaves me feeling very productive at the end, compared to if I can just sit down and work on one hard problem for hours. Still, I think it was a productive day: I babysat the d-i rc2 builds in the background, while working on fixing the debian-edu CD builds again, writing the we site changes for d-i rc2, setting up a page to track debian-edu CD build and autotest status, and in the background every hour or two I fixed another problem in the s390 autotest script and got a little bit closer to finishing that. The s390 autotest setup job has been running as a very low priority background task for 3 days, and would be done, except I rewrote most of it last night to more closely match a "real" s390 install.

Oh and when I wasn't working on all that, I tried to avoid getting sucked into several mooix bugs and discussions that have popped up all of a sudden after most mooix folk being inactive for most of the year. Now some misc notes and replies.

themes

Erich blogged about Debian branding. I may be in the minority, but one thing I like about Debian is that we make minimal changes to the default desktop, and in particular don't change from the default toolkit themes. This makes the Debian "brand" be whatever the standard look is. I think this fits in well with Debian's place as the de-facto standard distribution (um) used by a majority of the people who write this software. I've seen too many toolkit themes and skins put together by people who know what looks cool in a screenshot, but not what is actually usable. I trust at least one of our major desktops to take more care about usability than the average themer. Also, the idea of a militaristic theme based on the name "sarge" gives me and quite a few of my family members hives.

family blogging

It seems that several of my sibs have taken up blogging now, and the parents are still working to figure this stuff out. If this keeps up I should probably put together a family blog aggregator based on planet. If any family members are reading this and would like to receive any/all of the blogs by email, I can set that up in minutes for you.

busy, busy

d-i beta 3 has been slashdotted. Hppa folks have woken up, and I think the ia64 issues will be sorted out in a day or two.

Far too much time in town today for errands. 8 stops, ergh. I bought another usb keychain, it's 256 mb so I'll install Debian on it. Was looking for a decent PC joystick, but all I could find were thumb-blistering game pad things with 6 different pointing devices on them, and huge force-feedback joysticks. The games I have been wearing out my keyboard with to relieve stress were written before force-feedback, and I don't want a joystick larger than my laptop (hello Mr Freud). Looking for something small, cheap, and good that is well supported by linux (usb) and resembles a good old atari joystick, except it needs at least 3 buttons.

Lots, indeed tons, of interesting stuff is happening that I am not at a liberty to talk about. It's looking very likely that I'll be moving soon, don't know where to yet.

building d-i

I've been trying for several days to get a clean build of d-i to upload to incoming, but things have been in such flux that the build has broken every day because of something new. Today it was changes in wireless-tools and netcfg that required libiw be pulled into d-i, and evilness in mklibs that "reduced" libiw to a version twice as big that also pulled in all of libm. I hacked around that and got everything fitting on the root floppy again, and I think I finally have things sorted out to make a successful build tomorrow. If nothing changes in between..

d-i sparc is progressingly impressively well. They went from just kernel+busybox working, to it working all the way to debootstrap in one day.

We have still not figured out what to do about partman, but I am leaning toward provisionally making it the default for the next release and seeing what breaks. In retrospect, we should have done that as soon as beta 2 was out the door, as we did with grub. It will have to be decided one way or the other by this weekend.

bug hiding systems

OtherJoey's latest rant piqued my curiosity. Why would security supporting old versions of mozilla be so hard, surely they have a BTS that receives patches that could be backported, like other free software projects?

Picking a recent mozilla firefox security hole at random from my list of 14, I saw I'd found a nice bug, CAN-2005-2270 arbitrary code execution by remote attackers. Note that this hole has had a mozilla security announcement since mid-July, as well as 6+ separate advisories from third parties.

There were several links to the mozilla bug tracking system, I chose the first:

Access Denied

You are not authorized to access bug #294795. To see this bug, you must first login to an account with the appropriate permissions.

Please press Back and try again.

Amazingly, they want you to get a user account and log in just to see a bug report. This is normally the point where I decide I have something more pressing to attend to, and so will most people. But I really wanted to see this one, and I remembered bugmenot (which incidentially, mozilla has a plugin for).

Login details for bugzilla.mozilla.org

Account #1

anonymous@mailinator.com
digital

Awesome, so I can log in, and...

Access Denied

You are not authorized to access bug #294795.

Please press Back and try again.

That's right, this bug, which is for a security hole that was fixed two weeks ago, is not being dislosed until apparently, August 1st. Same is true for several others of the holes fixed in recent versions. That's two weeks for distibutions that have to backport these fixes to race against black hats to see who can track down the hole in all the other changes in the new mozilla release, and respectively fix and exploit it.

And so Ubuntu has decided to backport the new mozilla versions into their releases instead of backporting fixes, while Debian stable has decided to bow out of the race. Both understandable decisions in their own contexts.

This, IMHO, is a very good reason to find another web browser, both for myself and for those users for whom I maintain stable systems.

Bubble

The movie Bubble is set along the Ohio river in the Parkersville area where my Dad grew up, so it suprised me by being very personal and bringing up a lot of old memories of up there, my family in the area etc. I've not been up there since my grandmother died.

bts dependencies

Random hacking of the day -- finally patching debbugs for bug dependency support. Fun.

bts blockers support

AJ mentioned that he'd finished integrating my patches for bug blocking support into the Debian BTS. I hope this will prove useful in tracking both annoying blocking issues as well as transitions.

An example of the latter is the aalib transition. Every package that depends on aalib1 needs to be rebuilt to use libaa1. As part of this I filed bugs on everything that needed to be changed. I also filed a tracking bug on aalib itself, and I told the bts what bugs were blocking it.

Now I can easily just check the tracking bug every few days and watch the number of "blocked by" lines decrease to $SMALL_NUMBER, at which point I will declare the transition complete, remove the dummy transitional package, and upgrade the remaining no-longer-really-blockers to grave.

We could easily use this for all the other transitions that are currently going on, if someone has a list of bugs. For example, file a bug on general that we still have crud in /usr/doc and collect all the /usr/share/doc transition bugs as blockers. Which will also make it easy to find the packages nobody has filed a bug on yet for that transition.

I also plan to use it for coordinating some changes in the debian installer, where we often need to change a long list of things to deal with some general architectural bug (lack of unified serial console detection, udebs with broken dependencies, etc). More generally, many of the bugs filed against general are waiting for changes to different parts of Debian, which won't happen until someone files the individual bugs (with patches) to make it happen, and blockers can be used to track that work. We could also set up some yet more general bugs like "even if all RC bugs were are fixed, etch is not ready to release", and add various transition tracking bugs, todo items, etc as blockers to that etch release bug.

Of course the less grandiose use of blockers is just to let people know that their bugs are annoying you and wasting your time each time you have to read over bugs in your own bug list that you cannot fix.

Anyway, I think that's the spectrum of what that patch is capable of facilitating, which is why I wrote it, so I hope at least some of this comes to pass.

breaky bits

• rubber feet
• moleskin padding
• stickers
• pcmcia card slot filler
• molded plastic screen endpeices
• lid closers

The above are all the minor breaky bits that have broken off of my laptop in the several years I've owned it. The lid closers happened today when a chair collapsed, and it's the closest to something important to break off so far. At this point I have exactly one expendible breaky bit left:

• pcmcia card eject button

After that, I guess my streak of luck will be over and I'll lose the wifi antenna or the screen, or the hard drive, or the battery tray. All things that I'm rather suprised have survived intact so far..

Brazil and more d-i hacking (suprise..)

I mailed in my visa app for DebConf4 in Brazil today. Also picked up a bottom-of-the-line digital camera, perhaps a little lower than I should have aimed for, so I'm thinking about returning it. I just want to take snapshots, but this camera can barely manage that. Anyway, I'm already anticipating the whole Brazil trip, which starts early for me, on March 11th.

My rant on hotplug has drawn some attention in email, maybe something will improve in this area in Debian. Oddly, noone has commented on the first part of the rant, which was about hotplug itself..

I've been spending some time making the d-i retrievers deal with failure modes better, it's not very glamorous, but it's needed. Unfortunatly making d-i add grub boot items for existing hurd installs is harder than anticipated due to the hurd's use of the module command, so I have put that on the back burner. A lot of stuff has been breaking in d-i lately, and we really only have one effective week of development left before things need to simmer down in the ramp up for beta 4.

Jerry and Tomoko got hitched! I'm psyched for them.

books and music

I found something interesting on a peer to peer network today. No, it was not the source code to Windows. Searching for a recently released song, the first match had the song title, but then noted that it was actually a stump speech of one of the US's democratic presidential candidates. (I forget which one.) I didn't download it.

On a vagely related tangent, this talk gets a lot of things right about ebooks, IMHO. I have read books mostly on my laptop starting with my month in Honduras last year, when having a full library with me in the middle of nowhere was great. I'm currently 31% through Quicksilver, which has been a relatively slow slog so far.

My advice to anyone who has tried and failed to read entire books on the computer is to dedicate an entire virtual desktop to it, and don't use a GUI program with possibly distracting stuff. Especially don't use a GUI web browser. Go for the simplest interface you can. Make sure the screen is not too wide or set appopriate margins so your eye can scan entire lines easily. Since my laptop has a wide screen, I use w3m in a full height, half-width xterm.

The only thing I am missing is a good way to bookmark my place in w3m on the rare occasions when I restart X.

blog responses, window managers, tabs

One of the annoying things about blog aggregators is the high cost of writing a followup to some unimportant fallacy someone else wrote in their blog on the aggregator. It's much harder than hitting "r" in a mailing list.

If you are "ion" users, use a webbrowser written for ion by ion users.

So I'm not sure if it's worth my time to point out to Erich Schubert that gnome is not the world, and that one of X's great strengths is letting me run individual programs from gnome and kde in ion. Which is not a desktop environment and for which large quantities of special purpose programs (let alone special purpose web browsers!) will never be written. It's certianly not worth my time to properly link back to his silly blog post about it.

By the way, ion is a tabed window manager, which means that the window manager, not individual apps in it, takes care of drawing and managing tabs onscreen. This makes just as much sense to me as it does for a window manager to draw close, minimize, maximize buttons (although ion doesn't bother with those), or for the window manager and not the app to provide resize handles. Apps like xmms that provide their own close, resize, window shade, etc widgets end up looking strange and being hard to use in every environment.

It wasn't that long ago that window shading was a cutting edge new feature, so it might have made sense to implement it in an app back then, but it's in any self-respecting window manager now (except, arguably, ion ;-), so it's time to let go and let the window manager take care of it. I think the same will happen with tabs.

So I also have a hard time taking seriously Axel Beckert's winging a about specifics of galeon's internal implementation of tabs. Although I can't bear to properly link back to his blog posts about it either. If you find a window manager that does tabs well, your tab problems will be solved, not only for this browser, but for all applications.

bleh

My laptop's hard disk is failing. So it looks like I'm temporarily going back to the disk I bought this one to replace, since I thought that old disk was going to fail eventually. From 80 gb to 20 gb -- ouch -- luckily things will still just barely fit. Also luckily I started backing it up last weekend when it first started making loud noises rather than today, when it first started having DMA errors.

Besides making drives that fail after 100 days of uptime, Fujitsu has a fairly horrible warantee program; they won't send a replacement drive before I send mine in and they want to keep it for up to 45 days before replacing it. And they won't even provide a mailer. Totally useless, except as a way to get a new drive back from them at some point, to sell on ebay to get back some of the money for the non-Fujitsu drive I'll have to buy now.

Or am I just spoiled by other manufacturers overnighting a replacement drive and a prepaid mailer to send the bad one back in? Probably.

blah

Lots of little random things today (countrychooser sorting, taskel, partman uploads, redone the nasty nested table in the Ports Status Page), nothing exciting really.

The only remotely interesting thing today was that I had more irc messages than email when I logged in this morning.

Black Dog

This is very interesting. Drop by a friend's house or a library and instead of messing around with windows, just plug in a little USB stick, that is actually a Linux Powerpc computer running Debian. It uses autorun to run a windows binary that includes an X server and network interception and from then on out the machine is taken over and all programs run on the Black Dog machine.

No worries about keystoke loggers (except in hardware), or data leaking into the browser history, or anything. ssh. X. offlineimap. Drool.

Also includes some kind of biometric (fingerprint?) authentication in case it falls into the wrong hands. I have my doubts that works very well, but whatever.

Anyway, this is a great (and apparently, not entirely unique) idea, and I really hope it catches on and there's lots of competition and these end up on Ebay for $50 (instead of the current $300), and everyone who's stuck using computers at a public library can buy one.

For myself, I'm thinking I can pack three Debian architectures in a square foot for $500 now.

Birthday plans

Looks like I will be celebrating my 30th in Spain. I arrive in Madrid on my b-day.

Oddly the only presents I want are also only apparently only available for shipping to Europe, so that works out pretty well, in theory:

This is a pretty good kite design, my only other one in this design is fanny pack sized and works pretty well. If you ever wondered how to make my happy with just 2 euros, this is how.

Er, that was my idea, so it's only fair..

big truck

So a week ago I bought an enormous white diesel pickup truck. It's so big that it's got footrests to get into it; so big that I can see over those SUV's; so big that it can hold 6 people in the cab; so big that tractor-trailers don't seem very large anymore. Really, it's quite big.

(Why do I imagine I hear all the Europeans laughing at me?)

It's also a stick shift, which is something I'd mostly put off learning when I belatedly started driving around in my compact car a couple of years ago. I practiced a bit before I bought the truck, using another vehicle, and I wasn't entirely confident, but it seemed doable. I bought the truck in South Carolina and had to drive it 200 miles to get home. Somehow I managed this, at night, over the mountains, through fog and rain.

(Gosh, I like to make my life difficult, don't I.)

Once I got home, I seemed to lose what touch I'd had; I spent several days unsure I could drive it at all, not knowing enough to tell if I was just doing something wrong or if it was broken, and very stressed out about this literal white elephant. Today with William's help I figured out that the clutch was leaking hydralic fluid; we fixed this and with the clutch working properly I can drive it again. I'm even beginning to enjoy the perspective.

I'd hate this to be mistaken for an April Fool's post, so I will leave off explaining what I intend to do with a grossly oversized truck until another time.

beta4 release log

Wednesday       (Originally planned to be release day.)
morning         Told that powerpc checks out ok.
                Confirmation that bug #245560 breaks module parameter
                setting in depmod. It remains unfixed and is errataed.
11 am           Get fixed debootstrap into testing for ia64.
2 pm            Build from Tuesday night complete.
late            Told James Troup to finalise the release on ftp-master.

beta4 release log

Tuesday
early           Fixed cdebconf udeb uploaded.
11 am           The sparc CD initrd is missing cdrom.o. Fixed and resubmitted
                initrd build.
1 am            Wrote web page updates for beta4, posted test pages for
                overview.
11 pm           Discovered that hw-detect fails with discover 2. Put in
                hack to make it use discover1, and resubmitted build.
                Included cdebconf udeb in the build, since it was
                available.
late            Told that hppa CD fails to boot, fixed in debian-cd CVS.
                Told that debootstrap is still broken on ia64.
                Told that sparc framebuffer is broken. Errataed.

beta4 release log

Thursday
1 pm            Told that hppa fails to get to partitioner on netboot install.
2 pm            Hppa problem diagnosed to bad image build, affects other
                arches (?), new build started. Pending release cenceled.
7 pm            New build turns out to have idiotic mistake which fails to
                autobuild. Re-submitted with fix.
10:30 pm        Told that mips does not boot due to kernel bug.
10:40 pm        Dropped mips from main beta4 release, with the option
                to add it back in an update.
11 pm           Began keeping this log. Went back and added entries for
                stuff I could find.
3:35 am         New build accepted by ftp-master, begins autobuilding.
3:50 am         We learn that the makedev tty perms bug, thought to impact
                only unstable, has infected debootstrap.

beta4 release log

Sunday
morning         Lots of changes in svn that are not appropriate for beta
                release.
11:40 am        Branched svn.

beta4 release log

Saturday
11 am           Uploaded first image build, hoping to have usable images
                in 2 days.

beta4 release log

Monday
3 pm            Fix sparc image build, which was broken. Make "final"
                upload.
late            Told that cdebconf text frontend is broken, breaks access
                floppies.

beta4 release log

Friday          (Final deadline for release.)
11:10 am        All images from last night autobuilt except mipsel which
                failed.
11:50 am        Told that os-prober fails on all modern windows systems
                due to an incorect priority of ntfs-modules. Overrides added.
12:15 pm        Fixed debootstrap udebs available for all broken
                architectures.
12:30 pm        Fixed debootstrap udebs moved directly from incoming to
                testing. Phear the ftp-master behind the curtain.
2:44 pm         mipsel build in the archive
        dinstall!
4:30 pm     all CDs built successfully
4:45 pm     final check of i386 CD, looks good
4:50 pm     web site changes checked and committed
8:00 pm     web site updated
        sent out release announcement

I hope this rather unusual set of blog entries doesn't bother any aggregators. I need to reogranise my weblog to make this kind of thing excludable. Anyway, see the whole set here, and I hope that helps give some idea of what is currently involved in a d-i release.

d-i beta 2

It feels like this took too long, but beta 2 of d-i is in the process of being released for three architectures. Now we need to figure out what to do about the 2 and a half architectures that did not quite make this release, and where to go from here. Should be interesting.

A big thank you is due to everyone who made this d-i release possible. With especial thanks to the several ftp-masters who have helped us in a number of ways. You guys rock!

being productive at conferences

I've been thinking a lot about why I got so little work done on debian-installer at DebConf4. Apparently I'm not the only one to feel a bit disappointed or frustrated with the amount of work done there.

I'm sure that part of the problem is the absolute mythos that's grown up about all the things that were accomplished at DebConf3 and the subsequent Oldenburg developer's meeting. We came into DebConf4 with high expectations and plans that a lot of work would be done, and this created a lot of stress, which for me made it hard to do useful things, including socialise and have meetings.

Maybe less was really accomplished at DebConf3 and Oldenburg than we remember in the rosy glow of retrospect; I actually remember some very frustrating times at DebConf3 trying to find a box that I could boot d-i on to test it, and I know I didn't really do that much there on solid d-i development, though I did a lot on getting involved and up to speed on the project again. But then again, I remember doing a ton of stuff at Oldenburg, including usb, some pcmcia, and working floppies. And othes did the powerpc port, and a lot of other stuff too. And we had a lot of meetings, that seemed rather productive.

Maybe we really accomplished more at DebConf4 than is first apparent. Some things like Steve's BIDI work are now bearing fruit, but were really started there. It's can't be denied that Marga and others did a huge lot of testing, and so we know exactly where we are for the arches that we had available to test. Having the d-i people available to diagnose problems, but not doing the actual testing worked very well (until our testers turned into expert d-i veterans, anyway ;-). And there were lots of little fixes. And I saw more real life d-i installs by the uninitiated than ever before, and learned important things about usability problems, that led to stuff like the tasksel rewrite.

I think that part of it is that the nature of working on d-i has changed a lot since the last DebConf; back then you could pick anything and do some very simple work to get it from not working at all to kinda working. Very satisfying. Now it's more a matter of working hard to find something that's broken, and then a trivial fix. Or a very hard fix for a small problem. Or just lots and lots of testing that turns up some small problems. Important work, but not as high profile as "I added USB boot support to the installer last night!" Maybe if we had been able to focus on something like porting d-i to a new arch at DebConf4, we would have felt we accomplished more; unfortunatly s390's are not very portable, and noone thought to try to line up amd64 equipment either. Besides, the amd64 port seems to be going pretty well on its own.

I do think that DebConf4's schedule contributed to the problem. There were talks and bofs every day, and even if one had the willpower to skip some of them, likely someone one needed to work with was hard to find because they were at a talk (or locked in the Warthogs' dungeon, in some cases). And it was hard to skip everything, so most days were broken up into several peices, with not enough long chunks of productive hacking time. I think DebConf4 was the best organised DebConf yet, and really very well done all around, but this scheduling is to my mind a mistake. I'm interested to know if people outside d-i also had this kind of problem; I noticed that there seemed to be no organised RC bug squishing at DebConf4 either, and I think the idea of having a DebCamp before the Conf was basically lost for this conference.

So for various reasons this was perhaps not the best conference that could be for d-i. I've still very glad I attended it. I wonder if the d-i team should consider another Oldenburg like meeting in a while, and if so, what we should do to avoid some of the problems I discussed above.

beach trip to remember

Leaving Ocracoke, this seems to be a beach trip I won't soon forget. From the beginning, arriving via the Cedar Island ferry and not doing the long drive down the islands to the familiar Haterass-side terminal, things were not the same; then there was the flood that drowned half the campground and town. And also so many of us were not present this year, Anna, Jerry, Tomoko, Dad. I wasn't in a tent, having my camper. Then too I didn't even get in the water until late in the second day, it was so rainy and cold. Hmm, perhaps that's not so unusual after all. Other bits that were different for me included the Long Walk, and the Missing Cat. And here I am on the ferry to Hatterass, and they've changed that too (no more old new ferries, let alone old old, fellow Okracokers).

Despite all this, it was a classic Orcracoke experience. I've kinda complained before about these trips to the beach blurring together into a certian sameness, a single memory of 25 years of the same vacation. Will be interesting to see if this one will eventually join that gestalt.

BBS

Sometime over the years, I've managed to forget, or block out, most of my memories of the time in high school when I used BBSes, before the internet. Even though I still have friends from back then, and even, come to think, host a rack of computers at the house of a guy who was once a sysop. Those couple of years (how many?) are mostly a blank, and not something I think about much. I can't remember the first time I used a BBS. I can remember clearly and crisply the first time I used a unix system.

A bit of it came back when I didn't pirate the BBS Documentary, an epic 8 part series of interviews with the people who brought us the likes of XMODEM, fidonet, pcboard, and ansi art. Highly recommended. Amazing how just looking at so many of the people immediatly reminds me of the BBSers I knew at the time.

Its producer, Jason Scott, also runs textfiles.com, a huge collection of all sorts of text files from the 60's through 90's, which mostly originate from old BBSes. His historical bent in preserving and documenting this stuff really appeals.

I poked around the net today, managed to find some docs about one BBS I remembered using, and nothing about a couple of others. Logged into a telnet BBS or two and remembered just how angry fruit salad and silly to use those systems really were, and also how much they probably contributed to my enduring enjoyment of unix, muds, and email.

I thought it would be nice to not forget about this stuff, and so I coded up a reminder, merging the ancient plain text files with what's currently new and spiffy (weblogs natch), and creating this webblog which has a random text file posted to it each day. Of course, the subjects range from anarchy, to programming, to phone phreaking, to porn, so view at your own risk, just as if it were 1992, and you were were dialing a number your war dialer had found while you were away at school.

Would you like to play a game?

base-config and d-i leadership

In this blog entry, Daniel Silverstone uses a reducio ad absurdum argument in favor of keeping non-free. One I've heard before, that is not very convincing. Then he gets down to the mud-slinging.

Although I certianly enjoy being called a "charismatic leader who can make a large team pull together" -- mainly because there's a first time for everything, and the only time I've been called charasmatic before was when I rolled an 18/00 ... Er, where was I? Oh yes, despite that, Daniel's blog entry seems to be innarurate in a few areas.

Now I thought that Joey's main remit was the debian-installer system. The bit of the system which asks about non-free in Woody appears to be base-config which afaict isn't Joey's domain but rather that of the entire Debian Boot team.

Perhaps Daniel is not aware of some of the other significant software I maintain for Debian? Anyway..

As its changelog will attest, I wrote base-config in 2000, basing it on a little bit of ugly code (in the boot-floppies) written by Bruce Perens. While Petter Reinholdtsen has recently became a co-maintainer of that package, and others have contributed in various ways, I was almost entirely responsible for it through several Debian releases. This has always meant making large decisions and large changes in what users see when they install Debian. In a very real sense I "own" base-config much more than I do most of Debian-installer, which I didn't write. Until this August, my name was in the maintainer field for base-config; we decided to change it to point to debian-boot when Petter came on board.

Some of my other notable and controversial decisions about base-config have included making it not install standard priority packages by default (a bad decision, later reverted), putting "stable" in sources.list instead of the release code name, defaulting to shadow passwords, etc. So yes, I feel entirely justified in making large sweeping changes in what base-config does. I've been doing it for years and while base-config is not ideal, there have been few complaints and fewer interested in taking over some of the load.

Then again, it looks like the entire of Debian's Boot/Installer team defer almost entirely to Joey -- it's good to have a charismatic leader who can make a large team pull together, but that doesn't make it good to follow them unquestioningly

The reason you won't find much discussion on debian-boot about my decision to lower the priority of apt-setup/non-free is because I checked that change into CVS from a hacking room at DebConf3 in Oslo, surrounded by other d-i developers, and we discussed it there. (Petter was busy organising the conference, and missed it.) Since then we've had over 300 reports of d-i installs, and not one of them has complained about non-free not being available by default, so I feel we made the right decision.

Daniel's perception of me dominating decisions in debian-boot is shallow. There are many areas where I defer to other developers, like Denis Barbier for i18n and cdebconf issues, Christian Perrier for i18n and l10n issues, Bastian Blank for libdebian-installer, Santiago Garcia Mantinan for CD stuff, James Troup for archive stuff, David Nusinow and co for discover, Anton Zinoviev for partman, and numerous porters for their own architecture. We're a team with many specialists, and one of my specialisations happens to be the design and interaction of the system as a whole, that's all.

But yes, I do work double-full-time on d-i -- I'm winding up a 14 hour work day on it now -- and I can see how it would be easy to get the impression Daniel has gotten. We're rushing feverioushy to make the best damn installer we can, and we don't have a lot of time for the kind of senseless bickering and sniping that so many developers abuse Debian mailing lists to engage in.

Followups to debian-boot@lists.debian.org, please.

bamboo and snow

I'm always fascinated by what snow does to the bamboo here. This heavy one has made the bamboo grove implode, it's bent down all over the creek and driveway and just looks totally different than normal.

Nice to finally get some real winter weather. Wishing I had a digital camera right now..

backups

Before I leave the country, I like to do a full backup of my laptop, just in case. Even after compressing a lot of stuff and deleting various cruft, which freed up 5 very useful GB of disk space, I used 22 CDs for today's backup. Really must get a DVD writer, this is insane.

back

I suspect that I frustrated some readers who expected to hear a little more about Brazil in my blog, and didn't. Sorry. And thanks to everyone who cheered me up after that low point entry during DebConf4. The rest of the conf was fun.

So back home, and with the nasty cold I came home with sorta controlled by assorted medication, I'm free to type banal bits into the screen again.

Yesterday featured 15 hours of sleep -- as always I wish I could sleep on moving vehicles -- and other than that, well, I checked my email, recovered my gpg key (yay!), typed "upload" a few times on various packages, and I finished Stephenson's Confusion. Which I recommend if you're planning a trip involving 13 plane flights on 4 airlines over 2 hemispheres, but otherwise is merely something for a rainy week. So today was suprisingly busy: I got into town, mowed the lawn, did bills, bought a bed, tested skolelinux images, wrote a proposal (of which more later), failed to find a cheap computer locally without a bundled monitor, wrote two weeks of backlogged reports, fixed the i386 daily builds and almost caught back up on d-i, and more.

The overwhelming impression back up here, besides the standard weirdness of being back in America, is that the leaves are wet. And there are a huge lush burgeoning lot of them. We know Brazil for having a big rainforest in it, and forget that there's so much else there. It seems that while I was there I forgot how amazing an Appalachian summer can be.

autopackage: designed by monkeys?

Please note that this weblog entry concerns itself only with the autopackage file format, and not the software used to build or install autopackages. It would help to keep this distinction in mind when reading it.

From my point of view, the autopackage format is mainly notable for being an executable shell archive type format with as far as I can see, no formal design documentation. So to extract it without running it (for something like alien, or just because I didn't feel like trusting random code downloaded from the net today) will mean looking for magic values in a shell script and hoping they've not changed the shell script layout too much. Yuck. Didn't we learn anything from shar files?

The other interesting bit is that since they do full relocatability, the binary payload itself contains no path information.

joey@dragon:~/tmp/data> tail -c 98791 ~/autopackage-qt-1.0.x86.package | tar jvxf -
autopackage-frontend-qt
autopackage-frontend-qt.desktop
autopackage-manager-qt
autopackage-manager-qt.desktop
da.gmp
de.gmp
el.gmp
es.gmp
fr.gmp
nl.gmp
pt.gmp
sq_AL.gmp
sv.gmp

Where is the path info hidden? Why, in another shell script, deep in the package metadata, of course:

joey@dragon:~/tmp/meta> grep autopackage-frontend-qt *   
apkg-install-script:installExe ./autopackage-frontend-qt ./autopackage-manager-qt
apkg-install-script:installDesktop ".hidden" ./autopackage-frontend-qt.desktop
apkg-install-script:    copyFiles --silent ./extractinfo "$PREFIX/share/apps/autopackage-frontend-qt"

Of course there is no formal design for how any of these shell functions work, and no guarantee that they'll not change the names or add new ones later. Therefore, an autopackage package cannot be reasonably extracted by anything except autopackage or a reimplementation of it. And you cannot extract a package fully without executing it. And they'll have to keep all these unspecified bits working the same way, forever, if they want to keep supporting old packages. Didn't we learn anything from shared libraries? Worst. Package. Format. Ever.

In summary, autopackage is not a real package format, it's a bunch of shell scripts that depend on other shell scripts in the autopackage program to work, and Alien will never, ever support autopackage unless at least this last bit is changed.

at the end of my tether

500 feet of extension code can't be the safest thing, even duct taped. Oh well, very nice site in the "bowl" at the farm.

artwork designed for download?

Most art is not designed to be seen starting from the top, with the eye slowly scanning down to the bottom until the whole image is visible. Nor is it intended to first be seen at low resolution approximation, and then sharpened or zoomed in on. Rather, IIRC, the plan is generally to have the eye jump to a given significant feature, and then travel around the picture. Or have the viewer start out at a distance, and see more detail (or less) as they approach. So it's interesting to consider the idea of art that is designed to be revieled progressivly, by a download. I wonder if this has been done?

I'm reading the Da Vinci Code, which is a fun book to read just for the references. I've been jumping out to google repeatedly to look up strange sects (yes, they really do exist, and they have a hilariously earnest critique of the book -- google for the other side). Also paintings I'm unfamiliar with or have not looked at in a while, etc. Since it's past 1 am, my phone line is saturated with nightly downloads, so this panel of Bosch's "Garden of Earthly Delights" positively crawled down my screen, and the effect was quite different than if I'd seen it all at once.

It's nice to think that 0.3 k/s throughput might be an advantage in some situation.

arrivals

First random visitors at the farm for my new stay arrived at 1:30 am. The nice thing about being an introvert here is that interesting people come to you occasionally. It's also the bad thing of course. Still, I'd been meaning to post to the mailing list to invite people out, so I was in the right mood.

argh! argh! aaaarrgh!

So it seems that linux-kernel-di's dsc file is overflowing a static limit in apt, this makes apt unable to use the entire Sources file. I did not get this fixed (by dropping kernel versions from linux-kernel-di) before dinstall, because a) other changes had made linux-kernel-di unbuildable b) in my haste I got the version number wrong c) alioth was down at a key point in time d) it takes half an hour to build it and I had to built it 4 times in all e) someone picked up the phone at a very bad time. Like the title says, ARGGH!!

I have an opinion of apt's penchant for using hard-coded constants for buffer sizes, and of the pam LDAP stuff used on alioth, but it's not printable.

argh

Laptop power supply is surely dying on me. Actually it seems to be a short in the cord, which is good news; at least the laptop itself is not busted. I've rush ordered a new power supply.

I've been in a dry spell with reading for nearly the past month, just couldn't seem to find anything worthwhile, but that broke yesterday and I was up all night until 6 am reading Varley, finished that and now I've been reading Crowley. I had less than 4 hours of sleep, but it didn't bother me too badly today.

Before this most recent hardware annoyance, I did some random d-i work. The kind of thing where two hours go by, and one has done useful things that one can vaguely remember (or look up in the logs), but no one big significant thing that's really worth remembering.

In closing, I am so dependent on my exact desktop and configuration setup, that it's truely frightening. It's like moving into a house and never leaving.

another reason to love VIOP

root@chicken:/home/joey>route add phone lo

Stops the thing from ringing nicely.

another d-i release

I've very pleased to see that DCC PR1 is using the debian installer. I guess this means the folks at Progeny have finally moved away from installing Debian based systems using Anaconda, and I hope we'll see the kind of useful d-i contributions from DCC members that we've gotten from Ubuntu.

Anna

Anna is such a farm geek. Envy.

americana

Old guy in a turtle cap elegantly tap-shuffling to Appalachian influenced Reggae. Bluegrass jam sessions forming in the middle of the sidewalk. Normally downtown is a little quieter than this, but I like it this way.

America

When I read Australian Jeff Waugh's comments about his impressions of the US, I had some flashbacks to how I felt after returning here after my first long trip abroad.

I don't feel comfortable here. It's like a giddy nightmare set in a theme park devoted to "big". Everything is big - if not in size, then in volume or sheer in-your-face-ness. The cars are huge, though you could scarcely describe them as cars. The television in my hotel room is almost twice the size of our TV at home. The convenience store is a kidney-punch of choice and product. The Burger King girl yelled at me, very, very politely, with a searing-flesh smile.

I remember walking into a local supermarket the day after I got back from Honduras and being severly shocked at bigness of it all; having an identical reaction to Jeff's above. But I grew up here. For years, so much of living in the USA has been for me a process of coping and tuning out the insanities, trying to find the worthwhile things, and sometimes nearly giving up and feeling much happier abroad. So as I was leafing through my diary looking for my reaction to being back in the USA, I was suprised to find this, written a week after I got back:

Moon to the north-west, peeping out in a little clear-circle from the white clouds. Quarter-full crescent. Clouds covering half the sky and setting off the snow-covered hills, black and white. To the south, a big gap in the clouds, and orion and more stars shining out bright. Chewing on some crusty dry, clean snow. Klypso at my heels.

I had another beautiful night like that last night, out watching the blue moon. More and more there are to me two Americas, there's the America Jeff describes, and the special one I've sometimes managed to find. But I fear I have increasingly too much difficulty finding that side of this place to be able to introduce any visitors to it.

always coming home (part 2)

Leo was amazing the entire trip down. Sat in his carrier and didn't make a fuss. He's used to traveling now. When we got out to wortroot, he knew the place, even though he's not been there for two years. Quite a cat.

Wow, there are a lot of weeds out there.

Came into town this morning for a d-i irc meeting, and state street was blocked off. It's the rhythm and roots reunion festival. With no cash handy to pat the gate fee, I slipped in the back of the coffee shop and am listening to gospel singers in the front.

It's good to be back. If only for two days before yet another trip to Europe.

Always Coming Home

I'm part of a generation or group in America who moves fairly frequently. Except for those long, blissful years at the farm, in the past ten years I've moved nearly every year, and sometimes more than once per year. I moved into this house as a purposeful transition stage, and always intended to move out after a year. This proved more difficult than I expected, but here I am, moving out. I don't think this is particularly healthy or good on a number of levels but I don't try to justify this, it's just the way things are with me, at least for now.

Anyway, when I move, I always find myself thinking about what home is to me. It seems to me that home is not a specific house, or a city, other sort of place, but instead it's a region where I can see hills and mountains of the right shape, with mostly deciduious trees on them, where I can experience each kind of weather in its season, including big temperature differentials in a short time period, and especially including summer thunderstorms. Where things still feel a bit rough and unfinished. So I mostly feel at home in the Appalachian mountains, but occasionally, briefly, in other, disparate places such as certian parts of Honduras, California, and Brazil. The Appalachians are the place where I can always feel at home, so I find myself returning here again and again.

[...] I have a story to tell of where I went when I was young; but now I go nowhere, sitting like a stone in this place, in this ground, in this Valley. I have come where I was going.

-- LeGuin

Maybe one day. For now it's travel, travel, travel..

alpha joins the pack

A suprise UPS delivery today turned out to be an Alpha. Thanks, Matt! It was tricky to get it to netboot. None of the three tftp servers in Debian now work for all arches I have:

• tftpd: does not support i386 pxelinux
• tftpd-hpa: does not even respond to my alpha's tftp packets
• atftpd: fails on alpha and hppa

Anyway, already managed to fix some small bugs in d-i on alpha and in the next day or two, I hope to add fully automated install testing on alpha to the other 5 arches.

all this for a progress bar?

Over the past months we've been working on a big change in the debian installer, removing base-config from the installation process. Doing this has required a great many changes, some of them user-visible. The installer now configures the timezone, users and passwords, and apt all in the first stage instead of after rebooting. Some preseed values have been removed or changed, including base-config/early_command and base-config/late_command.

The biggest change of all, the the hardest to get working, has been moving of the selection of tasks and the installation of all packages into the first stage of the installer. Now tasksel pops up a debconf question that looks like any other question d-i asks, and the installation of packages is hidden behind a nice clean progress bar, and if any packages use debconf to ask questions, d-i will display those questions using its frontend too.

I'd include a screenshot, but really, it's just another progress bar, how boring is that?

By the way, if you maintain a debian package and it prompts without using debconf (when debconf is available), then your package will obviously break this, and it's well past time to fix it. I think that all the packages d-i installs do use debconf except for possibly a few like libc during upgrades. Upgrades are theoretically possible at this step for any packages debootstrap installs, so those will need to be fixed too.

This has been a long, long time coming. Some milestones include:

• 1998: Wichert Akkerman and others do the design of debconf, and first draft of its specification. Without debconf, none of this would be possible.
• 1999: At linuxworld, I see Stormix's Debian-based install. I remember discussing with them about how debconf could be used to clean up the install process. I don't remember what kind of hack they used to avoid the long dpkg log interspersed with the occsional question that a Debian install involved back then.
• 1999: I begin writing the debconf program. And promoting its use.
• 2000: Randolph Chung begins writing cdebconf; d-i begins.
• 2000: Randolph writes the passthrough frontend for debconf. At the time this was expected to be used by tools like aptitude (deity?) to integrate debconf into their GUIs, but it mostly just rotted for a long time unused. Looking back this was a contribution of unexpected significance.
• 2001: I visit Progeny, they show me their implementation of a progress bar running over a Debian installation. It's a special-purpose hack, but it works.
• 2001: debconf enters Debian policy
• 2003: debconf becomes the policy-accepted standard way to do installation prompting
• 2003: Adam Heath adds --status-fd to dpkg.
• 2005 (Jan): Matt Zimmerman dusts of the passthrough frontend and makes it usable to pass debconf questions from debconf running in a chroot up to cdebconf running in d-i.
• 2005 (Jan): Michael Vogt makes apt support status fds too.
• 2005 (Fall): Ubuntu pulls most of the above peices together and fixes some bugs, gets a progress bar running over apt with debconf passthrough.
• 2005 (Dec): Colin Watson and I develop debconf-apt-progress, rest of the pieces fall into place for using this stuff in Debian.

So yeah, this has been in er, progress for 8 years, and at least three Debian derived distributions have come up with thier own approaches in between with only the last one being quite similar to the end result, and the other two being rather dead. I don't know whether this is a study in how Debian is slow, or a study in how we do eventually come up with infrastructure that is done right and ends up being used by everyone. Or both.

alioth

I posted a little rant on alioth yesterday, seems a few people took that too seriously. First, I appreciate the work that's been done in setting up alioth, but I do still worry that it is not receiving enough maintenance and what that will mean for projects on it. Second, d-i won't move off it unless the project decides to, it's not just me making this decision. We'd have to find a better option anyway, and despite all the problems, alioth is hard to beat. We've hacked around the svn issues by various means, though this is not ideal, it will do for now.

I was much happier before the compromise, when cvs.debian.org Just Worked, for years and years at a time, though.

alien use patterns

Something interesting has changed in the past couple of years. I used to get my majority of alien bug reports and questions from Debian users. But nowadays, most of them are from users of rpm based distributions who are converting a deb into a rpm. Quite a few seem to be from developers who have a deb and want to make a quickie rpm to distribute.

This is an interesting shift. A few possible reasons for it might include:

• More developers using Debian.
• Increase in the percentage of users of RPM based distributions.
• Increasing clue amoung Debian users leads to less use of alien in places where it can be avoided; slight clue deficit amoung RPM users leads to the inverse.
• I generally do a worse job supporting alien on RPM based distributions (still doesn't support some newer versions of RPM that are not in Debian). So I hear more from the users of the less supported systems.

(In other news, use of alien to convert stampede packages is way down..)

alien -ick file.rpm

That combination of flags is so good that I wish I could claim it was intentional.

ah, progress

First there was email and usenet and we had double sigging and double posting.

Then there were moos and irc and we got spamming. The old, fun kind.

Now there are blogs and we have timestamp flooding.

admin

I've switched kite to use the stock Debian kernel instead of my own hand-compiled linus kernel. I'm very glad to do that, because there are so many unfixed security holes in the linus kernels and keeping up with them has gotten to be quite a pain. Since I still don't have a decent serial console on kite, there were some breathless moments as I waited for it to come back up, but the gamble worked.

And it also let me use stuff that I never got around to enabling in my own kernel. So I was able to set up raid 5 and put lvm on top of it. This is how kite should have been set up from the beginning, but I didn't know that stuff at the time; I've only learned it through working on supporting it in d-i. Enought excitement though: I think I will wait to switch the root filesystem to raid until some time when I am actually local to the box.

adding a raq

Debian hardware donations got me a cobalt raq to play with, so I added it to my automated test network. The wacky nfs stuff meant I had to write a few dozen lines of new code; aside from that the time from opening the box to first successful fully automated debian install on an architecture my automation scripts had never dealt with before was less than 4 hours. digress is still a bit of a pain to set up for new machines, but it's obviously getting better.

absolute paths: absolute bad idea

I wish I knew why people persist in hard-coding paths to executables in /bin and /usr/bin. This is just asking for trouble; the program moves from one place to another, or you get the path wrong somewhere and it breaks. It also prevents me from putting my own version of an executable in /usr/local/bin, or moving a binrary from /usr/bin to /bin because my system needs it on boot. So hardcoded paths are both fragile and rigid.

Oddly enough, UNIX was designed with a fix for this problem, known as the PATH variable, but few people seem to use it. Strangest of all is that I know that otherwise reasonable people will disagree with me, and continue using their hardcoded absolute paths.

This is one thing that python gets right.

a bad taste in the mouth: detailed Ubuntu patch review

Mark Shuttleworth is fond of saying "every Debian developer is also an Ubuntu developer". This Ubuntu developer feels somewhat unempowered and turned off by some of the things that appear in Ubuntu's versions of software that I maintain for Debian.

I took another look tonight at the omnibus patches, incorporating up to 1+ years worth of changes both relevant to Debian and not, that are the only way Ubuntu feeds most of their value added modifications back to Debian. Since Ununtu's patch archive still doesn't adhere to the standard convention of sorting packages by maintainer, I used the alternate archive provided by "utnubu", which does.

I maintain 40 or 50 packages for Debian, and of those, Ubuntu modifies 7 now. Last time I looked it was 3. Here's what's in those 7 patches.

• aalib's only change is to the changelog. They started to do things differently and changed their minds.

• base-config's patch is enormous. 53 thousand lines 1.8 megabytes. I have tried to pick the good bits from the patch before, but I gave up long ago, it's just not usable.

  Making base-config generally useful to distributions other than Debian is a pretty hard problem, and I decided a while ago that the correct solution was to refactor it all into d-i. To the extent that it helped me make that decision, seeing this patch has been useful, but that's all.

• bsdgames has some minor changes to add new acronyms to the wtf command.

  To make these changes they have for some reason converted the package to use the loathsome dpatch. Use of dpatch and kin are considered harmful by most people who NMU large quantities of packages for issues such as security. bsdgames containes sgid binaries, although I've audited them. Seeing one of my packages use dpatch was a sledgehammer blow, but I eventually continued on.

  The meat of the 216 line patch is adding 18 acronyms, of these --

  • 1 seems suitable for Debian, but not upstream.
  • 4 seem suitable to be added upstream.
  • 8 are non-acronym abbreviations for common words (ie, IM-speak), and thus out of scope for the wtf command, which according to its man page "translates acronyms for you". Ie, they're added bugs.
  • 2 are incorrectly spelt acronyms (it's FOAD, not FOD, and I am happy to be able to use the expression in this blog entry thankyou).
  • 1 is Ubuntu specific.
  • 2 I have no idea about.

  It's worth noting that my bsdgames packages does not add any acronyms, because I prefer to contribute such changes directly to upstream. My involvement in bsdgames began with porting it from a.out to elf, a little while ago; and excluding the debian/ directory, the whole Debian patch is only 135 lines. I last sent a detailed breakdown of the contents of the Debian patch to upstream about 2 months ago; I do it periodically to keep the patch down. Ubuntu has never contacted me regarding the larger (and uglier) patch they have accumulated in the past 7 months.

• debconf is notable in being maintained as much (or more) by an Ubuntu co-developer in Debian as it is by this Debian developer. I really appreciate his work. That said, the Ubuntu patch has some notable bits:

  • They modify the default debconf priority to high in several places. This is a bit weird since it actually defaults to high in newly installed Debian systems too, without those mods.
  • Ubuntu branding seems to call for commenting out the Debian logo even if you don't have a replacement Ubuntu logo handy.

  • Some gorgeously nasty hacking of dpkg-preconfigure. I think this is the best bit:

    exec "echo" or print STDERR "This is an amazing workaround that needs to be here!\n";

• Ubuntu's modifications of debhelper give me hives. Here is a program that, unmodified, is able to built nearly every package in Debian, and to which every divergence has the potential to break something or other, and yet Ubuntu for some reason needs to change it.

  The patches include:

  • Running scrollkeeper-update -q >/dev/null 2>&1. As if scrollkeeper-update -q not being quiet wouldn't be a bug. As if errors don't matter.
  • Build-Depending on file, a utility that debhelper does not use, AFAIK.
  • Some modifications to where X fonts are installed.
  • Modifications to where X man pages are installed, patch accepted.
  • Strange modifications to code dealing with /usr/X11R6/bin. AFAIK /usr/bin/X11 is a symlink to that directory so I don't see the point of moving things to there.

  • The worst bit is the striptranslations hack. If this program happens to be available, Ubuntu debhelper will run it at some point during a package build. That works ok until Debian adds a pkgstriptranslations that does something else and someone mix-n-matches software.

    I've pointed out to Ubuntu before that this is an accident waiting to happen; they have ignored me.

• The rbscrobbler patch patch is another dpatch between the eyes of good packaging practice. In another potentially security sensative application. (Which I've not really audited much.) Aside from that it seems to change some icons. Oh, and it does this:

  • Closes Malone #2925:
    • Connect the activate event of the password field with the btnOk_clicked callback

  I have no idea how to find "Malone #2925". It's not in their bugzilla. Does Ubuntu have multiple BTSes? The patch seems like it might apply to the Debian package, if I could tell what the bug was.

• The xaos patch is weird. And uses dpatch, grrrrrrr!

  Apparently this patch includes a merge of some upstream or third-party patches. But there is no indication of this at all in Ununtu's changelog.

  It's a huge patch with who knows what all in it.

• The xgalaga patch makes it build-depend on libdnet-dev. I don't know why.

  They include a second copy of the xgalaga-icon.xpm in the Debian directory. Hey guys, the same icon is one directory down, you knwo?

I'm left with pretty much a vagely bad taste in my mouth and a feelin that the three line patch I was able to glean from all this, plus the one possible one-liner bugfix (if I can find the bug) were not really worth the review of all these monolithic patches. That and the dpatch make it unlikely that I'll do this again. Ubuntu's left having to bring all these patches forward, prime busywork, but they made this bed..

I also don't feel one bit like an Ubuntu developer, thank you anyway Mark.

4th

I finally checked out the bike trail to downtown, it's a fairly easy ride if you're in shape for biking (I'm not), and a nice trail that I expect I'll use often. I ended up at my mom's, and had green food: Maggie's guacamole, and cookies with (no kidding) basil in them. Not at the same time except in fun, and that was nice.

I don't know if it's possible to get shell shocked from the 4th of July, but my new neighbors are doing their best. I was a bit annoyed at first, and I've really not gotten into the 4th in a long while, and have spent it out at the farm where I'd watch the fireflies and maybe hear a few bottlerockets. But hey, I have many fun memories of the 4th of July, and of doing dumb things with firecrackers, and I don't begrudge the fun. I have to think today of Duke, who died last year, far too young, and who made the 4th when I was a kid. I'll sit out here on the porch and watch a few more fireworks for you.

3 debhelper day

Today was a three debhelper day. Nice.

29

I celebrated my birthday early, yesterday with family out at the farm. Nice quiet, blissful day. Although at the end of it we began to move stuff out of my house, so I've officially started to move out already.

2.6!

d-i with the 2.6 kernel is so close I can taste it. In fact, images can be built now with only one hacked udeb, and it boots up and works fine all the way to partitioning. The partitioner does not see ide devices, but this is probably something stupid, since the kernel does.

Update, 3 pm April 14th: success!

Update: 21 April: Daily built CDs can be booted with "linux26", and it installs fine. A 2.6 kernel is not yet installed, and there are a few minor problems as documented in bug #243761.

11 keystrokes to Debian

Well, I counted keystrokes in a Debian install with DHCP and partman's spiffy streamlined autopartitoning. 11 keystrokes from the boot prompt to the "welcome to Debian" screen. 10 of them were "Enter"; the one extra was selecting the "Yes, please nuke my drive" button. I think perhaps two more keystrokes could be removed. Not bad.

The last time I did this experiment, I think d-i took 17 keystrokes or something like that.

base-config itself typically involves another dozen or so keystrokes, plus password entry. Of course the rest of the packages installs can involve many more..

.. and all through the house

At 4:50 am in my quiet house, a HP ia64 machine in my office, which has been quietly running a little-used Debian installation, spontaneously reboots. Five minutes later a loud roar comes from the server room next door, where a HP proliant with 6 drives and many loud fans has just turned on. While the ia64 is still wading through its pre-boot sequence, the proliant begins pulling software down over the network, and boots into the Debian installer. Minutes later, invisible hands scroll down and select "netboot" from a menu on the ia64 and it does the same. A few prompts are filled in by those same hands, flitting back and forth between the two machines, but they mostly seem able to do their tasks without intervention.

In a few minutes they're done and first the proliant and then the ia64 reboot. This time they're booting from disk drives that are now running software that was not present a few minutes before. On both a prompt comes up, asking for a password. Apparently, one is filled in. Both machines then proceed to suck down more software from the net, and install it.

Now the ia64 finishes first. Its ethernet light sparkles a few more times as it serves up web pages to the network. It reboots. Several minutes later, the proliant, which was downloading much more software, flips into graphics mode and displays a window with a big swirley logo and a login prompt. Soon it too reboots.

Over the next hour or so both computers repeat this process many times, with some variations. Sometimes one or the other will get stuck for half an hour and reset without finishing. At the end, the ia64 boots back into the installation it was running before, as if nothing had happened. And a while later the proliant turns itself off with a long whine.

In the morning, I wake up and check this with my morning email.

.. 2 bits

Yes, rumours are accurate, I did get a haircut. No, I can't believe it either.

(Thanks for the birthday wishes, they meant more than you know.)

I've now moved my blog into ikiwiki. I think that I've managed to do this without breaking (many) permalinks to old blog posts, and without flooding any aggregators.

.. Well, almost. Some of you may have seen a double post, which was due to me not having an url set right the first time. discussion

I realise this is a bit late in the game, but I've finally "learned" CSS. In quotes because I don't know an appreciable number of the available elements and, like probably a lot of people, I just cargo-cult around until I find something that works. My use of CSS is intentionally very miminal (see html version of this page).

It's amazing how easy it is to find bad docs about CSS still that encourage using absolute positioning, per-pixel layout, fixed page size, etc, and how hard it is to find answers to simple questions like "how do you embed a simple form in a horizontal list-based CSS navbar". Oh well..

I really hope that I'm about done with the web features side of ikiwiki, now that it uses CSS, is valid XHTML 1.0, and even has full text search thanks to Hyper Estraier.

I've added news feeds for moreutils, debhelper, alien, wmbattery, Words2Nums, sleepd, pdmenu and filters. New releases will be automatically posted to them using a wikiannounce program I threw together, which generates a markdown file that is fed to ikiwiki. Other news will be posted as I see fit.