a special announcement from RMS

A little grep has told me that RMS is going to announce something special on olduse.net in two weeks time, on 27 September 1983. Best not to speculate about the details, but this announcement will be something people are still talking about thirty years later. So it's an event you and your news reader won't want to miss.

I have set up an IRC channel #olduse.net on FreeNode, and we'll see who first spots the post!

a year of olduse.net

My olduse.net exhibit has now finished replaying the first year of historical Usenet posts from 30 years ago, in real time.

That was only twelve thousand messages, probably less than many people get on Facebook in a year.. but if you read along this year, you probably have a much better feel for what early Usenet was like. If you didn't, it's not too early to start, 9 years of Usenet's flowering lie ahead.

I don't know how many people followed along. I read .. not every message, but a large fraction of them. I see 40 or so unique NNTP connections per day, and some seem to stick around and read for quite a while, so I'm guessing there might be a few hundred people following on a weekly basis.

If you're not one of them, and don't read our olduse.net blog, here are a few of the year's highlights:

  • After announcing the exhibit it hit all the big tech sites like Hacker News and Slashdot, and Boing Boing. A quarter million people loaded the front page. For a while every one of those involved a login to my server to run tin. I have some really amusing who listings. Every post available initially was read an average of 100 times, which was probably more than they were read the first go around.
  • Beautiful ascii art usenet maps were posted as the network mushroomed; I collected them all. In a time-delayed collaboration with Mark, I produced a more modern version.
  • We watched sites struggle with the growth of usenet, and followed along as B-news was developed and deployed.
  • In 2011, Dennis Richie passed. We still enjoy his wit and insight on olduse.net.
  • We enjoyed the first posts of things like The Hacker's Dictionary and DEC Wars.
  • Some source code posted to usenet still compiles. I haven't found anything to add to Debian, yet. :)

I will probably be expiring the first year's messages before too long, to have a more uncluttered view. So if you wanted to read them, hurry up!

olduse.net 1982

Hard to believe I've consumed all of 1981's Usenet posts now on olduse.net, and it's been running for 7 months already.

Last night, there was a "very long" post, describing nearly every node on usenet in 1982. There had been a warning about this post the day before, since it would take many sites half an hour to download at 300 baud. It was handily formatted as a shell script, which created per-node files.

So, I ran this code nobody has run since 1982. It worked. I got files. I tossed them on the olduse.net wiki, and used some ikiwiki code TOVA contracted me to write just a few months ago, to make clickable links on my usenet map.

usenet map

The map data was contributed in another post a while back. By 1982, usenet is getting nearly impossible to map with 1982 technology of ascii art. I enjoyed throwing graphviz, git, wikis, and the web at it.

So, we have a collaboration across time, me and "Mark" and a lot of people who described their usenet nodes and piles of technology that make creating a mashup easy. Awesome!

I blog about stuff I find on the olduse.net blog. It's an open blog; Koldfront also blogs there, and we welcome other bloggers.

Some of the highlights for me have included:

As the space shuttle program is winding down, reading the excitement about the first shuttle flights, and the play-by-play coverage of a launch, posted to net.columbia by a high school student borrowing his dad's account. (A usegroup name that's hard to read without remembering its fate).

The announcements of the Motorola M68k, the IBM PC, and the CD-ROM.

world ipv6 launch Reading the TCP-IP digest, and Postel's plans for launching IPv4 soon, while the world IPv6 launch is being planned now. (The nay-sayers are especially fun to read. Including the guy who was concerned about the address space size, in 1981!)

Learning that nethack ascention tales have a history streching back 30 years, to rogue, and that the stories back then had much the same flavor as they do today.

Various celebrity sightings. Dennis Ritchie teaching C and Unix. Bill Joy talking vi. RMS talking .. nuclear politics?

The general development of usenet. B-news being rolled out, groups proliferating, many first inklings of what will be major problems and developments in 5 or 10 years. A shift in tone is already apparent, by now usenet is not only about announcements, there are already some flames.

oldusenet in a period terminal

Still 9 years to go!

date formats of a decade of usenet

I've finished importing the usenet archive for oldusenet. The fun part was parsing the dates to put the posts in order.

No date format was really required on usenet, and so a wide variery of formats were used. Some posts didn't have a Date, but a guess could be made from their Message-ID. Some posts had absurd dates (ie, 1969, 1995), others had dates that were correct in every way.. except the year was left out (oops). One early post had a date of "_____".

Still, this excerpt of my code managed to parse the rest and so gives a fairly complete picture of how messy dates can possibly be. Read and weep.

  p anyzone "%d %b %y %T"       "15 Jun 88 02:27:41 GMT"
, p anyzone "%a, %d %b %y %T"       "Thu, 22 Jun 89 20:02:03 GMT"
, p anyzone "%a, %d-%b-%y %T"       "Thu, 15-Jun-89 18:01:56 EDT"
, p anyzone "%d %b %y %T"       "8 Jan 90 14:07:27 -0400"
, p anyzone "%d %b %y %H:%M"        "4 Oct 89 19:56 GMT"
, p anyzone "%a, %d %b %y %H:%M"    "Thu, 23 May 91 02:13 PDT"
, p anyzone "%a, %d %b %Y %T"       "Thu, 23 May 1991 07:07:00 -0400"
, p anyzone "%a, %d %b %Y %H:%M"    "Sat, 18 May 1991 17:28 CDT"
, p anyzone "%d %b %Y %T"       "11 Apr 1991 12:02:01 GMT"
, p anyzone "%d-%b-%y %H:%M"        "24-Mar-90 14:22 CST"
, p anyzone "%d %b %y, %T"      "22 May 91, 16:31:37 EST"
, p anyzone "%d %b %Y %H:%M"        "30 June 1991 17:15 -0400"
, p anyzone "%a, %d %b T  %T"       "Fri, 8 Feb T  09:49:39 EST"

-- special cases
, p (tzconst est) "%a %b %d %T EST %Y"  "Tue Jan 11 12:44:36 EST 1983"
, p (tzconst est) "%a %b %d %T EST %y"  "Tue Jan 11 12:44:36 EST 83"
, p (tzconst edt) "%a %b %d %T EDT %Y"  "Tue Jan 11 12:44:36 EDT 1983"
, p (tzconst edt) "%a %b %d %T EDT %y"  "Tue Jan 11 12:44:36 EDT 83"
, p (tzconst utc) "%a %b %d %T GMT %Y"  "Thu Nov  1 23:14:37 GMT 1990"
, p (tzconst pdt) "%d %b %y %T -7"  "11 Jun 91 15:41:21 -7"

-- dates with no timezone specified are guessed
, p nozone "%d %b %y %T"        "9 Jan 90 09:33:59"
, p nozone "%d %b %Y %T"        "10 APR 1990 05:25:28"
, p nozone "%a %b %d %T %Y"     "Fri Feb  6 00:19:47 1981"
, p nozone "%a %b %d %T %y"     "Fri Feb  6 00:19:47 81"
, p nozone "%Y-%m-%d %T"        "1981-11-12 18:31:01"
, p nozone "%y-%m-%d %T"        "81-11-12 18:31:01"
, p nozone "%a, %d %b %y %T"        "Sat, 13 Apr 91 08:37:57"
, p nozone "%a, %d %b %Y %T"        "Sun, 16 Jun 1991 13:23:02"
, p nozone "%d %b, %Y %T"       "1 May, 1991 00:00:00"
, p nozone "%d %b %y %H:%M"     "8 Jan 88 18:03"
, p nozone "%a, %d %b %y %H:%M"     "Wed, 29 May 91 17:14"
, p nozone "1 %b %d %T %Y"      "1 Jan 08 20:59:08 1991"

-- this has to come near the end, as it matches greedily
, g nozone "%a %b %d %T %Y ("       "Wed Oct 27 17:02:46 1982 (Tuesday)"
, g nozone "%a, %d %b %y %T +"      "Tue, 21 May 91 16:46:01 +22323328"

-- extract date from message-id headers
-- (used for messages with no Date field)
, g nozone "<%Y%b%d.%H%M%S."        "<1989Jul6.214048.28313@jarvis.csri.toronto.edu>"

(Parsing the often ambiguous, malformed, etc timezones was fun all its own too, of course.)

announcing olduse.net

As I write this, it's the morning of June 5th, 1981. A few people scattered across the US are waking up, going in to work, sitting down at their terminal with a coffee, and reading Usenet. Usenet is only getting a trickle of posts each day -- it's still in that period where it's easy to read every message posted to it.

Many things lie in Usenet's future. It's still running A-News, which doesn't even have a real From header yet. Later this year it will switch over to B-News, and volume will begin to increase. In 1987 there will be The great renaming. And of course in 1994, the first spam will be posted to Usenet.

But that's all a long way off, here in 1981. Right now, they're talking about 500 mb disk drives that only cost $38000. And rms is inciting flames about nuclear proliferation. And Postel is publishing an RFC for the new Mail Transfer Protocol.

Good morning, Usenet. Who knows what will come next in this fledgeling electronic communications medium!

a ten year real-time historical exhibit

This morning, I'm announcing a new site: Olduse.net

It's Usenet, updated in real time as it was thirty years ago. Planned to be available for the next ten years, unless I run out of inodes (again).

If you missed it the first time around, this is your chance to follow Usenet's flowering.

made possible by

141 magtape usenet archive

  • Henry Spencer at the University of Toronto, Department of Zoology, who archived Usenet. Back when it was really uncool and really expensive. Our view onto Usenet is thus slightly centric to Canada and Zoology, but that's ok.
  • David Wiseman, who hauled 141 magtapes in a pickup truck.
  • Many who worked to rescue data off the tapes. Including from the deleted stuff at the ends.
  • Rich Skrenta, who somehow got a copy of the archive out from under the Google borg. Although one of the tar files is truncated. Just saying.
  • The creator of Telehack, who pointed me in the right direction, ending my multi-year quest to find the archive. And if you think this is neat, Telehack will blow you away.
  • The developers of Haskell, which enabled me to whip up a B-News to C-News converter, a custom uucp, date parsers for every crazy date format ever used on Usenet, and suitible queue data structures in a rock solid, maintainable way, in 500 lines of code written over 12 hours. When I realized I also needed an A-News to B-News converter, I knew it was worth it to have done things right, because that took only 43 more lines, and worked 100% on the first run! My code repository for olduse.net is here.

PS: You can post to olduse.net, but it won't show up for at least 30 years. :)