The newest pages in the codewiki are:

scriptreplay

In 2000, I patched script to dump out timing information, which let typescripts be replayed with realistic typing and output delays by a scriptreplay command I whipped up.

This was not a unique idea; Anthony Towns had some code to do something similar. And four months later, Satoru Takabayashi created ttyrec which does much the same. Of screenreplay, he says:

A friend of mine told me that Mr. Joey Hess implemented the same thing. If I had only known that earlier, I would have never reinvented the wheel...

In the end, ttyrec became the go-to tool for this purpose. Probably because it includes all the information in a single file, rather than the two files script uses (one for the terminal output and the other that I added for the timings)

In any case, it's common now to record terminal sessions and play them back in real time. One of the best known uses is by nethack.alt.org for replay and live watching of games.

I've had nothing to do with scriptreplay since I wrote it. Others have maintained it since. I tend to use ttyrec more myself, and have made a number of improvements to it, some of which have been included in the Debian package, but none of which seem to have gotten back to upstream as of yet. A number of my other patches to ttyrec, including using inotify to improve live playback of ttyrecord, are languishing in a git repository.

Posted
keysafe

Keysafe securely backs up a gpg secret key or other short secret to the cloud.

This is not intended for storing Debian Developer keys that yield root on ten million systems. It's about making it possible for users to use gpg who currently don't, and who would find it too hard to use paperkey to back up and restore their key as they reinstall their laptop.

https://keysafe.branchable.com

Posted
debug-me

A not yet implemented idea from Re: Debugging over email:

I've noticed that once I am in a position to run some commands in the environment that has the problem, it seems to be much easier to solve it than when I'm trying to get the user to debug it remotely. This must be partly psychological?

Partly, I think that the feeling of being at a remove from the system, makes it harder to think of what to do. And then there are the times where the user pastes some output of running some commands and I mentally skip right over an important part of it. Because I didn't think to run one of the commands myself.

debug-me lets a developer access your shell remotely, to debug a problem, avoiding a tedious back-and-forth by email. When you start debug-me, it starts a shell, and generates an URL which you can give to the developer (or developers) to connect them to the session.

It's not normally a good idea to let someone run commands in a shell on your computer. To make this as safe as possible, debug-me uses the GPG web of trust. Everything the developer sends to debug-me is signed with their GPG key, in a way that produces a GPG signed proof of what the developer saw, and what they did in the debug-me session. If the developer does something Evil, you have the neccessary proof to adjust their reputation.

As well as checking the GPG web of trust, debug-me can clone the git repository of the program that it's being used to debug, and check for GPG signed git tags and commits made using the developer's key. This helps you tell if the person connecting to debug-me is a developer of the program being debugged.

design

client-server

This needs to be client-server, because a end user's machine is probably buried behind NAT.

It should use HTTPS, because of stupid firewalls. Will need to use some form of long polling to get multiple messages promptly in both directions.

The user's client picks which server to use, connects, and gets an URL, which developers' clients can connect to in order to enter the session. Sharing that URL should be the only way for anyone to view the session (other than the person running the HTTP server).

Multiple clients can be connected, and all send inputs. The user's client will only accept inputs GPG signed with keys that the user has granted access.

When a new client connects, the whole session is sent to it, so it gets caught up and can start adding inputs to the chain.

Other clients can connect and only record the session. This allows a user to run debug-me on another computer, to record the developer activity proof there. This way a very Evil developer can't delete the user's only copy of the proof of their Evil.

The server can also record the session. This doesn't help preserve proof if the developer is running the server. But if there are multiple servers run by different people, the user's client could send the session to several of them to get it recorded.

proof

The developer activity proof takes the form of a chain of outputs and inputs. In other words, what the developer was shown, and what they did in response, and so on. Each input is GPG signed and includes the SHA256 hash of a previous output or input.

A debug-me session starts by generating an initial output. Probably this is a shell prompt. (To prevent replay attacks of debug-me sessions it should also include a timestamp and/or a random token.)

The developer enters a command, and their client hashes the last seen output and includes that in a GPG signed message with the command. The ouput of the command then arrives, and is hashed and included in the next GPG signed input message, and so on.

There are two kinds of inputs: Synchronous and asynchronous.

A synchronous input must include the hash of the output that it was made in response to, and will only be accepted by the user's client when that output is the last one that was sent. This way it's always clear what the state of the debug-me session was when the developer saw the output and entered a given input.

An asynchronous input must include the hash of the previous input. Asynchronous inputs are only used for sending signals, eg ctrl-c, not for sending keystrokes. This allows interruption of a running command at any time, without needing to catch up to its most recent output. (Especially useful if a buggy program is printing out a lot of output.)

It might seem like a good idea for a client to resend a rejected input, rebasing it on top of the new output. But, if a client did this, its user would have plausible deniability about what they intended that input to do. Perhaps they were not sending Y in response to "delete disk?", but to a previous question. So, inputs should not be resent. To prevent that, no message is sent to indicate when an input is rejected.

Synchronous input interacts badly with terminal echo, because sending an input causes an echo output, and if that output has not been received by the time the next letter is input, the next input sent will be rejected. To deal with this, an input can include an echo block. If the echo block has the same content as the sum of outputs sent since the output that the input is in response to, then the input will be accepted.

For example:

1. output "prompt>"
2. input "t" (previous: 1)
3. output "t" (previous: 2)
4. input "o" (previous: 1, echo: "t") -- valid because "t" = 3
5. output "o" (previous: 4)
6. input "p" (previous: 1, echo: "to") -- valid because "to" = 3+5
7. output: "p" (previous: 6)

This should be sufficient for shell access, and for using some editors. However, editors like vim that update a cursor position indicator as text is added will often have inputs rejected. debug-me could optionally beep or flash the screen when an input has been rejected.

interface

Interface is as close to a ssh session as possible.

The user can input stuff, just as well as the developer.

It can be important to match terminal sizes, to make sure the developer is seeing same thing as the user. debug-me can assist with this by drawing a box and having the developer resize their terminal to enclose it, when the terminal sizes vary.

It would be possible to have the user manually vet inputs and outputs, rather than generating a GPG signed proof of the developer's activity. But, this would significantly complicate things, by needing to display on the user's terminal what inputs and outputs remain to be vetted. The complication of implementing that does not seem worth it. Especially because manual vetting would introduce lag.

It might be nice to have an integrated chat, but again this complicates the terminal handing significantly. User and developer can just as well chat on IRC anyway. Or run screen within debug-me and use one window to chat in.

installation

For debug-me to be most useful, it will need to be included in major distributions. It should also be as easy as possible to install manually.

It may be helpful for programs to bundle debug-me, at least until the standalone debug-me is widely available. For example, I can imagine including it in the standalone git-annex bundles.

Posted
concurrent-output

Haskell library to let multiple threads and external processes concurrently output to the console, without it getting all garbled up.

Built on top of that is a way of defining multiple output regions, which are automatically laid out on the screen and can be individually updated by concurrent threads. Can be used for progress displays etc.

concurrent-output on Hackage

demo2.gif

aptdemo.gif

Used by: git-annex propellor etc

Posted