A not yet implemented idea from Re: Debugging over email:

I've noticed that once I am in a position to run some commands in the environment that has the problem, it seems to be much easier to solve it than when I'm trying to get the user to debug it remotely. This must be partly psychological?

Partly, I think that the feeling of being at a remove from the system, makes it harder to think of what to do. And then there are the times where the user pastes some output of running some commands and I mentally skip right over an important part of it. Because I didn't think to run one of the commands myself.

debug-me lets a developer access your shell remotely, to debug a problem, avoiding a tedious back-and-forth by email. When you start debug-me, it starts a shell, and generates an URL which you can give to the developer (or developers) to connect them to the session.

It's not normally a good idea to let someone run commands in a shell on your computer. To make this as safe as possible, debug-me uses the GPG web of trust. Everything the developer sends to debug-me is signed with their GPG key, in a way that produces a GPG signed proof of what the developer saw, and what they did in the debug-me session. If the developer does something Evil, you have the neccessary proof to adjust their reputation.

design

client-server

This needs to be client-server, because a end user's machine is probably buried behind NAT.

It should use HTTPS, because of stupid firewalls. Will need to use some form of long polling to get multiple messages promptly in both directions.

The user's client picks which server to use, connects, and gets an URL, which developers' clients can connect to in order to enter the session. Sharing that URL should be the only way for anyone to view the session (other than the person running the HTTP server).

Multiple clients can be connected, and all send inputs. The user's client will only accept inputs GPG signed with keys that the user has granted access.

When a new client connects, the whole session is sent to it, so it gets caught up and can start adding inputs to the chain.

Other clients can connect and only record the session. This allows a user to run debug-me on another computer, to record the developer activity proof there. This way a very Evil developer can't delete the user's only copy of the proof of their Evil.

The server can also record the session. This doesn't help preserve proof if the developer is running the server. But if there are multiple servers run by different people, the user's client could send the session to several of them to get it recorded.

proof

The developer activity proof takes the form of a chain of outputs and inputs. In other words, what the developer was shown, and what they did in response, and so on. Each input is GPG signed and includes the SHA256 hash of a previous output or input.

A debug-me session starts by generating an initial output. Probably this is a shell prompt. (To prevent replay attacks of debug-me sessions it should also include a timestamp and/or a random token.)

The developer enters a command, and their client hashes the last seen output and includes that in a GPG signed message with the command. The ouput of the command then arrives, and is hashed and included in the next GPG signed input message, and so on.

There are two kinds of inputs: Synchronous and asynchronous.

A synchronous input must include the hash of the output that it was made in response to, and will only be accepted by the user's client when that output is the last one that was sent. This way it's always clear what the state of the debug-me session was when the developer saw the output and entered a given input.

An asynchronous input must include the hash of the previous input. Asynchronous inputs are only used for sending signals, eg ctrl-c, not for sending keystrokes. This allows interruption of a running command at any time, without needing to catch up to its most recent output. (Especially useful if a buggy program is printing out a lot of output.)

It might seem like a good idea for a client to resend a rejected input, rebasing it on top of the new output. But, if a client did this, its user would have plausible deniability about what they intended that input to do. Perhaps they were not sending Y in response to "delete disk?", but to a previous question. So, inputs should not be resent. To prevent that, no message is sent to indicate when an input is rejected.

Synchronous input interacts badly with terminal echo, because sending an input causes an echo output, and if that output has not been received by the time the next letter is input, the next input sent will be rejected. To deal with this, an input can include an echo block. If the echo block has the same content as the sum of outputs sent since the output that the input is in response to, then the input will be accepted.

For example:

1. output "prompt>"
2. input "t" (previous: 1)
3. output "t" (previous: 2)
4. input "o" (previous: 1, echo: "t") -- valid because "t" = 3
5. output "o" (previous: 4)
6. input "p" (previous: 1, echo: "to") -- valid because "to" = 3+5
7. output: "p" (previous: 6)

This should be sufficient for shell access, and for using some editors. However, editors like vim that update a cursor position indicator as text is added will often have inputs rejected. debug-me could optionally beep or flash the screen when an input has been rejected.

proof truncation

A proof can be truncated at any point in the chain and still be a valid proof of what happened in the session up to that point.

One use case for this is that a developer might run a command to display known sensitive data (eg .ssh/id_rsa).

An interface to truncate a proof could also upload the truncated version of it to a server. There would need to be a way for the user to to remove the full proof from the server.

interface

Interface is as close to a ssh session as possible.

The user can input stuff, just as well as the developer.

It can be important to match terminal sizes, to make sure the developer is seeing same thing as the user. debug-me can assist with this by drawing a box and having the developer resize their terminal to enclose it, when the terminal sizes vary.

It would be possible to have the user manually vet inputs and outputs, rather than generating a GPG signed proof of the developer's activity. But, this would significantly complicate things, by needing to display on the user's terminal what inputs and outputs remain to be vetted. The complication of implementing that does not seem worth it. Especially because manual vetting would introduce lag.

It might be nice to have an integrated chat, but again this complicates the terminal handing significantly. User and developer can just as well chat on IRC anyway. Or run screen within debug-me and use one window to chat in.

installation

For debug-me to be most useful, it will need to be included in major distributions. It should also be as easy as possible to install manually.

It may be helpful for programs to bundle debug-me, at least until the standalone debug-me is widely available. For example, I can imagine including it in the standalone git-annex bundles.

other uses

Something like this could be used in a few other ways than debugging:

• Let anyone connect in watch-only mode. Could be used for broadcasting both debugging sessions and other activity.
• Moderated mode where N people connect and M of them need to approve an action before it's accepted. Could be used to add security, or for pair programming where both in the pair need to agree what to do.