This is my discussion blog. The way it works is that when any pages in this wiki have a discussion page created for them, the discussion pages show up below. Also, any comments on my blog posts also show up here.

tmux's API and ii

i have a similar concern in a project i'm working on now, which is to run a bunch of tests in one window and showing system logs in another window for convenience. i thought i would use tmux for this because the commandline interface is simple enough to be used like an API. my previous approach to this was to use a multi-interface design with simple functions like "alert", "prompt" or "log" that would send stuff to different places (terminal or gtk windows) depending on the environment, a bit like what debconf is doing, with all the crap that implies (generally: a suboptimal interface everywhere).

as for an irc client, you may be curious to try out ii: it's a simple FIFO and filesystem-based irc client that implements basically no user interface... a little nuts, but sounds like it would be the backend to the UI you are describing.

Comment by anarcat
flatpak?

this is awesome. :) i have used this in weird cases (which, unfortunately, now include debian jessie ;) and bundle at least one of those binaries next to my git-annex archives...

it seems there's a new player in this problem space - did you look at flatpak? the haskell folks have started to consider it...

Comment by anarcat
I2P is fit for anonymous filesharing by design

Trying out I2P is what I would recommend here, because especially if you are interested in filesharing or anonymous mail-services, this would probably be more cost-efficient, it is said to exist inside the FeeBSD ports-collection. So from my perspective TOR is nice for anonymous web-surfing, no idea about onion-services. Some big sites closed their services to TOR-users unfortunately, probably because of DDoS-attacks, some even don't accept connections from filtering proxies anymore. I2P is still in beta-status AFAIK, but probably one day it is going to be fit for real productive use and match up TOR. I2P is a different category, in so far as even the devolpers are anonymous and known by their pseudonyms only.

Comment by andreas.glaeser
climate change

2016 was a record-year globally, as far as I know, even warmer in average than 2015. Now, upon break of the new year, there is even snow and ice again in Berlin, and it is supposed to stay this way for three more days at least with temperatures below zero. This is not bad weather in my opinion, but more lovable weather indeed, because far less people are out there to get in your way. Sometimes bad weather is good weather in this sense, sometimes cool weather is good weather.

Comment by andreas.glaeser
Multi Pty programs
Voltron is a good example of this kind of program I think. I haven't actually used it, but it's a debugger interface that lets you spwawn new windows by using multiple ptys.
Comment by bburky
comment 5

The C API could look like this:

int openpty_managed(const char *addr,
    int *amaster, int *aslave, char *name,
    const struct termios *termp,
    const struct winsize *winp);

pid_t execvpepty_managed(const char *addr,
    const struct winsize *winp,
    const char *file, char *const argv[], char *const envp[]);

Compare with openpty(3) and forkpty(3).

Here addr is the address of the pty, eg "irc-client.channel.#haskell", and any parent address is automatically prefixed to it.

The implementation of these would use the socket environment variable to send requests to the pty manager. Mm, fd passing..

The winsize parameter lets a preferred window size be selected (to open an xterm with that size, or tile a buffer at that size).

It would probably be a good idea to add a parameter to both API calls for additional hints. For example, a pty manager might be able to lay out one buffer to the left of an other one, if provided a hint to do so. Or a pty might be non-interactive, and so a pty manager should not let the focus move to it. This needs to be something that's extensible. Could be as simple as char *const managehints[]

Comment by joey
comment 4

@josh interesting thoughts..

A pty-management library does seem like a good idea. There could be different pty-manager commands that open xterms, open screen buffers, etc. So the user runs something like:

in-xterms irc-client

in-screen irc-client

And irc-client could also be run in any shell that was started inside a pty-manager, talking to the outer pty-manager.

When the irc-client opens a pty, in-xterms would need to use something like xterm -e connect-pty to open a new xterm connected back to that pty. But when the irc-client runs the mail-client, in-xterms can just run xterm -e mail-client and doesn't need to relay.

(Leaving aside for the momemnt the problem that xterm -e does not propigate the exit status of the program it runs to its caller..)

While connect-pty would have some copying and syscall overhead (probably similar to screen), if a program found that excessive, it could run a child process, which would be connected up directly to the terminal (in the in-xterms case). That also provides a workaround for the singleton termnal in curses libraries.

As well as an environment variable containing the socket of the pty-manager, there could be one containing the address of the pty. If an irc-client opens a new pty to run a mail client, the mail client would see an address of eg "irc-client.mail-client", and it can use that address when it opens a new pty for an editor, so the editor gets an address of eg "irc-client.mail-client.editor". The pty manager can see a whole tree:

irc-client.channellist
irc-client.#haskell
irc-client.#haskell.userlist
irc-client.mail-client
irc-client.mail-client.message.happy new year
irc-client.mail-client.editor.file.tmpmsg
irc-client.mail-client.editor.help-window
irc-client.mail-client.editor.shell

Such human-friendly names in a tree would be useful for interactively managing ptys and could also inform tiling layouts. It also lets the pty manager set the default window title/screen buffer name to something reasonable.

Comment by joey
comment 3

Trying st with my ad-hoc benchmark, it managed to scroll around 4 thousand lines without noticable flicker. But I don't think st has a scroll buffer. Running screen in it to buffer the scrollback slowed it down to around 1-2 thousand lines, so comprable to vte.

Comment by joey
comment 2

I like the idea of unifying terminals, editors, IRC clients, mail clients, and other things that need multiple windows (whether tiled, tabbed, or just switched between). I don't know that I'd want to use terminal scrollback for that (I prefer editor-style or screen-style buffers with scroll positions), but many aspects of all those programs overlap. I can see why people use Emacs for everything, and I hope to see vim/neovim go in the same direction now that both have introduced coprocess support.

Editor buffers work nicely as a generalization, but they don't fit perfectly. For instance, embedding a shell in an editor needs to disable some of the editing functions for part but not all of the buffer. You want to use the editor to edit the next command line, but treat the prompt before it as read-only. (But you can use editor functions to search and copy text.)

I'd love to have a "libtmux" or "libscreen" that applications can embed. But I'd also like those applications to cooperate within a single thing that behaves like screen/tmux/vim, not to each have their own. What would such an abstraction look like?

What about a multi-pty-management library that can delegate to an outer multi-pty-management application. Imagine a library that lets you manage a pile of independent terminal-like buffers, and manage them like screen/tmux within a normal containing terminal. However, that library can also recognize when run inside an application that can handle such buffers itself (such as via an environment variable providing a communication socket). In that case, it'll contact that application, and delegate pty management to that application. The application can then logically group the buffers from each application together, and provide various functions for navigation, tiling, tabbing, switching, scrolling, searching, etc.

Comment by josh
Cool, yes, technical limitations

Firstly: yes, please :-) I've been using nmh for email lately and I love that I can open up a terminal and find something in my email archive for example, without disturbing the email I'm composing, or my inbox emptying process. Being able to connect multiple terminals is somewhat analogous to opening multiple tabs in the browser to the same webapp.

Second, odd that so many terminal emulaters are so slow. I run st (suckless terminal) and it seems to be able to handle about 100,000 lines of output per second (my notes file a few times over, no escape codes but some unicode) on my 6 year old desktop.

Third, most tiling window managers wreak havoc on existing output in terminals, even if they only resize it vertical dimension of the terminals. Tabbed should work fine though.

Comment by JasonWoof
comment 7

i have a similar concern in a project i'm working on now, which is to run a bunch of tests in one window and showing system logs in another window for convenience. i thought i would use tmux for this because the commandline interface is simple enough to be used like an API. my previous approach to this was to use a multi-interface design with simple functions like "alert", "prompt" or "log" that would send stuff to different places (terminal or gtk windows) depending on the environment, a bit like what debconf is doing, with all the crap that implies (generally: a suboptimal interface everywhere).

as for an irc client, you may be curious to try out ii: it's a simple FIFO and filesystem-based irc client that implements basically no user interface... a little nuts, but sounds like it would be the backend to the UI you are describing.

Comment by anarcat
comment 2
The latest release of dgit, 2.7, includes dgit-maint-merge(7), a tutorial building on the ideas in this blog post.
Comment by spwhitton
remove old batteries?

i am not an expert in the field, but it seems to me damaged / discharged batteries may actually be a liability. If you discharge a lead-acid battery too much, it gets damaged and charging it can't quite bring it back to its original capacity, as the terminals get full of crap. "Too much" varies from one battery to the other, but 11.1V is definitely too low, especially for a battery bank, where that voltage is more an average than the lowest charge point (which means some batteries are lower than 11V!).

The worst about damaged batteries in a battery bank is that they bring down other batteries too: the stronger batteries will try to charge the weaker batteries and discharge faster.

Another way to think about this is that every battery is also a resistance as well, even more so when it is damaged.

I can't explain, however, why the two new 12V batteries wouldn't be enough to power the house, unless you have too much of a power drain - I don't know the exact setup here (how much wattage is hooked to the circuit and the exact capacity of the individual batteries) so I can't guess much...

I hope that helps!

Comment by anarcat
Battery bank wiring

A few recommendations from what I see there, the simplest change I would do to improve the (dis)charging of the batteries would be move one cable to the other side of the bank, like this:

+---- house 
|           
+( 6v )-+( 6v )-
|              |
+( 6v )-+( 6v )-
|              |
+( 6v )-+( 6v )-
|              |
+( 6v )-+( 6v )-
|              |
+( 6v )-+( 6v )-
|              |
+(   new 12v  )-
|              |
+(   new 12v  )-
               -
       house----

That way the batteries "further away" will also get a decent charge. A good exercise is to calculate the wire length from plus to minus for all batteries. The topology above is not ideal, but usually quite enough. You should take care mixing old and new batteries, if you have a current clamp you can try monitoring the charging and see that all batteries are being utilized and no battery is being over charged.

Check with a voltmeter that the 6V packs are balanced by measuring the midpoints. A temporary solution that can help if they are not is connecting all the midpoints with a thin (like 2 mm2 or so) cable and letting them settle. If this doesn't help you may need to swap the pairs around to get them better matched

Just a few suggestions to get started, hope it helps!

Comment by jonatan
comment 7

Another bonus to using HMAC is that a large/distributed keysafe server cluster's nodes can all share the secret used to generate a request id, and then any node will accept a request id that was produced by any node. So that might be very useful in future scaling. Thanks for the help improving that part of this!

Comment by joey
comment 6

SYN cookies rely on knowing the client's IP address. Keysafe, as a TOR hidden service, does not have such a means to distinguish clients.

But, SYN cookie's use of a cryptographic hash makes me think of an HMAC. Could generate a request ID by (hmac secret random, random). Then the server can verify incoming request IDs using the secret.

That does seem to be simple enough to be worth implementing, to get rid of that first bloom filter. Thanks guys!

"mac PoW bucket bloom"? :)

Comment by joey
comment 5
Just a suggestion: Maybe something like the technique for syn cookies works better and faster than asymmetric cryptography.
Comment by mju
comment 3
I was thinking of asymmetric crypto, but yeah, using additional crypto does mean much more careful analysis.
Comment by josh
comment 3
Oh yeah, asymetric crypto would be the way to go.
Comment by joey
comment 2

That is indeed a good idea Josh. If the bloom filter memory use ever comes to be a problem that would be a good way to at half it.

It does add a fair bit of complexit. So many gotchas involved in using AES. Reusing an IV might allow key recovery attacks..

The 64 mb or so that would be freed is probably going to be peanuts on the average keysafe server (even if it's a fairly low-spec arm box), since they're supposed to be bare-metal servers only running keysafe.

Comment by joey