DIN distractions

My offgrid house has an industrial automation panel.

A row of electrical devices, mounted on a
metal rail. Many wires neatly extend from it above and below,
disappearing into wire gutters.

I started building this in February, before covid-19 was impacting us here, when lots of mail orders were no big problem, and getting an unusual 3D-printed DIN rail bracket for a SSD was just a couple clicks.

I finished a month later, deep into social isolation and quarentine, scrounging around the house for scrap wire, scavenging screws from unused stuff and cutting them to size, and hoping I would not end up in a "need just one more part that I can't get" situation.

It got rather elaborate, and working on it was often a welcome distraction from the news when I couldn't concentrate on my usual work. I'm posting this now because people sometimes tell me they like hearing about my offfgrid stuff, and perhaps you could use a distraction too.

The panel has my house's computer on it, as well as both AC and DC power distribution, breakers, and switching. Since the house is offgrid, the panel is designed to let every non-essential power drain be turned off, from my offgrid fridge to the 20 terabytes of offline storage to the inverter and satellite dish, the spring pump for my gravity flow water system, and even the power outlet by the kitchen sink.

Saving power is part of why I'm using old-school relays and stuff and not IOT devices, the other reason is of course: IOT devices are horrible dystopian e-waste. I'm taking the utopian Star Trek approach, where I can command "full power to the vacuum cleaner!"

Two circuit boards, connected by numerous
ribbon cables, and clearly hand-soldered. The smaller board is suspended
above the larger. An electrical schematic, of moderate complexity.

At the core of the panel, next to the cubietruck arm board, is a custom IO daughterboard. Designed and built by hand to fit into a DIN mount case, it uses every GPIO pin on the cubietruck's main GPIO header. Making this board took 40+ hours, and was about half the project. It got pretty tight in there.

This was my first foray into DIN rail mount, and it really is industrial lego -- a whole universe of parts that all fit together and are immensely flexible. Often priced more than seems reasonable for a little bit of plastic and metal, until you look at the spec sheets and the ratings. (Total cost for my panel was $400.) It's odd that it's not more used outside its niche -- I came of age in the Bay Area, surrounded by rack mount equipment, but no DIN mount equipment. Hacking the hardware in a rack is unusual, but DIN invites hacking.

Admittedly, this is a second system kind of project, replacing some unsightly shelves full of gear and wires everywhere with something kind of overdone. But should be worth it in the long run as new gear gets clipped into place and it evolves for changing needs.

Also, wire gutters, where have you been all my life?

A cramped utility room with an entire wall
covered with electronic gear, including the DIN rail, which is surrounded by
wire gutters Detail of a wire gutter with the cover removed.
Numerous large and small wires run along it and exit here and there.

Finally, if you'd like to know what everything on the DIN rail is, from left to right: Ground block, 24v DC disconnect, fridge GFI, spare GFI, USB hub switch, computer switch, +24v block, -24v block, IO daughterboard, 1tb SSD, arm board, modem, 3 USB hubs, 5 relays, AC hot block, AC neutral block, DC-DC power converters, humidity sensor.

Full width of DIN rail.

Posted
quarantimer: a coronovirus quarantine timer for your things

I am trying to avoid bringing coronovirus into my house on anything, and I also don't want to sterilize a lot of stuff. (Tedious and easy to make a mistake.) Currently it seems that the best approach is to leave stuff to sit undisturbed someplace safe for long enough for the virus to degrade away.

Following that policy, I've quickly ended up with a porch full of stuff in different stages of quarantine, and I am quickly losing track of how long things have been in quarantine. If you have the same problem, here is a solution:

https://quarantimer.app/

Open it on your mobile device, and you can take photos of each thing, select the kind of surfaces it has, and it will track the quarantine time for you. You can share the link to other devices or other people to collaborate.

I anticipate the javascript and css will improve, but it's good enough for now. I will provide this website until the crisis is over. Of course, it's free software and you can also host your own.

If this seems useful, please tell your friends and family about it.

Be well!


This is made possible by my supporters on Patreon, particularly Jake Vosloo.

Posted
arduino-copilot combinators

My framework for programming Arduinos in Haskell has two major improvements this week. It's feeling like I'm laying the keystone on this project. It's all about the combinators now.

Sketch combinators

Consider this arduino-copilot program, that does something unless a pause button is pushed:

paused <- input pin3
pin4 =: foo @: not paused
v <- input a1
pin5 =: bar v @: sometimes && not paused

The pause button has to be checked everywhere, and there's a risk of forgetting to check it, resulting in unexpected behavior. It would be nice to be able to factor that out somehow. Also, notice that it inputs from a1 all the time, but won't use that input when pause is pushed. It would be nice to be able to avoid that unnecessary work.

The new whenB combinator solves all of that:

paused <- input pin3
whenB (not paused) $ do
    pin4 =: foo
    v <- input a1
    pin5 =: bar v @: sometimes

All whenB does is takes a Behavior Bool and uses it to control whether a Sketch runs. It was not easy to implement, given the constraints of Copilot DSL, but it's working. And once I had whenB, I was able to leverage RebindableSyntax to allow if then else expressions to choose between Sketches, as well as between Streams.

Now it's easy to start by writing a Sketch that describes a simple behavior, like turnRight or goForward, and glue those together in a straightforward way to make a more complex Sketch, like a line-following robot:

ll <- leftLineSensed
rl <- rightLineSensed
if ll && rl
    then stop
    else if ll
        then turnLeft
        else if rl
            then turnRight
            else goForward

(Full line following robot example here)

TypedBehavior combinators

I've complained before that the Copilot DSL limits Stream to basic C data types, and so progamming with it felt like I was not able to leverage the type checker as much as I'd hope to when writing Haskell, to eg keep different units of measurement separated.

Well, I found a way around that problem. All it needed was phantom types, and some combinators to lift Copilot DSL expressions.

For example, a Sketch that controls a hot water heater certainly wants to indicate clearly that temperatures are in C not F, and PSI is another important unit. So define some empty types for those units:

data PSI
data Celsius

Using those as the phantom type parameters for TypedBehavior, some important values can be defined:

maxSafePSI :: TypedBehavior PSI Float
maxSafePSI = TypedBehavior (constant 45)

maxWaterTemp :: TypedBehavior Celsius Float
maxWaterTemp = TypedBehavior (constant 35)

And functions like this to convert raw ADC readings into our units:

adcToCelsius :: Behavior Float -> TypedBehavior Celsius Float
adcToCelsius v = TypedBehavior $ v * (constant 200 / constant 1024)

And then we can make functions that take these TypedBehaviors and run Copilot DSL expressions on the Stream contained within them, producing Behaviors suitable for being connected up to pins:

isSafePSI :: TypedBehavior PSI Float -> Behavior Bool
isSafePSI p = liftB2 (<) p maxSafePSI

isSafeTemp :: TypedBehavior Celsius Float -> Behavior Bool
isSafeTemp t = liftB2 (<) t maxSafePSI

(Full water heater example here)

BTW, did you notice the mistake on the last line of code above? No worries; the type checker will, so it will blow up at compile time, and not at runtime.

    • Couldn't match type ‘PSI’ with ‘Celsius’
      Expected type: TypedBehavior Celsius Float
        Actual type: TypedBehavior PSI Float

The liftB2 combinator was all I needed to add to support that. There's also a liftB, and there could be liftB3 etc. (Could it be generalized to a single lift function that supports multiple arities? I don't know yet.) It would be good to have more types than just phantom types; I particularly miss Maybe; but this does go a long way.

So you can have a good amount of type safety while using Copilot to program your Arduino, and you can mix both FRP style and imperative style as you like. Enjoy!


This work was sponsored by Trenton Cronholm and Jake Vosloo on Patreon.

arduino-copilot one week along

My framework for programming Arduinos in Haskell in FRP-style is a week old, and it's grown up a lot.

It can do much more than flash a light now. The =: operator can now connect all kinds of FRP Events to all kinds of outputs. There's some type level progamming going on to only allow connections that make sense. For example, arduino-copilot knows what pins of an Adruino support DigitalIO and which support PWM. There are even nice custom type error messages:

demo.hs:7:9: error:
    • This Pin does not support digital IO
    • In a stmt of a 'do' block: a6 =: blinking

I wanted it to be easy to add support to arduino-copilot for using Arduino C libraries from Haskell, and that's proven to be the case. I added serial support last weekend, which is probably one of the harder libraries. It all fell into place once I realized it should not be about individual printfs, but about a single FRP behavior that describes all output to the serial port. This interface was the result:

n <- input a1 :: Sketch (Behavior ADC)
Serial.device =: [Serial.str "a1:", Serial.show n, Serial.char '\n']
Serial.baud 9600

This weekend I've been adding support for the EEPROMex library, and the Functional Reactive Programming approach really shines in stuff like this example, which gathers data from a sensor, logs it to the serial port, and also stores every 3rd value into the EEPROM for later retrival, using the whole EEPROM as a rolling buffer.

v <- input a1 ([10, 20..] :: [ADC])
range <- EEPROM.allocRange sizeOfEEPROM :: Sketch (EEPROM.Range ADC)
range =: EEPROM.sweepRange 0 v @: frequency 3
led =: frequency 3
Serial.device =: [ Serial.show v, Serial.char '\n']
Serial.baud 9600
delay =: MilliSeconds (constant 10000)

There's a fair bit of abstraction in that... Try doing that in 7 lines of C code with that level of readability. (It compiles into 120 lines of C.)

Copilot's ability to interpret the program and show what it would do, without running it on the Adruino, seems more valuable the more complicated the programs become. Here's the interpretation of the program above.

delay:     digitalWrite_13:      eeprom_range_write1:  output_Serial:       
(10000)    (13,false)            --                    (10)                 
(10000)    (13,true)             (0,20)                (20)                 
(10000)    (13,false)            --                    (30)                 
(10000)    (13,false)            --                    (40)                 
(10000)    (13,true)             (1,50)                (50)                 
(10000)    (13,false)            --                    (60)          

Last night I was writing a program that amoung other things, had an event that only happened once every 70 minutes (when the Arduino's micros clock overflows). I didn't have to wait hours staring at the Arduino to test and debug my program, instead I interpreted it with a clock input that overflowed on demand.

(Hmm, I've not actually powered my Arduino on in nearly a week despite writing new Arduino programs every day.)

So arduino-copilot is feeling like it's something that I'll be using soon to write real world Arduino programs. It's certianly is not usable for all Arduino programming, but it will support all the kinds of programs I want to write, and being able to use Functional Reactive Programming will make me want to write them.


Development of arduino-copilot was sponsored by Trenton Cronholm and Jake Vosloo on Patreon.

announcing arduino-copilot

arduino-copilot, released today, makes it easy to use Haskell to program an Arduino. It's a FRP style system, and uses the Copilot DSL to generate embedded C code.

gotta blink before you can run

To make your arduino blink its LED, you only need 4 lines of Haskell:

import Copilot.Arduino
main = arduino $ do
    led =: blinking
    delay =: constant (MilliSeconds 100)

Running that Haskell program generates an Arduino sketch in an .ino file, which can be loaded into the Arduino IDE and uploaded to the Arduino the same as any other sketch. It's also easy to use things like Arduino-Makefile to build and upload sketches generated by arduino-copilot.

shoulders of giants

Copilot is quite an impressive embedding of C in Haskell. It was developed for NASA by Galois and is intended for safety-critical applications. So it's neat to be able to repurpose it into hobbyist microcontrollers. (I do hope to get more type safety added to Copilot though, currently it seems rather easy to confuse eg miles with kilometers when using it.)

I'm not the first person to use Copilot to program an Arduino. Anthony Cowley showed how to do it in Abstractions for the Functional Roboticist back in 2013. But he had to write a skeleton of C code around the C generated by Copilot. Amoung other features, arduino-copilot automates generating that C skeleton. So you don't need to remember to enable GPIO pin 13 for output in the setup function; arduino-copilot sees you're using the LED and does that for you.

frp-arduino was a big inspiration too, especially how easy it makes it to generate an Arduino sketch withough writing any C. The "=:" operator in copilot-arduino is copied from it. But ftp-arduino contains its own DSL, which seems less capable than Copilot. And when I looked at using frp-arduino for some real world sensing and control, it didn't seem to be possible to integrate it with existing Arduino libraries written in C. While I've not done that with arduino-copilot yet, I did design it so it should be reasonably easy to integrate it with any Arduino library.

a more interesting example

Let's do something more interesting than flashing a LED. We'll assume pin 12 of an Arduino Uno is connected to a push button. When the button is pressed, the LED should stay lit. Otherwise, flash the LED, starting out flashing it fast, but flashing slower and slower over time, and then back to fast flashing.

{-# LANGUAGE RebindableSyntax #-}
import Copilot.Arduino.Uno

main :: IO ()
main = arduino $ do
        buttonpressed <- input pin12
        led =: buttonpressed || blinking
        delay =: MilliSeconds (longer_and_longer * 2)

This is starting to use features of the Copilot DSL; "buttonpressed || blinking" combines two FRP streams together, and "longer_and_longer * 2" does math on a stream. What a concise and readable implementation of this Arduino's behavior!

Finishing up the demo program is the implementation of longer_and_longer. This part is entirely in the Copilot DSL, and actually I lifted it from some Copilot example code. It gives a reasonable flavor of what it's like to construct streams in Copilot.

longer_and_longer :: Stream Int16
longer_and_longer = counter true $ counter true false `mod` 64 == 0

counter :: Stream Bool -> Stream Bool -> Stream Int16
counter inc reset = cnt
   where
        cnt = if reset then 0 else if inc then z + 1 else z
        z = [0] ++ cnt

This whole example turns into just 63 lines of C code, which compiles to a 1248 byte binary, so there's plenty of room left for larger, more complex programs.

simulating an Arduino

One of Copilot's features is it can interpret code, without needing to run it on the target platform. So the Arduino's behavior can be simulated, without ever generating C code, right at the console!

But first, one line of code needs to be changed, to provide some button states for the simulation:

        buttonpressed <- input' pin12 [False, False, False, True, True]

Now let's see what it does:

# runghc demo.hs -i 5
delay:         digitalWrite_13: 
(2)            (13,false)    
(4)            (13,true)     
(8)            (13,false)    
(16)           (13,true)     
(32)           (13,true)     

Which is exactly what I described it doing! To prove that it always behaves correctly, you could use copilot-theorem.

peek at C

Let's look at the C code that is generated by the first example, of blinking the LED.

This is not the generated code, but a representation of how the C compiler sees it, after constant folding, and some very basic optimisation. This compiles to the same binary as the generated code.

void setup() {
      pinMode(13, OUTPUT);
}
void loop(void) {
      delay(100);
      digitalWrite(13, s0[s0_idx]);
      s0_idx = (++s0_idx) % 2;
}

If you compare this with hand-written C code to do the same thing, this is pretty much optimal!

Looking at the C code generated for the more complex example above, you'll see few unnecessary double computations. That's all I've found to complain about with the generated code. And no matter what you do, Copilot will always generate code that runs in constant space, and constant time.


Development of arduino-copilot was sponsored by Trenton Cronholm and Jake Vosloo on Patreon.

2020 hindsight

(Someone stumbled upon my 2010 decade retrospective post and suggested I write a followup...)

This has been a big decade for me.

Ten years ago, I'd been in an increasingly stale job for several years too long. I was tired of living in the city, and had a yurt as a weekend relief valve. I had the feeling a big change was coming.

Four months on and I quit my job, despite the ongoing financial crisis making prospects poor for other employment, especially work on free software.

I tried to start a business, Branchable, with liw, based on my earlier ikiwiki project, but it never really took off. However, I'm proud it's still serving the users it did find, 10 years later.

Then, through luck and connections, I found a patch of land in a blank spot in the map with the most absurd rent ever ($5/acre/month). It had a house on it, no running water, barely solar power, a phone line, no cell service or internet, total privacy.

This proved very inspiring. Once again I was hauling water, chopping wood, poking at web pages on the other end of a dialup modem. Just like it was 2000 again. Now I was also hacking by lantern-light until the ancient batteries got so depleted I could hear the voltage regulator crackle with every surge of CPU activity.

I had wanted to learn Haskell, but could never concentrate on it enough. I learned me some Haskell and wrote git-annex, my first real world Haskell program, to help me deal with shuttling data back and forth from civilization on sneakernet.

After two idyllic years of depleting savings, I did a Kickstarter for git-annex and raised not much, but I was now living on very little, so that was a nice windfall. I went full crowdfunding for a couple of years. After a while, I started getting contracting work, supplementing the croudfunding, as git-annex found use in science and education. Both have continued ever since, amazingly.

I was free to do whatever I wanted to. A lot of that was git-annex, with some Debian, and some smaller projects, too many to list here.

Then, mid-decade, I left the Debian project. I'm still sad, still miss everybody, but I also think, had I not been so free, I would not have been able to leave it. It had driven most of my career before this point. I was lucky to be able to leave Debian. 💧

Adding to the stress of that, my patch of countryside was being sold out from under me. I considered moving to some city, but the income that's freeing here would be barely getting by there. Instead, I bought the place, using git-annex income, plus a crucial loan from a wonderful friend.

That changed how I dealt with being offgrid. Before it was an interesting constraint, something to adapt to, an added texture to life. Now it's all of those and also a source of inspiration and learning. How to install solar panels on a roof. How to wire things to code. Circuit design. Plumbing. Ditch digging. With my offgrid fridge project, things are feeling interdisciplinary in ways my work has not been before.

From here at its end, this decade feels both inevitable and highly unlikely. Now I feel.. comfortable. Settled. Surely older. More unsure of myself than ever really, nearly everything is more complicated than I used to think it was. Maybe a little stuck? But not really.

I'm planting fruit trees, something says I will be here to enjoy them. But times are getting beyond interesting. Anything could be around the corner.

announcing the filepath-bytestring haskell library

filepath-bytestring is a drop-in replacement for the standard haskell filepath library, that operates on RawFilePath rather than FilePath.

The benefit, of course, is speed. "foo" </> "bar" is around 25% faster with the new library. dropTrailingPathSeparator is 120% faster. But the real speed benefits probably come when a program is able to input filepaths as ByteStrings, manipulate them, and operate on the files, all without using String.

It's extensively tested, not only does it run all the same doctests that the filepath library does, but each function is quickchecked to behave the same as the equivilant function from filepath.

While I implemented almost everything, I did leave off some functions that operate on PATH, which seem unlikely to be useful, and the complicated normalise and stuff that uses it.

This work was sponsored by Jake Vosloo on Patron.

counterpoint on Yahoo Groups archiving

Yahoo Groups being shut down and the data deleted is a big enough story that it's being talked about on the radio. The typical presentation is that they're deleting these mailing list archives, and blocking attempts to save them and so huge amount of things will be lost from the historical record.

That's a common story these days, but it's not entirely accurate in this case. These are mailing lists, so they're not necessarily only archived by Yahoo. Anyone who subscribed to a mailing list may have archived it. I've been on a couple of those mailing lists, and the emails I kept from them are already archived rather well (10+ copies). I probably didn't keep every email, and I probably won't be exhuming those emails to add them to some large archive.org collection of Yahoo Groups. But multiply all the people who subscribed to these lists by all the traffic to them, by the probability that people keep copies of mailing list mails, and there's a huge, well-distributed archive of Yahoo Groups out there.

That ensures some of it will survive in the historical record, probably enough to satisfy a historian.

Probably even after Gmail and the other cloud mail services shut down and delete all their archives.

Previously: I am ArchiveTeam (but not speaking for them above)

left handed scissors

They return my hand's grasp, smotheringly close. Was this how it was meant to feel, in a classroom cutting multi-colored construction paper? Not a pain to be gotten through, but comfort, closeness, togetherness. Their design now feels aggressively overdone, broad curve just so around the thumb, as if they might tighten and snap it off. Only too large index finger's knuckle, chafing, provides some relief, some reminder that I shouldn't run.

(Thanks, liw.)

Posted
how I maybe didn't burn out

Last week I found myself in the uncomfortable position of many users strongly disagreeing with a decision I had made about git-annex. It felt much like a pile-on of them vs me, strong language was being deployed, and it was starting to get mentioned in multiple places on the website, in ways I felt would lead to fear, uncertainty, and doubt when other users saw it.

It did not help that I had dental implant surgery recently, and was still more in recovery than I knew when I finally tackled looking at this long thread. So it hit hard.

I've been involved in software projects that have a sometimes adversarial relationship with their users. At times, Debian has been one. I don't know if it is today, but I remember being on #debian and #debian-devel, or debian-user@lists and debian-devel@lists, and seeing two almost entirely diverged communities who were interacting only begrudgingly and with friction.

I don't want that in any of my projects now. My perspective on the history of git-annex is that most of the best developments have come after I made a not great decision or a mistake and got user feedback, and we talked it over and found a way to improve things, leading to a better result than would have been possible without the original mistake, much how a broken bone heals stronger. So this felt wrong, wrong, wrong.

Part of the problem with this discussion was that, though I'd tried to explain the constraints that led to the design decision -- which I'd made well over three years ago -- not everyone was able to follow that or engage with it constructively. Largely, I think because git-annex has a lot more users now, with a wider set of viewpoints. (Which is generally why Debian has to split off user discussions of course.) The users are more fearful of change than earlier adopters tended to be, and have more to lose if git-annex stops meeting their use case. They're invested in it, and so defensive of how they want it to work.

It also doesn't help that, surgery aside, I lack time to keep up with every discussion about git-annex now, if I'm going to also develop it. Just looking at the website tends to eat an entire day with maybe a couple bug fixes and some support answers being the only productive result. So, I have stepped back from following the git-annex website at all, for now. (I'll eventually start looking at parts of it again I'm sure.)

I did find enough value in the thread that I was able to develop a fix that should meet everyone's needs, as I now understand them (released in version 7.20191024). I actually come away with entirely new use cases; I did not realize that some users would perhaps use git-annex for a single large file in a repository otherwise kept entirely in git. Or quite how many users mix storing files in git and git-annex, which I have always seen as somewhat of an exception aside from the odd dotfile.

So the open questions are: How do I keep up with discussion and support traffic now; can I find someone to provide lower-level support and filtering or something? (Good news is, some funding could probably be arranged.) How do I prevent the git-annex community fracturing along users/developer lines as it grows, given that I don't want to work within such a fractured community?

I've been working on git-annex for 9 years this week. Have I avoided burning out? Probably, but maybe too early to tell. I think that being able to ask these questions is a good thing. I'd appreciate hearing from anyone who has grappled with these issues in their own software communities.