censored Amazon review of Sandisk Ultra 32GB Micro SDHC Card

★ counterfeits in amazon pipeline

The 32 gb card I bought here at Amazon turned out to be fake. Within days I was getting read errors, even though the card was still mostly empty.

The logo is noticably blurry compared with a 32 gb card purchased elsewhere. Also, the color of the grey half of the card is subtly wrong, and the lettering is subtly wrong.

Amazon apparently has counterfiet stock in their pipeline, google "amazon counterfiet" for more.

You will not find this review on Sandisk Ultra 32GB Micro SDHC UHS-I Card with Adapter - 98MB/s U1 A1 - SDSQUAR-032G-GN6MA because it was rejected. As far as I can tell my review violates none of Amazon's posted guidelines. But it's specific about how to tell this card is counterfeit, and it mentions a real and ongoing issue that Amazon clearly wants to cover up.

usb drives with no phantom load

For a long time I've not had any network attached storage at home, because it's offgrid and power budget didn't allow it. But now I have 16 terabytes of network attached storage, that uses no power at all when it's not in use, and automatically spins up on demand.

I used a USB hub with per-port power control. But even with a USB drive's port powered down, there's a parasitic draw of around 3 watts per drive. Not a lot, but with 4 drives that's more power wasted than leaving a couple of ceiling lights on all the time. So I put all the equipment behind a relay too, so it can be fully powered down.

I'm using systemd for automounting the drives, and have it configured to power a drive's USB port on and off as needed using uhubctl. This was kind of tricky to work out how to do, but it works very well.

Here's the mount unit for a drive, media-joey-passport.mount:

[Unit]
Description=passport
Requires=startech-hub-port-4.service
After=startech-hub-port-4.service
[Mount]
Options=noauto
What=/dev/disk/by-label/passport
Where=/media/joey/passport

That's on port 4 of the USB hub, the startech-hub-port-4.service unit file is this:

[Unit]
Description=Startech usb hub port 4
PartOf=media-joey-passport.mount
[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/usr/sbin/uhubctl -a on -p 4 ; /bin/sleep 20
ExecStop=/usr/sbin/uhubctl -a off -p 4

The combination of PartOf with Requires and After in these units makes systemd start the port 4 service before mounting the drive, and stop it after unmounting. This was the hardest part to work out.

The sleep 20 is a bit unfortunate, it seems that it can take a few seconds for the drive to power up enough for the kernel to see it, and so without that the mount can fail, leaving the drive powered on indefinitely. Seems there ought to be a way to declare an additional dependency and avoid needing that sleep? Update: See my comment below for a better way.

Finally, the automount unit for the drive, media-joey-passport.automount:

[Unit]
Description=Automount passport
[Automount]
Where=/media/joey/passport
TimeoutIdleSec=300
[Install]
WantedBy=multi-user.target

The TimeoutIdleSec makes it unmount after around 5 minutes of not being used, and then its USB port gets powered off.

I decided to not automate the relay as part of the above, instead I typically turn it on for 5 hours or so, and use the storage whenever I want during that window. One advantage to that is cron jobs can't spin up the drives in the early morning hours.

Dear Ad Networks

In 1 week, I plan to benchmark all your advertisment delivery systems from IP address block 184.20/16.

Please note attached Intel microcode license may apply to your servers. If you don't want me benchmarking your ad servers, simply blacklist my IP block now.

Love, Joey

PS The benchmarking will continue indefinitely.

Posted
two security holes and a new library

For the past week and a half, I've been working on embargoed security holes. The embargo is over, and git-annex 6.20180626 has been released, fixing those holes. I'm also announcing a new Haskell library, http-client-restricted, which could be used to avoid similar problems in other programs.

Working in secret under a security embargo is mostly new to me, and I mostly don't like it, but it seems to have been the right call in this case. The first security hole I found in git-annex turned out to have a wider impact, affecting code in git-annex plugins (aka external special remotes) that uses HTTP. And quite likely beyond git-annex to unrelated programs, but I'll let their developers talk about that. So quite a lot of people were involved in this behind the scenes.

See also: The RESTLESS Vulnerability: Non-Browser Based Cross-Domain HTTP Request Attacks

And then there was the second security hole in git-annex, which took several days to notice, in collaboration with Daniel Dent. That one's potentially very nasty, allowing decryption of arbitrary gpg-encrypted files, although exploiting it would be hard. It logically followed from the first security hole, so it's good that the first security hole was under embagro long enough for us to think it all though.

These security holes involved HTTP servers doing things to exploit clients that connect to them. For example, a HTTP server that a client asks for the content of a file stored on it can redirect to a file:// on the client's disk, or to http://localhost/ or a private web server on the client's internal network. Once the client is tricked into downloading such private data, the confusion can result in private data being exposed. See the_advisory for details.

Fixing this kind of security hole is not necessarily easy, because we use HTTP libraries, often via an API library, which may not give much control over following redirects. DNS rebinding attacks can be used to defeat security checks, if the HTTP library doesn't expose the IP address it's connecting to.

I faced this problem in git-annex's use of the Haskell http-client library. So I had to write a new library, http-client-restricted. Thanks to the good design of the http-client library, particularly its Manager abstraction, my library extends it rather than needing to replace it, and can be used with any API library built on top of http-client.

I get the impression that a lot of other language's HTTP libraries need to have similar things developed. Much like web browsers need to enforce same-origin policies, HTTP clients need to be able to reject certain redirects according to the security needs of the program using them.

I kept a private journal while working on these security holes, and am publishing it now:

Posted
the single most important criteria when replacing Github

I could write a lot of things about the Github acquisition by Microsoft. About Github's embrace and extend of git, and how it passed unnoticed by people who now fear the same thing now that Microsoft is in the picture. About the stultifying effects of Github's centralization, and its retardant effect on general innovation in spaces around git and software development infrastructure.

Instead I'd rather highlight one simple criteria you can consider when you are evaluating any git hosting service, whether it's Gitlab or something self-hosted, or federated, or P2P[1], or whatever:

Consider all the data that's used to provide the value-added features on top of git. Issue tracking, wikis, notes in commits, lists of forks, pull requests, access controls, hooks, other configuration, etc.
Is that data stored in a git repository?

Github avoids doing that and there's a good reason why: By keeping this data in their own database, they lock you into the service. Consider if Github issues had been stored in a git repository next to the code. Anyone could quickly and easily clone the issue data, consume it, write alternative issue tracking interfaces, which then start accepting git pushes of issue updates and syncing all around. That would have quickly became the de-facto distributed issue tracking data format.

Instead, Github stuck it in a database, with a rate-limited API, and while this probably had as much to do with expediency, and a certain centralized mindset, as intentional lock-in at first, it's now become such good lock-in that Microsoft felt Github was worth $7 billion.

So, if whatever thing you're looking at instead of Github doesn't do this, it's at worst hoping to emulate that, or at best it's neglecting an opportunity to get us out of the trap we now find ourselves in.


[1] Although in the case of a P2P system which uses a distributed data structure, that can have many of the same benefits as using git. So, git-ssb, which stores issues etc as ssb messages, is just as good, for example.

fridge 0.1

Imagine something really cool, like a fridge connected to a powerwall, powered entirely by solar panels. What could be cooler than that?

How about a fridge powered entirely by solar panels without the powerwall? Zero battery use, and yet it still preserves your food.

That's much cooler, because batteries, even hyped ones like the powerwall, are expensive and innefficient and have limited cycles. Solar panels are cheap and efficient now. With enough solar panels that the fridge has power to cool down most days (even cloudy days), and a smart enough control system, the fridge itself becomes the battery -- a cold battery.

I'm live coding my fridge, with that goal in mind. You can follow along in this design thread on secure scuttlebutt, and my git commits, and you can watch real-time data from my fridge.

Over the past two days, which were not especially sunny, my 1 kilowatt of solar panels has managed to cool the fridge down close to standard fridge temperatures. The temperature remains steady overnight thanks to added thermal mass in the fridge. My food seems safe in it, despite it being powered off for 14 hours each night.

graph of fridge temperature, starting at 13C and trending downwards to 5C over 24 hours

(Numbers in this graph are running higher than the actual temps of food in the fridge, for reasons explained in the scuttlebutt thread.)

Of course, the longterm viability of a fridge that never draws from a battery is TBD; I'll know within a year if it works for me.

bunch of bananas resting on top of chest freezer fridge conversion

I've written about the coding side of this project before, in my haskell controlled offgrid fridge. The reactive-banana-automation library is working well in this application. My AIMS inverter control board and easy-peasy-devicetree-squeezy were other groundwork for this project.

more fun with reactive-banana-automation

My house knows when people are using the wifi, and keeps the inverter on so that the satellite internet is powered up, unless the battery is too low. When nobody is using the wifi, the inverter turns off, except when it's needed to power the fridge.

Sounds a little complicated, doesn't it? The code to automate that using my reactive-banana-automation library is almost shorter than the English description, and certianly clearer.

inverterPowerChange :: Sensors t -> MomentAutomation (Behavior (Maybe PowerChange))
inverterPowerChange sensors = do
    lowpower <- lowpowerMode sensors
    fridgepowerchange <- fridgePowerChange sensors
    wifiusers <- numWifiUsers sensors
    return $ react <$> lowpower <*> fridgepowerchange <*> wifiusers
where
    react lowpower fridgepowerchange wifiusers
            | lowpower = Just PowerOff
            | wifiusers > 0 = Just PowerOn
            | otherwise = fridgepowerchange

Of course, there are complexities under the hood, like where does numWifiUsers come from? (It uses inotify to detect changes to the DHCP leases file, and tracks when leases expire.) I'm up to 1200 lines of custom code for my house, only 170 lines of which are control code like the above.

But that code is the core, and it's where most of the bugs would be. The goal is to avoid most of the bugs by using FRP and Haskell the way I have, and the rest by testing.

For testing, I'm using doctest to embed test cases along with the FRP code. I designed reactive-banana-automation to work well with this style of testing. For example, here's how it determines when the house needs to be in low power mode, including the tests:

-- | Start by assuming we're not in low power mode, to avoid
-- entering it before batteryVoltage is available.
--
-- If batteryVoltage continues to be unavailable, enter low power mode for
-- safety.
-- 
-- >>> runner <- observeAutomation (runInverterUnless lowpowerMode) (mkSensors (pure ()))
-- >>> runner $ \sensors -> gotEvent (dhcpClients sensors) []
-- []
-- >>> runner $ \sensors -> sensorUnavailable (batteryVoltage sensors)
-- [InverterPower PowerOff]
-- >>> runner $ \sensors -> batteryVoltage sensors =: Volts 25
-- [InverterPower PowerOn]
-- >>> runner $ \sensors -> batteryVoltage sensors =: Volts 20
-- [InverterPower PowerOff]
-- >>> runner $ \sensors -> batteryVoltage sensors =: Volts 25
-- [InverterPower PowerOn]
-- >>> runner $ \sensors -> sensorUnavailable (batteryVoltage sensors)
-- [InverterPower PowerOff]
lowpowerMode :: Sensors t -> MomentAutomation (Behavior Bool)
lowpowerMode sensors = automationStepper False
    =<< fmap calc <$> getEventFrom (batteryVoltage sensors)
  where
    -- Below 24.0 (really 23.5 or so) is danger zone for lead acid.
    calc (Sensed v) = v < Volts 24.1
    calc SensorUnavailable = True

The sensor data is available over http, so I can run this controller code in test mode, on my laptop, and observe how it reacts to real-world circumstances.

joey@darkstar:~/src/homepower>./controller test
InverterPower PowerOn
FridgeRelay PowerOff

Previously: my haskell controlled offgrid fridge

Posted
my haskell controlled offgrid fridge

I'm preparing for a fridge upgrade, away from the tiny propane fridge to a chest freezer conversion. My home computer will be monitoring the fridge temperature and the state of my offgrid energy system, and turning the fridge on and off using a relay and the inverter control board I built earlier.

This kind of automation is a perfect fit for Functional Reactive Programming (FRP) since it's all about time-varying behaviors and events being combined together.

Of course, I want the control code to be as robust as possible, well tested, and easy to modify without making mistakes. Pure functional Haskell code.

There are many Haskell libraries for FRP, and I have not looked at most of them in any detail. I settled on reactive-banana because it has a good reputation and amazing testimonials.

"In the programming-language world, one rule of survival is simple: dance or die. This library makes dancing easy." – Simon Banana Jones

But, it's mostly used for GUI programming, or maybe some musical live-coding. There were no libraries for using reactive-banana for the more staid task of home automation, or anything like that. Also, using it involves a whole lot of IO code, so not great for testing.

So I built reactive-banana-automation on top of it to address my needs. I think it's a pretty good library, although I don't have a deep enough grokking of FRP to say that for sure.

Anyway, it's plenty flexible for my fridge automation needs, and I also wrote a motion-controlled light automation with it to make sure it could be used for something else (and to partly tackle the problem of using real-world time events when the underlying FRP library uses its own notion of time).

The code for my fridge is a work in progress since the fridge has not arrived yet, and because the question of in which situations an offgrid fridge should optimally run and not run is really rather complicated.

Here's a simpler example, for a non-offgrid fridge.

fridge :: Automation Sensors Actuators
fridge sensors actuators = do
        -- Create a Behavior that reflects the most recently reported
        -- temperature of the fridge.
        btemperature <- sensedBehavior (fridgeTemperature sensors)
        -- Calculate when the fridge should turn on and off.
        let bpowerchange = calcpowerchange <$> btemperature
        onBehaviorChangeMaybe bpowerchange (actuators . FridgePower)
  where
        calcpowerchange (Sensed temp)
                | temp `belowRange` allowedtemp = Just PowerOff
                | temp `aboveRange` allowedtemp = Just PowerOn
                | otherwise = Nothing
        calcpowerchange SensorUnavailable = Nothing
        allowedtemp = Range 1 4

And here the code is being tested in a reproducible fashion:

> runner <- observeAutomation fridge mkSensors
> runner $ \sensors -> fridgeTemperature sensors =: 6
[FridgePower PowerOn]
> runner $ \sensors -> fridgeTemperature sensors =: 3
[]
> runner $ \sensors -> fridgeTemperature sensors =: 0.5
[FridgePower PowerOff]

BTW, building a 400 line library and writing reams of control code for a fridge that has not been installed yet is what we Haskell programmers call "laziness".

Posted
AIMS inverter control via GPIO ports

I recently upgraded my inverter to a AIMS 1500 watt pure sine inverter (PWRI150024S). This is a decent inverter for the price, I hope. It seems reasonably efficient under load compared to other inverters. But when it's fully idle, it still consumes 4 watts of power.

That's almost as much power as my laptop, and while 96 watt-hours per day may not sound like a lot of power, some days in winter, 100 watt-hours is my entire budget for the day. Adding more batteries just to power an idle inverter would be the normal solution, probably. Instead, I want to have my house computer turn it off when it's not being used.

Which comes to the other problem with this inverter, since the power control is not a throw switch, but a button you have to press and hold for a second. And looking inside the inverter, this was not easily hacked to add a relay to control it.

The inverter has a RJ22 control port. AIMS also does not seem to document what the pins do, so I reverse engineered them.

Since the power is toggled, it's important that the computer be able to check if the inverter is currently running, to reliably get to the desired on/off state.

I designed (well, mostly cargo-culted) a circuit that uses 4n35 optoisolators to safely interface the AIMS with my cubietruck's GPIO ports, letting it turn the inverter on and off, and also check if it's currently running. Built this board, which is the first PCB I've designed and built myself.

The full schematic and haskell code to control the inverter are in the git repository https://git.joeyh.name/index.cgi/joey/homepower.git/tree/. My design notebook for this build is available in secure scuttlebutt along with power consumption measurements.

It works!

joey@darkstar:~>ssh house inverter status
off
joey@darkstar:~>ssh house inverter on
joey@darkstar:~>ssh house inverter status
on