Drought here since August. The small cistern ran dry a month ago, which has never happened before. The large cistern was down to some 900 gallons. I don't use anywhere near the national average of 400 gallons per day. More like 10 gallons. So could have managed for a few more months. Still, this was worrying, especially as the area moved from severe to extreme drought according to the US Drought Monitor.
Two days of solid rain fixed it, yay! The small cistern has already refilled, and the large will probably be full by tomorrow.
The winds preceeding that same rain storm fanned the flames that destroyed Gatlinburg. Earlier, fire got within 10 miles of here, although that may have been some kind of controlled burn.
Climate change is leading to longer duration weather events in this area. What tended to be a couple of dry weeks in the fall, has become multiple months of drought and weeks of fire. What might have been a few days of winter weather and a few inches of snow before the front moved through has turned into multiple weeks of arctic air, with multiple 1 ft snowfalls. What might have been a few scorching summer days has become a week of 100-110 degree temperatures. I've seen all that over the past several years.
After this, I'm adding "additional, larger cistern" to my todo list. Also "larger fire break around house".
On January 18th, I'll be presenting "Type driven configuration management with Propellor" at Linux.Conf.Au in Hobart, Tasmania. Abstract
Linux.Conf.Au is a wonderful conference, and I'm thrilled to be able to attend it again.
Update: My presentation on keysafe has also been accepted for the Security MiniConf at LCA, January 17th.
If your gpg key is too valuable for you to feel comfortable with backing it up to the cloud using keysafe, here's an alternative that might appeal more.
Keysafe can now back up some shares of the key to local media, and other shares to the cloud. You can arrange things so that the key can't be restored without access to some of the local media and some of the cloud servers, as well as your password.
For example, I have 3 USB sticks, and there are 3 keysafe servers. So let's make 6 shares total of my gpg secret key and require any 4 of them to restore it.
I plug in all 3 USB sticks and look at
mount to get the paths to them.
Then, run keysafe, to back up the key spread amoung all 6 locations.
keysafe --backup --totalshares 6 --neededshares 4 \ --add-storage-directory /media/sdc1 \ --add-storage-directory /media/sdd1 \ --add-storage-directory /media/sde1
Once it's done, I can remove the USB sticks, and distribute them to secure places.
To restore, I need at least one of the USB sticks. (If some of the servers are down, more USB sticks will be needed.) Again I tell keysafe the paths where USB stick(s) are mounted.
keysafe --restore --totalshares 6 --neededshares 4 \ --add-storage-directory /media/sdb1
Using keysafe this way, physical access to the USB sticks is the first level of defense, and hopefully you'll know if that's breached. The keysafe password is the second level of defense, and cracking that will take a lot of work. Leaving plenty of time to revoke your key, etc, if it comes to that.
I feel this is better than the methods I've been using before to back up my most important gpg keys. With paperkey, physical access to the printout immediately exposes the key. With Shamir Secret Sharing and manual distribution of shares, the only second line of defense is the much easier to crack gpg passphrase. Using OpenPGP smartcards is still a more secure option, but you'd need 3 smartcards to reach the same level of redundancy, and it's easier to get your hands on 3 USB sticks than 3 smartcards.
There's another benefit to using keysafe this way. It means that sometimes, the data stored on the keysafe servers is not sufficient to crack a key. There's no way to tell, so an attacker risks doing a lot of futile work.
If you're not using an OpenPGP smartcard, I encourage you to back up your gpg key with keysafe as described above.
Two of the three necessary keysafe servers are now in operation, and I hope to have a full complement of servers soon.
(This was sponsored by Thomas Hochstein on Patreon.)
My house entered full power saving mode with fall. Lantern light and all devices shutdown at bedtime.
But, it felt early to need to do this. Comparing with my logbook for last year, the batteries were indeed doing much worse.
I had added a couple of new batteries to the bank last winter, and they seemed to have helped at the time, although it's difficult to tell when you have a couple of good batteries amoung a dozen failing ones.
The bank was set up like this:
+---- house ---- | | +( 6v )-+( 6v )- | | +( 6v )-+( 6v )- | | +( 6v )-+( 6v )- | | +( 6v )-+( 6v )- | | +( 6v )-+( 6v )- | | +( new 12v )- | | +( new 12v )-
Tried as an experiement disconnecting all the bridges between the old 6v battery pairs. I expected this would mean only the new 12v ones would be in the circuit, and so I could see how well they powered the house. Instead, making this change left the house without any power at all!
On a hunch, I then reconnected one bridge, like this -- and power was restored.
+---- house ---- | | +( 6v )-+( 6v )- | | +( 6v ) ( 6v )- | | +( 6v ) ( 6v )- | | +( 6v ) ( 6v )- | | +( 6v ) ( 6v )- | | +( new 12v )- | | +( new 12v )-
My best guess of what's going on is that the wires forming the positive and negative rails are not making good connections (due to corrosion, rust, broken wires etc), and so batteries further down are providing less and less power. The new 12v ones may not have been able to push power up to the house at all.
(Or, perhaps having partially dead batteries hanging half-connected off the circuit has some effect that my meager electronics knowledge can't account for.)
So got longer cables to connect the new batteries directly to the house, bypassing all the old stuff. That's working great -- house power never dropped below 11.9v last night, vs 11.1v the night before.
The old battery bank might still be able to provide another day or so of power in a pinch, so I am going to keep them in there for now, but if I don't use them at all this winter I'll be recycling them. Astounding that those batteries were in use for 20 years.
After a month of development, keysafe 0.20160922 is released, and ready for beta testing. And it needs servers.
With this release, the whole process of backing up and restoring a gpg secret key to keysafe servers is implemented. Keysafe is started at desktop login, and will notice when a gpg secret key has been created, and prompt to see if it should back it up.
At this point, I recommend only using keysafe for lower-value secret keys, for several reasons:
- There could be some bug that prevents keysafe from restoring a backup.
- Keysafe's design has not been completely reviewed for security.
- None of the keysafe servers available so far or planned to be deployed soon meet all of the security requirements for a recommended keysafe server. While server security is only the initial line of defense, it's still important.
Currently the only keysafe server is one that I'm running myself. Two more keysafe servers are needed for keysafe to really be usable, and I can't run those.
If you're interested in running a keysafe server, read the keysafe server requirements and get in touch.
An interesting side problem in keysafe's design is that keysafe servers, which run as tor hidden services, allow anonymous data storage and retrieval. While each object is limited to 64 kb, what's to stop someone from making many requests and using it to store some big files?
The last thing I want is a git-annex keysafe special remote. ;-)
I've done a mash-up of three technologies to solve this, that I think is perhaps somewhat novel. Although it could be entirely old hat, or even entirely broken. (All I know so far is that the code compiles.) It uses proof of work, token buckets, and bloom filters.
Each request can have a proof of work attached to it, which is just a value that, when hashed with a salt, starts with a certain number of 0's. The salt includes the ID of the object being stored or retrieved.
The server maintains a list of token buckets. The first can be accessed without any proof of work, and subsequent ones need progressively more proof of work to be accessed.
Clients will start by making a request without a PoW, and that will often succeed, but when the first token bucket is being drained too fast by other load, the server will reject the request and demand enough proof of work to allow access to the second token bucket. And so on down the line if necessary. At the worst, a client may have to do 8-16 minutes of work to access a keysafe server that is under heavy load, which would not be ideal, but is acceptible for keysafe since it's not run very often.
If the client provides a PoW good enough to allow accessing the last token bucket, the request will be accepted even when that bucket is drained. The client has done plenty of work at this point, so it would be annoying to reject it. To prevent an attacker that is willing to burn CPU from abusing this loophole to flood the server with object stores, the server delays until the last token bucket fills back up.
So far so simple really, but this has a big problem: What prevents a proof of work from being reused? An attacker could generate a single PoW good enough to access all the token buckets, and flood the server with requests using it, and so force everyone else to do excessive amounts of work to use the server.
Guarding against that DOS is where the bloom filters come in. The server generates a random request ID, which has to be included in the PoW salt and sent back by the client along with the PoW. The request ID is added to a bloom filter, which the server can use to check if the client is providing a request ID that it knows about. And a second bloom filter is used to check if a request ID has been used by a client before, which prevents the DOS.
Of course, when dealing with bloom filters, it's important to consider what happens when there's a rare false positive match. This is not a problem with the first bloom filter, because a false positive only lets some made-up request ID be used. A false positive in the second bloom filter will cause the server to reject the client's proof of work. But the server can just request more work, or send a new request ID, and the client will follow along.
The other gotcha with bloom filters is that filling them up too far sets too many bits, and so false positive rates go up. To deal with this, keysafe just keeps count of how many request IDs it has generated, and once it gets to be too many to fit in a bloom filter, it makes a new, empty bloom filter and starts storing request IDs in it. The old bloom filter is still checked too, providing a grace period for old request IDs to be used. Using bloom filters that occupy around 32 mb of RAM, this rotation only has to be done every million requests of so.
But, that rotation opens up another DOS! An attacker could cause lots of request IDs to be generated, and so force the server to rotate its bloom filters too quickly, which would prevent any requests from being accepted. To solve this DOS, just use one more token bucket, to limit the rate that request IDs can be generated, so that the time it would take an attacker to force a bloom filter rotation is long enough that any client will have plenty of time to complete its proof of work.
This sounds complicated, and probably it is, but the implementation only took 333 lines of code. About the same number of lines that it took to implement the entire keysafe HTTP client and server using the amazing servant library.
There are a number of knobs that may need to be tuned to dial it in, including the size of the token buckets, their refill rate, the size of the bloom filters, and the number of argon2 iterations in the proof of work. Servers may eventually need to adjust those on the fly, so that if someone decides it's worth burning large quantities of CPU to abuse keysafe for general data storage, the server throttles down to a rate that will take a very long time to fill up its disk.
This protects against DOS attacks that fill up the keysafe server storage. It does not prevent a determined attacker, who has lots of CPU to burn, from flooding so many requests that legitimate clients are forced to do an expensive proof of work and then time out waiting for the server. But that's an expensive attack to keep running, and the proof of work can be adjusted to make it increasingly expensive.
With days beginning to shorten toward fall, my house is in initial power saving mode. Particularly, the internet gateway is powered off overnight. Still running electric lights until bedtime, and still using the inverter and other power without much conservation during the day.
Indeed, I had two laptops running cpu-melting keysafe benchmarks for much of today and one of them had to charge up from empty too. That's why the house power is a little low, at 11.0 volts now, despite over 30 amp-hours of power having been produced on this mostly clear day. (1 week average is 18.7 amp-hours)
September/October is the tricky time where it's easy to fall off a battery depletion cliff and be stuck digging out for a long time. So time to start dusting off the conservation habits after summer's excess.
I think this is the first time I've mentioned any details of living off grid with a bare minimum of PV capacity in over 4 years. Solar has a lot of older posts about it, and I'm going to note down the typical milestones and events over the next 8 months.
Three moments from earlier this week..
Sprawled under a tree after three hours of hiking with a heavy, water-filled pack, I look past my feet at six ranges of mountains behind mountains behind flowers.
From my campsite, I can see the rest of the path of the Appalachian Trail across the Roan balds, to Big Hump mountain. It seems close enough to touch, but not this trip. Good to have a goal.
Near sunset, land and sky merge as the mist moves in.
Keysafe securely backs up a gpg secret key or other short secret to the cloud. But not yet. Today's alpha release only supports storing the data locally, and I still need to finish tuning the argon2 hash difficulties with modern hardware. Other than that, I'm fairly happy with how it's turned out.
Keysafe is written in Haskell, and many of the data types in it keep track of the estimated CPU time needed to create, decrypt, and brute-force them. Running that through a AWS SPOT pricing cost model lets keysafe estimate how much an attacker would need to spend to crack your password.
(Above is for the password "makesad spindle stick")
If you'd like to be an early adopter, install it like this:
sudo apt-get install haskell-stack libreadline-dev libargon2-0-dev zenity stack install keysafe
~/.local/bin/keysafe --backup --store-local to back up a gpg key
I still need to tune the argon2 hash difficulty, and I need benchmark data to do so. If you have a top of the line laptop or server class machine that's less than a year old, send me a benchmark:
~/.local/bin/keysafe --benchmark | mail email@example.com -s benchmark
Bonus announcement: http://hackage.haskell.org/package/zxcvbn-c/ is my quick Haskell interface to the C version of the zxcvbn password strength estimation library.
PS: Past 50% of my goal on Patreon!
Have you ever thought about using a gpg key to encrypt something, but didn't due to worries that you'd eventually lose the secret key? Or maybe you did use a gpg key to encrypt something and lost the key. There are nice tools like paperkey to back up gpg keys, but they require things like printers, and a secure place to store the backups.
I feel that simple backup and restore of gpg keys (and encryption keys generally) is keeping some users from using gpg. If there was a nice automated solution for that, distributions could come preconfigured to generate encryption keys and use them for backups etc. I know this is a missing peice in the git-annex assistant, which makes it easy to generate a gpg key to encrypt your data, but can't help you back up the secret key.
So, I'm thinking about storing secret keys in the cloud. Which seems scary to me, since when I was a Debian Developer, my gpg key could have been used to compromise millions of systems. But this is not about developers, it's about users, and so trading off some security for some ease of use may be appropriate. Especially since the alternative is no security. I know that some folks back up their gpg keys in the cloud using DropBox.. We can do better.
I've thought up a design for this, called keysafe. The synopsis of how it works is:
The secret key is split into three shards, and each is uploaded to a server run by a different entity. Any two of the shards are sufficient to recover the original key. So any one server can go down and you can still recover the key.
A password is used to encrypt the key. For the servers to access your key, two of them need to collude together, and they then have to brute force the password. The design of keysafe makes brute forcing extra difficult by making it hard to know which shards belong to you.
Indeed the more people that use keysafe, the harder it becomes to brute-force anyone's key!
I could really use some additional reviews and feedback on the design by experts.