This is my discussion blog. The way it works is that when any pages in this wiki have a discussion page created for them, the discussion pages show up below. Also, any comments on my blog posts also show up here.

Spam filtering?

I wonder if your comment tripped a spam filter, rather than a coverup? 'Google "keywords" for more' is a phrase I see in a lot of comment-spam I get where they're trying to avoid rules that quarantine comments with links in them by instead including carefully selected search keywords which will turn up only the target site.

Your information about censored Amazon review of Sandisk Ultra 32GB Micro SDHC Card is very helpful, thanks! You could get it much higher in search results with a search results optimization service, Google "Wolfgang's Amazing Search Trick" for more information.

Comment by wolfgangmcq
re: udisks2

The documentation (quoted below) is a bit vague about exactly what it does, but it seems like does more than just disabling the port and probably does better at flushing caches than just unmounting.

power-off Arranges for the drive to be safely removed and powered off. On the OS side this includes ensuring that no process is using the drive, then requesting that in-flight buffers and caches are committed to stable storage. The exact steps for powering off the drive depends on the drive itself and the interconnect used. For drives connected through USB, the effect is that the USB device will be deconfigured followed by disabling the upstream hub port it is connected to. Note that as some physical devices contain multiple drives (for example 4-in-1 flash card reader USB devices) powering off one drive may affect other drives. As such there are not a lot of guarantees associated with performing this action. Usually the effect is that the drive disappears as if it was unplugged.

Comment by pabs3
re: udisks2

@pabs3, udisksctl power-off --block-device /dev/disk/by-label/passport does cause udev to remove the device file, it does not seem any better than udevadm trigger --action=remove in this situation though, because systemd has already unmounted the disk before that runs.

The only difference I notice is that it disconnects the hub port, but that leaves it powered on, so uhubctl is still needed to power off.

Unless it's better at getting caches flushed to the disk or something like that?

Comment by joey
udisks2?
Have you considered using udisksctl power-off to turn off the USB drives safely before shutting off power to their ports? That uses the same mechanisms as the "safely eject" GUI options and I think that would safer and might solve the timing issues.
Comment by pabs3
comment 2

f3 does not detect the card as a fake. Although it did have a write failure half way through the scan, perhaps due to overheating. Or due to a low quality though full capacity flash chip used in a fake.

And I could be wrong about that, despite the other indications. But a reviewer can of course be wrong about anything, that's not a reason to censor their review.

Comment by joey
Re: Relay questions

Yes, it's a good old fashioned relay, but it's under the computer's control.

For example, this relay https://www.sparkfun.com/products/13815 can be controlled by a GPIO port, consumes 0.6 watts of power to run and can switch 15 amps of AC current. I'm happily using several of them for other projects.

My USB hub is actually powered by 24V DC, which comes from my solar charge controller's load port, which is also switched by computer control. That line also powers a more beefy 24V industrial relay that I had lying around, which can switch a lot more power but does consume 5 watts when run. I'll probably downgrade that relay at some point but an extra 5 watts when the drives are running is not a big deal.

Comment by joey
Relay questions

When you mention relay, do you mean the electromechanical devices? If so, do you measure the current consumption of the relay coil when it's energized?

Also, what is the relay doing for you if it's manually actuated? Is it in place for future automation? It seems the relay could otherwise safely be a regular mechanical switch in this relatively low power application.

Comment by max
comment 1

Those things are a plague... It's surprising how many cards are fake! I've been maintaining the f3 package in Debian for a bit now and I've tested it against a few keys. Some show up as fake, but I'm pretty sure it misses a few. I'm curious to see what it would make of your key.

This is one of the reasons I don't buy at Amazon (anymore): on top of horrible worker conditions, returns are hard or impossible to carry out. At the corner store, I can and do go back and tell them their stuff is crap, and they actually care about fixing that because customers are in their face and won't come back if they don't fix it. :)

Comment by anarcat
comment 10

@grawity hmm, that's promising, the sysfs uvent file did not trigger udev, but "udevadm trigger --action=remove /sys/class/block/sdb1" does remove links to it in /dev/disk/by-label/, though /dev/sdb1 remains present.

And yeah, systemd's dependency on the device does work properly then, it delays mounting until the drive has spun up.

Ah, even easier, "udevadm trigger --action=remove /dev/disk/by-label/passport" does the same thing without needing to find the sysfs path.

So, I've changed the service file for the hub port to look like this:

[Unit]
Description=Startech usb hub port 4
PartOf=media-joey-passport.mount
[Service]
Type=oneshot
RemainAfterExit=true
ExecStart=/usr/sbin/uhubctl -a on -p 4
ExecStop=/bin/sh -c 'uhubctl -a off -p 4 ; udevadm trigger --action=remove /dev/disk/by-label/passport || true'

/bin/sh actually needed now since a failure of udevadm due to the label not existing etc needs to be ignored.

Kind of ugly that the service file for the hub port needs to know the label of the disk, but since I'm generating these service files not using systemd's limited templates, but from Haskell code, it was easy to extend to include that.

Comment by joey
With unison

Add that to your authorized_keys file on the host to restrict usage of the key to unison:

# Look at manpage sshd(8) for more information on options
command="unison -server",restrict ssh-rsa ...the key...
Comment by cassou
comment 9

udev keeps the disk device files in place. Attempting to access the device then leads to an IO error, and then udev notices the device is gone and removes its files

Udev maintains /dev/disk/* symlinks and systemd's .device units, but it relies 100% on receiving the kernel's uevents to trigger the maintenance. (For that matter, the actual /dev/sd* nodes are created by the kernel itself via devtmpfs, not by udev anymore.)

Therefore that's a kernel problem. (Or... maybe just the way uhubctl works? Does the kernel even know that the port is now powered off? I think it doesn't, because uhubctl sort of bypasses the whole USB stack.)

I wonder what happens if you try to fake a kernel uevent using udevadm trigger --action=remove /sys/<sysfs_path> after the poweroff. (Or echo remove > /sys/<sysfs_path>/uevent.) That won't remove the device from kernel, just poke udev about its supposed removal.

https://www.spinics.net/lists/linux-usb/msg157413.html

Aha. That's unfortunate. (Even though "the API can always be enlarged" I wouldn't hold my breath...)

Comment by grawity
comment 8

In systemd unit this line probably doesn't mean what you seem to think it does

No, it means exactly that. Multiple commands in ExecStart= separated by a standalone ; token have been supported for a very long time. (Though they don't always make sense logically – e.g. if you have Type=forking, which command is supposed to be the daemon? why aren't they ExecStartPre's instead? – but in a Type=oneshot this usage is fine.)

Comment by grawity
comment 6

@grawity, I tried the Requires on the device label, but it does not avoid the problem:

root@honeybee:/etc/systemd/system>ls /media/joey/archive-12
ls: cannot open directory '/media/joey/archive-12': No such device

Sep 09 13:33:50 honeybee systemd[1]: Starting Startech usb hub port 3...
Sep 09 13:33:50 honeybee uhubctl[23224]: Current status for hub 2-1 [0409:005a, USB 2.00, 4 ports]
Sep 09 13:33:50 honeybee uhubctl[23224]:   Port 3: 0000 off
Sep 09 13:33:50 honeybee uhubctl[23224]: Sent power on request
Sep 09 13:33:50 honeybee uhubctl[23224]: New status for hub 2-1 [0409:005a, USB 2.00, 4 ports]
Sep 09 13:33:50 honeybee uhubctl[23224]:   Port 3: 0101 power connect [1058:25ee]
Sep 09 13:33:50 honeybee systemd[1]: Started Startech usb hub port 3.
Sep 09 13:33:50 honeybee systemd[1]: Mounting archive-12...
Sep 09 13:33:50 honeybee systemd[1]: media-joey-archive\x2d12.mount: Mount process exited, code=exited status=32
Sep 09 13:33:50 honeybee systemd[1]: media-joey-archive\x2d12.mount: Failed with result 'exit-code'.
Sep 09 13:33:50 honeybee systemd[1]: Failed to mount archive-12.

Ah, I think I see why. After the usb port is shut down, udev keeps the disk device files in place. Attempting to access the device then leads to an IO error, and then udev notices the device is gone and removes its files. So systemd's dependencies are working, but it's seeing stale information.

Perhaps this is a bug in udev, that it's not noticing the usb port powered down. It seems to not get any kernel event for it at all, according to udevadm monitor.

I thought maybe a different workaround would be to dd one byte from the disk after powering it down, or udevadm trigger the device manually. Unfortunately, the kernel auto-powers the usb port back up when either of those is done. So I'll stick with the sleep for now..

Comment by joey
re: systemd unit syntax error

No, my syntax is correct and works. According to systemd.syntax(7):

Empty lines and lines starting with "#" or ";" are ignored, which 
may be used for commenting.

Nothing about use of those characters further into a line.

(Some of systemd's docs do contain examples of using sh -c and escaping semicolons, so perhaps this has changed since earlier versions. Multiple ExecStart lines is another way to run multiple commands.)

But in fact, the example I gave works. You can see the 20 second delay in the journal:

Sep 09 12:32:24 honeybee systemd[1]: Starting Startech usb hub port 4...
Sep 09 12:32:26 honeybee uhubctl[18819]: Current status for hub 2-1 [0409:005a, USB 2.00, 4 ports]
Sep 09 12:32:26 honeybee uhubctl[18819]:   Port 4: 0000 off
Sep 09 12:32:26 honeybee uhubctl[18819]: Sent power on request
Sep 09 12:32:26 honeybee uhubctl[18819]: New status for hub 2-1 [0409:005a, USB 2.00, 4 ports]
Sep 09 12:32:26 honeybee uhubctl[18819]:   Port 4: 0101 power connect [0480:0200]
Sep 09 12:32:46 honeybee systemd[1]: Started Startech usb hub port 4.
Comment by joey
systemd unit syntax error

In systemd unit this line probably doesn't mean what you seem to think it does:

ExecStart=/usr/sbin/uhubctl -a on -p 4 ; /bin/sleep 20

"/bin/sleep 20" there is probably irrelevant - a comment in an ini file.

Unless debian patches systemd that way, whole ExecStart= line isn't passed to shell anyway, so if that part was passed to command, it'd probably just exit on parsing options.

Something like this should work though:

ExecStart=/bin/bash -c "uhubctl -a on -p 4 ; /bin/sleep 20"
Comment by mk.fraggod
comment 2
As a side note, it sounds like you have several near-identical "hub-port-X" units (or at least they will become near-identical if the previous comment's suggestion with StopWhenUnneeded=true works out). These could be collapsed into a template startech-usb-hub@.service which uses %i in place of the port number everywhere.
Comment by grawity
comment 1

Seems there ought to be a way to declare an additional dependency and avoid needing that sleep?

Your .mount unit can use Requires=dev-disk-by\x2dlabel-passport.device.

(Actually I was quite sure that .mount units automatically depend on their What= device in any case, but I guess only an After= is implicit but a Requires= isn't.)

The combination of PartOf

I think it would be simpler to use StopWhenUnneeded=yes in the .service (once you have the aforementioned Requires=dev-disk in your .mount unit). Such a combination might be more reliable as well.

Comment by grawity
Thank you

Hi. I have Planet Debian in my feedreader, which is where I came across your blog. My wallpaper right now is a sunset taken from Roan Mountain. I don't know where I found the picture. It looks like a beautiful place. Thanks for posting it.

Ed H

Comment by edhamilton9