Posts about the Palm Pre

RSS
a tempest in a teapot

My sleepy midnight blog post on Palm Pre privacy was picked up by more than 60 news outlets. This is a postmortem of what such an experience is like from the inside.

I first learned that the media were picking up on the story when I got an email from a reporter wanting to do an interview. At the end he mentioned that he had just noticed my blog was submitted to slashdot. I spent half an hour doing an email interview, and by then the story was at the top of slashdot. So I had to switch gears to keeping my web server up.

I've been slashdotted several times before, and figured it would not be a problem since I have a static web site. And the server load was never a problem, but apache still slowed to a crawl. Turned out I had it configured to accept a very low number of simulantaneous clients. When I tried to set it to a more realistic value (like 150), apache crashed. I tried switching out apache2-mpm-worker for apache2-mpm-prefork, but that had the same problem. This apache problem was solved by the new apache2-mpm-event package, which handled slashdot with ease. Server response turned from molasses to snappy.

Two days later, apache has transmitted 6 gigabytes of data to 32 thousand curious visitors.

After reaching Slashdot, the story spread to lots of technical news sites. One of the better of these stories was on The Register, who actually interviewed me -- the only reporter to do so that day.

Overnight, the story spread from tech news sites to more established media, like the Washington Post, BBC, LA Times, MSNBC, and to the UK's Telegraph, Mirror, and Guardian. (None of these news sites contacted me.)

As the story filtered from site to site, I watched random wrong things accumulate in it -- such as me being called "Joey Hess, a mobile app developer". I knew the story had hit bottom when it reached The Inquirer, which referred to me as "A bogger (sic) called Josh (sic)".

By this time, the story came complete with parts of a press release from Palm, Inc, (which of course didn't address the true issue) and was sufficiently third-hand and vague as to do nothing but scare people.

Palm's stock price dropped that day, on news that they were scaling down their production plans. I doubt that was related.

On the second day I was interviewed by Wired. That day and today, the story spread to the non-English news in Austria, Turkey, Poland, France, Russia, Slovakia, Poland, Hungary, Italy, China, Hungary, Germany, Brazil, and The Netherlands.

At this point, the story seems to have died back in the news, and after all the noise, little has been accomplished. Palm insists there is no problem and that data collection can be turned off. I have not found a way to do so, without hacking WebOS.

Palm Pre privacy

I've been taking a closer look at the WebOS side of my Palm Pre tonight, and I noticed that it periodically uploads information to Palm, Inc.

The first thing sent is intended to be my GPS location. It's the same location I get if I open the map app on the Pre. Not very accurate in this case, but I've seen it be accurate enough to find my house before.

{ "errorCode": 0, "timestamp": 1249855555954.000000, "latitude": 36.594108, "longitude": -82.183260, "horizAccuracy": 2523, "heading": 0, "velocity": 0, "altitude": 0, "vertAccuracy": 0 }

Here they can tell every WebOS app I use, and for how long.

{ "appid": "com.palm.app.phone", "event": "close", "timestamp": 1250006362 }
{ "appid": "com.palm.app.messaging", "event": "launch", "timestamp": 1250006422 }
{ "appid": "com.palm.app.messaging", "event": "close", "timestamp": 1250006446 }

It sends the above info on a daily basis.

2009-08-10t09:15:10z    upload  /var/context/pending/1249895710-contextfile.gz.contextlog       ok      rdx-30681971
2009-08-11t09:15:10z    upload  /var/context/pending/1249982110-contextfile.gz.contextlog       ok      rdx-31306808

There is also some info that is recorded when a WebOS app crashes. Now, I've seen WebOS crash hard a time or two, but it turns out apps are crashing fairly frequently behind the scenes, and each such crash is logged and a system state snapshot taken. At least some of these are uploaded, though if things are crashing a whole lot it will be throttled.

2009-08-09T17:01:22Z    upload  /var/log/rdxd/pending/rdxd_log_59.tgz   OK      RDX-30246857
2009-08-09T17:05:36Z    upload  /var/log/rdxd/pending/rdxd_log_26.tgz   OK      RDX-30249465
2009-08-09T17:09:11Z    upload  /var/log/rdxd/pending/rdxd_log_56.tgz   OK      RDX-30252374
2009-08-09T17:11:46Z    upload  /var/log/rdxd/pending/rdxd_log_70.tgz   OK      RDX-30253958
2009-08-09T17:16:29Z    upload  /var/log/rdxd/pending/rdxd_log_67.tgz   ERR_UPLOAD_THROTTLED_DAILY      
2009-08-09T17:17:28Z    upload  /var/log/rdxd/pending/rdxd_log_51.tgz   ERR_UPLOAD_THROTTLED_DAILY      
2009-08-09T17:20:40Z    upload  /var/log/rdxd/pending/rdxd_log_21.tgz   ERR_UPLOAD_THROTTLED_DAILY

Each tarball contains a kernel dmesg, syslog, a manifest.txt listing all installed ipkg packages (including third-party apps), a backtrace of the crash, a df (from which they can tell I'm using Debian on the phone), and ps -f output listing all processes owned by root (but not by joey).

The uploading is handled by uploadd, which reads /etc/uploadd.conf:

[SERVER=rdx]
RepositoryURL=https://<HOST>/palmcsext/prefRequest?prefkey=APPLICATIONS,RDX_SRV
UploadURL=https://<HOST>/palmcsext/RDFileReceiver

[SERVER=context]
RepositoryURL=https://<HOST>/palmcsext/prefRequest?prefkey=APPLICATIONS,RDX_SRV
UploadURL=https://<HOST>//palmcsext/RDFileReceiver

The "HOST" this is sent to via https is ps.palmws.com.

My approach to disable this, which may not stick across WebOS upgrades, was to comment out the 'exec' line in /etc/event.d/uploadd and reboot. However, then I noticed a contextupload process running. This is started by dbus, so the best way to disable it seems to be: rm /usr/bin/contextupload

BTW, since Palm has lawyers, they have a privacy policy, which covers their ass fairly well regarding all this, without going into details or making clear that the above data is being uploaded.

Update: WebOS upgrades do re-enable the spyware; this has to be repeated after each upgrade.

Previously: Debian chroot on Palm Pre, debian desktop via vnc on the palm pre

debian desktop via vnc on the palm pre

On my Palm Pre, I'm running a PalmOS emulator, with a VNC client in it, connecting to a VNC server on the same phone, in order to get to an xterm. About the most roundabout way to get a prompt on a phone one can imagine, and also very wasteful of screen space because of that PalmOS emulator. I hope better methods become available.

  1. Install Classic. (This will only work for a week w/o being bought, BTW.)
  2. Login as root to Debian chroot on the Pre.
  3. Download PalmVNC: cd /media/internal/ClassicApps; wget http://palmvnc2.free.fr/stable/PalmVNC.prc
  4. apt-get install tightvncserver
  5. su to your non-root account on the Pre.
  6. Create a VNC password file containing an empty password by running: mkdir -p ~/.vnc; echo | vncpasswd -f > ~/.vnc/passwd; chmod 600 ~/.vnc/passwd
  7. Run tightvncserver -interface 127.0.0.1 -geometry 320x320
    (It only listens to localhost, so the empty password is ok (ish). The geometry is all that is visible on the Pre without scrolling.)
  8. Start Classic (or restart it).
  9. In Classic, select PalmVNC and tell it to connect to localhost, and change the display from 0 to 1. It will prompt for a password, just hit Ok for a blank one.
  10. Optionally, make VNC auto-start on boot, add to rc.local something like this: su joey -c 'cd; tightvncserver -interface 127.0.0.1 -geometry 320x320'

TODO:

  • Need way to send arrow keys!
  • xmonad configuration

Update: Here's another screenshot, of FBReader on the Pre. One of the missing apps on the Pre is a good offline e-book reader with no vendor lock-in. FBReader is what I use on my laptop, so it's great to have it on my phone. Especially since I have room for my entire e-book library on the Pre's generous flash drive.

Previously: Palm Pre, ssh to Debian chroot on Palm Pre

ssh to debian chroot on palm pre

Following setting up a Debian chroot on the Pre, I wanted to be able to ssh into the chroot. This sets that up, including starting Debian's ssh on boot.

Actually, I went a step further and hooked Debian's rc3.d init scripts into the Pre's webOS boot process, so all services installed in Debian will start on boot. So as a bonus, cron, at, etc will also work.

Since the Pre's EVDO IP address changes all the time, dynamic DNS is needed if you want to ssh in over EVDO. I used ez-ipupdate to handle that. (If you didn't want to ssh via EVDO, you could omit that part, and add "-i eth0" to both the firewall rules below.)

  1. Use novacom and the cable to get a root shell, and chroot /media/debian
  2. Install ssh: apt-get install openssh-server openssh-client
  3. Hook Debian init scripts into webOS startup:
     cat > /media/pre-root/etc/event.d/debian_start <<EOF
 # start Debian services after webOS boot is complete to avoid delaying that
 start on stopped finish
 script
     echo "Starting Debian services..."
     chroot /media/debian /etc/init.d/rc 2
 end script
 EOF
  4. Edit /etc/rc.local and add these commands to it:
     # Add firewall rule to allow SSH access on port 22,
 # both wifi and EVDO.
 iptables -D INPUT -p tcp --dport 22 -j ACCEPT || true
 iptables -I INPUT -p tcp --dport 22 -j ACCEPT
  5. If you want to easily be able to determine the phone's address while it's on wifi, you can use mDNS (hostname.local): apt-get install avahi-daemon
  6. Install ez-ipupdate: apt-get ez-ipupdate
  7. Follow the prompts to configure ez-ipupdate to use dyndns or your preferred dyanmic DNS service. When it asks if ppp is used, say yes.
  8. Edit /etc/ez-ipupdate/default.conf and make sure the interface is set to 'ppp0' (See Debian bug #536512 for why you need to do this currently.)
  9. Hook ez-ipupdate into webOS's ip-up script, so it is run whenever the address changes:
     HOOK=/media/pre-root/etc/ppp/ip-up.d/09ez-ipupdate-debian
 echo '#!/bin/sh' > $HOOK
 echo '/usr/sbin/chroot /media/debian /etc/ppp/ip-up.d/ez-ipupdate >/dev/null 2>&1 || true' >> $HOOK
 chmod a+x $HOOK
  10. Reboot the Pre. Now is a good time to learn that Orange+Sym+R on the keypad is equivilant to ctrl-alt-del. :-) The ssh daemon should be started automatically.

TODO:

  • Enable avahi for mdns for wifi.
  • Cleanly stop Debian services on shutdown. Problem with doing this is that I can't just run '/etc/init.d/rc 0', because that will kill every process in webOS too, and hang the phone. It'd need a very custom runlevel to be set up in Debian.
palm pre

Pre

I got a Palm Pre because I want a Linux smartphone that is not locked down (like the G1), on which I can play with Debian and even my own kernel. But I also want one that seems likely to be widely viable going forward, and there I have my doubts about OpenMoko, sadly.

(Or, I got a Palm Pre because I dreamt last night that I got one, and decided my subconcious must have made a decision after dithering for years. And because Madduck rules.)

Debian chroot on Palm Pre

This is a quick installation of a Debian chroot on the Palm Pre phone. Similar methods are explained at the Pre Dev Wiki. But they have you download a prebuilt Debian image and modify the Palm webOS system to install ssh; I preferred to use debootstrap and keep webOS changes to a minimum.

Also, since the Pre uses LVM, its media partition can be resized on the fly. That allows for a larger Debian filesystem, and I think is a nicer method than the loopback filesystem approach documented elsewhere, but do read the warnings about Palm, LVM, and upgrades here.

On the Pre:

  1. Enable dev mode by opening the Pre's Launcher and typing the following into it: upupdowndownleftrightleftrightbastart
  2. Plug the Pre into your laptop, tell it to charge via USB, while doing the following on the laptop.

On a Linux laptop:

  1. Run debootstrap to build an armel chroot: sudo debootstrap --foreign --arch=armel sid /tmp/debian
  2. Download novacom, cd to it, and build it: sudo apt-get install libusb-dev; make
  3. Transfer the armel chroot from the laptop to the Pre: (cd /tmp; sudo tar czv debian) | sudo ./novacom put file:///tmp/debian.tar.gz
  4. Run sudo ./novacom to get root shell on the Pre.
  5. In that shell, use LVM to resize the existing /media/internal volume down to 1 GB, and create a new 6 GB one for Debian:
     cp -a /media/internal /opt
 umount /media/internal
 lvresize -L 1G /dev/mapper/store-media
 mkdosfs -F 32 /dev/mapper/store-media
 mount /media/internal
 mv /opt/internal/ /opt/internal/. /media/internal
 rmdir /opt/internal
 lvcreate -l 100%FREE -n debian store
 mkfs.ext3 /dev/store/debian
 mkdir /media/debian
 echo "/dev/mapper/store-debian /media/debian auto noatime 0 0" >> /etc/fstab
 mount /media/debian
  6. Then in the Pre's root shell, you can unpack Debian, set up some bind mounts to allow accessing the Pre's filesystems from inside Debian, and chroot in to finish debootstrap:
     cd /media
 tar zxvf /tmp/debian.tar.gz
 mkdir /media/debian/media/pre-root /media/debian/media/internal
 echo "/ /media/debian/media/pre-root bind defaults,bind 0 0" >> /etc/fstab
 echo "/var /media/debian/media/pre-root/var bind defaults,bind 0 0" >> /etc/fstab
 echo "/var/log /media/debian/media/pre-root/var/log bind defaults,bind 0 0" >> /etc/fstab
 echo "/tmp /media/debian/tmp bind defaults,bind 0 0" >> /etc/fstab
 echo "/proc /media/debian/proc bind defaults,bind 0 0" >> /etc/fstab
 echo "/dev /media/debian/dev bind defaults,bind 0 0" >> /etc/fstab
 echo "/dev/pts /media/debian/dev/pts bind defaults,bind 0 0" >> /etc/fstab
 echo "/sys /media/debian/sys bind defaults,bind 0 0" >> /etc/fstab
 echo "/media/internal /media/debian/media/internal bind defaults,bind 0 0" >> /etc/fstab
 mount -a
 chroot debian
 /debootstrap/debootstrap --second-stage
 ln -sf /proc/mounts /etc/mtab
 ln -sf /media/pre-root/etc/resolv.conf /etc/resolv.conf
 apt-get clean
  7. Now customise the Debian chroot as usual.

How this could be made easier:

  • Make a debian package of novacom.
  • Installation script? debcoexist? d-i?

TODO:

  • Install openssh in Debian and configure it to start at boot.
  • Set up VNC, install VNC app in webOS to allow accessing a Debian desktop.