state of testing's security

Now that we're in the freeze, I can kind of get a taste of what it must be like to work on fixing security holes in stable, and can try to compare this with my half a year of working on the security of testing.

We have the testing-proposed-updates queue, which allows security fixes to be built directly against testing, so currently the main thing everyone has thought blocks packages from testing -- getting dependencies of security fixes into testing -- is not a problem (well, mostly; not all security fixes go through t-p-u; but if one doesn't and is blocked by something, we can do it through t-p-u instead).

To counterbalance this, we're in a freeze, all changes should be minimal to fix a bug or security hole, and they have to be reviewed by a member of the release team. I've been doing a lot of reviewing of security fixes, as well as a lot of work tracking down and applying backports of security fixes for holes that are fixed in new upstream releases that we cannot accept into testing due to their other changes.

Watching the list of security issues in testing has been interesting. Since the freeze began, it's been near a high water mark for holes fixed in unstable but not testing.

A lot of this is due to a few massive security hole sources such as ethereal and the linux kernel, for which it's been difficult to get security fixes backported, autobuilt on all arches (t-p-u still has some autobuilder issues apparently), and into testing. Factoring those out halves the number of holes. If we were not in a freeze, most of these holes would already be fixed in testing.

The number of completly unfixed holes has been holding stready from before the freeze, or possibly going up a bit. I think this number is more correlated to the severity of security holes, and the frequency of incoming holes than anything else, and the freeze has not seemed to change the way the numbers are moving for this one.

It is suprising that release critical bug fixing efforts have not dropped it more, but many of the unfixed holes are actually minor security holes that are (wrongly or rightly) not considered release critical. Then too, the efforts of many of us who would be working on these holes has been redirected to the backporting and patch reviewing activities I mentioned.

I probably don't have enough experience and information to say for sure, but my gut feeling is that it's quite a bit harder to manage security fixes for a frozen release than it is for regular testing. The ease of getting most security fixes into testing, even if we have to push in significant upstream changes to do it, seems to outweigh the issues with unfullfilled dependencies blocking propigation of security fixes to testing.

With that said, the t-p-u (or testing-security) queue is a very good thing and after sarge is released I really hope we can still have one to use for pushing in targeted security fixes when there are dependency issues. Best of both worlds..

Anyway, I hope that the stable security team can find some time to work on security issues in testing now that it's frozen, as they promised they'd be able to do. We can certianly use their expertise with backporting and the other kinds of problems that come from doing security support for a frozen distribution.

(If you found this at all interesting, you might enjoy my talk at the next DebConf. If I ever find the time to work on it.)