Friday, March 25, 2011

Debian ftpmaster Meeting — The wanna-build/buildd part

I've joined the Debian ftpmaster team in the Linuxhotel in Essen-Horst and so far my coding/hacking has been quite productive (it wasn't on dak after all). Linuxhotel has both a nice working and holiday atmosphere. Albeit I'm not taking much time off anyway.
  • Reenabled mipsel d-i autobuilding. (#618989)
  • Added support to filter the buildd overview pages by out-of-date/uncompiled. (#555527)
  • Adapted the wanna-build triggers (i.e. the scripts that import an archive into wanna-build and which are called by dak instances, for instance) to not start processing immediately but flag that a push happened. The real work is then done by a cronjob that loops through the various flags until there's nothing to do anymore. That avoids losing triggers on the way due to locking. (#602841)
  • All buildds (regardless whether they are running lenny or squeeze) are now running sbuild/buildd 0.61.0. Of course there are quite some patches on top of the upstream version. Packages are available in our repository.
  • Autosigning: adjusted buildd to pass a keyid to sbuild and to arrange for the then-signed .changes to be uploaded (configurable per dist in .builddrc); this involved some hackery in sbuild to actually cope correctly with a keyid passed on the CLI and to sign the package at the right time in the build process
  • Updated the unit tests of the build log importer: mocking more objects (especially the PostgreSQL log database; the tests were broken ever since pkg_history was added as a table) and testing that the actual content we write to disk matches up with our expectations
  • Added support for MIME encoded build logs to the build log importer. The log is still transmitted by mail from the buildd to the admins/security team and to the central log host. However it's now gzip-compressed, which shouldn't cause "this mail is too big" bounces anymore and also save some unneeded traffic for our buildd host sponsors. Furthermore .changes files are now attached to the mail instead of placed somewhere within the log, so it's also easier to sign packages without relying on regular expressions identifying the right portion within the log.
  • Added initial support for arch:all autobuilding to the database, wanna-build and buildd. The merging still needs more thinking as the cases in which an arch:all needs to be built still need to be determined. (Also it needs a Packages file for all the arch:all packages in a suite because it's not guaranteed that the newest arch:all is listed in any of the arch-specific Packages files.)
  • Adjusted my own scripts for build processing (which are used by a few others) to at least ignore autosigned logs.  It still needs to grow deMIME abilities, though.
Autosigning will get its own posting later on, unless Joerg gets there first.  There is currently one buildd (zandonai/s390) that has working autosigning for all suites on ftp-master (but not for security, backports or edu).  More will be added in the next days.

1 comment:

  1. Yay for auto-signing :) Thanks for working on this and getting it finally implemented.