tag:blogger.com,1999:blog-50488904635143042082024-03-19T08:54:30.512+01:00Philipp Kern's Debian blogPhilipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.comBlogger46125tag:blogger.com,1999:blog-5048890463514304208.post-4468952190276162412020-08-23T08:53:00.001+02:002020-08-23T08:53:04.177+02:00Self-service buildd givebacks now use Salsa auth<p>As client certificates are on the way out and Debian's SSO solution is effectively not maintained any longer, I switched self-service buildd givebacks over to Salsa authentication. It lives again at <a href="https://buildd.debian.org/auth/giveback.cgi">https://buildd.debian.org/auth/giveback.cgi</a>. For authorization you still need to be in the "debian" group for now, i.e. be a regular Debian member.</p><p>For convenience the <a href="https://buildd.debian.org/status/">package status</a> web interface now features an additional column "Actions" with generated "giveback" links.</p><p>Please remember to file bugs if you give builds back because of flakiness of the package rather than the infrastructure and resist the temptation to use this excessively to let your package migrate. We do not want to end up with packages that require multiple givebacks to actually build in stable, as that would hold up both security and stable updates needlessly and complicate development.<br /></p>Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Munich, Germany48.1351253 11.581980519.824891463821153 -23.5742695 76.445359136178837 46.7382305tag:blogger.com,1999:blog-5048890463514304208.post-15193359350995547602019-12-09T21:19:00.000+01:002019-12-09T21:19:54.185+01:00Voting for systemdI have <a href="https://www.debian.org/vote/2019/vote_002">voted</a> putting <a href="https://www.debian.org/vote/2019/vote_002#textf">Proposal F</a> first, <a href="https://www.debian.org/vote/2019/vote_002#textb">Proposal B</a> second and everything else after Further Discussion. I think if something truly better than systemd comes around, people in Debian will not stand in the way of people making it work - even despite this GR passing. That's how systemd started out after all. But the fact that we also want to support inferior old ways holds us back.<br />
<br />
At the point where Debian decided on the question of upstart vs. systemd, to me upstart was not a valid contender anymore. I had to deal professionally with it and no-one really know how to hold it in the right way so that modifications to upstart jobs did not break the boot. For systemd - despite the complexity everyone mentions as the problem - I only recall one major one where a certain machine type did not boot anymore and that was actually due to a regression in udev. If we would be discussing this as we debate the next serious contender to systemd, I would vote differently. Proposal F, being a statement about development <i>du jour</i>, thus makes sense to me. We should aim for the best possible integration with what we have. While I acknowledge that others try to keep old things working, they should also not hold the distribution hostage to a lofty "only if you provide two implementations you can use a feature". A distribution is always opinionated. We pick a default C library, a default compiler, a default desktop environment, a default installer. Some people prefer to diverge from that and within boundaries we support this. To be honest, as an example: If sysvinit had active maintainers, they could come up with technical solutions to provide missing init scripts from sysvinit. systemd started off no differently, overriding some of the init scripts using unit files. There are only few points of centralized contention that everyone needs to agree on (for instance as a pre-depends of the init metapackage). Most notably the systemd maintainers carefully worked through all of these issues of being a parallel init system and did not pressure others (again maybe except for few centralized touchpoints). <br />
<br />
A distribution without deep systemd integration is just unappealing to me at this point. It's a well working system with <a href="https://manpages.debian.org/unstable/systemd/systemd.1.en.html">excellent documentation in the form of manpages</a> that's easy enough to coerce to do my bidding in a non-surprising way. Limiting ourselves to units autogenerated from init scripts is not cool and just makes the whole thing harder to debug as there are non-obvious moving pieces in shell scripts. Plus we cannot actually benefit from the other improvements like hardening you can just flip on with a flag and declarative configuration of users and temporary files and directories. Let's not forget that there are still people trying to argue that a dependency on libsystemd is not ok - a library that effectively does not do anything when systemd is not active, while we accepted many other somewhat fringe libraries as core (libselinux) in an opinionated distribution. If the argument is that packages behave different under compilation with systemd support that is probably something that can be addressed with patches or those active maintainers need to come up with alternative approaches to spawn things the new way within their ecosystem. For instance by fixing inetd to do the right thing for socket-activated services.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0München, Deutschland48.1351253 11.58198049999998647.965637799999996 11.259256999999986 48.3046128 11.904703999999986tag:blogger.com,1999:blog-5048890463514304208.post-25765825286553428232019-08-21T00:54:00.000+02:002020-01-30T08:02:43.906+01:00Alpha: Self-service buildd givebacksBuilds on Debian's build farm sometimes fail transiently. Sometimes those failures are legitimate flakes, for instance when an in-progress build happens to exhaust its resources because of other builds on the same machine. Until now, you always needed to mail the buildd, wanna-build admins or the Release Team directly in order to get the builds re-queued.<br />
<br />
As an alpha trial I implemented self-service givebacks as a web script. As SSO for Debian developers is now a thing, it is trivial to add authentication in a way that a role account can use to act on your behalf. While at work this would all be an RPC service, I figured that a little CGI script would do the job just as well. So lo and behold, accessing <br />
https://auth.buildd.debian.org/giveback.cgi?pkg=<package>&suite=<suite>&arch=<arch>
with the right parameters set:<br />
<br />
<pre>You are authenticated as pkern. ✓
Working on package fife, suite sid and architecture mipsel. ✓
Package version 0.4.2-1 in state Build-Attempted, can be given back. ✓
Successfully given back the package. ✓
</pre>
<br />
Note that you need to be a Debian developer with a valid SSO client certificate to access this service.<br />
<br />
So why do I say <i>alpha</i>? We still expect Debian developers to act responsibly when looking at build failures. A lot of times there is a legitimate bug in the package and the last thing we would like to see as a project is someone addressing flakiness by continuously retrying a build. Access to this service is logged. Most people coming to us today did their due diligence and tried reproducing the issue on a porterbox. We still expect these things to happen but this aims to cut on the round-trip time until an admin gets around to process your request, which have been longer than necessary recently. We will audit the logs and see if particular packages stand out.<br />
<br />
There can also still be bugs. Please file them against <a href="https://bugs.debian.org/buildd.debian.org" rel="nofollow">buildd.debian.org</a> when you see them. Please include a copy of the output, which includes validation and important debugging information when requests are rejected. Also this all only works for packages in <i>Build-Attempted</i>. If the build has been marked as <i>Failed</i> (which is a manual process), you still need to mail us. And lastly the API can still change. Luckily the state change can only happen once, so it's not much of a problem for the GET request to be retried. But it should likely move to POST anyhow. In that case I will update this post to reflect the new behavior.<br />
<br />
Thanks to DSA for making sure that I run the service sensibly using a dedicated role account as well as WSGI and doing the work to set up the necessary bits.<br />
<br />
<b>Updated 2020-01-30: The script moved from buildd.debian.org/auth to auth.buildd.debian.org.</b>Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Munich, Germany48.1351253 11.58198049999998647.965637799999996 11.259256999999986 48.3046128 11.904703999999986tag:blogger.com,1999:blog-5048890463514304208.post-57846581825278822562016-12-29T23:50:00.001+01:002016-12-29T23:50:08.356+01:00Automating the installation of Debian on z/VM instancesI got tired of manually fetching installation images via either FTP or by manually transferring files to z/VM to test s390x installs. Hence it was about time to automate it. Originally I wanted to instrument an installation via <a href="https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lgdd/lgdd_r_vmcp_cmd.html">vmcp</a> from another instance on the same host but I figured that I cannot really rely on a secondary instance when I need it and went the <a href="https://packages.debian.org/sid/s3270">s3270</a>/<a href="http://x3270.bgp.nu/x3270-script.html">x3270-script</a> way instead.<br />
<br />
The <a href="https://gist.github.com/pkern/79e5e2ec3224381df8525fbfdd60351d#file-reinstall-sh">resulting script</a> isn't something I'm particularly proud of, especially as it misses error handling that really should be there. But this is not expect <span style="background-color: white; color: #252525; font-family: sans-serif; font-size: 14px;">–</span> instead you operate on whole screens of data and z/VM is not particularly helpful in telling you that you just completed your logon either. Anyway, it seems to work for me. It downloads the most recent stable or daily image if they are not present yet, uploads them via DFT to CMS and makes sure that the installation does not terminate when the script disconnects. Sadly DFT is pretty slow, so I'm stuck with 70 kB/s and about five minutes of waiting until kernel and initrd are finally uploaded. Given that installations themselves are usually blazingly fast on System z, I'm not too annoyed by that, though.<br />
<br />
I previously wrote about a <a href="http://debblog.philkern.de/2015/08/automating-3270-part-of-debian-system-z.html">parmfile example</a> that sets enough options to bring debian-installer to the point of a working network console via SSH without further prompting. It's a little unfortunate that s390-netdevice needs to be preseeded with the hardware addresses of the network card in all cases, even if only one is available. I should go and fix that. For now this means that the parmfile will be dependent on the actual VM system definition. With that in mind there is an <a href="https://gist.github.com/pkern/79e5e2ec3224381df8525fbfdd60351d#file-reinstall-example-sh">example script</a> in the same gist that writes out a parmfile and then calls the reinstall script mentioned above. Given that debian-installer now supports HTTPS (so far only in the daily images) you can even do a reasonably secure bootstrapping of the network console credentials and preseeding settings.<br />
<br />
If you put this pretty generic <a href="https://gist.github.com/pkern/79e5e2ec3224381df8525fbfdd60351d#file-preseed-s390x-cfg">preseed configuration file</a> onto a securely accessible webserver and reference it from the parmfile, you can also skip the more tedious questions at the beginning of debian-installer. A secure transport is encouraged as preseed files can do anything to your installation process. Unfortunately it seems that there is no way to preseed SSH keys for the resulting installation yet, neither for the created user nor for root. So I haven't achieved my desired target of a fully automated installation just yet. Debian's Jenkins setup just went with <a href="https://anonscm.debian.org/cgit/qa/jenkins.debian.net.git/tree/d-i-preseed-cfgs/debian_sid_daily_gnome_preseed.cfg#n102">insecure defaults</a>, but given that my sponsored VMs are necessarily connected to the public Internet that seemed like a bad idea to me. I suppose one way out would be to IP/password ACL the preseed file. Another one to somehow get SSH key support into <a href="https://anonscm.debian.org/cgit/d-i/user-setup.git/tree/">user-setup</a>.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Munich, Germany48.1351253 11.58198059999995247.965637799999996 11.259257099999951 48.3046128 11.904704099999952tag:blogger.com,1999:blog-5048890463514304208.post-71692120552419144222015-10-04T23:08:00.004+02:002015-10-04T23:08:50.313+02:00Root on LVM on Debian s390x, new Hercules<br />
<div>
Two s390x changes landed in Debian unstable today:</div>
<ul>
<li>A <a href="https://packages.qa.debian.org/z/zipl-installer/news/20151004T181857Z.html">new version of zipl-installer</a> that will <a href="http://anonscm.debian.org/cgit/d-i/zipl-installer.git/commit/?id=87176b057ef2ab61bb5bf420b38d4e0fd2484fc5">infer the root filesystem to pass to the kernel using mapdevfs</a> instead of enforcing a single DASD partition for it.</li>
<li>A <a href="https://packages.qa.debian.org/s/sysconfig/news/20151004T180438Z.html">new version of sysconfig-hardware</a> that contains a hook that activates all configured disks in the initramfs instead of a single one passed in as the root parameter in the kernel command-line. Thanks to Stephen Powell for the <a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=621080">patch</a>.</li>
</ul>
<div>
With this it should be possible to install Debian on s390x with root on LVM. I'd be happy to hear feedback about installations with any configuration, be it root on a single DASD or root on LVM. Unless you set both <span style="font-family: Courier New, Courier, monospace;">mirror/udeb/suite</span> and <span style="font-family: Courier New, Courier, monospace;">mirror/suite</span> to <span style="font-family: Courier New, Courier, monospace;">unstable</span> you'll need to wait until the changes are in testing, though. (The debian-installer build does not matter as zipl-installer is not part of the initrd and sysconfig-hardware is part of the installation.)</div>
<div>
<br /></div>
<div>
Furthermore I uploaded a new version of <a href="http://www.hercules-390.eu/">Hercules</a> - a z/Architecture emulator - to get a few more years of maintenance into Debian. See its <a href="http://www.hercules-390.eu/hercnew.html">upstream changelog</a> for details on the changes (old 3.07 → new 3.11).</div>
<div>
<br /></div>
<div>
At this point <a href="http://www.qemu.org/">qemu</a> at master is also usable for s390x emulation. It is much faster than Hercules, but it uses newfangled I/O subsystems like virtio. Hence we will need to do some more patching to make debian-installer just work. One patch for netcfg is in to <a href="http://anonscm.debian.org/cgit/d-i/netcfg.git/commit/?id=85e727bf5a21cb9af9ac36293d34a518cffa0ce1">support virtio networking correctly</a>, but then it forces the user to configure a DASD. (Which would be as wrong if Fibre Channel were to be used.) In the end qemu and KVM on s390x look so much like a normal x86 VM that we could drop most of the special-casing of s390x (netcfg-static instead of netcfg; network-console instead of using the VM console; DASD configuration instead of simply using virtio-blk devices; I guess we get to keep zIPL for booting).</div>
Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com4Munich, Germany48.1351253 11.58198059999995247.965637799999996 11.259257099999951 48.3046128 11.904704099999952tag:blogger.com,1999:blog-5048890463514304208.post-13981888308365980142015-09-19T21:45:00.001+02:002015-09-19T21:45:05.858+02:00Working with z/VM DIRMAINT as a mere userTwo helpful CMS commands to know if the users on the z/VM host you connect to are managed using DIRMAINT:<br />
<br />
<ul>
<li><span style="font-family: Courier New, Courier, monospace;">DIRMAINT PW</span>: This will ask you for your current password and then for the new one. This way you can change your own user password instead of having to ask the administrator for it.</li>
<li><span style="font-family: Courier New, Courier, monospace;">DIRMAINT REVIEW</span>: If you want to see the directory entry for your user (i.e. the configuration of your VM), you can issue this command. It will put the directory onto the card stack, from which you can retrieve it onto your A disk using <span style="font-family: Courier New, Courier, monospace;">RETRIEVE <number> <USERNAME> DIRECT A</span>. The number is listed after <span style="font-family: Courier New, Courier, monospace;">RDR FILE</span> in CP's output.</li>
</ul>
<div>
<span style="font-family: Courier New, Courier, monospace;">REVIEW</span> is something I always expire from my mind after a few weeks of not using a mainframe. And then I do not usually find it quickly, even knowing where to look. (For some reason <span style="font-family: Courier New, Courier, monospace;">GET</span> won't work unless you are a privileged user of the system.)</div>
<div>
<br /></div>
<div>
I guess by now most of the systems use DIRMAINT, but it's an IBM product that requires a separate license in addition to z/VM. Hence on some systems the user directory is still maintained by hand. In this case the password is written verbatim into the file and the administrator needs to change it manually for you.</div>
Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Munich, Germany48.1351253 11.58198059999995247.965637799999996 11.259257099999951 48.3046128 11.904704099999952tag:blogger.com,1999:blog-5048890463514304208.post-88801094455071621352015-08-30T19:36:00.002+02:002015-08-30T19:36:36.630+02:00Automating the 3270 part of a Debian System z installIf you try to install Debian on System z within z/VM you might be annoyed at the various prompts it shows before it lets you access the network console via SSH. We can do better. From within CMS copy the default EXEC and default PARMFILE:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">COPYFILE DEBIAN EXEC A DEBAUTO EXEC A</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: x-small;">COPYFILE PARMFILE DEBIAN A PARMFILE DEBAUTO A</span><br />
<br />
Now edit <span style="font-family: Courier New, Courier, monospace;">DEBAUTO EXEC A</span> and replace the DEBIAN in <span style="font-family: Courier New, Courier, monospace;">'PUNCH PARMFILE DEBIAN * (NOHEADER'</span> with DEBAUTO. This will load the alternate kernel parameters file into the card reader, while still loading the original kernel and initrd files.<br />
<br />
Replace <span style="font-family: Courier New, Courier, monospace;">PARMFILE DEBAUTO A</span>'s content with this (note the 80 character column limit):<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">ro locale=C </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">s390-netdevice/choose_networktype=qeth s390-netdevice/qeth/layer2=true </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">s390-netdevice/qeth/choose=0.0.fffc-0.0.fffd-0.0.fffe </span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;">netcfg/get_ipaddress=</span><b style="font-family: 'Courier New', Courier, monospace; font-size: x-small;"><IPADDR></b><span style="font-family: 'Courier New', Courier, monospace; font-size: xx-small;"> netcfg/get_netmask=255.255.255.0 </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">netcfg/get_gateway=<b><GW></b> netcfg/get_nameservers=<b><FIRST-DNS></b> </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">netcfg/confirm_static=true netcfg/get_hostname=debian </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">netcfg/get_domain= </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">network-console/authorized_keys_url=http://www.kern.lc/id_rsa.pub </span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">preseed/url=http://www.kern.lc/preseed-s390x.cfg </span><br />
<br />
Replace <IPADDR>, <GW>, and <FIRST-DNS> to suit your local network config. You might also need to change the netmask, which I left in for clarity about the format. Adjust the device address of your OSA network card. If it's in layer 3 mode (very likely) you should set <span style="font-family: Courier New, Courier, monospace;">layer2=false</span>. Note that mixed case matters, hence you will want to <span style="font-family: Courier New, Courier, monospace;">SET CASE MIXED</span> in xedit.<br />
<br />
Then there are the two URLs that need to be changed. The <span style="font-family: Courier New, Courier, monospace;">authorized_keys_url</span> file contains your SSH public key and is fetched unencrypted and unauthenticated, so be careful what networks you traverse with your request (HTTPS is not supported by debian-installer in Debian).<br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">preseed/url</span> is needed for installation parameters that do not fit the parameters file - there is an upper character limit that's about two lines longer than my example. This is why this example only contains the bare minimum for the network part, everything else goes into this preseeding file. It file can optionally be protected with a MD5 checksum in <span style="font-family: Courier New, Courier, monospace;">preseed/url/checksum</span>.<br />
<br />
Both URLs need to be very short. I thought that there was a way to specify a line continuation, but in my tests I was unable to produce one. Hence it needs to fit on one line, including the key. You might want to use an IPv4 as the hostname.<br />
<br />
To skip the initial boilerplate prompts and to skip straight to the user and disk setup you can use this as <span style="font-family: Courier New, Courier, monospace;">preseed.cfg</span>:<br />
<br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">d-i debian-installer/locale string en_US</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">d-i debian-installer/country string US</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">d-i debian-installer/language string en</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">d-i time/zone US/Eastern</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><span style="font-size: xx-small;">d-i mirror/country manual</span></span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">d-i mirror/http/mirror string httpredir.debian.org</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">d-i mirror/http/directory string /debian</span><br />
<span style="font-family: Courier New, Courier, monospace; font-size: xx-small;">d-i mirror/http/proxy string</span><br />
<br /><div>
I'm relatively certain that the DASD disk setup part cannot be automated yet. But the other bits of the installation should be preseedable just like on non-mainframe hardware.</div>
Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Munich, Germany48.1351253 11.58198059999995247.965637799999996 11.259257099999951 48.3046128 11.904704099999952tag:blogger.com,1999:blog-5048890463514304208.post-52202801902591189012015-02-18T19:15:00.001+01:002015-02-18T19:15:32.644+01:00Caveats of the HP MicroServer Gen8If you intend to buy the HP MicroServer Gen8 as a home server there are a few caveats that I didn't find on the interwebs before I bought the device:<br />
<ul>
<li>Even though the main chassis fan is now fixed in AHCI mode with recent BIOS versions, there is still an annoying PSU fan that's tiny and high frequency. You cannot control it and the PSU seems to be custom-built.</li>
<li>The BIOS does not support ACPI S3 (suspend-to-RAM) at all. Apparently it being a server BIOS they chose to not include the code paths in the BIOS needed to properly turn off devices and turn them back on. This means that it's not possible to simply suspend it and have it woken up when your media center boots.</li>
<li>In contrast to the older AMD-based MicroServers the Gen8 comes with iLO, which will consume quite a few watts just for being present even if you don't use it. I read figures of about ten watts. It also cannot be turned off, as it does system management like fan control.</li>
<li>The HDD cages are not vibration proof or decoupled.</li>
</ul>
<div>
If you try to boot FreeBSD with its zfsloader you will likely need to apply a <a href="https://github.com/kuriyama/freebsd/compare/freebsd:stable/10...hp-bios-workaround-10.diff">workaround patch</a>, because the BIOS seems to do something odd. Linux works as expected.</div>
Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com4Munich, Germany48.1351253 11.58198059999995247.965637799999996 11.259257099999951 48.3046128 11.904704099999952tag:blogger.com,1999:blog-5048890463514304208.post-43505392913064088702014-10-14T00:28:00.001+02:002015-12-06T22:23:05.740+01:00pbuilder and pam_tmpdirIt turns out that my recent woes with pbuilder were all due to <a href="https://packages.debian.org/sid/libpam-tmpdir">libpam-tmpdir</a> being installed (at least two old bug reports exist about this issue: #<a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=576425">576425</a> and #<a href="https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=725434">725434</a>). I rather like my private temporary directory that cannot be accessed by other (potential) users on the same system. Previously I used a hook to fix this up by ensuring that the directory actually exists in the chroot, but somehow that recently broke.<br />
<br />
A rather crude but working solution seems to be "<span style="font-family: "courier new" , "courier" , monospace;">session required pam_env.so user_readenv=1</span>" in /etc/pam.d/sudo and "<span style="font-family: "courier new" , "courier" , monospace;">TMPDIR=/tmp</span>" in /root/.pam_environment. One could probably skip pam_tmpdir.so for root, but I did not want to start fighting with pam-auth-update as this is in /etc/pam.d/common-session*.<br />
<br />
<b>Update:</b> Roberto C. Sánchez comments that "<span style="font-family: Courier New, Courier, monospace;">export TMPDIR=/tmp</span>" in <span style="font-family: Courier New, Courier, monospace;">/root/.pbuilderrc</span> also works.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Munich, Germany48.1351253 11.58198059999995247.965637799999996 11.259257099999951 48.3046128 11.904704099999952tag:blogger.com,1999:blog-5048890463514304208.post-56392723781524356892013-11-29T20:14:00.001+01:002013-11-29T20:14:27.513+01:00On PDiffs Usefulness for Stable<a href="http://noone.org/blog/English/Computer/Debian/PDiffs%20are%20still%20useful.html">Axel just said</a> that PDiffs are really useful for stable because it changes seldom. The truth is that <a href="http://ftp.de.debian.org/debian/dists/stable/main/binary-amd64/Packages.diff/Index">there are no PDiffs for stable</a>. I consider this a bug because you need to download a huge blob on every point release even though only few stanzas changed.<br />
<br />
The real problem of PDiffs is the count that needs to be applied. We run "dinstall" much more often now than we used to and we cannot currently skip diffs in the middle when looking at a dak-maintained repository. It would be more useful if you could fetch a single diff from some revision to the current state and apply this instead of those (download, apply, hash check)+ cycles. 56 available index diffs are also not particularly helpful in this regard. They cover 14 days with four dinstalls each, but it takes a very long time to apply them, even on modern machines.<br />
<br />
I know that there was ongoing work improving PDiffs that wanted to use git to generate the intermediate diffs. The current algorithm used in dak does not require old versions except one to be kept, but keeping everything is the far other end of the spectrum where you have a git repository that potentially gets huge unless you rewrite it. It should be possible to just implement a rolling ring buffer of Packages files to diff against, reprepro already generates the PDiffs in the right way, the Index format is flexible enough to support this. So someone would just need to make that bit of work in dak.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Munich, Germany48.1351253 11.58198059999995247.965637799999996 11.259257099999951 48.3046128 11.904704099999952tag:blogger.com,1999:blog-5048890463514304208.post-92124776522246732232013-09-16T23:39:00.000+02:002013-09-16T23:39:04.733+02:00buildd.debian.org<p>Did you notice that it was gone? It's back now. Phew.</p>
<p>Apparently an upgrade to wheezy was scheduled for the passed week-end. This went badly with the RAID controller acting up after the upgrade. So the move to a new machine (or rather VM) was… sped up. And because various policies changed, we needed to move to a dedicated database server with a new authentication scheme, get rid of the /org symlink, get rid of the buildd_* accounts, cope with a new PHP, cope with rlimits that broke a Python CGI (thanks, KiBi!), triggers that weren't tickled, and various other stuff.</p>
<p>Thanks to DSA for handling the underlying infrastructure despite some communication issues during heavy fire fighting.</p>Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Munich, Germany48.1367203 11.57675399999993747.9672328 11.254030499999937 48.3062078 11.899477499999938tag:blogger.com,1999:blog-5048890463514304208.post-5402025696896564882013-03-03T23:44:00.002+01:002013-03-03T23:44:45.117+01:00git-annex: encrypted remotesDue to the <a href="http://debblog.philkern.de/2013/03/psa-lvm-pvmove-and-ssds.html">data loss I blogged about</a>, I had to reverse engineer the encryption used by <a href="http://git-annex.branchable.com/">git-annex</a> for its <a href="http://git-annex.branchable.com/encryption/">encrypted</a> <a href="http://git-annex.branchable.com/special_remotes/">special remotes</a>. The file system on which the content lived has a bullet hole of 8 GB in it, which was helpfully discarded by pvmove. It's pretty unhappy about that fact, parts of the git repository are unusable and directories cannot be accessed anymore. git-annex cannot possibly run anymore.<br />
<br />
However, I was still able to access the git-annex branch within said git repository (using porcelain). This branch contains a file called <span style="font-family: Courier New, Courier, monospace;">remote.log</span> which contains the keys of the special remotes. There's one per remote, encrypted to a GPG key of your choice and all files within that remote are encrypted with the same symmetric key.<br />
<br />
One small detail stopped me from getting the decryption right the first time, though. It seems that git-annex uses randomness generated by GPG and armored into base64. In my naïveté I spotted the base64 and decoded it. Instead it's used verbatim: the first 256 bytes as HMAC key (which reduces randomness to 192 bytes) and the remaining bytes for the symmetric key used by GPG (which will do another key derivation for <a href="http://en.wikipedia.org/wiki/CAST-128">CAST5</a> with it). A <a href="http://git-annex.branchable.com/bugs/encryption_key_is_surprising/">bug about that</a> just hit the git-annex wiki.<br />
<br />
With that knowledge in mind I wrote <a href="https://gist.github.com/pkern/5078559">a little tool</a> that's able to generate encrypted content keys from the plain ones used in the symlinks. That helps you to locate the file in the encrypted remote. Fetch it and then use the tool to decrypt the file in question with the right key.<br />
<br />
The lesson: Really backup the git repository used with git-annex and especially remote.log. I'm now missing most of the metadata but for some more important files it's luckily still present. Recovery of the file content does not depend on it if you can deduce the filename from the content. If you have many little files it might be a bit futile without it, though.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Karlsruhe, Germany49.009148 8.379944448.842521500000004 8.057220899999999 49.1757745 8.7026679tag:blogger.com,1999:blog-5048890463514304208.post-61315771024342183442013-03-02T15:43:00.001+01:002013-03-02T16:40:13.457+01:00PSA: LVM, pvmove and SSDsIf you use LVM with Wheezy on a solid-state drive, you really want to install the <a href="http://packages.qa.debian.org/l/lvm2/news/20130226T163916Z.html">latest lvm2 update</a> (i.e. 2.02.95-6, which contains the <a href="http://packages.qa.debian.org/l/lvm2/news/20121119T113246Z.html">changes of -5</a>). Otherwise, if you set <span style="font-family: Courier New, Courier, monospace;">issue_discards=1</span> in <span style="font-family: Courier New, Courier, monospace;">/etc/lvm/lvm.conf</span>, you will experience <a href="https://bugzilla.redhat.com/show_bug.cgi?id=832392">severe data loss when using pvmove</a>. Happened to me twice, once I didn't care (chroot data being lost), the second time (today) I did. Not fun, especially when the backup of the data was scheduled for the same day.<br />
<br />
One has to wonder why it takes three months for a bug that trashes data to reach testing. (Obviously I know the answer, but they're not particularly good reasons.) Other distributions, like Ubuntu, were much quicker to notice and incorporate that fix. And in the case of the named distribution not because they auto-synced it from unstable. <i>If somebody notices such a grave bug, please yell at people to get the fix out there to our users. Thanks.</i>Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Karlsruhe, Germany49.009148 8.379944448.842521500000004 8.057220899999999 49.1757745 8.7026679tag:blogger.com,1999:blog-5048890463514304208.post-38214939181136519162013-02-08T23:49:00.000+01:002013-02-08T23:49:25.790+01:00Mozilla's directionAm I the only one who's disappointed with the route Mozilla's taking and left wondering what the direction is? First they <a href="http://blog.lizardwrangler.com/2012/07/06/thunderbird-stability-and-community-innovation/">killed off the development of Thunderbird</a> because, as we all know, people mainly use webmail these days. Then they presented us their view that the big Certificate Authorities are <i>too big to fail</i>, as CAs gravely violated our trust (c.f. <a href="http://lwn.net/Articles/480279/">Trustwave and their MitM authority</a>). And "now" <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=600919">they're also blocking the introduction of new formats</a> into their browser because they cannot be the one who innovates. Instead Microsoft and Apple obviously need to take the lead in introducing a format into their browsers because otherwise it wouldn't be useful. Even though it's safe to say that Chrome and Firefox make up for more than half of the desktop browser market share. It might be that Chrome's nibbling from Firefox's, still IE seems to be in decline and Safari is rather a further mention than something many people would care strongly about.<br />
<br />
There were of course some valid reasons for not supporting WebP yet. But most of them got fixed in the meantime and all we hear is the referal to proprietary vendors who need to move first. If I'd want to depend on such vendors I'd go with proprietary operating systems. (Having to deal with hardware products of proprietary vendors at $dayjob is enough.) So what's up Mozilla? The solution is to ignore your users and tag bugs with patches <i>wontfix</i>?<br />
<br />
The only real advantage of Firefox over Chromium these days is the vast amount of plugins and extensions (e.g. <a href="http://5digits.org/pentadactyl/">Pentadactyl</a>, for which there is no real equivalent available). Another sad fact is that you need to pull Firefox from a 3rd party repository (even though packages are coming from the 2nd party) to get a current version onto your Debian system to work with the web. But then it's not Mozilla who's to blame here. Maybe we should've introduced one Iceweasel version that's allowed to have reverse-dependencies and one that cannot.<br />
<br />
(This post might contain hyberboles, which should be considered as such.)Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com3Neuried, Deutschland48.4489197 7.802360600000042748.2803987 7.4796371000000423 48.617440699999996 8.1250841000000431tag:blogger.com,1999:blog-5048890463514304208.post-39211883304903830512012-11-22T23:10:00.000+01:002012-11-23T00:27:12.616+01:00OpenClonkThe Free and Open successor to the <a href="https://en.wikipedia.org/wiki/Clonk">Clonk</a> computer game series <a href="http://www.openclonk.org/">OpenClonk</a> is now available in Debian sid and Ubuntu raring: just use <span style="font-family: "Courier New",Courier,monospace;">sudo apt-get install openclonk</span> to install it (backports to Ubuntu precise and quantal can be found in the <a href="https://launchpad.net/~openclonkdevteam/+archive/release">OpenClonk Releases PPA</a>).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen="allowfullscreen" frameborder="0" height="236" src="https://www.youtube.com/embed/ykJZ9WsevGE" width="420"><a href="https://www.youtube.com/watch?v=ykJZ9WsevGE">OpenClonk Beyond the Rocks Trailer</a></iframe></div>
<br />
It's still under development, but as you can see it's also already quite functional. The engine was made available freely under the ISC license after the original author of Clonk was no longer interested in creating new successors. Many of the old volunteer contributors then picked it up. Be aware that same computer multiplayer, which was available in the older parts of the series, is gone. If you play on the network, you need to ensure that both of you have the same version of the game.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Karlsruhe, Germany49.009148 8.379944448.674755000000005 7.7482304 49.343541 9.0116584tag:blogger.com,1999:blog-5048890463514304208.post-76709617484627106992012-10-02T15:04:00.000+02:002013-03-18T21:59:22.985+01:00How to maintain an OpenWRT installationI wondered how to properly maintain an <a href="https://openwrt.org/">OpenWRT</a> installation that needs to track trunk for one reason or another. For instance the OpenWRT release pace is quite slow and "new" devices or various fixes are only available in this particular branch.<br />
<br />
Now the OpenWRT project does offer <a href="http://downloads.openwrt.org/snapshots/trunk/">snapshot builds</a>. These are images built with the default config for the particular device type. You can even upgrade most models using <a href="http://wiki.openwrt.org/doc/howto/generic.sysupgrade">sysupgrade</a>, which preserves a list of configuration files you specify when flashing the new image, instead of doing a dreadful tftp upload. But you still lose all the packages you installed and accumulate a bit of cruft when reinstalling them. If you do not flash new images, you will be unable to install kernel modules after a while, because the kernel version they're based upon was updated.<br />
<br />
So what's suggested is this:<br />
<ul>
<li>You <a href="https://dev.openwrt.org/wiki/GetSource">checkout the trunk from svn</a>. You enable all the feeds you need (e.g. packages and luci) and install the packages you want. (See <span style="font-family: "Courier New",Courier,monospace;">scripts/feeds</span>.)</li>
<li>Run <span style="font-family: "Courier New",Courier,monospace;">make menuconfig</span> and select the target system type and the device profile.</li>
<li><span style="font-family: "Courier New",Courier,monospace;">make defconfig</span> will now give you the default configuration for the selected profile. It is important to use this as the starting point.</li>
<li>Again invoke menuconfig and select all the packages you need. You can get the current list of packages installed on your device with <span style="font-family: "Courier New",Courier,monospace;">opkg list_installed</span>. You can search in menuconfig as you would in a kernel build config screen.</li>
<li>If you're happy with the configuration, save the difference to the default configuration to a file: <span style="font-family: "Courier New",Courier,monospace;">scripts/diffconfig.sh > my-own-config</span></li>
<li>Now start the build with <span style="font-family: "Courier New",Courier,monospace;">make</span>. With the package configuration I picked I need about 5 GB of disk space. It will start off compiling a toolchain, the kernel and then the packages you selected. If something seems to go wrong, you probably want to restart the build with <span style="font-family: "Courier New",Courier,monospace;">make V=s</span> to see the build messages (the default is pretty quiet). Some <span style="font-family: "Courier New",Courier,monospace;">-jX</span> option for parallel building might shorten your waiting time.</li>
<li>Finally you will get an image in bin/<system type>. You want to grab the squashfs & sysupgrade image.</li>
<li>Check /etc/sysupgrade.conf on the live system if files are missing. /etc/config, passwd, shadow and some other files are preserved even if they are not listed. Those are then the only files that are copied into a temporary ramdisk during upgrade and written back to the filesystem post-flash. You need to keep this file up-to-date if you include new packages that have non-<a href="http://wiki.openwrt.org/doc/uci"><acronym title="Unified Configuration Interface">UCI</acronym></a> configuration.</li>
<li>For actually executing the upgrade process, you can either use the luci web interface or start sysupgrade over SSH. For the latter you will have to copy the image yourself. If you go via luci, you should check the md5 hash presented to you against the one of the image on your disk.</li>
<li>If you want to update your image, just <span style="font-family: "Courier New",Courier,monospace;">svn update</span> (in the hope that the OpenWRT developers did not break anything in trunk), get the new defconfig, append your saved configuration delta (the file "my-own-config" above) to .config and run <span style="font-family: "Courier New",Courier,monospace;">make oldconfig</span>. This will keep track of feature changes in the default configuration (like shadow password support). <b>Update 2013-03-18:</b> To avoid problems with unclean trees you should call <span style="font-family: Courier New, Courier, monospace;">make distclean</span> before running <span style="font-family: Courier New, Courier, monospace;">make defconfig</span> again. Make sure you have the config diff ready, because cleaning the tree will drop the <span style="font-family: Courier New, Courier, monospace;">.config</span>.</li>
</ul>
The advantage of building an image that includes all the packages is that you don't get to reinstall them and due to the use of squashfs, all files provided by those packages are actually kept in compressed form in flash. The image I built has a size of about 7 MB and contains "heavyweight" packages like isc-dhcpd, bind, openvpn (using openssl), bash, vim-tiny, tcpdump-mini, luci, and a webserver with TLS support.<br />
<br />
As always: Take great care when you're flashing devices. The devices I bought cannot be bricked, but this might not be true for all devices that run OpenWRT. You might need and go doing a tftp image upload if something goes wrong. Check the OpenWRT wiki for instructions and possibly make your computer ready for them before starting to play with your box. For my <a href="http://wiki.openwrt.org/toh/buffalo/wzr-hp-ag300h">Buffalo WZR-HP-AG300H</a> the sysupgrade procedure is stable.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com3Karlsruhe, Germany49.009148 8.379944448.925825 8.2220158999999988 49.092471 8.5378729tag:blogger.com,1999:blog-5048890463514304208.post-86288368734892056342012-09-23T23:31:00.001+02:002012-09-23T23:31:30.297+02:00 Call for testing: Upcoming Squeeze point release 6.0.6We just sent out a new <a href="http://lists.debian.org/debian-stable-announce/2012/09/msg00000.html">call for testing</a> for the next point release of <a href="http://www.debian.org/releases/squeeze/">Debian Squeeze</a>. The last one was back in May, hence there are a bunch of updates. Please test the packages in <i>squeeze-proposed-updates</i> on some machines running <i>squeeze</i> if possible, so that we don't screw up your production machines with bad updates in a week. The point release is scheduled for September 29th, i.e. next Saturday. Don't forget to copy the <a href="http://lists.debian.org/debian-release/">debian-release</a> mailing list when you encounter regressions. Thanks for your efforts.<br />
<br />
If you want to receive these notices by mail, please subscribe to the <a href="http://lists.debian.org/debian-stable-announce/">debian-stable-announce</a> mailing list.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Karlsruhe, Germany49.009148 8.379944448.925825 8.2220158999999988 49.092471 8.5378729tag:blogger.com,1999:blog-5048890463514304208.post-73972370239245236042012-09-22T00:04:00.000+02:002012-09-22T00:51:12.929+02:00IPv6 support in debian-installer, take 2The IPv6 patch set for <a href="http://packages.qa.debian.org/n/netcfg.html">netcfg</a> (part of <a href="http://www.debian.org/devel/debian-installer">debian-installer</a>) has <a href="http://packages.qa.debian.org/n/netcfg/news/20120917T210000Z.html">landed in Debian unstable</a>. In follow-up uploads I diverged from the Ubuntu patch set a little bit:<br />
<ul>
<li>I <a href="http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=commitdiff;h=be33e473cb752d4895d196a3d5a9a67b2487240a">dropped</a> the use of <a href="http://tools.ietf.org/html/rfc3315#section-9.4"><acronym title="DHCP Unique Identifier Based on Link-layer Address">DUID-LL</acronym></a> as the client identifier used by the <a href="http://tools.ietf.org/html/rfc3315">DHCPv6</a> client. While basically a good idea, because it's predictable, it's also against the RFC to do that. We simply don't know if the network interface is permanently attached to the device. If the <acronym title="Network Interface Card">NIC</acronym> that's used for DUID-LL generation is reused for installation in another box, the same identifier would be generated and stored on the machine. That's what <a href="http://tools.ietf.org/html/rfc3315#section-9.2"><acronym title="DUID Based on Link-layer Address Plus Time">DUID-LLT</acronym></a> (the default in other software) avoids. As much as I loathe the replacement of the current <acronym title="Media Access Control">MAC</acronym> address-based scheme of DHCPv4 with DUID-LLT, it seems to be more correct to use that instead of DUID-LL, because that identifier is designed to be decoupled from the presence of the NIC. If you want predictable addressing, just enable <acronym title="Stateless Address Autoconfiguration">SLAAC</acronym> in your broadcast domain and use the <a href="http://en.wikipedia.org/wiki/IPv6_address#Modified_EUI-64">EUI-64 address</a>, which already contains the current MAC address.</li>
<li>If SLAAC is used, netcfg will activate <a href="http://tools.ietf.org/html/rfc4941">IPv6 privacy extensions</a> in the installed system by default. They can be turned off by editing <span style="font-family: "Courier New",Courier,monospace;">/etc/network/interfaces</span> post-installation. This will only affect outgoing traffic, incoming traffic can be addressed at the stable EUI-64 address. (We don't use a randomized interface identifier like Windows does.)</li>
</ul>
Thanks to Michael Tokarev's <a href="http://packages.qa.debian.org/b/busybox/news/20120920T084755Z.html">recent upload of busybox</a>, we now also have a more featureful ping and ping6 in debian-installer's environment.<br />
<br />
Known bugs:<br />
<ul>
<li>As far as I can see, the DUID generated in the debian-installer environment is not copied into the installed system. Given the low prevalence of stateful DHCPv6 for servers I don't consider this a blocker, though.</li>
<li>If stateful DHCPv6 is requested by the router advertisement (<i>other config</i> flag being set), the DHCPv6 client will not time out and continuously retry to get an address.</li>
<li>According to <a href="http://bugs.debian.org/688273">Debian bug #688273</a> preseeding of <span style="font-family: "Courier New",Courier,monospace;">netcfg/use_autoconfig</span> does not work correctly. <span style="font-family: "Courier New",Courier,monospace;">netcfg/disable_dhcp</span> needs to be re-added as a deprecated preseeding option and mapped onto the <span style="font-family: "Courier New",Courier,monospace;">use_autoconfig</span> value until somebody comes up with a better scheme.</li>
<li>We should state in the installation guide that ftp.ipv6.debian.org and http.debian.net can both be used for IPv6-only installation. The installer does not show in the mirror list if a mirror is IPv6-capable.</li>
</ul>
If you could take another look at a <a href="http://d-i.debian.org/daily-images/">current d-i daily</a>, install a system with it and look if the installed system has a correct network configuration, that would still be appreciated. I also need somebody to test the GNU/kFreeBSD image. I fixed it up to build there but the result might be entirely wrong.<br />
<br />
My sincere thanks go to Bernhard Schmidt and Karsten Merker, who successfully completed installations over IPv6 in various environments!
Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Karlsruhe, Germany49.009148 8.379944448.8423165 8.0640874 49.175979500000004 8.6958013999999988tag:blogger.com,1999:blog-5048890463514304208.post-88888265812318098402012-09-03T18:11:00.000+02:002012-09-03T18:11:11.942+02:00IPv6 support in debian-installerI tried to continue <a href="http://packages.qa.debian.org/n/netcfg.html">netcfg</a>'s journey to support IPv6 in <a href="http://www.debian.org/devel/debian-installer">debian-installer</a>. <a href="http://hezmatt.org/~mpalmer/blog/">Matt Palmer</a> wrote a large patch set many moons ago (kudos!) and Colin Watson polished it and included it into Ubuntu. The reason for it not being merged into Debian first was <a href="http://packages.qa.debian.org/i/ifupdown.html">ifupdown</a>. Version 0.7 finally supports IPv6 and only <a href="http://packages.qa.debian.org/i/ifupdown/news/20120531T213331Z.html">entered Debian unstable</a> by the end of May. Due to a bunch of changes to netcfg, one reason being <a href="http://home-sorina.blogspot.de/2012/08/status-update-d-i-improve-network-config.html">a Summer of Code project</a> on it, I had to forward-port those 50 patches and add two bug fixes on top. Hence it's possible that I introduced some breakage, even if the result works well for me (apart from one <a href="http://en.wikipedia.org/wiki/DHCPv6">DHCPv6</a> oddity that cannot be seen from within <a href="http://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine">KVM</a>). The tree can be found in the <a href="http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=shortlog;h=refs/heads/people/pkern/ipv6">people/pkern/ipv6 branch</a>. I was unable to check if Ubuntu has introduced any additional patches on top of the old patch set, as I'm not used enough to <a href="http://packages.qa.debian.org/b/bzr.html">bzr</a>.<br />
<br />
This <a href="http://people.debian.org/~pkern/d-i/netcfg-ipv6/mini.iso">mini.iso</a> (<a href="http://people.debian.org/~pkern/d-i/netcfg-ipv6/mini.iso.sig">detached signature</a>) netinst image contains a debian-installer current as of today and two additional patched udebs: netcfg with the aforementioned IPv6 patch set and <a href="http://packages.qa.debian.org/b/busybox.html">busybox</a> with ping6 enabled. I'd appreciate if you could go and test it in one (or more) of the following environments: IPv4-only (DHCPv4 / static), wireless (IPv4 / IPv6 as desired, there has been some refactoring), stateless IPv6 autoconfiguration (it even supports <a href="http://tools.ietf.org/html/rfc6106">RDNSS and DNSSL</a>), stateless IPv6 autoconfiguration with stateless DHCPv6, and stateful DHCPv6. It will try to configure dual stack if possible. Please note that many Debian mirrors are not yet IPv6-enabled. Even prominent ftp.CC.debian.org hosts like ftp.de.debian.org still do not support it. So you'll have to pick one carefully if you want to try IPv6-only installation. (Which worked for me!)<br />
<br />
If you have any feedback for me, please leave it here as a comment or mail me at pkern@d.o. Thanks!Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com2Karlsruhe, Germany49.009148 8.379944448.334874000000006 7.1165169 49.683422 9.6433719tag:blogger.com,1999:blog-5048890463514304208.post-91738612711900687412012-08-14T17:42:00.003+02:002012-08-14T17:42:59.355+02:00Debian s390x installation now workingWith a <a href="http://d-i.debian.org/daily-images/s390x/daily/generic/">current daily build of debian-installer</a> you should now be able to install a VM with Debian s390x. I'm told that even installation within <a href="http://packages.qa.debian.org/h/hercules.html">hercules</a> works again. d-i's beta 1 for wheezy has a <a href="http://bugs.debian.org/681760">broken busybox</a> and is hence unusable. But there were more issues:<br />
<ul>
<li><a href="http://packages.qa.debian.org/z/zipl-installer.html">zipl-installer</a> was blacklisted from building on s390x by Packages-arch-specific which caused <a href="http://packages.qa.debian.org/n/nobootloader.html">nobootloader</a> to be selected during installation.</li>
<li><a href="http://packages.qa.debian.org/b/base-installer.html">base-installer</a> was unable to pick the correct kernel image due to it not having any rules for this architecture.</li>
<li>A <a href="http://anonscm.debian.org/gitweb/?p=d-i/netcfg.git;a=commitdiff;h=be8a052bf38fc121620fff90ea277eafb795713f;hp=5fc24621988db4012cff5b3c1ffc4c8d177152d1">netcfg fix</a> was needed that removes a 50s timeout while d-i tries to arping the configured gateway. This check seems to be pretty new and if it fails the link is still considered up and the installation continues.
The network driver commonly used on s390(x) (qeth) uses layer 3
configuration by default, which means that ARP is meaningless. The IPv4
and IPv6 addresses are configured in the network adapter which then does all the ARP if necessary (i.e. not within the same machine).</li>
</ul>
The resulting installation even boots. But it shares some problems with squeeze's s390 port:<br />
<ul>
<li>udev's persistent network device naming is broken. It includes a volatile device ID that is incremented with each reboot. This is now tracked in <a href="http://bugs.debian.org/684766">Debian bug #684766</a>.</li>
<li>The TERM variable is set to <i>linux</i> during boot-up instead of <i>dumb</i> (the s390(x) terminals are either line- or form-based, except for one <acronym title="Hardware Management Console">HMC</acronym> ASCII terminal). Together with the new fancy coloured LSB init output in wheezy the screen gets even more noisy with all those escape sequences. It's yet unclear where this should be fixed. The kernel provides the init system with a <span style="font-family: "Courier New",Courier,monospace;">TERM=linux</span> default and it's not yet fixed up afterwards (either in initramfs-tools, or busybox, or sysvinit).</li>
</ul>
Tip of the day: If you see funny characters in x3270, then you maybe selected the wrong EBCDIC code page. If you use Linux you want to select CP1047 instead of the default CP037 in Options → Character Set (→ Euro). That's the target of the in-kernel ASCII to EBCDIC converter.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Karlsruhe, Germany49.009148 8.379944448.925956000000006 8.2220158999999988 49.09234 8.5378729tag:blogger.com,1999:blog-5048890463514304208.post-60152994628564284912012-06-22T20:45:00.000+02:002012-06-25T10:26:25.688+02:00The upcoming DebConf 12<b>Update 2:</b> <a href="http://debian-administration.org/users/rkd/weblog/20">Richard</a> and especially <a href="http://morayallan.livejournal.com/12139.html">Moray</a> debunked my statistics and given that they're actually on the DebConf team, they simply know it better what to look at and which numbers cannot be trusted. So look over to Moray's post for some numbers about DC11 and DC12. <br />
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRKy4hOkbzr2VO7fy3aNgsK0M7OYsjqczMGy63Mz7g-KO0Gtm-uqnLQQ3icofphYFcNlBs5xgWgI17Yh1JkzGNv8NuqrHu_WuB1q77QoChcHsYrfz3Y3CBJOmpuCa4qD_8A9qd75jHKsU2/s1600/attendee-count.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRKy4hOkbzr2VO7fy3aNgsK0M7OYsjqczMGy63Mz7g-KO0Gtm-uqnLQQ3icofphYFcNlBs5xgWgI17Yh1JkzGNv8NuqrHu_WuB1q77QoChcHsYrfz3Y3CBJOmpuCa4qD_8A9qd75jHKsU2/s320/attendee-count.png" width="284" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<a href="http://debconf-data.alioth.debian.org/stats/">Oops</a>. At least the average count of days per person seems slightly higher. (But then the stats may likely be off, given that any part of the day counts as full.)<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyTgR6wjGGscY2lqJOOOIMvepE6QFzbBpa8Iibq-V6JfY5y1C42dbxRRYUQjpzJlukaRJeM8oj0RSbVfbLKUS4M8XReBWouAdTPM9hv0f2S2Ml0N057qY_gy0SGMVVnZbnO5h5qXXdh260/s1600/days-per-person.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiyTgR6wjGGscY2lqJOOOIMvepE6QFzbBpa8Iibq-V6JfY5y1C42dbxRRYUQjpzJlukaRJeM8oj0RSbVfbLKUS4M8XReBWouAdTPM9hv0f2S2Ml0N057qY_gy0SGMVVnZbnO5h5qXXdh260/s320/days-per-person.png" width="284" /></a></div>
<a href="http://debconf12.debconf.org/">This year's DebConf</a> will have remote participation through video streams and IRC chat, as usual. But they will be late at night for Europeans. Despite those hurdles, let's make this conference a success! The same procedure as every year. ;-)<br />
<br />
<b>Update:</b> I don't have any privileged access to Pentabarf and hence I was just working on the exported data at the link above. When perusing the "Statistics for ONLY people who have both dates in Penta" I get this:<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSM4HQaIyXnKICLTdiTUh_NHP8Vh_gNTcM4THiyfidUJ38P0Br8nBq1JwRNDOdF91guwLbBMKrsqihN_JbO1-ST1N714fB892zS0pIATQQcVIONW3pE-h-MkTWcXM5rrYmQwuqsf_TPUXQ/s1600/attendee-count-both-dates.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhSM4HQaIyXnKICLTdiTUh_NHP8Vh_gNTcM4THiyfidUJ38P0Br8nBq1JwRNDOdF91guwLbBMKrsqihN_JbO1-ST1N714fB892zS0pIATQQcVIONW3pE-h-MkTWcXM5rrYmQwuqsf_TPUXQ/s320/attendee-count-both-dates.png" width="284" /></a></div>
That's because this year seems to have a much higher percentage of people filling out both fields (~58% for DC10/DC11 and 85% for DC12). I'm still unsure if Penta was set to filter out those who did not reconfirm. But after all the others would not be very meaningful for room planning.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com6Karlsruhe, Germany49.009148 8.379944448.925825 8.2220158999999988 49.092471 8.5378729tag:blogger.com,1999:blog-5048890463514304208.post-81282431848910296832012-06-10T16:34:00.003+02:002012-06-10T16:38:54.888+02:00s390x accepted as release architectureYay, so we made it: <a href="https://lists.debian.org/debian-devel-announce/2012/06/msg00003.html">s390x got added as a release architecture</a>. What this means:<br />
<ul>
<li>If the package already exists on that architecture and fails to build, that issue is deemed RC. You can debug the issue on the porterbox (<a href="http://db.debian.org/machines.cgi?host=zelenka">zelenka</a>), chroot sid_s390x. Build dependencies are installed by a team, just follow the <a href="http://dsa.debian.org/doc/install-req/">request guidelines</a>. </li>
<li>If your package is not installable on s390x, it will not migrate to testing anymore. So if a package does not exist on s390x, you need to make sure not to generate a dependency on it or prevent your package from building on s390x (by e.g. build-depending on a package that's not available on s390x or by getting it into Packages-arch-specific).</li>
</ul>
This will also help other 64bit big-endian ports (like powerpc64 and sparc64) to enter the archive more easily, as most issues left are indeed related to endianness, not to specialities of the System z hardware.<br />
<ul>
</ul>
Many thanks go to <a href="http://www.aurel32.net/">Aurélien Jarno</a>, without whom this would not have been possible. I also want to take this opportunity to thank all our s390(x) machine sponsors: <a href="http://www.zivit.de/"><acronym title="Zentrum für Informationsverarbeitung und Informationstechnik">ZIVIT</acronym></a>, <a href="http://www.iic.kit.edu/"><acronym title="Informatics Innovation Center">IIC</acronym>@<acronym title="Karlsruhe Institute of Technology">KIT</acronym></a> and <a href="http://www.marist.edu/">Marist College</a>. There are not many mainframe owners who let free software projects work on their machines.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Karlsruhe, Germany37.09024 -95.71289111.6301275 -136.1425785 62.5503525 -55.2832035tag:blogger.com,1999:blog-5048890463514304208.post-19059958215778977952012-06-07T16:19:00.001+02:002012-06-07T16:23:26.824+02:00Adding zFCP drives on Debian GNU/LinuxIf you want to add System z <acronym title="z Fibre Channel Protocol">zFCP</acronym> drives to a Debian GNU/Linux system, you first need to make the <acronym title="Host Bus Adapter">HBA</acronym> known to the system. For this you add an (initially empty) file with the (lowercase) device ID of the zFCP HBA to <span style="font-family: "Courier New",Courier,monospace;">/etc/sysconfig/hardware</span>. There should already be files for the <acronym title="Extended Count Key Data">ECKD</acronym>/<acronym title="Fixed Block Architecture">FBA</acronym> disks and the network adapters.<br />
<br />
Then you need to list the drives to initialize by placing the following array in it with a list of <acronym title="World Wide Port Name">WWPN</acronym>:<acronym title="Logical Unit Number">LUN</acronym>, seperated by spaces within the parentheses:<br />
<br />
<span style="font-family: "Courier New",Courier,monospace; font-size: 0.9em;">ZFCP_DEVICES=(0x1234567890abcdef:0x4000400000000000)</span><br />
<br />
Please keep in mind that the WWPN and LUN need to be specified in hex and again entirely in lower case. This will cause <span style="font-family: "Courier New",Courier,monospace;">hwup ccw 0.0.4000</span> (with 4000 being the <acronym title="Channel Path Identifier">CHPID</acronym>) to instruct the HBA to add the drive after setting it online. Then you should regenerate your initrd, so that it happens on-boot, by running <span style="font-family: "Courier New",Courier,monospace;">update-initramfs -u -k all</span>.<br />
<br />
With current kernels the available WWPNs should be probed and listed in <span style="font-family: "Courier New",Courier,monospace;">/sys/bus/ccw/devices/0.0.4000</span> without further intervention after the HBA is initialized.<br />
<br />
Sadly there's no way to set up a zFCP disk in debian-installer.Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Karlsruhe, Germany49.008084 8.40375627.584074999999995 -32.0259315 70.432093 48.8334435tag:blogger.com,1999:blog-5048890463514304208.post-6263466778288783382012-05-17T18:07:00.002+02:002012-05-18T00:16:59.420+02:00Lazyweb question: How to avoid leaking process info?Dear Lazyweb,<br />
<br />
is there a simple way to block some users who login with SSH to read /proc/<pid>/cmdline of processes they don't own? Or better yet: don't see these pids at all?<br />
<br />
I know that there are PID namespaces, but they seem to require a special PID 1. Seems hard to get for a simple SSH login. (I wouldn't mind changing a user's shell. But brittle shell startup scripts wouldn't cut it.) systemd-nspawn wants to boot a full Linux distribution in a container and even then I'd be unsure how to wire it up so that it cannot be skipped. I wouldn't mind a read-only bind mount of the outermost Linux installation into a chroot environment, as long as the parent SSH process can get the user jailed into it securely. (No need for someone to be root in the chroot.)<br />
<br />
I know that there are RBAC frameworks, but they're cumbersome to use. I don't need file labelling or path-based access control, as I do trust the Linux file permissions for this. I think SMACK wouldn't help here, AppArmor isn't really useable in Debian testing and TOMOYO is a tad crazy to use with its domain transitions through process invocations.<br />
<br />
I bet that grsecurity would have something for me up its sleeve. But it's not in a Debian kernel. Even though I know how to compile my own kernel I'd only do that if everything else fails.<br />
<br />
Thanks in advance for your help.<br />
<br />
<b>UPDATE: </b>That was quick, thanks to everyone who participated! Vasiliy Kulikov came up with a <a href="http://git.kernel.org/?p=linux/kernel/git/stable/linux-stable.git;a=commit;h=0499680a42141d86417a8fbaa8c8db806bea1201">kernel patch</a> to my problem (a <span style="font-family: "Courier New",Courier,monospace;">hidepid</span> mount option for procfs) that landed in 3.3. I tested it with the kernel in experimental and it works just fine and as expected. With <span style="font-family: "Courier New",Courier,monospace;">hidepid</span> set to 1, it will still leak the process count and their euids and egids. With <span style="font-family: "Courier New",Courier,monospace;">hidepid</span> set to 2, you only see your own processes, unless you're root. For <span style="font-family: "Courier New",Courier,monospace;">ps</span> there's no visible distinction between the two. So to test it you can just invoke this as root on a host running 3.3+:<br />
<blockquote style="font-family: "Courier New",Courier,monospace;">
mount -o remount,hidepid=1 /proc</blockquote>
There's even a <a href="http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=669028">backport request</a> in the Debian <acronym title="Bug Tracking System">BTS</acronym> to get the feature into the wheezy kernel (3.2).Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com11Karlsruhe, Germany49.009148 8.379944448.925825 8.2220158999999988 49.092471 8.5378729tag:blogger.com,1999:blog-5048890463514304208.post-46176645056730883762012-04-03T14:56:00.000+02:002012-04-03T14:56:07.845+02:00The state of Debian s390xWhen we added s390x to the main archive, coming from <a href="http://www.debian-ports.org/">Debian Ports</a>, we were unlucky. A <a href="http://packages.qa.debian.org/g/glib2.0/news/20111118T193351Z.html">new glib release</a> had assumptions that weren't true on 64bit big endian architectures and it entered the archive just a few days before we made the initial import. This weekend we finally got <a href="http://packages.qa.debian.org/g/glib2.0/news/20120330T073522Z.html">a new major release</a> into Debian unstable that fixed these issues. So we're <a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhlAFJEBYRljpWDn2Mq3Ar6iz76RsCCmNboxCdt301U7LQa_eJ9A7enSI7PS17gBUzwUAAGoTgUA1wh33GaAV_Ye7718nurTIqyMXTUgyctxXaLAcMTQt5lQm2vVGmK9W-hVZDKjjMaHDGK/s1600/graph-week-big.png">almost on par with s390</a> now. It all untangled quite nicely after <a href="http://packages.qa.debian.org/g/glib-networking.html">glib-networking</a> was able to complete its testsuite. Only one build-dependency loop between <a href="http://packages.qa.debian.org/n/nautilus.html">nautilus</a> and <a href="http://packages.qa.debian.org/t/tracker.html">tracker</a> had to be broken manually.<br />
<br />
So what's left? There's a bunch of <a href="http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=s390x;users=debian-s390@lists.debian.org">usertagged bugs</a> (with both general FTBFSes and arch-specific issues; kudos to <a href="http://blog.aurel32.net/">Aurélien Jarno</a> providing a lot of patches) and we still need to file some, like <a href="http://packages.qa.debian.org/i/iceweasel.html">iceweasel</a> segfaulting during its build. That's important because another bunch of packages needs it to build (well, mozjs and/or xulrunner, or some package that needs those).Philipp Kernhttp://www.blogger.com/profile/12612857680528965620noreply@blogger.com0Bad Wildbad, Germany48.751608 8.549273148.667854999999996 8.3913446 48.835361 8.7072016000000012