Colin Watson: Free software activity in July 2025

About 90% of my Debian contributions this month were
sponsored by Freexian.
You can also support my work directly via
Liberapay or GitHub
Sponsors.
DebConf
I attended DebConf for the first time in 11 years (my last one was DebConf
14 in Portland). It was great! For once I had a conference where I had a
fairly light load of things I absolutely had to do, so I was able to spend
time catching up with old friends, making some new friends, and doing some
volunteering – a bit of Front Desk, and quite a lot of video team work where
I got to play with sound desks and such. Apparently one of the BoFs (“birds
of a feather”, i.e. relatively open discussion sessions) where I was
talkmeister managed to break the automatic video cutting system by starting
and ending precisely on time, to the second, which I’m told has never
happened before. I’ll take that.
I gave a talk about
Debusine,
along with helping Enrico run a Debusine
BoF.
We still need to process some of the feedback from this, but are generally
pretty thrilled about the reception. My personal highlight was getting a
shout-out in a talk from
CERN
(in the slide starting at 32:55).
Other highlights for me included a Python team
BoF,
Ian’s tag2upload
talk
and some very useful follow-up discussions, a session on archive-wide
testing,
a somewhat brain-melting whiteboard session about the “multiarch interpreter
problem”,
several
useful
discussions
about salsa.debian.org, Matthew’s talk on how Wikimedia automates their Debian package
builds,
and many others. I hope I can start attending regularly again!
OpenSSH
Towards the end of a release cycle, people tend to do more upgrade testing,
and this sometimes results in interesting problems. Manfred Stock reported
“No new SSH connections possible during large part of upgrade to Debian
Trixie”, and after a little testing in a
container I confirmed that this was a reproducible problem that would have
affected many people upgrading from Debian 12 (bookworm), with potentially
severe consequences for people upgrading remote systems. In fact, there
were two independent problems that each led to much the same symptom:

OpenSSH 9.8 split the monolithic sshd listener process into two
pieces: a minimal network listener (still called sshd), and an
sshd-session process dealing with each individual session. (OpenSSH
10.0 further split sshd-session, adding an sshd-auth process that
deals with the user authentication phase of the protocol.) This hardens
the OpenSSH server by using different address spaces for privileged and
unprivileged code.
Before this change, when sshd received an incoming connection, it
forked and re-executed itself with some special parameters to deal with
it. After this change, it forks and executes sshd-session instead,
and sshd no longer accepts the parameters it used to accept for this.
Debian package upgrades happen in two phases: first we unpack the new
files onto disk, and then we run some package-specific configuration
steps which usually include things like restarting services. (I’m
simplifying, but this is good enough for this post.) Normally this is
fine, and in fact desirable: the old service keeps on working, and this
approach often allows breaking what would otherwise be difficult cycles
by ensuring that the system is in a more coherent state before trying to
restart services. However, in this case, unpacking the new files onto
disk immediately means that new SSH connections no longer work: the old
sshd receives the connection and tries to hand it off to a
freshly-executed copy of the new sshd binary on disk, which no longer
supports this.
If you’re just upgrading OpenSSH on its own or with a small number of
other packages, this isn’t much of a problem as the listener will be
restarted quite soon; but if you’re upgrading from bookworm to trixie,
there may be a long gap when you can’t SSH to the system any more, and
if something fails in the middle of the upgrade then you could be in trouble.
So, what to do? I considered keeping a copy of the old sshd around
temporarily and patching the new sshd to re-execute it if it’s being
run to handle an incoming connection, but that turned out to fail in my
first test: dependencies are normally only checked when configuring a
package, so it’s possible to unpack openssh-server before unpacking a
newer libc6 that it depends on, at which point you can’t execute the
new sshd at all. (That also means that the approach of restarting the
service at unpack time instead of configure time is a non-starter.) We
needed a different idea.
dpkg, the core Debian package manager, has a specialized facility
called “diversions”: you can tell it that when it’s unpacking a
particular file it should put it somewhere else instead. This is
normally used by administrators when they want to install a
locally-modified version of a particular file at their own risk, or by
packages that knowingly override a file normally provided by some other
package. However, in this case it turns out to be useful for
openssh-server to temporarily divert one of its own files! When
upgrading from before 9.8, it now diverts /usr/sbin/sshd to
/usr/sbin/sshd.session-split before the new version is unpacked, then
removes the diversion and moves the new file into place once it’s ready
to restart the service; this reduces the period when incoming
connections fail to a minimum. (We actually have to pretend that the
diversion is being performed on behalf of a slightly different package
since we’re using dpkg-divert in a strange way here, but it all works.)

Most OpenSSH processes, including sshd, check for a compatible version
of the OpenSSL library when they start up. This check used to be very
picky, among other things requiring both the major and minor number to
match. OpenSSL 3 has a better versioning
policy,
and so OpenSSH 9.4p1 relaxed this
check.
Unfortunately, bookworm shipped with OpenSSH 9.2p1, which means that as
soon as you unpack the new libssl3 during an upgrade (actually
libssl3t64 due to the 64-bit time_t
transition), sshd
stops working. This couldn’t be fixed by a change in trixie; we needed
to change bookworm in advance of the upgrade so that it would tolerate
newer versions of OpenSSL. And time was tight if we wanted to maximize
the chance that people would apply that stable update before upgrading
to trixie; there isn’t going to be another point release of Debian 12
before the release of Debian 13.
Fortunately, there’s a
stable-updates
mechanism for exactly this sort of thing, and the stable release
managers kindly accepted my proposal to fix this there.

The net result is that if you apply updates to bookworm (including
stable-updates / bookworm-updates, which is enabled by default) before
starting the upgrade to trixie, everything should be fine. Many thanks to
Manfred for reporting this with just enough time to spare that we were able
to fix it before Debian 13 is released in a few days!
debmirror
I did my twice-yearly refresh of debmirror’s mirror_size
documentation,
and applied a patch from Christoph Goehre
to improve mirroring of installer files.
madison-lite
I proposed renaming this project along
with the rmadison tool in devscripts, although I’m not yet sure what a
good replacement name would be.
Python team
I upgraded python-expandvars, python-typing-extensions (in experimental),
and webtest to new upstream versions.
I backported fixes for some security vulnerabilities to unstable:

python-urllib3: CVE-2025-50181,
CVE-2025-50182

I fixed or helped to fix a number of release-critical bugs:

bitstruct: autopkgtest regression: invalid command ‘test’
django-pipeline: autopkgtest failure
(contributed supporting fix
upstream)
pnopaste: Fails to install with debconf noninteractive
frontend (suggested possible patch)
py3dns: autopkgtest regression: ‘96.7.128.186’ !=
‘93.184.215.14’ (contributed
upstream)
python-marshmallow-dataclass: autopkgtest depends on removed package python-marshmallow-enum
python-pkgconfig: autopkgtest regression: list index out of range
python-txrequests: autopkgtest regression:
twisted.trial.unittest.FailTest: 200 != 404

I fixed some other bugs, mostly Severity: important:

afew: Unable to remove tags (reviewed
and merged
MR)
ipy: FTBFS with the nocheck build profile
paramiko: Does not correctly handle OpenSSH 10 version
python-django-storages: FTBFS with the nocheck build profile
python-icalendar: Depends on a transitional
package (and follow-up fixes for missing
build-dependencies in python-recurring-ical-events, python-x-wr-timezone,
and todoman)
python-libais: Stop calling python3 setup.py
test (contributed supporting fix
upstream)

I reinstated python3-mastodon’s build-dependency on and recommendation of
python3-blurhash, now that the latter has been fixed to use the correct
upstream source.