systemd-confext Two Years On: Versioned Overlays for /etc, Reloaded (asg2025)

Chaos Computer Club - recent audio-only feed

systemd-confext is a lightweight overlay mechanism for /etc, allowing you to drop in a configuration extension ("confext") bundle and let systemd make it visible to your service as though it was already shipped with the base image. Building on the same extension magic as systemd-sysext, confext also introduces extra features tailored for the /etc use case, such as vpick-ing the newest version and the ability to pick up config revisions with a `systemctl reload`. This talk presents the changes to systemd-confext since [its debut at All Systems Go! 2023](https://cfp.all-systems-go.io/all-systems-go-2023/talk/XLQNDJ/), the lessons learned along the way to make it work, and how we leverage this capability at Microsoft already to deliver configuration payloads in production. Immutable Linux distributions offer stability and reproducibility, but at the cost of configuration changes needing time to build. A small configuration or system change can then require complete redeployment of the entire OS, adding friction to development work and impacting customers. This is not acceptable for certain Linux environments at Microsoft, where only seconds of planned downtime budget exist every year. systemd-confext is intended to be a signed, `dm-verity`-protected, live configuration update mechanism meant to address this issue. It provides a way to make quick additions in a secure and reliable way with minimum customer impact. Two years later, confext now supports host-level payloads to /etc and also works in the individual namespaces of services and portable units via `ExtensionImages=` and `ExtensionDirectories=` too. One of the recent significant additions is that systemd-confext has been integrated with `systemctl reload`, giving it the ability to pick up new configuration revisions during a reload, not just a restart. Combined with services that implement notify-reload, it is possible to simply drop a new versioned extension into the watched directory to trigger the reload flow and update the service config. We'll review how these features can be used and include a demo of a couple of use cases, including path-activated units to deploy config payloads in production at Microsoft. We'll also briefly discuss the namespace and mounting changes needed in the systemd codebase to make this integration work. Licensed to the public under https://creativecommons.org/licenses/by/4.0/de/ about this event: https://cfp.all-systems-go.io/all-systems-go-2025/talk/GSRYLR/