Patroni 4.1.0 causes issues because of systemd notify - Mailing list pgsql-pkg-debian

From Hauke Bruno Wollentin
Subject Patroni 4.1.0 causes issues because of systemd notify
Date
Msg-id emf4c21a82-984c-4bc4-91e8-79bdf35bc018@79dd1d3b.com
Whole thread Raw
List pgsql-pkg-debian
Hi all,

while spinning up new Patroni cluster with version 4.1.0 (which has been added to the Debian repo 2 days ago), my clusters won’t bootstrap properly anymore.

At first, logs looks clear but anyways, all systemd operations like start/stop/restart will run into a timeout, causing the service to stop which then will break the cluster at all.

It seems like the new systemd notify feature is causing this, since using Patroni 4.0.7 with `Type=simple` in the systemd unit file works pretty fine.

I’m on Debian 12 and psql 17.

This is the patroni log after bootstrapping. Everything works fine until the 30s timeout of the systemd unit kicks in:

Oct 22 17:43:09 hetzner-lab01 patroni[13066]: Success. You can now start the database server using:
Oct 22 17:43:09 hetzner-lab01 patroni[13066]:     /usr/lib/postgresql/17/bin/pg_ctl -D /var/lib/postgresql/patroni/data -l logfile start
Oct 22 17:43:09 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:09,768 INFO: postmaster pid=13083
Oct 22 17:43:09 hetzner-lab01 patroni[13084]: localhost:5432 - no response
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.780 CEST [13083] LOG:  starting PostgreSQL 17.6 (Debian 17.6-2.pgdg12+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14+deb12u1) 12.2.0, 64-bit
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.780 CEST [13083] LOG:  listening on IPv4 address "0.0.0.0", port 5432
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.780 CEST [13083] LOG:  listening on IPv6 address "::", port 5432
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.782 CEST [13083] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432”
Oct 22 17:43:09 hetzner-lab01 patroni[13087]: 2025-10-22 17:43:09.789 CEST [13087] LOG:  database system was shut down at 2025-10-22 17:43:09 CEST
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.798 CEST [13083] LOG:  database system is ready to accept connections
Oct 22 17:43:09 hetzner-lab01 systemd[1]: patroni.service: Got notification message from PID 13083, but reception only permitted for main PID 13059
Oct 22 17:43:10 hetzner-lab01 patroni[13091]: localhost:5432 - accepting connections
Oct 22 17:43:10 hetzner-lab01 patroni[13093]: localhost:5432 - accepting connections
Oct 22 17:43:10 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:10,814 INFO: establishing a new patroni heartbeat connection to postgres
Oct 22 17:43:10 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:10,918 INFO: running post_bootstrap
Oct 22 17:43:11 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:11,225 INFO: initialized a new cluster
Oct 22 17:43:11 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:11,373 INFO: no action. I am (hetzner-lab01), the leader with the lock
Oct 22 17:43:21 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:21,276 INFO: no action. I am (hetzner-lab01), the leader with the lock
Oct 22 17:43:31 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:31,368 INFO: no action. I am (hetzner-lab01), the leader with the lock
Oct 22 17:43:37 hetzner-lab01 systemd[1]: patroni.service: start operation timed out. Terminating.
Oct 22 17:43:37 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:37.987 CEST [13083] LOG:  received fast shutdown request
Oct 22 17:43:37 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:37.989 CEST [13083] LOG:  aborting any active transactions
Oct 22 17:43:37 hetzner-lab01 patroni[13095]: 2025-10-22 17:43:37.990 CEST [13095] FATAL:  terminating connection due to administrator command
Oct 22 17:43:37 hetzner-lab01 systemd[1]: patroni.service: Got notification message from PID 13083, but reception only permitted for main PID 13059
Oct 22 17:43:37 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:37.992 CEST [13083] LOG:  background worker "logical replication launcher" (PID 13090) exited with exit code 1
Oct 22 17:43:37 hetzner-lab01 patroni[13085]: 2025-10-22 17:43:37.994 CEST [13085] LOG:  shutting down
Oct 22 17:43:37 hetzner-lab01 patroni[13085]: 2025-10-22 17:43:37.996 CEST [13085] LOG:  checkpoint starting: shutdown immediate
Oct 22 17:43:38 hetzner-lab01 patroni[13085]: 2025-10-22 17:43:38.014 CEST [13085] LOG:  checkpoint complete: wrote 17 buffers (0.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.005 s, sync=0.004 s, total=0.020 s; sync files=13, longest=0.002 s, average=0.001 s; distance=36 kB, estimate=36 kB; lsn=0/152D6F0, redo lsn=0/152D6F0
Oct 22 17:43:38 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:38.020 CEST [13083] LOG:  database system is shut down
Oct 22 17:43:39 hetzner-lab01 systemd[1]: patroni.service: Failed with result ‘timeout’.
Oct 22 17:43:39 hetzner-lab01 systemd[1]: Failed to start patroni.service - Runners to orchestrate a high-availability PostgreSQL.

Any recommendations how to fix/workaround this?

cheers,
Hauke

🥷 Hauke Bruno Wollentin

pgsql-pkg-debian by date:

Previous
From: apt.postgresql.org Repository Update
Date:
Subject: patroni updated to version 4.1.0-1.pgdg+1
Next
From: Kristo Marijo
Date:
Subject: Trixie postgresql-18-partman debian package