Re: Remove Deprecated Exclusive Backup Mode - Mailing list pgsql-hackers

Greetings,

* Magnus Hagander (magnus@hagander.net) wrote:
> On Wed, Jul 1, 2020 at 2:47 PM Stephen Frost <sfrost@snowman.net> wrote:
> > This would presumably be for the exclusive API as a way to make it not
> > completely broken, maybe.
> >
> > If we wanted to try and make this work in a non-exclusive manner then
> > we'd need to do something like have the user save some info out at
> > pg_start_backup time that they then provide at pg_stop_backup time, so
> > we can match up the specific backup and write the appropriate WAL
> > message.
>
> We don't even need to make it an exclusive mode -- we can allow
> *nonexclusive* backups but remove the requirement to run start and stop
> backup in the same connection, which I believe is the problem that people
> have with the exclusive mode. So how about something like this:
>
> 1. Make pg_start_backup() return the backup label file as well. We can add
> an extra column to the output without breaking backwards compatibility. And
> since we have all the information for the file at pg_start_backup() time,
> the user can then write that into the backup. We clearly document that this
> should *not* be written as "backup_label" in the data directory. We can
> even define what it should be instead.
>
> 2. Make pg_start_backup() also return a "cookie" value with a
> unique identifier if asked to run in "disconnectable mode". Store this
> identifier in shared state somewhere in the backend.

Seems like we could possibly just make this be the WAL position the
backup started at, since we use that as the finishing location...?  I
get that users could screw up passing the value back in and get a
corrupted backup, but it'd avoid us having to stick anything in shared
memory, wouldn't it?

> 3. Make a version of pg_stop_backup that takes this cookie as a parameter,
> and that way supports stopping a "disconnectable backup".
>
> 4. Scream loudly in the logs if we shut down postgres with any open
> "disconnectable backups" in progress. (to let people know they screwed up)

I'm less sure how useful that'd be, but ok.

> 5. Perhaps provide a row in pg_stat_progress_basebackup, or in it's own
> view, to show these "disconnectable mode backups"?
> 5b. In fact, maybe provide something like that for the current
> non-exclusive ones as well, in case people have hung sessions.

Ideally we'd provide a way for external backup tools to update said
progress with information of their own like the % complete, if they wish
to...

> The weak spot in this is still the "don't write it as backup label", but we
> can document that. And that would allow us to do non-exclusive base backups
> without requiring maintaining the connection. And we can completely get rid
> of the exclusive ones.

If this helps us get rid of exclusive backup mode then that's certainly
helpful, though inventing yet another method of doing backup makes me
cringe to think of the documentation complexity since it doesn't seem
like we'd really be reducing that.  Now, if we also removed the existing
non-exclusive method and then had a *single* backup method with clear
documentation as to how to use it properly, that'd be a marked
improvement overall.

Thanks,

Stephen