From fbd1260730a7e9a22033b24db2a1a8e7c4d58ef7 Mon Sep 17 00:00:00 2001 From: Peter Geoghegan Date: Sat, 22 Apr 2023 12:41:00 -0700 Subject: [PATCH v4 5/9] Move Interpreting XID stamps from tuple headers. Move handling of 32-bit XID comparisons/physical wraparound from "Routine Vacuuming" to chapter about transaction internals. This is intended to be fairly close to a mechanical change. It isn't entirely mechanical, though, since the original wording has been slightly modified for it to work in context. TODO fix xact.sgml indentation. The new content is indented correctly already, but the existing content will need to be re-indented to match in a later commit. As always, structuring things this way is intended to make life a little bit easier for doc translators. --- doc/src/sgml/maintenance.sgml | 80 +++++++--------------------------- doc/src/sgml/xact.sgml | 82 ++++++++++++++++++++++++++++++++++- 2 files changed, 96 insertions(+), 66 deletions(-) diff --git a/doc/src/sgml/maintenance.sgml b/doc/src/sgml/maintenance.sgml index 83fa7ba8b..970c4a848 100644 --- a/doc/src/sgml/maintenance.sgml +++ b/doc/src/sgml/maintenance.sgml @@ -446,75 +446,25 @@ analyze threshold = analyze base threshold + analyze scale factor * number of tu wraparound - - wraparound - of transaction IDs - + + wraparound + of transaction IDs + - PostgreSQL's - MVCC transaction semantics - depend on being able to compare transaction ID (XID) - numbers: a row version with an insertion XID greater than the current - transaction's XID is in the future and should not be visible - to the current transaction. But since transaction IDs have limited size - (32 bits) a cluster that runs for a long time (more - than 4 billion transactions) would suffer transaction ID - wraparound: the XID counter wraps around to zero, and all of a sudden - transactions that were in the past appear to be in the future — which - means their output become invisible. In short, catastrophic data loss. - (Actually the data is still there, but that's cold comfort if you cannot - get at it.) To avoid this, it is necessary to vacuum every table - in every database at least once every two billion transactions. + PostgreSQL's MVCC transaction semantics depend on + being able to compare transaction + ID numbers (XID) to determine + whether or not the row is visible to each query's MVCC snapshot + (see ). But since + on-disk storage of transaction IDs in heap pages uses a truncated + 32-bit representation to save space (rather than the full 64-bit + representation), it is necessary to vacuum every table in every + database at least once every two billion + transactions (though far more frequent vacuuming is typical). - - The reason that periodic vacuuming solves the problem is that - VACUUM will mark rows as frozen, indicating that - they were inserted by a transaction that committed sufficiently far in - the past that the effects of the inserting transaction are certain to be - visible to all current and future transactions. - Normal XIDs are - compared using modulo-232 arithmetic. This means - that for every normal XID, there are two billion XIDs that are - older and two billion that are newer; another - way to say it is that the normal XID space is circular with no - endpoint. Therefore, once a row version has been created with a particular - normal XID, the row version will appear to be in the past for - the next two billion transactions, no matter which normal XID we are - talking about. If the row version still exists after more than two billion - transactions, it will suddenly appear to be in the future. To - prevent this, PostgreSQL reserves a special XID, - FrozenTransactionId, which does not follow the normal XID - comparison rules and is always considered older - than every normal XID. - Frozen row versions are treated as if the inserting XID were - FrozenTransactionId, so that they will appear to be - in the past to all normal transactions regardless of wraparound - issues, and so such row versions will be valid until deleted, no matter - how long that is. - - - - - In PostgreSQL versions before 9.4, freezing was - implemented by actually replacing a row's insertion XID - with FrozenTransactionId, which was visible in the - row's xmin system column. Newer versions just set a flag - bit, preserving the row's original xmin for possible - forensic use. However, rows with xmin equal - to FrozenTransactionId (2) may still be found - in databases pg_upgrade'd from pre-9.4 versions. - - - Also, system catalogs may contain rows with xmin equal - to BootstrapTransactionId (1), indicating that they were - inserted during the first phase of initdb. - Like FrozenTransactionId, this special XID is treated as - older than every normal XID. - - - controls how old an XID value has to be before rows bearing that XID will be diff --git a/doc/src/sgml/xact.sgml b/doc/src/sgml/xact.sgml index b467660ee..8a1f9fd6f 100644 --- a/doc/src/sgml/xact.sgml +++ b/doc/src/sgml/xact.sgml @@ -22,6 +22,8 @@ single-statement transactions. + + Virtual Transaction IDs Every transaction is identified by a unique VirtualTransactionId (also called @@ -46,10 +48,13 @@ started, particularly if the transaction started with statements that only performed database reads. + + + Permanent Transaction IDs The internal transaction ID type xid is 32 bits wide - and wraps around every + and wraps around every 4 billion transactions. A 32-bit epoch is incremented during each wraparound. There is also a 64-bit type xid8 which includes this epoch and therefore does not wrap around during the @@ -69,6 +74,80 @@ linkend="guc-track-commit-timestamp"/> is enabled. + + <type>TransactionId</type> comparison rules + + The system often needs to compare t_xmin + and t_xmax fields for MVCC snapshot + visibility checks. + + + + We use a truncated 32-bit representation of transaction IDs, rather than + using the full 64-bit representation. The 2.1 billion XIDs + distance invariant must be preserved because transaction + IDs stored in heap row headers use a truncated 32-bit representation + (rather than the full 64-bit representation). Since all unfrozen + transaction IDs from heap tuple headers must be from + the same transaction ID epoch (or from a space in the 64-bit + representation that spans two adjoining transaction ID epochs), there + isn't any need to store a separate epoch field in each tuple header. + This scheme has the advantage of requiring much less space than a design + that stores an XID epoch alongside each XID stored in each heap tuple + header. It has the disadvantage of constraining the system's ability to + allocate new XIDs (in the worst case scenario where transaction ID + exhaustion occurs). + + + + VACUUM will mark tuple headers + frozen, indicating that all eligible rows on the + page were inserted by a transaction that committed sufficiently far in + the past that the effects of the inserting transaction are certain to be + visible to all current and future transactions. Normal XIDs are compared + using modulo-232 arithmetic. This means that + for every normal XID, there are two billion XIDs that are + older and two billion that are newer; + another way to say it is that the normal XID space is circular with no + endpoint. Therefore, once a row version has been created with a + particular normal XID, the row version will appear to be in the + past for the next two billion transactions, no matter which + normal XID we are talking about. If the row version still exists after + more than two billion transactions, it will suddenly appear to be in the + future. To prevent this, PostgreSQL reserves a + special XID, FrozenTransactionId, which does not + follow the normal XID comparison rules and is always considered older + than every normal XID. Frozen row versions are treated as if the + inserting XID were FrozenTransactionId, so that they + will appear to be in the past to all normal transactions + regardless of wraparound issues, and so such row versions will be valid + until deleted, no matter how long that is. + + + + + In PostgreSQL versions before 9.4, freezing was + implemented by actually replacing a row's insertion XID + with FrozenTransactionId, which was visible in the + row's xmin system column. Newer versions just set a flag + bit, preserving the row's original xmin for possible + forensic use. However, rows with xmin equal + to FrozenTransactionId (2) may still be found + in databases pg_upgrade'd from pre-9.4 versions. + + + Also, system catalogs may contain rows with xmin equal + to BootstrapTransactionId (1), indicating that they were + inserted during the first phase of initdb. + Like FrozenTransactionId, this special XID is treated as + older than every normal XID. + + + + + + + Global Transaction Identifiers In addition to vxid and xid, prepared transactions are also assigned Global Transaction @@ -77,6 +156,7 @@ prepared transactions. The mapping of GID to xid is shown in pg_prepared_xacts. + -- 2.40.1