Re: PG 18 release notes draft committed - Mailing list pgsql-hackers

On Wed, Sep 17, 2025 at 6:58 AM Bruce Momjian <bruce@momjian.us> wrote:
> > In light of all this, I propose that we change the current feature
> > description, from:
> >
> > "This allows multi-column btree indexes to be used by queries that
> > only equality-reference the second or later indexed columns."
> >
> > to:
> >
> > "This allows multi-column btree indexes to be used by queries that
> > only specify conditions on the second or later indexed columns."
>
> I think your new text is inaccurate because you state here that the
> first column can be referenced and skip-scan still be used:

It's true that that's also possible. Skip scan will always "fill-in"
*any* index column that lacks a = condition that comes from the query
itself, by adding a skip array for that column during nbtree
preprocessing (note that a column that has a < or a > condition counts
as not having an = condition, and so will get its own skip array). A
skip array is an artificial "= ANY(<all possible column values>)"
condition/constraint that makes up for the fact that the user's query
did not provide us with a conventional = condition/constraint. nbtree
preprocessing always does this to the extent required to enable
repositioning the scan using *all* of the keys that actually come from
the query itself.

So we'll reliably read only those B-Tree leaf pages that might have
matching index tuples, no matter the details -- even with absurdly
complicated/unrealistic index scans. Note that adding a skip array
doesn't necessarily make us skip (skipping only happens when the scan
finds that it actually allows us to skip over something, otherwise we
just step to the next page on the leaf level as before). Skipping is
purely a runtime choice (adding skip arrays merely enables us to make
this choice, but it still has to make sense for us to do it).

To be clear, I don't think that the release notes need to go into
anything like this level of detail. My point is just that there are
just about no limitations. This isn't useful because we actually
expect users to have really complicated index scans, with varied
operators/conditions on only a subset of index columns (though some
may); it's useful because users don't have to think about it at all.

> I think we need to highlight new cases where indexes can now be used by
> skip scan:
>
> *  missing early indexed column references
> *  early indexed column references that use non-equality comparisons and
>    the comparisons are not sufficiently restrictive on their own to use
>    the index.

There isn't that much difference between these 2 things, from an
implementation point of view. They're both cases where a query/scan
initially lacks an = condition on a column where it'd be useful to
have one, so as to be able to use at least one later condition that
comes from the query to reposition the scan.

If there is an inequality condition on a column that gets a skip array
(because it didn't initially have a = condition), then the skip array
will only generate values that satisfy the inequalities. For example,
a skip array on "x" will only generate the values 1 and 2 given a qual
such as "WHERE x BETWEEN 1 AND 2 AND y = 66". The runtime behavior is
very similar to what we'd get in Postgres 17 for a qual "WHERE x =
ANY('{1, 2}') AND y = 66".

> And, at the same time, not fall into the trip of saying the later column
> references must be equality-only.

Right. This later condition could be a simple scalar equality or
inequality condition, it could be an IN() condition, or a row compare
inequality -- it could be anything.

> Here is what I have so far:
>
>         Previously, multi-column btree indexes could only be used by
>         queries that either equality-referenced the first indexed column
>         or referenced that column in a restrictive-enough way for index
>         lookups to be efficient.

It's hard to pin this down, since it has always been possible to make
*some* use of index columns, even in these tricky cases (we at least
didn't have to go to the heap to eliminate non-matching rows). Skip
scan just makes the B-Tree code find the most efficient way of
*navigating through the index*, by skipping over groups of
provably-irrelevant index leaf pages. This makes it much more likely
that the optimizer will actually choose such an index scan in the
first place.

I suggest the following alternative, which has the merit of being a
bit less verbose:

"Skip scan allows B-Tree index scans to find the most efficient way of
navigating through a multicolumn index when one or more of its columns
initially lacks a = condition and comes before a column that is
directly used by the query. Such an index scan can now be broken down
into multiple "index searches" by generating an implementation level =
condition on any underspecified columns. This allows the scan to skip
over irrelevant sections of the index, though only when the generated
= condition is on a column that has relatively few distinct values."

Note that "index searches" is a term that now appears in EXPLAIN ANALYZE output.

> I apologize for people who got the wrong impression of the feature and I
> hope they see this email thread or the updated text.

It's tough to get this right. There are many ways in which this could
be unnecessarily confusing, or misleading.

--
Peter Geoghegan