Thread: Upsert error "column reference is ambiguous"

Regarding upsert syntax.

psql (16.8 (Ubuntu 16.8-0ubuntu0.24.04.1), server 14.13 (Ubuntu 
14.13-0ubuntu0.22.04.1))
=> CREATE TABLE t (k INTEGER, v INTEGER);
=> CREATE UNIQUE INDEX t_k ON t (k);
=> INSERT INTO t VALUES (1,1);
INSERT 0 1
=> INSERT INTO t VALUES (1,1) ON CONFLICT (k) DO UPDATE SET v=v+1;
ERROR:  column reference "v" is ambiguous

Please convince me that this is not a bug.

If I understand correctly, in the expression "v+1", both EXCLUDED.v 
and t.v are present as the unqualified name "v". This is always the 
case and it is never possible to reference an unqualified field name 
in the expression of a conflict action.

Thus, any query with an unqualified name is statically known to be 
invalid. It is not a b/c break to remove EXCLUDED.v from the list of 
unqualified fields in a new major release of PG, thus allowing it to DWIM.

I'm a maintainer of MediaWiki. Some kind person contributed PostgreSQL 
support many years ago so now I am required to maintain it in 
perpetuity. The work seems out of proportion to the benefit, but 
that's the industry I guess. A handful of users benefit, such as 
wiki.postgresql.org.

Our application has an upsert method which takes the assignment 
"v=v+1" as a string. It is feasible to split it on the equals sign 
into the destination field and expression components, but it is not 
feasible to parse the expression or to require callers to supply an 
AST tree for the expressions they give us. It is not feasible to 
require callers to prefix all field names with the table name.

We currently emulate upsert on PostgreSQL using several awkward and 
inefficient queries. It would be nice to be able to use PostgreSQL's 
native upsert feature. But unless someone here has an idea for a 
workaround, I think this field name resolution policy is a total 
blocker. We can implement upsert on MySQL and SQLite but on PostgreSQL 
it will remain emulated.

-- Tim Starling

thus allowing it to DWIM.

Tim Starling <tstarling@wikimedia.org> writes:
> Regarding upsert syntax.
> => INSERT INTO t VALUES (1,1) ON CONFLICT (k) DO UPDATE SET v=v+1;
> ERROR:  column reference "v" is ambiguous

> Please convince me that this is not a bug.

It's not a bug.

> If I understand correctly, in the expression "v+1", both EXCLUDED.v 
> and t.v are present as the unqualified name "v". This is always the 
> case and it is never possible to reference an unqualified field name 
> in the expression of a conflict action.

Correct: it's not clear whether you mean to use "v" from the new
desired-to-be-inserted row or "v" from the existing row.

> Thus, any query with an unqualified name is statically known to be 
> invalid. It is not a b/c break to remove EXCLUDED.v from the list of 
> unqualified fields in a new major release of PG, thus allowing it to DWIM.

Even if I were on board with arbitrarily adopting one of the two
possible interpretations, it's far from obvious to me that most people
would agree that "v" should mean the value from the existing row,
rather than the new value.  Better to make them say which they want.

            regards, tom lane

On 28/4/25 20:54, Tom Lane wrote:
> Even if I were on board with arbitrarily adopting one of the two
> possible interpretations, it's far from obvious to me that most people
> would agree that "v" should mean the value from the existing row,
> rather than the new value.  Better to make them say which they want.

OK sure, no way to tell, but if every other DBMS does it the same way 
then that might be a hint.

Also, I'm just saying, the upsert feature is fully useless to me with 
this name resolution policy.

In the single-row case, there's no need for EXCLUDED at all, because 
the client knows everything about the excluded row. Recall my example:

INSERT INTO t VALUES (1,1) ON CONFLICT (k) DO UPDATE SET v=v+1;

If I meant SET v=EXCLUDED.v+1 I would have just written v=2. The 
default policy (in other DBMSes) follows by analogy from the 
single-row case.

-- Tim Starling

On Mon, 2025-04-28 at 21:22 +1000, Tim Starling wrote:
> On 28/4/25 20:54, Tom Lane wrote:
> > Even if I were on board with arbitrarily adopting one of the two
> > possible interpretations, it's far from obvious to me that most people
> > would agree that "v" should mean the value from the existing row,
> > rather than the new value.  Better to make them say which they want.
>
> OK sure, no way to tell, but if every other DBMS does it the same way
> then that might be a hint.

Which DBMS that supports INSERT .. ON CONFLICT do you have in mind?

> Also, I'm just saying, the upsert feature is fully useless to me with
> this name resolution policy.

Because you cannot write EXCLUDED?

> In the single-row case, there's no need for EXCLUDED at all, because
> the client knows everything about the excluded row. Recall my example:
>
> INSERT INTO t VALUES (1,1) ON CONFLICT (k) DO UPDATE SET v=v+1;
>
> If I meant SET v=EXCLUDED.v+1 I would have just written v=2. The
> default policy (in other DBMSes) follows by analogy from the
> single-row case.

Actually, for many people, the DWIM would be the other way around:

INSERT INTO tab (col)
SELECT something FROM othertab
ON CONFLICT (id)
/* "col" should get set to "something" */
DO UPDATE SET col = col;

I can follow your reasoning above, but if the SQL parser tried to
guess the user's intention like that, it is likely to go wrong
sometimes.  As Tom said, better force the user to be explicit.

Yours,
Laurenz Albe

On Mon, Apr 28, 2025 at 12:56 AM Tim Starling <tstarling@wikimedia.org> wrote:
> Our application has an upsert method which takes the assignment
> "v=v+1" as a string. It is feasible to split it on the equals sign
> into the destination field and expression components, but it is not
> feasible to parse the expression or to require callers to supply an
> AST tree for the expressions they give us. It is not feasible to
> require callers to prefix all field names with the table name.

You can use an alias for the target table name. Is it feasible to
require callers to prefix all field names with a generic table name
alias?

--
Peter Geoghegan

Tim Starling <tstarling@wikimedia.org> writes:
> On 28/4/25 20:54, Tom Lane wrote:
>> Even if I were on board with arbitrarily adopting one of the two
>> possible interpretations, it's far from obvious to me that most people
>> would agree that "v" should mean the value from the existing row,
>> rather than the new value.  Better to make them say which they want.

> OK sure, no way to tell, but if every other DBMS does it the same way 
> then that might be a hint.

AFAIK, "ON CONFLICT" is a Postgres-ism.  Exactly which constructs
in exactly which other databases are you citing as precedent?

> In the single-row case, there's no need for EXCLUDED at all, because 
> the client knows everything about the excluded row.

Laurenz already provided the counter-example of an INSERT/SELECT,
but there's also the possibility of the INSERT supplying a computed
default value for a column, e.g., CURRENT_TIMESTAMP.  So you won't
get far with that argument.

I do actually have some sympathy for your proposal after thinking
about it a bit more, but the argument I would use is "the behavior
of the ON CONFLICT UPDATE SET list should be as much as possible like
the behavior of an ordinary UPDATE's SET list".  Since "v = v+1" would
refer to the existing row's "v" in regular UPDATE, it's sensible to
let that happen here too.  Of course the counter-argument is that this
should be compared not to a trivial UPDATE, but an "UPDATE ... FROM
othertable" where the othertable supplies some conflicting column
name(s).  In that situation we're going to make you resolve the
conflict by qualifying the column names.  The only thing that makes
that not a precise parallel is that EXCLUDED is not something the user
wrote into the query explicitly, so there's no opportunity to
substitute different column aliases, as a FROM clause would allow.
Perhaps that justifies demoting it to second-class citizenship whereby
EXCLUDED has to be qualified but the target table doesn't.  (I don't
find this argument hugely compelling, but it's an argument.)

BTW, I did wonder how hard it would be to make such a change.
On first glance it seems to be a one-liner:

diff --git a/src/backend/parser/analyze.c b/src/backend/parser/analyze.c
index 1f4d6adda52..f11727adbaa 100644
--- a/src/backend/parser/analyze.c
+++ b/src/backend/parser/analyze.c
@@ -1306,7 +1306,7 @@ transformOnConflictClause(ParseState *pstate,
          * Add the EXCLUDED pseudo relation to the query namespace, making it
          * available in the UPDATE subexpressions.
          */
-        addNSItemToQuery(pstate, exclNSItem, false, true, true);
+        addNSItemToQuery(pstate, exclNSItem, false, true, false);
 
         /*
          * Now transform the UPDATE subexpressions.

So this isn't about implementation difficulty but about whether
we think it's a good idea.

            regards, tom lane

AFAIK, "ON CONFLICT" is a Postgres-ism.  Exactly which constructs
in exactly which other databases are you citing as precedent?

On 28/4/25 23:54, Tom Lane wrote:
> AFAIK, "ON CONFLICT" is a Postgres-ism.  Exactly which constructs
> in exactly which other databases are you citing as precedent?

There's a list here:

<https://wiki.postgresql.org/wiki/UPSERT#UPSERT_as_implemented_in_practice>

Since that page was written in 2014, SQLite added upsert support, 
consciously following PG's syntax, except that unqualified names 
resolve to target rows.

My code would be like

function upsert( $table, $names, $values, $key, $set ) {
     if ( $this->type === 'mysql' ) {
         $conflict = 'ON DUPLICATE KEY UPDATE';
     } else {
         $conflict = "ON CONFLICT ($key) DO UPDATE SET";
     }
     return $this->query( "INSERT INTO $table ($names) " .
         "VALUES ($values) $conflict $set" );
}

The parameters are a little bit more structured than that, but that 
gives you the idea.

MediaWiki has supported MySQL's ON DUPLICATE KEY UPDATE since 2013, 
and we've always had the conflict target parameter $key since then as 
a helper for emulation. So it's trivial to produce either MySQL and 
SQLite syntax.

-- Tim Starling

On 28/4/25 23:30, Peter Geoghegan wrote:
> You can use an alias for the target table name. Is it feasible to
> require callers to prefix all field names with a generic table name
> alias?

No, primarily because MySQL does not support such an alias.

-- Tim Starling

> On Apr 28, 2025, at 15:36, Tim Starling <tstarling@wikimedia.org> wrote:
> function upsert( $table, $names, $values, $key, $set ) {
>    if ( $this->type === 'mysql' ) {
>        $conflict = 'ON DUPLICATE KEY UPDATE';
>    } else {
>        $conflict = "ON CONFLICT ($key) DO UPDATE SET";
>    }
>    return $this->query( "INSERT INTO $table ($names) " .
>        "VALUES ($values) $conflict $set" );

I'll mention that you can do this without ON CONFLICT in PostgreSQL in a way that, while not nearly as clean as ON
CONFLICT,isn't a huge hack, either: 

"DO $$ BEGIN INSERT INTO $table($names) VALUES($values); EXCEPTION WHEN integrity_constraint_violation THEN UPDATE
$tableSET $set WHERE $key=$values[0]; END; $$ LANGUAGE plpgsql;" 

It does require knowing which of the VALUES is the key value being inserted (pseudocode syntax above), but if that is
stylizedto always be the first value, that does not seem insurmountable. 

> On Apr 28, 2025, at 15:58, Christophe Pettus <xof@thebuild.com> wrote:
> It does require knowing which of the VALUES is the key value being inserted (pseudocode syntax above) [...]

The instant after I hit send, I realized that information is available to the function by lining up the $names and
$valuesarray, since the name of the key column is passed in. 

On Tue, 2025-04-29 at 08:36 +1000, Tim Starling wrote:
> My code would be like
>
> function upsert( $table, $names, $values, $key, $set ) {
>      if ( $this->type === 'mysql' ) {
>          $conflict = 'ON DUPLICATE KEY UPDATE';
>      } else {
>          $conflict = "ON CONFLICT ($key) DO UPDATE SET";
>      }
>      return $this->query( "INSERT INTO $table ($names) " .
>          "VALUES ($values) $conflict $set" );
> }
>
> The parameters are a little bit more structured than that, but that
> gives you the idea.

Another litle "if" to cater for PostgreSQL's "EXCLUDED." would be
such a big problem?

Yours,
Laurenz Albe

On 29/4/25 16:36, Laurenz Albe wrote:
> On Tue, 2025-04-29 at 08:36 +1000, Tim Starling wrote:
>> My code would be like
>>
>> function upsert( $table, $names, $values, $key, $set ) {
>>       if ( $this->type === 'mysql' ) {
>>           $conflict = 'ON DUPLICATE KEY UPDATE';
>>       } else {
>>           $conflict = "ON CONFLICT ($key) DO UPDATE SET";
>>       }
>>       return $this->query( "INSERT INTO $table ($names) " .
>>           "VALUES ($values) $conflict $set" );
>> }
>>
>> The parameters are a little bit more structured than that, but that
>> gives you the idea.
> 
> Another litle "if" to cater for PostgreSQL's "EXCLUDED." would be
> such a big problem?

I don't understand what you mean. EXCLUDED is not needed. "$table." 
needs to be prefixed to every column reference in the string $set. How 
do you find the column references amongst the string literals, 
function calls, etc.? You would need to parse the expression.

This is a public interface and there may be callers in code that I 
don't have access to.

Part of the reason for wanting to replace the existing emulation with 
a native upsert is to simplify the code. Parsing the expression is 
definitely not a simplification.

-- Tim Starling

On Tue, 29 Apr 2025 at 01:54, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I do actually have some sympathy for your proposal after thinking
> about it a bit more, but the argument I would use is "the behavior
> of the ON CONFLICT UPDATE SET list should be as much as possible like
> the behavior of an ordinary UPDATE's SET list".  Since "v = v+1" would
> refer to the existing row's "v" in regular UPDATE, it's sensible to
> let that happen here too.  Of course the counter-argument is that this
> should be compared not to a trivial UPDATE, but an "UPDATE ... FROM
> othertable" where the othertable supplies some conflicting column
> name(s).  In that situation we're going to make you resolve the
> conflict by qualifying the column names.  The only thing that makes
> that not a precise parallel is that EXCLUDED is not something the user
> wrote into the query explicitly, so there's no opportunity to
> substitute different column aliases, as a FROM clause would allow.
> Perhaps that justifies demoting it to second-class citizenship whereby
> EXCLUDED has to be qualified but the target table doesn't.  (I don't
> find this argument hugely compelling, but it's an argument.)

Not arguing for or against, but... I think there are some cases where
it would be more dangerous to relax this.  Here's one case where not
qualifying the column can be dangerous:

create table a1 (a int);
insert into a1 values(1),(2);
create table a2 (a int);
insert into a2 values(1);
select * from a1 where a in(select a from a2); -- as expected.

-- application changes, a2.a isn't needed anymore. column gets dropped
but someone forgets to update a query in the app...
alter table a2 drop column a;
select * from a1 where a in(select a from a2); -- silently returns
unexpected results.

If the original author of that query had been thoughtful enough to
qualify the column in the subquery then someone would probably have
gotten an error and fixed it. The moral of that story is that
sometimes forcing the query author to qualify the column is a good
idea. (not that there's much we can do about that one...)

Now the question is, do any similar hazards exist with ON CONFLICT DO
UPDATE? I don't think so as any columns being dropped will disappear
from the insert target table and the EXCLUDED work table at the same
time.

Another thought is that you can have an UPDATE with a RETURNING
clause. An unqualified column defaults to NEW even though you could
argue it's ambiguous due to OLD (as of 80feb727c). Likely we were
forced into making it work that way through not wanting to force
everyone to rewrite their RETURNING statements when upgrading to v18.
The moral of that story is, UPDATE isn't exactly consistent already
about when it requires column qualifications. Maybe it's weird to
insist that users qualify columns with their ON CONFLICT UPDATE SET
when RETURNING is happy to assume you must have meant NEW.

David

David Rowley <dgrowleyml@gmail.com> writes:
> Another thought is that you can have an UPDATE with a RETURNING
> clause. An unqualified column defaults to NEW even though you could
> argue it's ambiguous due to OLD (as of 80feb727c). Likely we were
> forced into making it work that way through not wanting to force
> everyone to rewrite their RETURNING statements when upgrading to v18.
> The moral of that story is, UPDATE isn't exactly consistent already
> about when it requires column qualifications. Maybe it's weird to
> insist that users qualify columns with their ON CONFLICT UPDATE SET
> when RETURNING is happy to assume you must have meant NEW.

That's an analogy I hadn't thought of, and it does seem on-point.
You might be right that we would not have done it like that if
we'd invented RETURNING's ability to support "OLD" at the get-go.
Nonetheless, it's there now and is a pretty similar precedent.

            regards, tom lane

This is a public interface and there may be callers in code that I don't have access to.