Thread: Add LINE: hint when schemaname.typename is a non-existent schema
The attached patch adds a LINE: ... hint when schemaname.typename
results in a schema which does not exist. I came across this when a
missing comma in a SELECT list resulted in an error without a location
in a query a few thousand lines long.
Before:
(postgres@[local]:5432 14:41:25) [postgres]> select test.id 'all' as
example from test;
ERROR: 3F000: schema "test" does not exist
LOCATION: get_namespace_oid, namespace.c:2826
After:
(postgres@[local]:5433 14:42:32) [postgres]> select test.id 'all' as
example from test;
ERROR: 3F000: schema "test" does not exist
LINE 1: select test.id 'all' as example from test;
^
LOCATION: LookupTypeName, parse_type.c:171
-Ryan Kelly
Attachment
On Mon, Feb 2, 2015 at 2:44 PM, Ryan Kelly <rpkelly22@gmail.com> wrote: > The attached patch adds a LINE: ... hint when schemaname.typename > results in a schema which does not exist. I came across this when a > missing comma in a SELECT list resulted in an error without a location > in a query a few thousand lines long. > > Before: > > (postgres@[local]:5432 14:41:25) [postgres]> select test.id 'all' as > example from test; > ERROR: 3F000: schema "test" does not exist > LOCATION: get_namespace_oid, namespace.c:2826 > > After: > > (postgres@[local]:5433 14:42:32) [postgres]> select test.id 'all' as > example from test; > ERROR: 3F000: schema "test" does not exist > LINE 1: select test.id 'all' as example from test; > ^ > LOCATION: LookupTypeName, parse_type.c:171 > > -Ryan Kelly Please add your patch to https://commitfest.postgresql.org/action/commitfest_view/open so do we don't forget about it. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Ryan Kelly <rpkelly22@gmail.com> writes:
> The attached patch adds a LINE: ... hint when schemaname.typename
> results in a schema which does not exist. I came across this when a
> missing comma in a SELECT list resulted in an error without a location
> in a query a few thousand lines long.
I think this is a good problem to attack, but the proposed patch seems
a bit narrow and brute-force. Surely this isn't the only place where
we're missing an errposition pointer on a "no such schema" error?
It's probably not reasonable to add a pstate argument to
LookupExplicitNamespace itself, given that namespace.c isn't part
of the parser. But you could imagine adding an interface function
in the parser that calls LookupExplicitNamespace and then throws a
position-aware error on failure, and then making sure that all schema
lookups in the parser go through that.
regards, tom lane
On Tue, Feb 3, 2015 at 11:10 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Ryan Kelly <rpkelly22@gmail.com> writes: >> The attached patch adds a LINE: ... hint when schemaname.typename >> results in a schema which does not exist. I came across this when a >> missing comma in a SELECT list resulted in an error without a location >> in a query a few thousand lines long. > > I think this is a good problem to attack, but the proposed patch seems > a bit narrow and brute-force. Surely this isn't the only place where > we're missing an errposition pointer on a "no such schema" error? I managed to find four code paths that failed to report an error position if the schema did not exist: - Schema-qualified types (e.g. select schemaname.typename 'foo') - Schema-qualified operators (e.g. select 1 operator(schemaname.+) 1) - Schema-qualified table names in CREATE TABLE (e.g. create table schemaname.tablename (colname integer)) - Schema-qualified function calls (e.g. select schemaname.funcname()) > It's probably not reasonable to add a pstate argument to > LookupExplicitNamespace itself, given that namespace.c isn't part > of the parser. But you could imagine adding an interface function > in the parser that calls LookupExplicitNamespace and then throws a > position-aware error on failure, and then making sure that all schema > lookups in the parser go through that. While searching for other instances of this problem, I found that using a schema that does not exist in conjunction with a collation reported the position. That code uses setup_parser_errposition_callback and the documentation for that function seems to indicate that the usage of it in my newly attached patch are consistent. I do not know, however, if this is the cleanest approach or if I should actually create a function like validate_explicit_namespace to handle this (and I'm also not sure where such a function should live, if I do create it). -Ryan Kelly
Attachment
Ryan Kelly <rpkelly22@gmail.com> writes:
> On Tue, Feb 3, 2015 at 11:10 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> It's probably not reasonable to add a pstate argument to
>> LookupExplicitNamespace itself, given that namespace.c isn't part
>> of the parser. But you could imagine adding an interface function
>> in the parser that calls LookupExplicitNamespace and then throws a
>> position-aware error on failure, and then making sure that all schema
>> lookups in the parser go through that.
> While searching for other instances of this problem, I found that using
> a schema that does not exist in conjunction with a collation reported the
> position. That code uses setup_parser_errposition_callback and the
> documentation for that function seems to indicate that the usage of it in my
> newly attached patch are consistent. I do not know, however, if this is the
> cleanest approach or if I should actually create a function like
> validate_explicit_namespace to handle this (and I'm also not sure where
> such a function should live, if I do create it).
Yeah, setup_parser_errposition_callback would work too. I'm not sure
offhand which I like better. One thing to keep in mind is that the
callback approach results in adding an error cursor position to *any*
error thrown while the callback is active, not only "schema not found".
There are pluses and minuses to that. I've seen error cursors attached
to very bizarre internal problems that (more or less by chance) showed up
while the parser was looking up a table name, but weren't really connected
to the table name at all. OTOH, most of the time you'd just as soon not
be too picky about what conditions you provide a cursor for.
The main place I'd question what you did is the callback placement around
make_oper_cache_key --- that might work, but it seems very bizarre, and
perhaps more than usually prone to the "cursor given for unrelated
problem" issue. Perhaps better to push it down inside that function
so it surrounds just the namespace lookup call.
Also the diffs in parse_utilcmd.c are very confusing and seem to change
more code than is necessary --- why did that happen?
regards, tom lane
The following review has been posted through the commitfest application: make installcheck-world: not tested Implements feature: tested, failed Spec compliant: not tested Documentation: not tested Tom suggested few changes already which I too think author needs to address. So marking it "Waiting on Author". However, I see following, example does not work well. postgres=# create or replace function f1(a abc.test.id%type) returns int as $$ select 1; $$ language sql; ERROR: schema "abc" does not exist Is that expected? I guess we need it at all places in parse_*.c where we will look for namespace. Please fix. Also, like Tom's suggestion on make_oper_cache_key, can we push down this inside func_get_detail() as well, just to limit it for namespace lookup? However, patch is not getting applied cleanly on latest sources. Need rebase. > On Tom comments on parse_utilcmd.c: I guess the block is moved after the pstate and CreateStmtContext are setup properly. I guess, we can move just after pstate setup, so that it will result into minimal changes? Can we have small test-case? Or will it be too much for this feature? The new status of this patch is: Waiting on Author
In the interest of moving forward, I have updated this patch because Ryan has been inactive for over a month now. Tom Lane wrote: > Yeah, setup_parser_errposition_callback would work too. I'm not sure > offhand which I like better. One thing to keep in mind is that the > callback approach results in adding an error cursor position to *any* > error thrown while the callback is active, not only "schema not found". > There are pluses and minuses to that. I've seen error cursors attached > to very bizarre internal problems that (more or less by chance) showed up > while the parser was looking up a table name, but weren't really connected > to the table name at all. OTOH, most of the time you'd just as soon not > be too picky about what conditions you provide a cursor for. I think we can live with cursor positions in some weird corner cases. If we later find out that we don't like it for some reason, we can reduce the scope that this applies to. > The main place I'd question what you did is the callback placement around > make_oper_cache_key --- that might work, but it seems very bizarre, and > perhaps more than usually prone to the "cursor given for unrelated > problem" issue. Perhaps better to push it down inside that function > so it surrounds just the namespace lookup call. Agreed; the attached patch does it that way. (I notice that we have the pstate as first arg in many places; I put at the end for make_oper_cache_key, together with location. Is there some convention to have it as first arg?) > Also the diffs in parse_utilcmd.c are very confusing and seem to change > more code than is necessary --- why did that happen? The reason appears to be that Ryan wanted to have the pstate set, but that was only created after looking other things up, so he moved a largish block down; this was pretty bogus AFAICT. The attached patch fixes this by first creating the pstate, then doing the namespace lookup, then doing the rest of the setup. It's a bit disappointing to see so little changes in regression expected output ... -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachment
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Agreed; the attached patch does it that way. (I notice that we have the
> pstate as first arg in many places; I put at the end for
> make_oper_cache_key, together with location. Is there some convention
> to have it as first arg?)
Yes, parser-related functions always have pstate as first arg if they
need it at all. Please follow that convention.
regards, tom lane
Álvaro, I think, there are few open questions here and thus marking it back to "Waiting on Author". Please have your views on the review comments already posted. Also make changes as Tom suggested about placing pstate at the beginning. I am more concerned about this: 1. postgres=# create or replace function f1(a abc.test.id%type) returns int as $$ select 1; $$ language sql; ERROR: schema "abc" does not exist Is that expected? 2. Also what about pushing setup_parser_errposition_callback() inside func_get_detail() as well, just to limit it for namespacelookup? Thanks The new status of this patch is: Waiting on Author
Jeevan Chalke <jeevan.chalke@gmail.com> writes:
> I am more concerned about this:
> 1.
> postgres=# create or replace function
> f1(a abc.test.id%type) returns int as
> $$ select 1; $$
> language sql;
> ERROR: schema "abc" does not exist
> Is that expected?
Yes, or at least, if it's not what we want it's not this patch's fault.
That behavior is pre-existing.
> Also what about pushing setup_parser_errposition_callback() inside func_get_detail() as well, just to limit it for
namespacelookup?
Adding a pstate parameter to func_get_detail would be rather invasive;
not sure it's worth it.
regards, tom lane
Jeevan Chalke wrote: > Álvaro, > > I think, there are few open questions here and thus marking it back to "Waiting on Author". > > Please have your views on the review comments already posted. For some reason I missed your previous email. > Also make changes as Tom suggested about placing pstate at the beginning. Sure. > 1. > postgres=# create or replace function > f1(a abc.test.id%type) returns int as > $$ select 1; $$ > language sql; > ERROR: schema "abc" does not exist > > Is that expected? Type resolution for function arguments happens at execution time, in interpret_function_parameter_list() as called by CreateFunction(). We don't have a pstate at that point which is why location is not reported. Fixing that seems like a whole new project. > 2. > Also what about pushing setup_parser_errposition_callback() inside func_get_detail() as well, just to limit it for namespacelookup? Hm, that sounds more reasonable. Actually it means we have to drill all the way down to FuncnameGetCandidates. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Jeevan, Thanks for the review. Jeevan Chalke wrote: > I think, there are few open questions here and thus marking it back to "Waiting on Author". > > Please have your views on the review comments already posted. > Also make changes as Tom suggested about placing pstate at the beginning. Pushed with that one change. > postgres=# create or replace function > f1(a abc.test.id%type) returns int as > $$ select 1; $$ > language sql; > ERROR: schema "abc" does not exist > > Is that expected? We can leave this for a future patch. > 2. > Also what about pushing setup_parser_errposition_callback() inside func_get_detail() as well, just to limit it for namespacelookup? Seemed messy enough that I left this as-is; and as Tom said elsewhere, I think it's okay to have cursor position in other errors too. At the very least, the user will know for certain what's the function being processed that caused whatever failure. -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services