Thread: reading row in backend
I'm just upgrading a database to v7.0, but I am rather unlucky in that in each of 2 large (ca. 20000 rows) tables, 1 row is too big eg: ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140 Now, it would help me if I could actually get a clue as to which row this is. The error message comes from hio.c:118 at which point one has a HeapTuple. Is there a way of extracting a piece of the row in the tuple to be able to identify it? It seems that a HeapTuple starts with a HeapTupleHeader, so the data I suppose starts at tuple->t_data[sizeof(HeapTupleHeaderData)] and goes on for tuple->t_len-sizeof(HeapTupleHeaderData) bytes, but how is it represented? Any pointers appreciated! (These are huge COPY statements, so after the error I'm left with an empty table - I'd just like to chop the 12 bytes off!) Cheers, Patrick
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
> ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140
> Now, it would help me if I could actually get a clue as to which row this
> is.
Um ... doesn't that line number in the COPY source data help any?
regards, tom lane
On Mon, May 15, 2000 at 06:55:50PM -0400, Tom Lane wrote: > Patrick Welche <prlw1@newn.cam.ac.uk> writes: > > ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140 > > > Now, it would help me if I could actually get a clue as to which row this > > is. > > Um ... doesn't that line number in the COPY source data help any? Unfortunately not - in fact the 2nd large tuple allegedly has a smaller line number than the first :( Am I barking up the wrong tree if I think that the HeapTuple contains the actual data? The first number in it would the primary key... Cheers, Patrick
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
>> Um ... doesn't that line number in the COPY source data help any?
> Unfortunately not - in fact the 2nd large tuple allegedly has a smaller
> line number than the first :(
Hmm, are you claiming that COPY is reporting a bogus line number?
If so, that needs to be looked into.
> Am I barking up the wrong tree if I think that the HeapTuple contains the
> actual data? The first number in it would the primary key...
Type HeapTuple is a pointer to a HeapTupleData, which is just
administrative overhead. The t_data field of the HeapTupleData
points at the actual tuple (a HeapTupleHeaderData followed by
null-values bitmap and then the user data). See include/access/htup.h
regards, tom lane
On Mon, May 15, 2000 at 07:22:30PM -0400, Tom Lane wrote: > Patrick Welche <prlw1@newn.cam.ac.uk> writes: > >> Um ... doesn't that line number in the COPY source data help any? > > > Unfortunately not - in fact the 2nd large tuple allegedly has a smaller > > line number than the first :( > > Hmm, are you claiming that COPY is reporting a bogus line number? > If so, that needs to be looked into. Yup: ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140 PQendcopy: resetting connection ERROR: copy: line 26714, Tuple is too big: size 8164, max size 8140 PQendcopy: resetting connection % wc /home/prlw1/db.out3631449 19833180 186847533 /home/prlw1/db.out and the second line is in a table that is over half way through the file. I'll take a look after some sleep.. > > Am I barking up the wrong tree if I think that the HeapTuple contains the > > actual data? The first number in it would the primary key... > > Type HeapTuple is a pointer to a HeapTupleData, which is just > administrative overhead. The t_data field of the HeapTupleData > points at the actual tuple (a HeapTupleHeaderData followed by > null-values bitmap and then the user data). See include/access/htup.h I think my question really is, is the user data meant to be just text? When I tried printing a block, I just got jibberish - it could well be the way I tried to print it, but I don't know what to expect.. Cheers, Patrick
Patrick Welche <prlw1@newn.cam.ac.uk> writes:
>> Hmm, are you claiming that COPY is reporting a bogus line number?
>> If so, that needs to be looked into.
> Yup:
> ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140
> PQendcopy: resetting connection
> ERROR: copy: line 26714, Tuple is too big: size 8164, max size 8140
> PQendcopy: resetting connection
> % wc /home/prlw1/db.out
> 3631449 19833180 186847533 /home/prlw1/db.out
> and the second line is in a table that is over half way through the file.
Looks reasonable enough to me. The line numbers are within the data
being fed to that copy command --- copy has no way to know that it's
part of some larger script...
regards, tom lane
On Mon, May 15, 2000 at 08:44:03PM -0400, Tom Lane wrote: > Patrick Welche <prlw1@newn.cam.ac.uk> writes: > >> Hmm, are you claiming that COPY is reporting a bogus line number? > >> If so, that needs to be looked into. > > > Yup: > > > ERROR: copy: line 57552, Tuple is too big: size 8152, max size 8140 > > PQendcopy: resetting connection > > ERROR: copy: line 26714, Tuple is too big: size 8164, max size 8140 > > PQendcopy: resetting connection > > > % wc /home/prlw1/db.out > > 3631449 19833180 186847533 /home/prlw1/db.out > > > and the second line is in a table that is over half way through the file. > > Looks reasonable enough to me. The line numbers are within the data > being fed to that copy command --- copy has no way to know that it's > part of some larger script... Ah! Thank you! So it's 57552 rows down from "COPY" ! OK Cheers, Patrick