Thread: size of data stored in gist index

size of data stored in gist index

From
Gerhard Heift
Date:
Hello,

I try to index histograms in my table. For this I uses the cube contrib
module in which I removed the dimension check. If the cube has more than
255 dimensions, for example 256 ^= 4 + 4 + 256 * 2 * 8 = 4104 bytes, this
data can not be stored in the gist index any more. If I try it, I get the
following error:

PANIC:  failed to add item to index page in "histogram_idx"

Do I have to compress the data in any way or is it possible, to store index
data with this huge size?

Thanks,
  Gerhard

Attachment

Re: size of data stored in gist index

From
Tom Lane
Date:
Gerhard Heift <ml-postgresql-20081012-3518@gheift.de> writes:
> I try to index histograms in my table. For this I uses the cube contrib
> module in which I removed the dimension check. If the cube has more than
> 255 dimensions, for example 256 ^= 4 + 4 + 256 * 2 * 8 = 4104 bytes, this
> data can not be stored in the gist index any more. If I try it, I get the
> following error:

> PANIC:  failed to add item to index page in "histogram_idx"

> Do I have to compress the data in any way or is it possible, to store index
> data with this huge size?

Well, if you're going to turn cube into an unlimited-size datatype,
it would behoove you to make its compress and decompress routines
do something.

Still, it seems like gist ought to defend itself a bit better against
ill-considered datatypes.  Maybe put a check in gistFormTuple to verify
that the tuple isn't larger than can fit on one page?  Or is there a
better place to check it?

            regards, tom lane