Re: Manipulating complex types as non-contiguous structures in-memory - Mailing list pgsql-hackers
| From | Tom Lane |
|---|---|
| Subject | Re: Manipulating complex types as non-contiguous structures in-memory |
| Date | |
| Msg-id | 13296.1424381532@sss.pgh.pa.us Whole thread Raw |
| In response to | Re: Manipulating complex types as non-contiguous structures in-memory (Tom Lane <tgl@sss.pgh.pa.us>) |
| Responses |
Re: Manipulating complex types as non-contiguous structures in-memory
|
| List | pgsql-hackers |
Here's an 0.4 version, in which I've written some user docs, refactored
the array-specific code into a more reasonable arrangement, and adjusted
a lot of the built-in array functions to support expanded arrays directly.
This is about as far as I feel a need to take the latter activity, at
least for now; there are a few remaining operations that might be worth
converting but it's not clear they'd really offer much benefit.
I think this is actually now a serious candidate to be committed as-is,
not just a prototype. What we lack though is a clear understanding of
the performance characteristics.
regards, tom lane
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index d8c5287..e5b7b4b 100644
*** a/doc/src/sgml/storage.sgml
--- b/doc/src/sgml/storage.sgml
*************** comparison table, in which all the HTML
*** 503,510 ****
<acronym>TOAST</> pointers can point to data that is not on disk, but is
elsewhere in the memory of the current server process. Such pointers
obviously cannot be long-lived, but they are nonetheless useful. There
! is currently just one sub-case:
! pointers to <firstterm>indirect</> data.
</para>
<para>
--- 503,511 ----
<acronym>TOAST</> pointers can point to data that is not on disk, but is
elsewhere in the memory of the current server process. Such pointers
obviously cannot be long-lived, but they are nonetheless useful. There
! are currently two sub-cases:
! pointers to <firstterm>indirect</> data and
! pointers to <firstterm>expanded</> data.
</para>
<para>
*************** and there is no infrastructure to help w
*** 519,524 ****
--- 520,562 ----
</para>
<para>
+ Expanded <acronym>TOAST</> pointers are useful for complex data types
+ whose on-disk representation is not especially suited for computational
+ purposes. As an example, the standard varlena representation of a
+ <productname>PostgreSQL</> array includes dimensionality information, a
+ nulls bitmap if there are any null elements, then the values of all the
+ elements in order. When the element type itself is variable-length, the
+ only way to find the <replaceable>N</>'th element is to scan through all the
+ preceding elements. This representation is appropriate for on-disk storage
+ because of its compactness, but for computations with the array it's much
+ nicer to have an <quote>expanded</> or <quote>deconstructed</>
+ representation in which all the element starting locations have been
+ identified. The <acronym>TOAST</> pointer mechanism supports this need by
+ allowing a pass-by-reference Datum to point to either a standard varlena
+ value (the on-disk representation) or a <acronym>TOAST</> pointer that
+ points to an expanded representation somewhere in memory. The details of
+ this expanded representation are up to the data type, though it must have
+ a standard header and meet the other API requirements given
+ in <filename>src/include/utils/expandeddatum.h</>. C-level functions
+ working with the data type can choose to handle either representation.
+ Functions that do not know about the expanded representation, but simply
+ apply <function>PG_DETOAST_DATUM</> to their inputs, will automatically
+ receive the traditional varlena representation; so support for an expanded
+ representation can be introduced incrementally, one function at a time.
+ </para>
+
+ <para>
+ <acronym>TOAST</> pointers to expanded values are further broken down
+ into <firstterm>read-write</> and <firstterm>read-only</> pointers.
+ The pointed-to representation is the same either way, but a function that
+ receives a read-write pointer is allowed to modify the referenced value
+ in-place, whereas one that receives a read-only pointer must not; it must
+ first create a copy if it wants to make a modified version of the value.
+ This distinction and some associated conventions make it possible to avoid
+ unnecessary copying of expanded values during query execution.
+ </para>
+
+ <para>
For all types of in-memory <acronym>TOAST</> pointer, the <acronym>TOAST</>
management code ensures that no such pointer datum can accidentally get
stored on disk. In-memory <acronym>TOAST</> pointers are automatically
diff --git a/doc/src/sgml/xtypes.sgml b/doc/src/sgml/xtypes.sgml
index 2459616..ac0b8a2 100644
*** a/doc/src/sgml/xtypes.sgml
--- b/doc/src/sgml/xtypes.sgml
*************** CREATE TYPE complex (
*** 300,305 ****
--- 300,376 ----
</para>
</note>
+ <para>
+ Another feature that's enabled by <acronym>TOAST</> support is the
+ possibility of having an <firstterm>expanded</> in-memory data
+ representation that is more convenient to work with than the format that
+ is stored on disk. The regular or <quote>flat</> varlena storage format
+ is ultimately just a blob of bytes; it cannot for example contain
+ pointers, since it may get copied to other locations in memory.
+ For complex data types, the flat format may be quite expensive to work
+ with, so <productname>PostgreSQL</> provides a way to <quote>expand</>
+ the flat format into a representation that is more suited to computation,
+ and then pass that format in-memory between functions of the data type.
+ </para>
+
+ <para>
+ To use expanded storage, a data type must define an expanded format that
+ follows the rules given in <filename>src/include/utils/expandeddatum.h</>,
+ and provide functions to <quote>expand</> a flat varlena value into
+ expanded format and <quote>flatten</> the expanded format back to the
+ regular varlena representation. Then ensure that all C functions for
+ the data type can accept either representation, possibly by converting
+ one into the other immediately upon receipt. This does not require fixing
+ all existing functions for the data type at once, because the standard
+ <function>PG_DETOAST_DATUM</> macro is defined to convert expanded inputs
+ into regular flat format. Therefore, existing functions that work with
+ the flat varlena format will continue to work, though slightly
+ inefficiently, with expanded inputs; they need not be converted until and
+ unless better performance is important.
+ </para>
+
+ <para>
+ C functions that know how to work with an expanded representation
+ typically fall into two categories: those that can only handle expanded
+ format, and those that can handle either expanded or flat varlena inputs.
+ The former are easier to write but may be less efficient overall, because
+ converting a flat input to expanded form for use by a single function may
+ cost more than is saved by operating on the expanded format.
+ When only expanded format need be handled, conversion of flat inputs to
+ expanded form can be hidden inside an argument-fetching macro, so that
+ the function appears no more complex than one working with traditional
+ varlena input.
+ To handle both types of input, write an argument-fetching function that
+ will detoast external, short-header, and compressed varlena inputs, but
+ not expanded inputs. Such a function can be defined as returning a
+ pointer to a union of the flat varlena format and the expanded format.
+ Callers can use the <function>VARATT_IS_EXPANDED_HEADER()</> macro to
+ determine which format they received.
+ </para>
+
+ <para>
+ The <acronym>TOAST</> infrastructure not only allows regular varlena
+ values to be distinguished from expanded values, but also
+ distinguishes <quote>read-write</> and <quote>read-only</> pointers to
+ expanded values. C functions that only need to examine an expanded
+ value, or will only change it in safe and non-semantically-visible ways,
+ need not care which type of pointer they receive. C functions that
+ produce a modified version of an input value are allowed to modify an
+ expanded input value in-place if they receive a read-write pointer, but
+ must not modify the input if they receive a read-only pointer; in that
+ case they have to copy the value first, producing a new value to modify.
+ A C function that has constructed a new expanded value should always
+ return a read-write pointer to it. Also, a C function that is modifying
+ a read-write expanded value in-place should take care to leave the value
+ in a sane state if it fails partway through.
+ </para>
+
+ <para>
+ For examples of working with expanded values, see the standard array
+ infrastructure, particularly
+ <filename>src/backend/utils/adt/array_expanded.c</>.
+ </para>
+
</sect2>
</sect1>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 867035d..860ad78 100644
*** a/src/backend/access/common/heaptuple.c
--- b/src/backend/access/common/heaptuple.c
***************
*** 60,65 ****
--- 60,66 ----
#include "access/sysattr.h"
#include "access/tuptoaster.h"
#include "executor/tuptable.h"
+ #include "utils/expandeddatum.h"
/* Does att's datatype allow packing into the 1-byte-header varlena format? */
*************** heap_compute_data_size(TupleDesc tupleDe
*** 93,105 ****
for (i = 0; i < numberOfAttributes; i++)
{
Datum val;
if (isnull[i])
continue;
val = values[i];
! if (ATT_IS_PACKABLE(att[i]) &&
VARATT_CAN_MAKE_SHORT(DatumGetPointer(val)))
{
/*
--- 94,108 ----
for (i = 0; i < numberOfAttributes; i++)
{
Datum val;
+ Form_pg_attribute atti;
if (isnull[i])
continue;
val = values[i];
+ atti = att[i];
! if (ATT_IS_PACKABLE(atti) &&
VARATT_CAN_MAKE_SHORT(DatumGetPointer(val)))
{
/*
*************** heap_compute_data_size(TupleDesc tupleDe
*** 108,118 ****
*/
data_length += VARATT_CONVERTED_SHORT_SIZE(DatumGetPointer(val));
}
else
{
! data_length = att_align_datum(data_length, att[i]->attalign,
! att[i]->attlen, val);
! data_length = att_addlength_datum(data_length, att[i]->attlen,
val);
}
}
--- 111,131 ----
*/
data_length += VARATT_CONVERTED_SHORT_SIZE(DatumGetPointer(val));
}
+ else if (atti->attlen == -1 &&
+ VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(val)))
+ {
+ /*
+ * we want to flatten the expanded value so that the constructed
+ * tuple doesn't depend on it
+ */
+ data_length = att_align_nominal(data_length, atti->attalign);
+ data_length += EOH_get_flat_size(DatumGetEOHP(val));
+ }
else
{
! data_length = att_align_datum(data_length, atti->attalign,
! atti->attlen, val);
! data_length = att_addlength_datum(data_length, atti->attlen,
val);
}
}
*************** heap_fill_tuple(TupleDesc tupleDesc,
*** 203,212 ****
*infomask |= HEAP_HASVARWIDTH;
if (VARATT_IS_EXTERNAL(val))
{
! *infomask |= HEAP_HASEXTERNAL;
! /* no alignment, since it's short by definition */
! data_length = VARSIZE_EXTERNAL(val);
! memcpy(data, val, data_length);
}
else if (VARATT_IS_SHORT(val))
{
--- 216,241 ----
*infomask |= HEAP_HASVARWIDTH;
if (VARATT_IS_EXTERNAL(val))
{
! if (VARATT_IS_EXTERNAL_EXPANDED(val))
! {
! /*
! * we want to flatten the expanded value so that the
! * constructed tuple doesn't depend on it
! */
! ExpandedObjectHeader *eoh = DatumGetEOHP(values[i]);
!
! data = (char *) att_align_nominal(data,
! att[i]->attalign);
! data_length = EOH_get_flat_size(eoh);
! EOH_flatten_into(eoh, data, data_length);
! }
! else
! {
! *infomask |= HEAP_HASEXTERNAL;
! /* no alignment, since it's short by definition */
! data_length = VARSIZE_EXTERNAL(val);
! memcpy(data, val, data_length);
! }
}
else if (VARATT_IS_SHORT(val))
{
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index f8c1401..ebcbbc4 100644
*** a/src/backend/access/heap/tuptoaster.c
--- b/src/backend/access/heap/tuptoaster.c
***************
*** 37,42 ****
--- 37,43 ----
#include "catalog/catalog.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
+ #include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/typcache.h"
*************** heap_tuple_fetch_attr(struct varlena * a
*** 130,135 ****
--- 131,149 ----
result = (struct varlena *) palloc(VARSIZE_ANY(attr));
memcpy(result, attr, VARSIZE_ANY(attr));
}
+ else if (VARATT_IS_EXTERNAL_EXPANDED(attr))
+ {
+ /*
+ * This is an expanded-object pointer --- get flat format
+ */
+ ExpandedObjectHeader *eoh;
+ Size resultsize;
+
+ eoh = DatumGetEOHP(PointerGetDatum(attr));
+ resultsize = EOH_get_flat_size(eoh);
+ result = (struct varlena *) palloc(resultsize);
+ EOH_flatten_into(eoh, (void *) result, resultsize);
+ }
else
{
/*
*************** heap_tuple_untoast_attr(struct varlena *
*** 196,201 ****
--- 210,224 ----
attr = result;
}
}
+ else if (VARATT_IS_EXTERNAL_EXPANDED(attr))
+ {
+ /*
+ * This is an expanded-object pointer --- get flat format
+ */
+ attr = heap_tuple_fetch_attr(attr);
+ /* flatteners are not allowed to produce compressed/short output */
+ Assert(!VARATT_IS_EXTENDED(attr));
+ }
else if (VARATT_IS_COMPRESSED(attr))
{
/*
*************** heap_tuple_untoast_attr_slice(struct var
*** 263,268 ****
--- 286,296 ----
return heap_tuple_untoast_attr_slice(redirect.pointer,
sliceoffset, slicelength);
}
+ else if (VARATT_IS_EXTERNAL_EXPANDED(attr))
+ {
+ /* pass it off to heap_tuple_fetch_attr to flatten */
+ preslice = heap_tuple_fetch_attr(attr);
+ }
else
preslice = attr;
*************** toast_raw_datum_size(Datum value)
*** 344,349 ****
--- 372,381 ----
return toast_raw_datum_size(PointerGetDatum(toast_pointer.pointer));
}
+ else if (VARATT_IS_EXTERNAL_EXPANDED(attr))
+ {
+ result = EOH_get_flat_size(DatumGetEOHP(value));
+ }
else if (VARATT_IS_COMPRESSED(attr))
{
/* here, va_rawsize is just the payload size */
*************** toast_datum_size(Datum value)
*** 400,405 ****
--- 432,441 ----
return toast_datum_size(PointerGetDatum(toast_pointer.pointer));
}
+ else if (VARATT_IS_EXTERNAL_EXPANDED(attr))
+ {
+ result = EOH_get_flat_size(DatumGetEOHP(value));
+ }
else if (VARATT_IS_SHORT(attr))
{
result = VARSIZE_SHORT(attr);
diff --git a/src/backend/executor/execQual.c b/src/backend/executor/execQual.c
index fec76d4..7bdc201 100644
*** a/src/backend/executor/execQual.c
--- b/src/backend/executor/execQual.c
*************** ExecEvalArrayCoerceExpr(ArrayCoerceExprS
*** 4246,4252 ****
{
ArrayCoerceExpr *acoerce = (ArrayCoerceExpr *) astate->xprstate.expr;
Datum result;
- ArrayType *array;
FunctionCallInfoData locfcinfo;
result = ExecEvalExpr(astate->arg, econtext, isNull, isDone);
--- 4246,4251 ----
*************** ExecEvalArrayCoerceExpr(ArrayCoerceExprS
*** 4263,4276 ****
if (!OidIsValid(acoerce->elemfuncid))
{
/* Detoast input array if necessary, and copy in any case */
! array = DatumGetArrayTypePCopy(result);
ARR_ELEMTYPE(array) = astate->resultelemtype;
PG_RETURN_ARRAYTYPE_P(array);
}
- /* Detoast input array if necessary, but don't make a useless copy */
- array = DatumGetArrayTypeP(result);
-
/* Initialize function cache if first time through */
if (astate->elemfunc.fn_oid == InvalidOid)
{
--- 4262,4273 ----
if (!OidIsValid(acoerce->elemfuncid))
{
/* Detoast input array if necessary, and copy in any case */
! ArrayType *array = DatumGetArrayTypePCopy(result);
!
ARR_ELEMTYPE(array) = astate->resultelemtype;
PG_RETURN_ARRAYTYPE_P(array);
}
/* Initialize function cache if first time through */
if (astate->elemfunc.fn_oid == InvalidOid)
{
*************** ExecEvalArrayCoerceExpr(ArrayCoerceExprS
*** 4300,4314 ****
*/
InitFunctionCallInfoData(locfcinfo, &(astate->elemfunc), 3,
InvalidOid, NULL, NULL);
! locfcinfo.arg[0] = PointerGetDatum(array);
locfcinfo.arg[1] = Int32GetDatum(acoerce->resulttypmod);
locfcinfo.arg[2] = BoolGetDatum(acoerce->isExplicit);
locfcinfo.argnull[0] = false;
locfcinfo.argnull[1] = false;
locfcinfo.argnull[2] = false;
! return array_map(&locfcinfo, ARR_ELEMTYPE(array), astate->resultelemtype,
! astate->amstate);
}
/* ----------------------------------------------------------------
--- 4297,4310 ----
*/
InitFunctionCallInfoData(locfcinfo, &(astate->elemfunc), 3,
InvalidOid, NULL, NULL);
! locfcinfo.arg[0] = result;
locfcinfo.arg[1] = Int32GetDatum(acoerce->resulttypmod);
locfcinfo.arg[2] = BoolGetDatum(acoerce->isExplicit);
locfcinfo.argnull[0] = false;
locfcinfo.argnull[1] = false;
locfcinfo.argnull[2] = false;
! return array_map(&locfcinfo, astate->resultelemtype, astate->amstate);
}
/* ----------------------------------------------------------------
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 753754d..a05d8b1 100644
*** a/src/backend/executor/execTuples.c
--- b/src/backend/executor/execTuples.c
***************
*** 88,93 ****
--- 88,94 ----
#include "nodes/nodeFuncs.h"
#include "storage/bufmgr.h"
#include "utils/builtins.h"
+ #include "utils/expandeddatum.h"
#include "utils/lsyscache.h"
#include "utils/typcache.h"
*************** ExecCopySlot(TupleTableSlot *dstslot, Tu
*** 812,817 ****
--- 813,864 ----
return ExecStoreTuple(newTuple, dstslot, InvalidBuffer, true);
}
+ /* --------------------------------
+ * ExecMakeSlotContentsReadOnly
+ * Mark any R/W expanded datums in the slot as read-only.
+ *
+ * This is needed when a slot that might contain R/W datum references is to be
+ * used as input for general expression evaluation. Since the expression(s)
+ * might contain more than one Var referencing the same R/W datum, we could
+ * get wrong answers if functions acting on those Vars thought they could
+ * modify the expanded value in-place.
+ *
+ * For notational reasons, we return the same slot passed in.
+ * --------------------------------
+ */
+ TupleTableSlot *
+ ExecMakeSlotContentsReadOnly(TupleTableSlot *slot)
+ {
+ /*
+ * sanity checks
+ */
+ Assert(slot != NULL);
+ Assert(slot->tts_tupleDescriptor != NULL);
+ Assert(!slot->tts_isempty);
+
+ /*
+ * If the slot contains a physical tuple, it can't contain any expanded
+ * datums, because we flatten those when making a physical tuple. This
+ * might change later; but for now, we need do nothing unless the slot is
+ * virtual.
+ */
+ if (slot->tts_tuple == NULL)
+ {
+ Form_pg_attribute *att = slot->tts_tupleDescriptor->attrs;
+ int attnum;
+
+ for (attnum = 0; attnum < slot->tts_nvalid; attnum++)
+ {
+ slot->tts_values[attnum] =
+ MakeExpandedObjectReadOnly(slot->tts_values[attnum],
+ slot->tts_isnull[attnum],
+ att[attnum]->attlen);
+ }
+ }
+
+ return slot;
+ }
+
/* ----------------------------------------------------------------
* convenience initialization routines
diff --git a/src/backend/executor/nodeSubqueryscan.c b/src/backend/executor/nodeSubqueryscan.c
index 3f66e24..e5d1e54 100644
*** a/src/backend/executor/nodeSubqueryscan.c
--- b/src/backend/executor/nodeSubqueryscan.c
*************** SubqueryNext(SubqueryScanState *node)
*** 56,62 ****
--- 56,70 ----
* We just return the subplan's result slot, rather than expending extra
* cycles for ExecCopySlot(). (Our own ScanTupleSlot is used only for
* EvalPlanQual rechecks.)
+ *
+ * We do need to mark the slot contents read-only to prevent interference
+ * between different functions reading the same datum from the slot. It's
+ * a bit hokey to do this to the subplan's slot, but should be safe
+ * enough.
*/
+ if (!TupIsNull(slot))
+ slot = ExecMakeSlotContentsReadOnly(slot);
+
return slot;
}
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 4b86e91..daa9f69 100644
*** a/src/backend/executor/spi.c
--- b/src/backend/executor/spi.c
*************** SPI_pfree(void *pointer)
*** 1014,1019 ****
--- 1014,1040 ----
pfree(pointer);
}
+ Datum
+ SPI_datumTransfer(Datum value, bool typByVal, int typLen)
+ {
+ MemoryContext oldcxt = NULL;
+ Datum result;
+
+ if (_SPI_curid + 1 == _SPI_connected) /* connected */
+ {
+ if (_SPI_current != &(_SPI_stack[_SPI_curid + 1]))
+ elog(ERROR, "SPI stack corrupted");
+ oldcxt = MemoryContextSwitchTo(_SPI_current->savedcxt);
+ }
+
+ result = datumTransfer(value, typByVal, typLen);
+
+ if (oldcxt)
+ MemoryContextSwitchTo(oldcxt);
+
+ return result;
+ }
+
void
SPI_freetuple(HeapTuple tuple)
{
diff --git a/src/backend/utils/adt/Makefile b/src/backend/utils/adt/Makefile
index 20e5ff1..d1ed33f 100644
*** a/src/backend/utils/adt/Makefile
--- b/src/backend/utils/adt/Makefile
*************** endif
*** 16,25 ****
endif
# keep this list arranged alphabetically or it gets to be a mess
! OBJS = acl.o arrayfuncs.o array_selfuncs.o array_typanalyze.o \
! array_userfuncs.o arrayutils.o ascii.o bool.o \
! cash.o char.o date.o datetime.o datum.o dbsize.o domains.o \
! encode.o enum.o float.o format_type.o formatting.o genfile.o \
geo_ops.o geo_selfuncs.o inet_cidr_ntop.o inet_net_pton.o int.o \
int8.o json.o jsonb.o jsonb_gin.o jsonb_op.o jsonb_util.o \
jsonfuncs.o like.o lockfuncs.o mac.o misc.o nabstime.o name.o \
--- 16,26 ----
endif
# keep this list arranged alphabetically or it gets to be a mess
! OBJS = acl.o arrayfuncs.o array_expanded.o array_selfuncs.o \
! array_typanalyze.o array_userfuncs.o arrayutils.o ascii.o \
! bool.o cash.o char.o date.o datetime.o datum.o dbsize.o domains.o \
! encode.o enum.o expandeddatum.o \
! float.o format_type.o formatting.o genfile.o \
geo_ops.o geo_selfuncs.o inet_cidr_ntop.o inet_net_pton.o int.o \
int8.o json.o jsonb.o jsonb_gin.o jsonb_op.o jsonb_util.o \
jsonfuncs.o like.o lockfuncs.o mac.o misc.o nabstime.o name.o \
diff --git a/src/backend/utils/adt/array_expanded.c b/src/backend/utils/adt/array_expanded.c
index ...f89a533 .
*** a/src/backend/utils/adt/array_expanded.c
--- b/src/backend/utils/adt/array_expanded.c
***************
*** 0 ****
--- 1,356 ----
+ /*-------------------------------------------------------------------------
+ *
+ * array_expanded.c
+ * Basic functions for manipulating expanded arrays.
+ *
+ * Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/utils/adt/array_expanded.c
+ *
+ *-------------------------------------------------------------------------
+ */
+ #include "postgres.h"
+
+ #include "access/tupmacs.h"
+ #include "utils/array.h"
+ #include "utils/lsyscache.h"
+ #include "utils/memutils.h"
+
+
+ /* "Methods" required for an expanded object */
+ static Size EA_get_flat_size(ExpandedObjectHeader *eohptr);
+ static void EA_flatten_into(ExpandedObjectHeader *eohptr,
+ void *result, Size allocated_size);
+
+ static const ExpandedObjectMethods EA_methods =
+ {
+ EA_get_flat_size,
+ EA_flatten_into
+ };
+
+
+ /*
+ * expand_array: convert an array Datum into an expanded array
+ *
+ * The expanded object will be a child of parentcontext.
+ *
+ * Caller can provide element type's representational data; we do that because
+ * caller is often in a position to cache it across repeated calls. If the
+ * caller can't do that, pass zeroes for elmlen/elmbyval/elmalign.
+ */
+ Datum
+ expand_array(Datum arraydatum, MemoryContext parentcontext,
+ int elmlen, bool elmbyval, char elmalign)
+ {
+ ArrayType *array;
+ ExpandedArrayHeader *eah;
+ MemoryContext objcxt;
+ MemoryContext oldcxt;
+
+ /*
+ * Allocate private context for expanded object. We start by assuming
+ * that the array won't be very large; but if it does grow a lot, don't
+ * constrain aset.c's large-context behavior.
+ */
+ objcxt = AllocSetContextCreate(parentcontext,
+ "expanded array",
+ ALLOCSET_SMALL_MINSIZE,
+ ALLOCSET_SMALL_INITSIZE,
+ ALLOCSET_DEFAULT_MAXSIZE);
+
+ /* Set up expanded array header */
+ eah = (ExpandedArrayHeader *)
+ MemoryContextAlloc(objcxt, sizeof(ExpandedArrayHeader));
+
+ EOH_init_header(&eah->hdr, &EA_methods, objcxt);
+ eah->ea_magic = EA_MAGIC;
+
+ /*
+ * Detoast and copy original array into private context, as a flat array.
+ * We flatten it even if it's in expanded form; it's not clear that adding
+ * a special-case path for that would be worth the trouble.
+ *
+ * Note that this coding risks leaking some memory in the private context
+ * if we have to fetch data from a TOAST table; however, experimentation
+ * says that the leak is minimal. Doing it this way saves a copy step,
+ * which seems worthwhile, especially if the array is large enough to need
+ * external storage.
+ */
+ oldcxt = MemoryContextSwitchTo(objcxt);
+ array = DatumGetArrayTypePCopy(arraydatum);
+ MemoryContextSwitchTo(oldcxt);
+
+ eah->ndims = ARR_NDIM(array);
+ /* note these pointers point into the fvalue header! */
+ eah->dims = ARR_DIMS(array);
+ eah->lbound = ARR_LBOUND(array);
+
+ /* save array's element-type data for possible use later */
+ eah->element_type = ARR_ELEMTYPE(array);
+ if (elmlen)
+ {
+ /* Caller provided representational data */
+ eah->typlen = elmlen;
+ eah->typbyval = elmbyval;
+ eah->typalign = elmalign;
+ }
+ else
+ {
+ /* No, so look it up */
+ get_typlenbyvalalign(eah->element_type,
+ &eah->typlen,
+ &eah->typbyval,
+ &eah->typalign);
+ }
+
+ /* we don't make a deconstructed representation now */
+ eah->dvalues = NULL;
+ eah->dnulls = NULL;
+ eah->dvalueslen = 0;
+ eah->nelems = 0;
+ eah->flat_size = 0;
+
+ /* remember we have a flat representation */
+ eah->fvalue = array;
+ eah->fstartptr = ARR_DATA_PTR(array);
+ eah->fendptr = ((char *) array) + ARR_SIZE(array);
+
+ /* return a R/W pointer to the expanded array */
+ return EOHPGetRWDatum(&eah->hdr);
+ }
+
+ /*
+ * get_flat_size method for expanded arrays
+ */
+ static Size
+ EA_get_flat_size(ExpandedObjectHeader *eohptr)
+ {
+ ExpandedArrayHeader *eah = (ExpandedArrayHeader *) eohptr;
+ int nelems;
+ int ndims;
+ Datum *dvalues;
+ bool *dnulls;
+ Size nbytes;
+ int i;
+
+ Assert(eah->ea_magic == EA_MAGIC);
+
+ /* Easy if we have a valid flattened value */
+ if (eah->fvalue)
+ return ARR_SIZE(eah->fvalue);
+
+ /* If we have a cached size value, believe that */
+ if (eah->flat_size)
+ return eah->flat_size;
+
+ /*
+ * Compute space needed by examining dvalues/dnulls. Note that the result
+ * array will have a nulls bitmap if dnulls isn't NULL, even if the array
+ * doesn't actually contain any nulls now.
+ */
+ nelems = eah->nelems;
+ ndims = eah->ndims;
+ Assert(nelems == ArrayGetNItems(ndims, eah->dims));
+ dvalues = eah->dvalues;
+ dnulls = eah->dnulls;
+ nbytes = 0;
+ for (i = 0; i < nelems; i++)
+ {
+ if (dnulls && dnulls[i])
+ continue;
+ nbytes = att_addlength_datum(nbytes, eah->typlen, dvalues[i]);
+ nbytes = att_align_nominal(nbytes, eah->typalign);
+ /* check for overflow of total request */
+ if (!AllocSizeIsValid(nbytes))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
+ errmsg("array size exceeds the maximum allowed (%d)",
+ (int) MaxAllocSize)));
+ }
+
+ if (dnulls)
+ nbytes += ARR_OVERHEAD_WITHNULLS(ndims, nelems);
+ else
+ nbytes += ARR_OVERHEAD_NONULLS(ndims);
+
+ /* cache for next time */
+ eah->flat_size = nbytes;
+
+ return nbytes;
+ }
+
+ /*
+ * flatten_into method for expanded arrays
+ */
+ static void
+ EA_flatten_into(ExpandedObjectHeader *eohptr,
+ void *result, Size allocated_size)
+ {
+ ExpandedArrayHeader *eah = (ExpandedArrayHeader *) eohptr;
+ ArrayType *aresult = (ArrayType *) result;
+ int nelems;
+ int ndims;
+ int32 dataoffset;
+
+ Assert(eah->ea_magic == EA_MAGIC);
+
+ /* Easy if we have a valid flattened value */
+ if (eah->fvalue)
+ {
+ Assert(allocated_size == ARR_SIZE(eah->fvalue));
+ memcpy(result, eah->fvalue, allocated_size);
+ return;
+ }
+
+ /* Else allocation should match previous get_flat_size result */
+ Assert(allocated_size == eah->flat_size);
+
+ /* Fill result array from dvalues/dnulls */
+ nelems = eah->nelems;
+ ndims = eah->ndims;
+
+ if (eah->dnulls)
+ dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, nelems);
+ else
+ dataoffset = 0; /* marker for no null bitmap */
+
+ /* We must ensure that any pad space is zero-filled */
+ memset(aresult, 0, allocated_size);
+
+ SET_VARSIZE(aresult, allocated_size);
+ aresult->ndim = ndims;
+ aresult->dataoffset = dataoffset;
+ aresult->elemtype = eah->element_type;
+ memcpy(ARR_DIMS(aresult), eah->dims, ndims * sizeof(int));
+ memcpy(ARR_LBOUND(aresult), eah->lbound, ndims * sizeof(int));
+
+ CopyArrayEls(aresult,
+ eah->dvalues, eah->dnulls, nelems,
+ eah->typlen, eah->typbyval, eah->typalign,
+ false);
+ }
+
+ /*
+ * Argument fetching support code
+ */
+
+ /*
+ * DatumGetExpandedArray: get a writable expanded array from an input argument
+ */
+ ExpandedArrayHeader *
+ DatumGetExpandedArray(Datum d)
+ {
+ ExpandedArrayHeader *eah;
+
+ /* If it's a writable expanded array already, just return it */
+ if (VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(d)))
+ {
+ eah = (ExpandedArrayHeader *) DatumGetEOHP(d);
+ Assert(eah->ea_magic == EA_MAGIC);
+ return eah;
+ }
+
+ /*
+ * If it's a non-writable expanded array, copy it, extracting the element
+ * representational data to save a catalog lookup.
+ */
+ if (VARATT_IS_EXTERNAL_EXPANDED_RO(DatumGetPointer(d)))
+ {
+ eah = (ExpandedArrayHeader *) DatumGetEOHP(d);
+ Assert(eah->ea_magic == EA_MAGIC);
+ d = expand_array(d, CurrentMemoryContext,
+ eah->typlen, eah->typbyval, eah->typalign);
+ return (ExpandedArrayHeader *) DatumGetEOHP(d);
+ }
+
+ /* Else expand the hard way */
+ d = expand_array(d, CurrentMemoryContext, 0, 0, 0);
+ return (ExpandedArrayHeader *) DatumGetEOHP(d);
+ }
+
+ /*
+ * As above, when caller has the ability to cache element type info
+ */
+ ExpandedArrayHeader *
+ DatumGetExpandedArrayX(Datum d,
+ int elmlen, bool elmbyval, char elmalign)
+ {
+ ExpandedArrayHeader *eah;
+
+ /* If it's a writable expanded array already, just return it */
+ if (VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(d)))
+ {
+ eah = (ExpandedArrayHeader *) DatumGetEOHP(d);
+ Assert(eah->ea_magic == EA_MAGIC);
+ Assert(eah->typlen == elmlen);
+ Assert(eah->typbyval == elmbyval);
+ Assert(eah->typalign == elmalign);
+ return eah;
+ }
+
+ /* Else expand using caller's data */
+ d = expand_array(d, CurrentMemoryContext, elmlen, elmbyval, elmalign);
+ return (ExpandedArrayHeader *) DatumGetEOHP(d);
+ }
+
+ /*
+ * DatumGetAnyArray: return either an expanded array or a detoasted varlena
+ * array. The result must not be modified in-place.
+ */
+ AnyArrayType *
+ DatumGetAnyArray(Datum d)
+ {
+ ExpandedArrayHeader *eah;
+
+ /*
+ * If it's an expanded array (RW or RO), return the header pointer.
+ */
+ if (VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(d)))
+ {
+ eah = (ExpandedArrayHeader *) DatumGetEOHP(d);
+ Assert(eah->ea_magic == EA_MAGIC);
+ return (AnyArrayType *) eah;
+ }
+
+ /* Else do regular detoasting as needed */
+ return (AnyArrayType *) PG_DETOAST_DATUM(d);
+ }
+
+ /*
+ * Create the Datum/isnull representation of an expanded array object
+ * if we didn't do so previously
+ */
+ void
+ deconstruct_expanded_array(ExpandedArrayHeader *eah)
+ {
+ if (eah->dvalues == NULL)
+ {
+ MemoryContext oldcxt = MemoryContextSwitchTo(eah->hdr.eoh_context);
+ Datum *dvalues;
+ bool *dnulls;
+ int nelems;
+
+ dnulls = NULL;
+ deconstruct_array(eah->fvalue,
+ eah->element_type,
+ eah->typlen, eah->typbyval, eah->typalign,
+ &dvalues,
+ ARR_HASNULL(eah->fvalue) ? &dnulls : NULL,
+ &nelems);
+
+ /*
+ * Update header only after successful completion of this step. If
+ * deconstruct_array fails partway through, worst consequence is some
+ * leaked memory in the object's context. If the caller fails at a
+ * later point, that's fine, since the deconstructed representation is
+ * valid anyhow.
+ */
+ eah->dvalues = dvalues;
+ eah->dnulls = dnulls;
+ eah->dvalueslen = eah->nelems = nelems;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ }
diff --git a/src/backend/utils/adt/array_userfuncs.c b/src/backend/utils/adt/array_userfuncs.c
index 5c20d0c..755c50e 100644
*** a/src/backend/utils/adt/array_userfuncs.c
--- b/src/backend/utils/adt/array_userfuncs.c
***************
*** 20,33 ****
/*
* fetch_array_arg_replace_nulls
*
! * Fetch an array-valued argument; if it's null, construct an empty array
! * value of the proper data type. Also cache basic element type information
! * in fn_extra.
*/
! static ArrayType *
fetch_array_arg_replace_nulls(FunctionCallInfo fcinfo, int argno)
{
! ArrayType *v;
ArrayMetaState *my_extra;
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
--- 20,33 ----
/*
* fetch_array_arg_replace_nulls
*
! * Fetch an array-valued argument in expanded form; if it's null, construct an
! * empty array value of the proper data type. Also cache basic element type
! * information in fn_extra.
*/
! static ExpandedArrayHeader *
fetch_array_arg_replace_nulls(FunctionCallInfo fcinfo, int argno)
{
! ExpandedArrayHeader *eah;
ArrayMetaState *my_extra;
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
*************** fetch_array_arg_replace_nulls(FunctionCa
*** 63,73 ****
/* Now we can collect the array value */
if (PG_ARGISNULL(argno))
! v = construct_empty_array(my_extra->element_type);
else
! v = PG_GETARG_ARRAYTYPE_P(argno);
! return v;
}
/*-----------------------------------------------------------------------------
--- 63,80 ----
/* Now we can collect the array value */
if (PG_ARGISNULL(argno))
! eah = construct_empty_expanded_array(my_extra->element_type,
! CurrentMemoryContext,
! my_extra->typlen,
! my_extra->typbyval,
! my_extra->typalign);
else
! eah = PG_GETARG_EXPANDED_ARRAYX(argno,
! my_extra->typlen,
! my_extra->typbyval,
! my_extra->typalign);
! return eah;
}
/*-----------------------------------------------------------------------------
*************** fetch_array_arg_replace_nulls(FunctionCa
*** 78,106 ****
Datum
array_append(PG_FUNCTION_ARGS)
{
! ArrayType *v;
Datum newelem;
bool isNull;
! ArrayType *result;
int *dimv,
*lb;
int indx;
ArrayMetaState *my_extra;
! v = fetch_array_arg_replace_nulls(fcinfo, 0);
isNull = PG_ARGISNULL(1);
if (isNull)
newelem = (Datum) 0;
else
newelem = PG_GETARG_DATUM(1);
! if (ARR_NDIM(v) == 1)
{
/* append newelem */
int ub;
! lb = ARR_LBOUND(v);
! dimv = ARR_DIMS(v);
ub = dimv[0] + lb[0] - 1;
indx = ub + 1;
--- 85,113 ----
Datum
array_append(PG_FUNCTION_ARGS)
{
! ExpandedArrayHeader *eah;
Datum newelem;
bool isNull;
! Datum result;
int *dimv,
*lb;
int indx;
ArrayMetaState *my_extra;
! eah = fetch_array_arg_replace_nulls(fcinfo, 0);
isNull = PG_ARGISNULL(1);
if (isNull)
newelem = (Datum) 0;
else
newelem = PG_GETARG_DATUM(1);
! if (eah->ndims == 1)
{
/* append newelem */
int ub;
! lb = eah->lbound;
! dimv = eah->dims;
ub = dimv[0] + lb[0] - 1;
indx = ub + 1;
*************** array_append(PG_FUNCTION_ARGS)
*** 110,116 ****
(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
errmsg("integer out of range")));
}
! else if (ARR_NDIM(v) == 0)
indx = 1;
else
ereport(ERROR,
--- 117,123 ----
(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
errmsg("integer out of range")));
}
! else if (eah->ndims == 0)
indx = 1;
else
ereport(ERROR,
*************** array_append(PG_FUNCTION_ARGS)
*** 120,129 ****
/* Perform element insertion */
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
! result = array_set(v, 1, &indx, newelem, isNull,
-1, my_extra->typlen, my_extra->typbyval, my_extra->typalign);
! PG_RETURN_ARRAYTYPE_P(result);
}
/*-----------------------------------------------------------------------------
--- 127,137 ----
/* Perform element insertion */
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
! result = array_set_element(EOHPGetRWDatum(&eah->hdr),
! 1, &indx, newelem, isNull,
-1, my_extra->typlen, my_extra->typbyval, my_extra->typalign);
! PG_RETURN_DATUM(result);
}
/*-----------------------------------------------------------------------------
*************** array_append(PG_FUNCTION_ARGS)
*** 134,146 ****
Datum
array_prepend(PG_FUNCTION_ARGS)
{
! ArrayType *v;
Datum newelem;
bool isNull;
! ArrayType *result;
int *dimv,
*lb;
int indx;
ArrayMetaState *my_extra;
isNull = PG_ARGISNULL(0);
--- 142,155 ----
Datum
array_prepend(PG_FUNCTION_ARGS)
{
! ExpandedArrayHeader *eah;
Datum newelem;
bool isNull;
! Datum result;
int *dimv,
*lb;
int indx;
+ int lb0;
ArrayMetaState *my_extra;
isNull = PG_ARGISNULL(0);
*************** array_prepend(PG_FUNCTION_ARGS)
*** 148,161 ****
newelem = (Datum) 0;
else
newelem = PG_GETARG_DATUM(0);
! v = fetch_array_arg_replace_nulls(fcinfo, 1);
! if (ARR_NDIM(v) == 1)
{
/* prepend newelem */
! lb = ARR_LBOUND(v);
! dimv = ARR_DIMS(v);
indx = lb[0] - 1;
/* overflow? */
if (indx > lb[0])
--- 157,171 ----
newelem = (Datum) 0;
else
newelem = PG_GETARG_DATUM(0);
! eah = fetch_array_arg_replace_nulls(fcinfo, 1);
! if (eah->ndims == 1)
{
/* prepend newelem */
! lb = eah->lbound;
! dimv = eah->dims;
indx = lb[0] - 1;
+ lb0 = lb[0];
/* overflow? */
if (indx > lb[0])
*************** array_prepend(PG_FUNCTION_ARGS)
*** 163,170 ****
(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
errmsg("integer out of range")));
}
! else if (ARR_NDIM(v) == 0)
indx = 1;
else
ereport(ERROR,
(errcode(ERRCODE_DATA_EXCEPTION),
--- 173,183 ----
(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
errmsg("integer out of range")));
}
! else if (eah->ndims == 0)
! {
indx = 1;
+ lb0 = 1;
+ }
else
ereport(ERROR,
(errcode(ERRCODE_DATA_EXCEPTION),
*************** array_prepend(PG_FUNCTION_ARGS)
*** 173,186 ****
/* Perform element insertion */
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
! result = array_set(v, 1, &indx, newelem, isNull,
-1, my_extra->typlen, my_extra->typbyval, my_extra->typalign);
/* Readjust result's LB to match the input's, as expected for prepend */
! if (ARR_NDIM(v) == 1)
! ARR_LBOUND(result)[0] = ARR_LBOUND(v)[0];
! PG_RETURN_ARRAYTYPE_P(result);
}
/*-----------------------------------------------------------------------------
--- 186,204 ----
/* Perform element insertion */
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
! result = array_set_element(EOHPGetRWDatum(&eah->hdr),
! 1, &indx, newelem, isNull,
-1, my_extra->typlen, my_extra->typbyval, my_extra->typalign);
/* Readjust result's LB to match the input's, as expected for prepend */
! Assert(result == EOHPGetRWDatum(&eah->hdr));
! if (eah->ndims == 1)
! {
! /* This is ok whether we've deconstructed or not */
! eah->lbound[0] = lb0;
! }
! PG_RETURN_DATUM(result);
}
/*-----------------------------------------------------------------------------
diff --git a/src/backend/utils/adt/arrayfuncs.c b/src/backend/utils/adt/arrayfuncs.c
index 79aefaf..9b3d58e 100644
*** a/src/backend/utils/adt/arrayfuncs.c
--- b/src/backend/utils/adt/arrayfuncs.c
*************** bool Array_nulls = true;
*** 42,47 ****
--- 42,53 ----
*/
#define ASSGN "="
+ #define AARR_FREE_IF_COPY(array,n) \
+ do { \
+ if (!VARATT_IS_EXPANDED_HEADER(array)) \
+ PG_FREE_IF_COPY(array, n); \
+ } while (0)
+
typedef enum
{
ARRAY_NO_LEVEL,
*************** static void ReadArrayBinary(StringInfo b
*** 93,102 ****
int typlen, bool typbyval, char typalign,
Datum *values, bool *nulls,
bool *hasnulls, int32 *nbytes);
! static void CopyArrayEls(ArrayType *array,
! Datum *values, bool *nulls, int nitems,
! int typlen, bool typbyval, char typalign,
! bool freedata);
static bool array_get_isnull(const bits8 *nullbitmap, int offset);
static void array_set_isnull(bits8 *nullbitmap, int offset, bool isNull);
static Datum ArrayCast(char *value, bool byval, int len);
--- 99,114 ----
int typlen, bool typbyval, char typalign,
Datum *values, bool *nulls,
bool *hasnulls, int32 *nbytes);
! static Datum array_get_element_expanded(Datum arraydatum,
! int nSubscripts, int *indx,
! int arraytyplen,
! int elmlen, bool elmbyval, char elmalign,
! bool *isNull);
! static Datum array_set_element_expanded(Datum arraydatum,
! int nSubscripts, int *indx,
! Datum dataValue, bool isNull,
! int arraytyplen,
! int elmlen, bool elmbyval, char elmalign);
static bool array_get_isnull(const bits8 *nullbitmap, int offset);
static void array_set_isnull(bits8 *nullbitmap, int offset, bool isNull);
static Datum ArrayCast(char *value, bool byval, int len);
*************** ReadArrayStr(char *arrayStr,
*** 939,945 ****
* the values are not toasted. (Doing it here doesn't work since the
* caller has already allocated space for the array...)
*/
! static void
CopyArrayEls(ArrayType *array,
Datum *values,
bool *nulls,
--- 951,957 ----
* the values are not toasted. (Doing it here doesn't work since the
* caller has already allocated space for the array...)
*/
! void
CopyArrayEls(ArrayType *array,
Datum *values,
bool *nulls,
*************** CopyArrayEls(ArrayType *array,
*** 997,1004 ****
Datum
array_out(PG_FUNCTION_ARGS)
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
! Oid element_type = ARR_ELEMTYPE(v);
int typlen;
bool typbyval;
char typalign;
--- 1009,1016 ----
Datum
array_out(PG_FUNCTION_ARGS)
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
! Oid element_type = AARR_ELEMTYPE(v);
int typlen;
bool typbyval;
char typalign;
*************** array_out(PG_FUNCTION_ARGS)
*** 1014,1021 ****
*
* +2 allows for assignment operator + trailing null
*/
- bits8 *bitmap;
- int bitmask;
bool *needquotes,
needdims = false;
int nitems,
--- 1026,1031 ----
*************** array_out(PG_FUNCTION_ARGS)
*** 1027,1032 ****
--- 1037,1043 ----
int ndim,
*dims,
*lb;
+ ARRAY_ITER ARRAY_ITER_VARS(iter);
ArrayMetaState *my_extra;
/*
*************** array_out(PG_FUNCTION_ARGS)
*** 1061,1069 ****
typalign = my_extra->typalign;
typdelim = my_extra->typdelim;
! ndim = ARR_NDIM(v);
! dims = ARR_DIMS(v);
! lb = ARR_LBOUND(v);
nitems = ArrayGetNItems(ndim, dims);
if (nitems == 0)
--- 1072,1080 ----
typalign = my_extra->typalign;
typdelim = my_extra->typdelim;
! ndim = AARR_NDIM(v);
! dims = AARR_DIMS(v);
! lb = AARR_LBOUND(v);
nitems = ArrayGetNItems(ndim, dims);
if (nitems == 0)
*************** array_out(PG_FUNCTION_ARGS)
*** 1094,1109 ****
needquotes = (bool *) palloc(nitems * sizeof(bool));
overall_length = 1; /* don't forget to count \0 at end. */
! p = ARR_DATA_PTR(v);
! bitmap = ARR_NULLBITMAP(v);
! bitmask = 1;
for (i = 0; i < nitems; i++)
{
bool needquote;
/* Get source element, checking for NULL */
! if (bitmap && (*bitmap & bitmask) == 0)
{
values[i] = pstrdup("NULL");
overall_length += 4;
--- 1105,1122 ----
needquotes = (bool *) palloc(nitems * sizeof(bool));
overall_length = 1; /* don't forget to count \0 at end. */
! ARRAY_ITER_SETUP(iter, v);
for (i = 0; i < nitems; i++)
{
+ Datum itemvalue;
+ bool isnull;
bool needquote;
/* Get source element, checking for NULL */
! ARRAY_ITER_NEXT(iter, i, itemvalue, isnull, typlen, typbyval, typalign);
!
! if (isnull)
{
values[i] = pstrdup("NULL");
overall_length += 4;
*************** array_out(PG_FUNCTION_ARGS)
*** 1111,1122 ****
}
else
{
- Datum itemvalue;
-
- itemvalue = fetch_att(p, typbyval, typlen);
values[i] = OutputFunctionCall(&my_extra->proc, itemvalue);
- p = att_addlength_pointer(p, typlen, p);
- p = (char *) att_align_nominal(p, typalign);
/* count data plus backslashes; detect chars needing quotes */
if (values[i][0] == '\0')
--- 1124,1130 ----
*************** array_out(PG_FUNCTION_ARGS)
*** 1149,1165 ****
overall_length += 2;
/* and the comma */
overall_length += 1;
-
- /* advance bitmap pointer if any */
- if (bitmap)
- {
- bitmask <<= 1;
- if (bitmask == 0x100)
- {
- bitmap++;
- bitmask = 1;
- }
- }
}
/*
--- 1157,1162 ----
*************** ReadArrayBinary(StringInfo buf,
*** 1534,1552 ****
Datum
array_send(PG_FUNCTION_ARGS)
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
! Oid element_type = ARR_ELEMTYPE(v);
int typlen;
bool typbyval;
char typalign;
- char *p;
- bits8 *bitmap;
- int bitmask;
int nitems,
i;
int ndim,
! *dim;
StringInfoData buf;
ArrayMetaState *my_extra;
/*
--- 1531,1548 ----
Datum
array_send(PG_FUNCTION_ARGS)
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
! Oid element_type = AARR_ELEMTYPE(v);
int typlen;
bool typbyval;
char typalign;
int nitems,
i;
int ndim,
! *dim,
! *lb;
StringInfoData buf;
+ ARRAY_ITER ARRAY_ITER_VARS(iter);
ArrayMetaState *my_extra;
/*
*************** array_send(PG_FUNCTION_ARGS)
*** 1583,1642 ****
typbyval = my_extra->typbyval;
typalign = my_extra->typalign;
! ndim = ARR_NDIM(v);
! dim = ARR_DIMS(v);
nitems = ArrayGetNItems(ndim, dim);
pq_begintypsend(&buf);
/* Send the array header information */
pq_sendint(&buf, ndim, 4);
! pq_sendint(&buf, ARR_HASNULL(v) ? 1 : 0, 4);
pq_sendint(&buf, element_type, sizeof(Oid));
for (i = 0; i < ndim; i++)
{
! pq_sendint(&buf, ARR_DIMS(v)[i], 4);
! pq_sendint(&buf, ARR_LBOUND(v)[i], 4);
}
/* Send the array elements using the element's own sendproc */
! p = ARR_DATA_PTR(v);
! bitmap = ARR_NULLBITMAP(v);
! bitmask = 1;
for (i = 0; i < nitems; i++)
{
/* Get source element, checking for NULL */
! if (bitmap && (*bitmap & bitmask) == 0)
{
/* -1 length means a NULL */
pq_sendint(&buf, -1, 4);
}
else
{
- Datum itemvalue;
bytea *outputbytes;
- itemvalue = fetch_att(p, typbyval, typlen);
outputbytes = SendFunctionCall(&my_extra->proc, itemvalue);
pq_sendint(&buf, VARSIZE(outputbytes) - VARHDRSZ, 4);
pq_sendbytes(&buf, VARDATA(outputbytes),
VARSIZE(outputbytes) - VARHDRSZ);
pfree(outputbytes);
-
- p = att_addlength_pointer(p, typlen, p);
- p = (char *) att_align_nominal(p, typalign);
- }
-
- /* advance bitmap pointer if any */
- if (bitmap)
- {
- bitmask <<= 1;
- if (bitmask == 0x100)
- {
- bitmap++;
- bitmask = 1;
- }
}
}
--- 1579,1626 ----
typbyval = my_extra->typbyval;
typalign = my_extra->typalign;
! ndim = AARR_NDIM(v);
! dim = AARR_DIMS(v);
! lb = AARR_LBOUND(v);
nitems = ArrayGetNItems(ndim, dim);
pq_begintypsend(&buf);
/* Send the array header information */
pq_sendint(&buf, ndim, 4);
! pq_sendint(&buf, AARR_HASNULL(v) ? 1 : 0, 4);
pq_sendint(&buf, element_type, sizeof(Oid));
for (i = 0; i < ndim; i++)
{
! pq_sendint(&buf, dim[i], 4);
! pq_sendint(&buf, lb[i], 4);
}
/* Send the array elements using the element's own sendproc */
! ARRAY_ITER_SETUP(iter, v);
for (i = 0; i < nitems; i++)
{
+ Datum itemvalue;
+ bool isnull;
+
/* Get source element, checking for NULL */
! ARRAY_ITER_NEXT(iter, i, itemvalue, isnull, typlen, typbyval, typalign);
!
! if (isnull)
{
/* -1 length means a NULL */
pq_sendint(&buf, -1, 4);
}
else
{
bytea *outputbytes;
outputbytes = SendFunctionCall(&my_extra->proc, itemvalue);
pq_sendint(&buf, VARSIZE(outputbytes) - VARHDRSZ, 4);
pq_sendbytes(&buf, VARDATA(outputbytes),
VARSIZE(outputbytes) - VARHDRSZ);
pfree(outputbytes);
}
}
*************** array_send(PG_FUNCTION_ARGS)
*** 1650,1662 ****
Datum
array_ndims(PG_FUNCTION_ARGS)
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
/* Sanity check: does it look like an array at all? */
! if (ARR_NDIM(v) <= 0 || ARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
! PG_RETURN_INT32(ARR_NDIM(v));
}
/*
--- 1634,1646 ----
Datum
array_ndims(PG_FUNCTION_ARGS)
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
/* Sanity check: does it look like an array at all? */
! if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
! PG_RETURN_INT32(AARR_NDIM(v));
}
/*
*************** array_ndims(PG_FUNCTION_ARGS)
*** 1666,1672 ****
Datum
array_dims(PG_FUNCTION_ARGS)
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
char *p;
int i;
int *dimv,
--- 1650,1656 ----
Datum
array_dims(PG_FUNCTION_ARGS)
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
char *p;
int i;
int *dimv,
*************** array_dims(PG_FUNCTION_ARGS)
*** 1680,1693 ****
char buf[MAXDIM * 33 + 1];
/* Sanity check: does it look like an array at all? */
! if (ARR_NDIM(v) <= 0 || ARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
! dimv = ARR_DIMS(v);
! lb = ARR_LBOUND(v);
p = buf;
! for (i = 0; i < ARR_NDIM(v); i++)
{
sprintf(p, "[%d:%d]", lb[i], dimv[i] + lb[i] - 1);
p += strlen(p);
--- 1664,1677 ----
char buf[MAXDIM * 33 + 1];
/* Sanity check: does it look like an array at all? */
! if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
! dimv = AARR_DIMS(v);
! lb = AARR_LBOUND(v);
p = buf;
! for (i = 0; i < AARR_NDIM(v); i++)
{
sprintf(p, "[%d:%d]", lb[i], dimv[i] + lb[i] - 1);
p += strlen(p);
*************** array_dims(PG_FUNCTION_ARGS)
*** 1704,1723 ****
Datum
array_lower(PG_FUNCTION_ARGS)
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
int reqdim = PG_GETARG_INT32(1);
int *lb;
int result;
/* Sanity check: does it look like an array at all? */
! if (ARR_NDIM(v) <= 0 || ARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
! if (reqdim <= 0 || reqdim > ARR_NDIM(v))
PG_RETURN_NULL();
! lb = ARR_LBOUND(v);
result = lb[reqdim - 1];
PG_RETURN_INT32(result);
--- 1688,1707 ----
Datum
array_lower(PG_FUNCTION_ARGS)
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
int reqdim = PG_GETARG_INT32(1);
int *lb;
int result;
/* Sanity check: does it look like an array at all? */
! if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
! if (reqdim <= 0 || reqdim > AARR_NDIM(v))
PG_RETURN_NULL();
! lb = AARR_LBOUND(v);
result = lb[reqdim - 1];
PG_RETURN_INT32(result);
*************** array_lower(PG_FUNCTION_ARGS)
*** 1731,1752 ****
Datum
array_upper(PG_FUNCTION_ARGS)
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
int reqdim = PG_GETARG_INT32(1);
int *dimv,
*lb;
int result;
/* Sanity check: does it look like an array at all? */
! if (ARR_NDIM(v) <= 0 || ARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
! if (reqdim <= 0 || reqdim > ARR_NDIM(v))
PG_RETURN_NULL();
! lb = ARR_LBOUND(v);
! dimv = ARR_DIMS(v);
result = dimv[reqdim - 1] + lb[reqdim - 1] - 1;
--- 1715,1736 ----
Datum
array_upper(PG_FUNCTION_ARGS)
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
int reqdim = PG_GETARG_INT32(1);
int *dimv,
*lb;
int result;
/* Sanity check: does it look like an array at all? */
! if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
! if (reqdim <= 0 || reqdim > AARR_NDIM(v))
PG_RETURN_NULL();
! lb = AARR_LBOUND(v);
! dimv = AARR_DIMS(v);
result = dimv[reqdim - 1] + lb[reqdim - 1] - 1;
*************** array_upper(PG_FUNCTION_ARGS)
*** 1761,1780 ****
Datum
array_length(PG_FUNCTION_ARGS)
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
int reqdim = PG_GETARG_INT32(1);
int *dimv;
int result;
/* Sanity check: does it look like an array at all? */
! if (ARR_NDIM(v) <= 0 || ARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
! if (reqdim <= 0 || reqdim > ARR_NDIM(v))
PG_RETURN_NULL();
! dimv = ARR_DIMS(v);
result = dimv[reqdim - 1];
--- 1745,1764 ----
Datum
array_length(PG_FUNCTION_ARGS)
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
int reqdim = PG_GETARG_INT32(1);
int *dimv;
int result;
/* Sanity check: does it look like an array at all? */
! if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
! if (reqdim <= 0 || reqdim > AARR_NDIM(v))
PG_RETURN_NULL();
! dimv = AARR_DIMS(v);
result = dimv[reqdim - 1];
*************** array_length(PG_FUNCTION_ARGS)
*** 1788,1796 ****
Datum
array_cardinality(PG_FUNCTION_ARGS)
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
! PG_RETURN_INT32(ArrayGetNItems(ARR_NDIM(v), ARR_DIMS(v)));
}
--- 1772,1780 ----
Datum
array_cardinality(PG_FUNCTION_ARGS)
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
! PG_RETURN_INT32(ArrayGetNItems(AARR_NDIM(v), AARR_DIMS(v)));
}
*************** array_get_element(Datum arraydatum,
*** 1825,1831 ****
char elmalign,
bool *isNull)
{
- ArrayType *array;
int i,
ndim,
*dim,
--- 1809,1814 ----
*************** array_get_element(Datum arraydatum,
*** 1850,1859 ****
arraydataptr = (char *) DatumGetPointer(arraydatum);
arraynullsptr = NULL;
}
else
{
! /* detoast input array if necessary */
! array = DatumGetArrayTypeP(arraydatum);
ndim = ARR_NDIM(array);
dim = ARR_DIMS(array);
--- 1833,1854 ----
arraydataptr = (char *) DatumGetPointer(arraydatum);
arraynullsptr = NULL;
}
+ else if (VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(arraydatum)))
+ {
+ /* expanded array: let's do this in a separate function */
+ return array_get_element_expanded(arraydatum,
+ nSubscripts,
+ indx,
+ arraytyplen,
+ elmlen,
+ elmbyval,
+ elmalign,
+ isNull);
+ }
else
{
! /* detoast array if necessary, producing normal varlena input */
! ArrayType *array = DatumGetArrayTypeP(arraydatum);
ndim = ARR_NDIM(array);
dim = ARR_DIMS(array);
*************** array_get_element(Datum arraydatum,
*** 1903,1908 ****
--- 1898,1985 ----
}
/*
+ * Implementation of array_get_element() for an expanded array
+ */
+ static Datum
+ array_get_element_expanded(Datum arraydatum,
+ int nSubscripts, int *indx,
+ int arraytyplen,
+ int elmlen, bool elmbyval, char elmalign,
+ bool *isNull)
+ {
+ ExpandedArrayHeader *eah;
+ int i,
+ ndim,
+ *dim,
+ *lb,
+ offset;
+ Datum *dvalues;
+ bool *dnulls;
+
+ eah = (ExpandedArrayHeader *) DatumGetEOHP(arraydatum);
+ Assert(eah->ea_magic == EA_MAGIC);
+
+ /* sanity-check caller's info against object */
+ Assert(arraytyplen == -1);
+ Assert(elmlen == eah->typlen);
+ Assert(elmbyval == eah->typbyval);
+ Assert(elmalign == eah->typalign);
+
+ ndim = eah->ndims;
+ dim = eah->dims;
+ lb = eah->lbound;
+
+ /*
+ * Return NULL for invalid subscript
+ */
+ if (ndim != nSubscripts || ndim <= 0 || ndim > MAXDIM)
+ {
+ *isNull = true;
+ return (Datum) 0;
+ }
+ for (i = 0; i < ndim; i++)
+ {
+ if (indx[i] < lb[i] || indx[i] >= (dim[i] + lb[i]))
+ {
+ *isNull = true;
+ return (Datum) 0;
+ }
+ }
+
+ /*
+ * Calculate the element number
+ */
+ offset = ArrayGetOffset(nSubscripts, dim, lb, indx);
+
+ /*
+ * Deconstruct array if we didn't already. Note that we apply this even
+ * if the input is nominally read-only: it should be safe enough.
+ */
+ deconstruct_expanded_array(eah);
+
+ dvalues = eah->dvalues;
+ dnulls = eah->dnulls;
+
+ /*
+ * Check for NULL array element
+ */
+ if (dnulls && dnulls[offset])
+ {
+ *isNull = true;
+ return (Datum) 0;
+ }
+
+ /*
+ * OK, get the element. It's OK to return a pass-by-ref value as a
+ * pointer into the expanded array, for the same reason that regular
+ * array_get_element can return a pointer into flat arrays: the value is
+ * assumed not to change for as long as the Datum reference can exist.
+ */
+ *isNull = false;
+ return dvalues[offset];
+ }
+
+ /*
* array_get_slice :
* This routine takes an array and a range of indices (upperIndex and
* lowerIndx), creates a new array structure for the referred elements
*************** array_get_slice(Datum arraydatum,
*** 2083,2089 ****
*
* Result:
* A new array is returned, just like the old except for the one
! * modified entry. The original array object is not changed.
*
* For one-dimensional arrays only, we allow the array to be extended
* by assigning to a position outside the existing subscript range; any
--- 2160,2168 ----
*
* Result:
* A new array is returned, just like the old except for the one
! * modified entry. The original array object is not changed,
! * unless what is passed is a read-write reference to an expanded
! * array object; in that case the expanded array is updated in-place.
*
* For one-dimensional arrays only, we allow the array to be extended
* by assigning to a position outside the existing subscript range; any
*************** array_set_element(Datum arraydatum,
*** 2166,2171 ****
--- 2245,2264 ----
if (elmlen == -1 && !isNull)
dataValue = PointerGetDatum(PG_DETOAST_DATUM(dataValue));
+ if (VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(arraydatum)))
+ {
+ /* expanded array: let's do this in a separate function */
+ return array_set_element_expanded(arraydatum,
+ nSubscripts,
+ indx,
+ dataValue,
+ isNull,
+ arraytyplen,
+ elmlen,
+ elmbyval,
+ elmalign);
+ }
+
/* detoast input array if necessary */
array = DatumGetArrayTypeP(arraydatum);
*************** array_set_element(Datum arraydatum,
*** 2355,2360 ****
--- 2448,2702 ----
}
/*
+ * Implementation of array_set_element() for an expanded array
+ *
+ * Note: as with any operation on a read/write expanded object, we must
+ * take pains not to leave the object in a corrupt state if we fail partway
+ * through.
+ */
+ static Datum
+ array_set_element_expanded(Datum arraydatum,
+ int nSubscripts, int *indx,
+ Datum dataValue, bool isNull,
+ int arraytyplen,
+ int elmlen, bool elmbyval, char elmalign)
+ {
+ ExpandedArrayHeader *eah;
+ Datum *dvalues;
+ bool *dnulls;
+ int i,
+ ndim,
+ dim[MAXDIM],
+ lb[MAXDIM],
+ offset;
+ bool dimschanged,
+ newhasnulls;
+ int addedbefore,
+ addedafter;
+ char *oldValue;
+
+ /* Convert to R/W object if not so already */
+ if (!VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(arraydatum)))
+ arraydatum = expand_array(arraydatum, CurrentMemoryContext,
+ elmlen, elmbyval, elmalign);
+
+ eah = (ExpandedArrayHeader *) DatumGetEOHP(arraydatum);
+ Assert(eah->ea_magic == EA_MAGIC);
+
+ /* sanity-check caller's info against object */
+ Assert(arraytyplen == -1);
+ Assert(elmlen == eah->typlen);
+ Assert(elmbyval == eah->typbyval);
+ Assert(elmalign == eah->typalign);
+
+ /*
+ * Copy dimension info into local storage. This allows us to modify the
+ * dimensions if needed, while not messing up the expanded value if we
+ * fail partway through.
+ */
+ ndim = eah->ndims;
+ Assert(ndim >= 0 && ndim <= MAXDIM);
+ memcpy(dim, eah->dims, ndim * sizeof(int));
+ memcpy(lb, eah->lbound, ndim * sizeof(int));
+ dimschanged = false;
+
+ /*
+ * if number of dims is zero, i.e. an empty array, create an array with
+ * nSubscripts dimensions, and set the lower bounds to the supplied
+ * subscripts.
+ */
+ if (ndim == 0)
+ {
+ /*
+ * Allocate adequate space for new dimension info. This is harmless
+ * if we fail later.
+ */
+ Assert(nSubscripts > 0 && nSubscripts <= MAXDIM);
+ eah->dims = (int *) MemoryContextAllocZero(eah->hdr.eoh_context,
+ nSubscripts * sizeof(int));
+ eah->lbound = (int *) MemoryContextAllocZero(eah->hdr.eoh_context,
+ nSubscripts * sizeof(int));
+
+ /* Update local copies of dimension info */
+ ndim = nSubscripts;
+ for (i = 0; i < nSubscripts; i++)
+ {
+ dim[i] = 0;
+ lb[i] = indx[i];
+ }
+ dimschanged = true;
+ }
+ else if (ndim != nSubscripts)
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("wrong number of array subscripts")));
+
+ /*
+ * Deconstruct array if we didn't already. (Someday maybe add a special
+ * case path for fixed-length, no-nulls cases, where we can overwrite an
+ * element in place without ever deconstructing. But today is not that
+ * day.)
+ */
+ deconstruct_expanded_array(eah);
+
+ /*
+ * Copy new element into array's context, if needed (we assume it's
+ * already detoasted, so no junk should be created). If we fail further
+ * down, this memory is leaked, but that's reasonably harmless.
+ */
+ if (!eah->typbyval && !isNull)
+ {
+ MemoryContext oldcxt = MemoryContextSwitchTo(eah->hdr.eoh_context);
+
+ dataValue = datumCopy(dataValue, false, eah->typlen);
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ dvalues = eah->dvalues;
+ dnulls = eah->dnulls;
+
+ newhasnulls = ((dnulls != NULL) || isNull);
+ addedbefore = addedafter = 0;
+
+ /*
+ * Check subscripts (this logic matches original array_set_element)
+ */
+ if (ndim == 1)
+ {
+ if (indx[0] < lb[0])
+ {
+ addedbefore = lb[0] - indx[0];
+ dim[0] += addedbefore;
+ lb[0] = indx[0];
+ dimschanged = true;
+ if (addedbefore > 1)
+ newhasnulls = true; /* will insert nulls */
+ }
+ if (indx[0] >= (dim[0] + lb[0]))
+ {
+ addedafter = indx[0] - (dim[0] + lb[0]) + 1;
+ dim[0] += addedafter;
+ dimschanged = true;
+ if (addedafter > 1)
+ newhasnulls = true; /* will insert nulls */
+ }
+ }
+ else
+ {
+ /*
+ * XXX currently we do not support extending multi-dimensional arrays
+ * during assignment
+ */
+ for (i = 0; i < ndim; i++)
+ {
+ if (indx[i] < lb[i] ||
+ indx[i] >= (dim[i] + lb[i]))
+ ereport(ERROR,
+ (errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
+ errmsg("array subscript out of range")));
+ }
+ }
+
+ /* Now we can calculate linear offset of target item in array */
+ offset = ArrayGetOffset(nSubscripts, dim, lb, indx);
+
+ /* Physically enlarge existing dvalues/dnulls arrays if needed */
+ if (dim[0] > eah->dvalueslen)
+ {
+ /* We want some extra space if we're enlarging */
+ int newlen = dim[0] + dim[0] / 8;
+
+ eah->dvalues = dvalues = (Datum *)
+ repalloc(dvalues, newlen * sizeof(Datum));
+ if (dnulls)
+ eah->dnulls = dnulls = (bool *)
+ repalloc(dnulls, newlen * sizeof(bool));
+ eah->dvalueslen = newlen;
+ }
+
+ /*
+ * If we need a nulls bitmap and don't already have one, create it, being
+ * sure to mark all existing entries as not null.
+ */
+ if (newhasnulls && dnulls == NULL)
+ eah->dnulls = dnulls = (bool *)
+ MemoryContextAllocZero(eah->hdr.eoh_context,
+ eah->dvalueslen * sizeof(bool));
+
+ /*
+ * We now have all the needed space allocated, so we're ready to make
+ * irreversible changes. Be very wary of allowing failure below here.
+ */
+
+ /* Flattened value will no longer represent array accurately */
+ eah->fvalue = NULL;
+ /* And we don't know the flattened size either */
+ eah->flat_size = 0;
+
+ /* Update dimensionality info if needed */
+ if (dimschanged)
+ {
+ eah->ndims = ndim;
+ memcpy(eah->dims, dim, ndim * sizeof(int));
+ memcpy(eah->lbound, lb, ndim * sizeof(int));
+ }
+
+ /* Reposition items if needed, and fill addedbefore items with nulls */
+ if (addedbefore > 0)
+ {
+ memmove(dvalues + addedbefore, dvalues, eah->nelems * sizeof(Datum));
+ for (i = 0; i < addedbefore; i++)
+ dvalues[i] = (Datum) 0;
+ if (dnulls)
+ {
+ memmove(dnulls + addedbefore, dnulls, eah->nelems * sizeof(bool));
+ for (i = 0; i < addedbefore; i++)
+ dnulls[i] = true;
+ }
+ eah->nelems += addedbefore;
+ }
+
+ /* fill addedafter items with nulls */
+ if (addedafter > 0)
+ {
+ for (i = 0; i < addedafter; i++)
+ dvalues[eah->nelems + i] = (Datum) 0;
+ if (dnulls)
+ {
+ for (i = 0; i < addedafter; i++)
+ dnulls[eah->nelems + i] = true;
+ }
+ eah->nelems += addedafter;
+ }
+
+ /* Grab old element value for pfree'ing, if needed. */
+ if (!eah->typbyval && (dnulls == NULL || !dnulls[offset]))
+ oldValue = (char *) DatumGetPointer(dvalues[offset]);
+ else
+ oldValue = NULL;
+
+ /* And finally we can insert the new element. */
+ dvalues[offset] = dataValue;
+ if (dnulls)
+ dnulls[offset] = isNull;
+
+ /*
+ * Free old element if needed; this keeps repeated element replacements
+ * from bloating the array's storage. If the pfree somehow fails, it
+ * won't corrupt the array.
+ */
+ if (oldValue)
+ {
+ /* Don't try to pfree a part of the original flat array */
+ if (oldValue < eah->fstartptr || oldValue >= eah->fendptr)
+ pfree(oldValue);
+ }
+
+ /* Done, return standard TOAST pointer for object */
+ return EOHPGetRWDatum(&eah->hdr);
+ }
+
+ /*
* array_set_slice :
* This routine sets the value of a range of array locations (specified
* by upper and lower subscript values) to new values passed as
*************** array_set(ArrayType *array, int nSubscri
*** 2734,2741 ****
* the function fn(), and if nargs > 1 then argument positions after the
* first must be preset to the additional values to be passed. The
* first argument position initially holds the input array value.
- * * inpType: OID of element type of input array. This must be the same as,
- * or binary-compatible with, the first argument type of fn().
* * retType: OID of element type of output array. This must be the same as,
* or binary-compatible with, the result type of fn().
* * amstate: workspace for array_map. Must be zeroed by caller before
--- 3076,3081 ----
*************** array_set(ArrayType *array, int nSubscri
*** 2749,2762 ****
* the array are OK however.
*/
Datum
! array_map(FunctionCallInfo fcinfo, Oid inpType, Oid retType,
! ArrayMapState *amstate)
{
! ArrayType *v;
ArrayType *result;
Datum *values;
bool *nulls;
- Datum elt;
int *dim;
int ndim;
int nitems;
--- 3089,3100 ----
* the array are OK however.
*/
Datum
! array_map(FunctionCallInfo fcinfo, Oid retType, ArrayMapState *amstate)
{
! AnyArrayType *v;
ArrayType *result;
Datum *values;
bool *nulls;
int *dim;
int ndim;
int nitems;
*************** array_map(FunctionCallInfo fcinfo, Oid i
*** 2764,2778 ****
int32 nbytes = 0;
int32 dataoffset;
bool hasnulls;
int inp_typlen;
bool inp_typbyval;
char inp_typalign;
int typlen;
bool typbyval;
char typalign;
! char *s;
! bits8 *bitmap;
! int bitmask;
ArrayMetaState *inp_extra;
ArrayMetaState *ret_extra;
--- 3102,3115 ----
int32 nbytes = 0;
int32 dataoffset;
bool hasnulls;
+ Oid inpType;
int inp_typlen;
bool inp_typbyval;
char inp_typalign;
int typlen;
bool typbyval;
char typalign;
! ARRAY_ITER ARRAY_ITER_VARS(iter);
ArrayMetaState *inp_extra;
ArrayMetaState *ret_extra;
*************** array_map(FunctionCallInfo fcinfo, Oid i
*** 2781,2792 ****
elog(ERROR, "invalid nargs: %d", fcinfo->nargs);
if (PG_ARGISNULL(0))
elog(ERROR, "null input array");
! v = PG_GETARG_ARRAYTYPE_P(0);
!
! Assert(ARR_ELEMTYPE(v) == inpType);
! ndim = ARR_NDIM(v);
! dim = ARR_DIMS(v);
nitems = ArrayGetNItems(ndim, dim);
/* Check for empty array */
--- 3118,3128 ----
elog(ERROR, "invalid nargs: %d", fcinfo->nargs);
if (PG_ARGISNULL(0))
elog(ERROR, "null input array");
! v = PG_GETARG_ANY_ARRAY(0);
! inpType = AARR_ELEMTYPE(v);
! ndim = AARR_NDIM(v);
! dim = AARR_DIMS(v);
nitems = ArrayGetNItems(ndim, dim);
/* Check for empty array */
*************** array_map(FunctionCallInfo fcinfo, Oid i
*** 2833,2841 ****
nulls = (bool *) palloc(nitems * sizeof(bool));
/* Loop over source data */
! s = ARR_DATA_PTR(v);
! bitmap = ARR_NULLBITMAP(v);
! bitmask = 1;
hasnulls = false;
for (i = 0; i < nitems; i++)
--- 3169,3175 ----
nulls = (bool *) palloc(nitems * sizeof(bool));
/* Loop over source data */
! ARRAY_ITER_SETUP(iter, v);
hasnulls = false;
for (i = 0; i < nitems; i++)
*************** array_map(FunctionCallInfo fcinfo, Oid i
*** 2843,2860 ****
bool callit = true;
/* Get source element, checking for NULL */
! if (bitmap && (*bitmap & bitmask) == 0)
! {
! fcinfo->argnull[0] = true;
! }
! else
! {
! elt = fetch_att(s, inp_typbyval, inp_typlen);
! s = att_addlength_datum(s, inp_typlen, elt);
! s = (char *) att_align_nominal(s, inp_typalign);
! fcinfo->arg[0] = elt;
! fcinfo->argnull[0] = false;
! }
/*
* Apply the given function to source elt and extra args.
--- 3177,3184 ----
bool callit = true;
/* Get source element, checking for NULL */
! ARRAY_ITER_NEXT(iter, i, fcinfo->arg[0], fcinfo->argnull[0],
! inp_typlen, inp_typbyval, inp_typalign);
/*
* Apply the given function to source elt and extra args.
*************** array_map(FunctionCallInfo fcinfo, Oid i
*** 2899,2915 ****
errmsg("array size exceeds the maximum allowed (%d)",
(int) MaxAllocSize)));
}
-
- /* advance bitmap pointer if any */
- if (bitmap)
- {
- bitmask <<= 1;
- if (bitmask == 0x100)
- {
- bitmap++;
- bitmask = 1;
- }
- }
}
/* Allocate and initialize the result array */
--- 3223,3228 ----
*************** array_map(FunctionCallInfo fcinfo, Oid i
*** 2928,2934 ****
result->ndim = ndim;
result->dataoffset = dataoffset;
result->elemtype = retType;
! memcpy(ARR_DIMS(result), ARR_DIMS(v), 2 * ndim * sizeof(int));
/*
* Note: do not risk trying to pfree the results of the called function
--- 3241,3248 ----
result->ndim = ndim;
result->dataoffset = dataoffset;
result->elemtype = retType;
! memcpy(ARR_DIMS(result), AARR_DIMS(v), ndim * sizeof(int));
! memcpy(ARR_LBOUND(result), AARR_LBOUND(v), ndim * sizeof(int));
/*
* Note: do not risk trying to pfree the results of the called function
*************** construct_empty_array(Oid elmtype)
*** 3092,3097 ****
--- 3406,3429 ----
}
/*
+ * construct_empty_expanded_array: make an empty expanded array
+ * given only type information. (elmlen etc can be zeroes if not known.)
+ */
+ ExpandedArrayHeader *
+ construct_empty_expanded_array(Oid element_type,
+ MemoryContext parentcontext,
+ int elmlen, bool elmbyval, char elmalign)
+ {
+ ArrayType *array = construct_empty_array(element_type);
+ Datum d;
+
+ d = expand_array(PointerGetDatum(array), parentcontext,
+ elmlen, elmbyval, elmalign);
+ pfree(array);
+ return (ExpandedArrayHeader *) DatumGetEOHP(d);
+ }
+
+ /*
* deconstruct_array --- simple method for extracting data from an array
*
* array: array object to examine (must not be NULL)
*************** array_contains_nulls(ArrayType *array)
*** 3229,3264 ****
Datum
array_eq(PG_FUNCTION_ARGS)
{
! ArrayType *array1 = PG_GETARG_ARRAYTYPE_P(0);
! ArrayType *array2 = PG_GETARG_ARRAYTYPE_P(1);
Oid collation = PG_GET_COLLATION();
! int ndims1 = ARR_NDIM(array1);
! int ndims2 = ARR_NDIM(array2);
! int *dims1 = ARR_DIMS(array1);
! int *dims2 = ARR_DIMS(array2);
! Oid element_type = ARR_ELEMTYPE(array1);
bool result = true;
int nitems;
TypeCacheEntry *typentry;
int typlen;
bool typbyval;
char typalign;
! char *ptr1;
! char *ptr2;
! bits8 *bitmap1;
! bits8 *bitmap2;
! int bitmask;
int i;
FunctionCallInfoData locfcinfo;
! if (element_type != ARR_ELEMTYPE(array2))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("cannot compare arrays of different element types")));
/* fast path if the arrays do not have the same dimensionality */
if (ndims1 != ndims2 ||
! memcmp(dims1, dims2, 2 * ndims1 * sizeof(int)) != 0)
result = false;
else
{
--- 3561,3596 ----
Datum
array_eq(PG_FUNCTION_ARGS)
{
! AnyArrayType *array1 = PG_GETARG_ANY_ARRAY(0);
! AnyArrayType *array2 = PG_GETARG_ANY_ARRAY(1);
Oid collation = PG_GET_COLLATION();
! int ndims1 = AARR_NDIM(array1);
! int ndims2 = AARR_NDIM(array2);
! int *dims1 = AARR_DIMS(array1);
! int *dims2 = AARR_DIMS(array2);
! int *lbs1 = AARR_LBOUND(array1);
! int *lbs2 = AARR_LBOUND(array2);
! Oid element_type = AARR_ELEMTYPE(array1);
bool result = true;
int nitems;
TypeCacheEntry *typentry;
int typlen;
bool typbyval;
char typalign;
! ARRAY_ITER ARRAY_ITER_VARS(it1);
! ARRAY_ITER ARRAY_ITER_VARS(it2);
int i;
FunctionCallInfoData locfcinfo;
! if (element_type != AARR_ELEMTYPE(array2))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("cannot compare arrays of different element types")));
/* fast path if the arrays do not have the same dimensionality */
if (ndims1 != ndims2 ||
! memcmp(dims1, dims2, ndims1 * sizeof(int)) != 0 ||
! memcmp(lbs1, lbs2, ndims1 * sizeof(int)) != 0)
result = false;
else
{
*************** array_eq(PG_FUNCTION_ARGS)
*** 3293,3303 ****
/* Loop over source data */
nitems = ArrayGetNItems(ndims1, dims1);
! ptr1 = ARR_DATA_PTR(array1);
! ptr2 = ARR_DATA_PTR(array2);
! bitmap1 = ARR_NULLBITMAP(array1);
! bitmap2 = ARR_NULLBITMAP(array2);
! bitmask = 1; /* use same bitmask for both arrays */
for (i = 0; i < nitems; i++)
{
--- 3625,3632 ----
/* Loop over source data */
nitems = ArrayGetNItems(ndims1, dims1);
! ARRAY_ITER_SETUP(it1, array1);
! ARRAY_ITER_SETUP(it2, array2);
for (i = 0; i < nitems; i++)
{
*************** array_eq(PG_FUNCTION_ARGS)
*** 3308,3349 ****
bool oprresult;
/* Get elements, checking for NULL */
! if (bitmap1 && (*bitmap1 & bitmask) == 0)
! {
! isnull1 = true;
! elt1 = (Datum) 0;
! }
! else
! {
! isnull1 = false;
! elt1 = fetch_att(ptr1, typbyval, typlen);
! ptr1 = att_addlength_pointer(ptr1, typlen, ptr1);
! ptr1 = (char *) att_align_nominal(ptr1, typalign);
! }
!
! if (bitmap2 && (*bitmap2 & bitmask) == 0)
! {
! isnull2 = true;
! elt2 = (Datum) 0;
! }
! else
! {
! isnull2 = false;
! elt2 = fetch_att(ptr2, typbyval, typlen);
! ptr2 = att_addlength_pointer(ptr2, typlen, ptr2);
! ptr2 = (char *) att_align_nominal(ptr2, typalign);
! }
!
! /* advance bitmap pointers if any */
! bitmask <<= 1;
! if (bitmask == 0x100)
! {
! if (bitmap1)
! bitmap1++;
! if (bitmap2)
! bitmap2++;
! bitmask = 1;
! }
/*
* We consider two NULLs equal; NULL and not-NULL are unequal.
--- 3637,3644 ----
bool oprresult;
/* Get elements, checking for NULL */
! ARRAY_ITER_NEXT(it1, i, elt1, isnull1, typlen, typbyval, typalign);
! ARRAY_ITER_NEXT(it2, i, elt2, isnull2, typlen, typbyval, typalign);
/*
* We consider two NULLs equal; NULL and not-NULL are unequal.
*************** array_eq(PG_FUNCTION_ARGS)
*** 3374,3381 ****
}
/* Avoid leaking memory when handed toasted input. */
! PG_FREE_IF_COPY(array1, 0);
! PG_FREE_IF_COPY(array2, 1);
PG_RETURN_BOOL(result);
}
--- 3669,3676 ----
}
/* Avoid leaking memory when handed toasted input. */
! AARR_FREE_IF_COPY(array1, 0);
! AARR_FREE_IF_COPY(array2, 1);
PG_RETURN_BOOL(result);
}
*************** btarraycmp(PG_FUNCTION_ARGS)
*** 3435,3465 ****
static int
array_cmp(FunctionCallInfo fcinfo)
{
! ArrayType *array1 = PG_GETARG_ARRAYTYPE_P(0);
! ArrayType *array2 = PG_GETARG_ARRAYTYPE_P(1);
Oid collation = PG_GET_COLLATION();
! int ndims1 = ARR_NDIM(array1);
! int ndims2 = ARR_NDIM(array2);
! int *dims1 = ARR_DIMS(array1);
! int *dims2 = ARR_DIMS(array2);
int nitems1 = ArrayGetNItems(ndims1, dims1);
int nitems2 = ArrayGetNItems(ndims2, dims2);
! Oid element_type = ARR_ELEMTYPE(array1);
int result = 0;
TypeCacheEntry *typentry;
int typlen;
bool typbyval;
char typalign;
int min_nitems;
! char *ptr1;
! char *ptr2;
! bits8 *bitmap1;
! bits8 *bitmap2;
! int bitmask;
int i;
FunctionCallInfoData locfcinfo;
! if (element_type != ARR_ELEMTYPE(array2))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("cannot compare arrays of different element types")));
--- 3730,3757 ----
static int
array_cmp(FunctionCallInfo fcinfo)
{
! AnyArrayType *array1 = PG_GETARG_ANY_ARRAY(0);
! AnyArrayType *array2 = PG_GETARG_ANY_ARRAY(1);
Oid collation = PG_GET_COLLATION();
! int ndims1 = AARR_NDIM(array1);
! int ndims2 = AARR_NDIM(array2);
! int *dims1 = AARR_DIMS(array1);
! int *dims2 = AARR_DIMS(array2);
int nitems1 = ArrayGetNItems(ndims1, dims1);
int nitems2 = ArrayGetNItems(ndims2, dims2);
! Oid element_type = AARR_ELEMTYPE(array1);
int result = 0;
TypeCacheEntry *typentry;
int typlen;
bool typbyval;
char typalign;
int min_nitems;
! ARRAY_ITER ARRAY_ITER_VARS(it1);
! ARRAY_ITER ARRAY_ITER_VARS(it2);
int i;
FunctionCallInfoData locfcinfo;
! if (element_type != AARR_ELEMTYPE(array2))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("cannot compare arrays of different element types")));
*************** array_cmp(FunctionCallInfo fcinfo)
*** 3495,3505 ****
/* Loop over source data */
min_nitems = Min(nitems1, nitems2);
! ptr1 = ARR_DATA_PTR(array1);
! ptr2 = ARR_DATA_PTR(array2);
! bitmap1 = ARR_NULLBITMAP(array1);
! bitmap2 = ARR_NULLBITMAP(array2);
! bitmask = 1; /* use same bitmask for both arrays */
for (i = 0; i < min_nitems; i++)
{
--- 3787,3794 ----
/* Loop over source data */
min_nitems = Min(nitems1, nitems2);
! ARRAY_ITER_SETUP(it1, array1);
! ARRAY_ITER_SETUP(it2, array2);
for (i = 0; i < min_nitems; i++)
{
*************** array_cmp(FunctionCallInfo fcinfo)
*** 3510,3551 ****
int32 cmpresult;
/* Get elements, checking for NULL */
! if (bitmap1 && (*bitmap1 & bitmask) == 0)
! {
! isnull1 = true;
! elt1 = (Datum) 0;
! }
! else
! {
! isnull1 = false;
! elt1 = fetch_att(ptr1, typbyval, typlen);
! ptr1 = att_addlength_pointer(ptr1, typlen, ptr1);
! ptr1 = (char *) att_align_nominal(ptr1, typalign);
! }
!
! if (bitmap2 && (*bitmap2 & bitmask) == 0)
! {
! isnull2 = true;
! elt2 = (Datum) 0;
! }
! else
! {
! isnull2 = false;
! elt2 = fetch_att(ptr2, typbyval, typlen);
! ptr2 = att_addlength_pointer(ptr2, typlen, ptr2);
! ptr2 = (char *) att_align_nominal(ptr2, typalign);
! }
!
! /* advance bitmap pointers if any */
! bitmask <<= 1;
! if (bitmask == 0x100)
! {
! if (bitmap1)
! bitmap1++;
! if (bitmap2)
! bitmap2++;
! bitmask = 1;
! }
/*
* We consider two NULLs equal; NULL > not-NULL.
--- 3799,3806 ----
int32 cmpresult;
/* Get elements, checking for NULL */
! ARRAY_ITER_NEXT(it1, i, elt1, isnull1, typlen, typbyval, typalign);
! ARRAY_ITER_NEXT(it2, i, elt2, isnull2, typlen, typbyval, typalign);
/*
* We consider two NULLs equal; NULL > not-NULL.
*************** array_cmp(FunctionCallInfo fcinfo)
*** 3604,3611 ****
result = (ndims1 < ndims2) ? -1 : 1;
else
{
! /* this relies on LB array immediately following DIMS array */
! for (i = 0; i < ndims1 * 2; i++)
{
if (dims1[i] != dims2[i])
{
--- 3859,3865 ----
result = (ndims1 < ndims2) ? -1 : 1;
else
{
! for (i = 0; i < ndims1; i++)
{
if (dims1[i] != dims2[i])
{
*************** array_cmp(FunctionCallInfo fcinfo)
*** 3613,3624 ****
break;
}
}
}
}
/* Avoid leaking memory when handed toasted input. */
! PG_FREE_IF_COPY(array1, 0);
! PG_FREE_IF_COPY(array2, 1);
return result;
}
--- 3867,3892 ----
break;
}
}
+ if (result == 0)
+ {
+ int *lbound1 = AARR_LBOUND(array1);
+ int *lbound2 = AARR_LBOUND(array2);
+
+ for (i = 0; i < ndims1; i++)
+ {
+ if (lbound1[i] != lbound2[i])
+ {
+ result = (lbound1[i] < lbound2[i]) ? -1 : 1;
+ break;
+ }
+ }
+ }
}
}
/* Avoid leaking memory when handed toasted input. */
! AARR_FREE_IF_COPY(array1, 0);
! AARR_FREE_IF_COPY(array2, 1);
return result;
}
*************** array_cmp(FunctionCallInfo fcinfo)
*** 3633,3652 ****
Datum
hash_array(PG_FUNCTION_ARGS)
{
! ArrayType *array = PG_GETARG_ARRAYTYPE_P(0);
! int ndims = ARR_NDIM(array);
! int *dims = ARR_DIMS(array);
! Oid element_type = ARR_ELEMTYPE(array);
uint32 result = 1;
int nitems;
TypeCacheEntry *typentry;
int typlen;
bool typbyval;
char typalign;
- char *ptr;
- bits8 *bitmap;
- int bitmask;
int i;
FunctionCallInfoData locfcinfo;
/*
--- 3901,3918 ----
Datum
hash_array(PG_FUNCTION_ARGS)
{
! AnyArrayType *array = PG_GETARG_ANY_ARRAY(0);
! int ndims = AARR_NDIM(array);
! int *dims = AARR_DIMS(array);
! Oid element_type = AARR_ELEMTYPE(array);
uint32 result = 1;
int nitems;
TypeCacheEntry *typentry;
int typlen;
bool typbyval;
char typalign;
int i;
+ ARRAY_ITER ARRAY_ITER_VARS(iter);
FunctionCallInfoData locfcinfo;
/*
*************** hash_array(PG_FUNCTION_ARGS)
*** 3680,3707 ****
/* Loop over source data */
nitems = ArrayGetNItems(ndims, dims);
! ptr = ARR_DATA_PTR(array);
! bitmap = ARR_NULLBITMAP(array);
! bitmask = 1;
for (i = 0; i < nitems; i++)
{
uint32 elthash;
/* Get element, checking for NULL */
! if (bitmap && (*bitmap & bitmask) == 0)
{
/* Treat nulls as having hashvalue 0 */
elthash = 0;
}
else
{
- Datum elt;
-
- elt = fetch_att(ptr, typbyval, typlen);
- ptr = att_addlength_pointer(ptr, typlen, ptr);
- ptr = (char *) att_align_nominal(ptr, typalign);
-
/* Apply the hash function */
locfcinfo.arg[0] = elt;
locfcinfo.argnull[0] = false;
--- 3946,3969 ----
/* Loop over source data */
nitems = ArrayGetNItems(ndims, dims);
! ARRAY_ITER_SETUP(iter, array);
for (i = 0; i < nitems; i++)
{
+ Datum elt;
+ bool isnull;
uint32 elthash;
/* Get element, checking for NULL */
! ARRAY_ITER_NEXT(iter, i, elt, isnull, typlen, typbyval, typalign);
!
! if (isnull)
{
/* Treat nulls as having hashvalue 0 */
elthash = 0;
}
else
{
/* Apply the hash function */
locfcinfo.arg[0] = elt;
locfcinfo.argnull[0] = false;
*************** hash_array(PG_FUNCTION_ARGS)
*** 3709,3725 ****
elthash = DatumGetUInt32(FunctionCallInvoke(&locfcinfo));
}
- /* advance bitmap pointer if any */
- if (bitmap)
- {
- bitmask <<= 1;
- if (bitmask == 0x100)
- {
- bitmap++;
- bitmask = 1;
- }
- }
-
/*
* Combine hash values of successive elements by multiplying the
* current value by 31 and adding on the new element's hash value.
--- 3971,3976 ----
*************** hash_array(PG_FUNCTION_ARGS)
*** 3735,3741 ****
}
/* Avoid leaking memory when handed toasted input. */
! PG_FREE_IF_COPY(array, 0);
PG_RETURN_UINT32(result);
}
--- 3986,3992 ----
}
/* Avoid leaking memory when handed toasted input. */
! AARR_FREE_IF_COPY(array, 0);
PG_RETURN_UINT32(result);
}
*************** hash_array(PG_FUNCTION_ARGS)
*** 3756,3766 ****
* When matchall is false, return true if any members of array1 are in array2.
*/
static bool
! array_contain_compare(ArrayType *array1, ArrayType *array2, Oid collation,
bool matchall, void **fn_extra)
{
bool result = matchall;
! Oid element_type = ARR_ELEMTYPE(array1);
TypeCacheEntry *typentry;
int nelems1;
Datum *values2;
--- 4007,4017 ----
* When matchall is false, return true if any members of array1 are in array2.
*/
static bool
! array_contain_compare(AnyArrayType *array1, AnyArrayType *array2, Oid collation,
bool matchall, void **fn_extra)
{
bool result = matchall;
! Oid element_type = AARR_ELEMTYPE(array1);
TypeCacheEntry *typentry;
int nelems1;
Datum *values2;
*************** array_contain_compare(ArrayType *array1,
*** 3769,3782 ****
int typlen;
bool typbyval;
char typalign;
- char *ptr1;
- bits8 *bitmap1;
- int bitmask;
int i;
int j;
FunctionCallInfoData locfcinfo;
! if (element_type != ARR_ELEMTYPE(array2))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("cannot compare arrays of different element types")));
--- 4020,4031 ----
int typlen;
bool typbyval;
char typalign;
int i;
int j;
+ ARRAY_ITER ARRAY_ITER_VARS(it1);
FunctionCallInfoData locfcinfo;
! if (element_type != AARR_ELEMTYPE(array2))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("cannot compare arrays of different element types")));
*************** array_contain_compare(ArrayType *array1,
*** 3809,3816 ****
* worthwhile to use deconstruct_array on it. We scan array1 the hard way
* however, since we very likely won't need to look at all of it.
*/
! deconstruct_array(array2, element_type, typlen, typbyval, typalign,
! &values2, &nulls2, &nelems2);
/*
* Apply the comparison operator to each pair of array elements.
--- 4058,4075 ----
* worthwhile to use deconstruct_array on it. We scan array1 the hard way
* however, since we very likely won't need to look at all of it.
*/
! if (VARATT_IS_EXPANDED_HEADER(array2))
! {
! /* This should be safe even if input is read-only */
! deconstruct_expanded_array(&(array2->xpn));
! values2 = array2->xpn.dvalues;
! nulls2 = array2->xpn.dnulls;
! nelems2 = array2->xpn.nelems;
! }
! else
! deconstruct_array(&(array2->flt),
! element_type, typlen, typbyval, typalign,
! &values2, &nulls2, &nelems2);
/*
* Apply the comparison operator to each pair of array elements.
*************** array_contain_compare(ArrayType *array1,
*** 3819,3828 ****
collation, NULL, NULL);
/* Loop over source data */
! nelems1 = ArrayGetNItems(ARR_NDIM(array1), ARR_DIMS(array1));
! ptr1 = ARR_DATA_PTR(array1);
! bitmap1 = ARR_NULLBITMAP(array1);
! bitmask = 1;
for (i = 0; i < nelems1; i++)
{
--- 4078,4085 ----
collation, NULL, NULL);
/* Loop over source data */
! nelems1 = ArrayGetNItems(AARR_NDIM(array1), AARR_DIMS(array1));
! ARRAY_ITER_SETUP(it1, array1);
for (i = 0; i < nelems1; i++)
{
*************** array_contain_compare(ArrayType *array1,
*** 3830,3856 ****
bool isnull1;
/* Get element, checking for NULL */
! if (bitmap1 && (*bitmap1 & bitmask) == 0)
! {
! isnull1 = true;
! elt1 = (Datum) 0;
! }
! else
! {
! isnull1 = false;
! elt1 = fetch_att(ptr1, typbyval, typlen);
! ptr1 = att_addlength_pointer(ptr1, typlen, ptr1);
! ptr1 = (char *) att_align_nominal(ptr1, typalign);
! }
!
! /* advance bitmap pointer if any */
! bitmask <<= 1;
! if (bitmask == 0x100)
! {
! if (bitmap1)
! bitmap1++;
! bitmask = 1;
! }
/*
* We assume that the comparison operator is strict, so a NULL can't
--- 4087,4093 ----
bool isnull1;
/* Get element, checking for NULL */
! ARRAY_ITER_NEXT(it1, i, elt1, isnull1, typlen, typbyval, typalign);
/*
* We assume that the comparison operator is strict, so a NULL can't
*************** array_contain_compare(ArrayType *array1,
*** 3909,3925 ****
}
}
- pfree(values2);
- pfree(nulls2);
-
return result;
}
Datum
arrayoverlap(PG_FUNCTION_ARGS)
{
! ArrayType *array1 = PG_GETARG_ARRAYTYPE_P(0);
! ArrayType *array2 = PG_GETARG_ARRAYTYPE_P(1);
Oid collation = PG_GET_COLLATION();
bool result;
--- 4146,4159 ----
}
}
return result;
}
Datum
arrayoverlap(PG_FUNCTION_ARGS)
{
! AnyArrayType *array1 = PG_GETARG_ANY_ARRAY(0);
! AnyArrayType *array2 = PG_GETARG_ANY_ARRAY(1);
Oid collation = PG_GET_COLLATION();
bool result;
*************** arrayoverlap(PG_FUNCTION_ARGS)
*** 3927,3934 ****
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
! PG_FREE_IF_COPY(array1, 0);
! PG_FREE_IF_COPY(array2, 1);
PG_RETURN_BOOL(result);
}
--- 4161,4168 ----
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
! AARR_FREE_IF_COPY(array1, 0);
! AARR_FREE_IF_COPY(array2, 1);
PG_RETURN_BOOL(result);
}
*************** arrayoverlap(PG_FUNCTION_ARGS)
*** 3936,3943 ****
Datum
arraycontains(PG_FUNCTION_ARGS)
{
! ArrayType *array1 = PG_GETARG_ARRAYTYPE_P(0);
! ArrayType *array2 = PG_GETARG_ARRAYTYPE_P(1);
Oid collation = PG_GET_COLLATION();
bool result;
--- 4170,4177 ----
Datum
arraycontains(PG_FUNCTION_ARGS)
{
! AnyArrayType *array1 = PG_GETARG_ANY_ARRAY(0);
! AnyArrayType *array2 = PG_GETARG_ANY_ARRAY(1);
Oid collation = PG_GET_COLLATION();
bool result;
*************** arraycontains(PG_FUNCTION_ARGS)
*** 3945,3952 ****
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
! PG_FREE_IF_COPY(array1, 0);
! PG_FREE_IF_COPY(array2, 1);
PG_RETURN_BOOL(result);
}
--- 4179,4186 ----
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
! AARR_FREE_IF_COPY(array1, 0);
! AARR_FREE_IF_COPY(array2, 1);
PG_RETURN_BOOL(result);
}
*************** arraycontains(PG_FUNCTION_ARGS)
*** 3954,3961 ****
Datum
arraycontained(PG_FUNCTION_ARGS)
{
! ArrayType *array1 = PG_GETARG_ARRAYTYPE_P(0);
! ArrayType *array2 = PG_GETARG_ARRAYTYPE_P(1);
Oid collation = PG_GET_COLLATION();
bool result;
--- 4188,4195 ----
Datum
arraycontained(PG_FUNCTION_ARGS)
{
! AnyArrayType *array1 = PG_GETARG_ANY_ARRAY(0);
! AnyArrayType *array2 = PG_GETARG_ANY_ARRAY(1);
Oid collation = PG_GET_COLLATION();
bool result;
*************** arraycontained(PG_FUNCTION_ARGS)
*** 3963,3970 ****
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
! PG_FREE_IF_COPY(array1, 0);
! PG_FREE_IF_COPY(array2, 1);
PG_RETURN_BOOL(result);
}
--- 4197,4204 ----
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
! AARR_FREE_IF_COPY(array1, 0);
! AARR_FREE_IF_COPY(array2, 1);
PG_RETURN_BOOL(result);
}
*************** makeArrayResultAny(ArrayBuildStateAny *a
*** 5213,5243 ****
Datum
array_larger(PG_FUNCTION_ARGS)
{
! ArrayType *v1,
! *v2,
! *result;
!
! v1 = PG_GETARG_ARRAYTYPE_P(0);
! v2 = PG_GETARG_ARRAYTYPE_P(1);
!
! result = ((array_cmp(fcinfo) > 0) ? v1 : v2);
!
! PG_RETURN_ARRAYTYPE_P(result);
}
Datum
array_smaller(PG_FUNCTION_ARGS)
{
! ArrayType *v1,
! *v2,
! *result;
!
! v1 = PG_GETARG_ARRAYTYPE_P(0);
! v2 = PG_GETARG_ARRAYTYPE_P(1);
!
! result = ((array_cmp(fcinfo) < 0) ? v1 : v2);
!
! PG_RETURN_ARRAYTYPE_P(result);
}
--- 5447,5465 ----
Datum
array_larger(PG_FUNCTION_ARGS)
{
! if (array_cmp(fcinfo) > 0)
! PG_RETURN_DATUM(PG_GETARG_DATUM(0));
! else
! PG_RETURN_DATUM(PG_GETARG_DATUM(1));
}
Datum
array_smaller(PG_FUNCTION_ARGS)
{
! if (array_cmp(fcinfo) < 0)
! PG_RETURN_DATUM(PG_GETARG_DATUM(0));
! else
! PG_RETURN_DATUM(PG_GETARG_DATUM(1));
}
*************** generate_subscripts(PG_FUNCTION_ARGS)
*** 5262,5268 ****
/* stuff done only on the first call of the function */
if (SRF_IS_FIRSTCALL())
{
! ArrayType *v = PG_GETARG_ARRAYTYPE_P(0);
int reqdim = PG_GETARG_INT32(1);
int *lb,
*dimv;
--- 5484,5490 ----
/* stuff done only on the first call of the function */
if (SRF_IS_FIRSTCALL())
{
! AnyArrayType *v = PG_GETARG_ANY_ARRAY(0);
int reqdim = PG_GETARG_INT32(1);
int *lb,
*dimv;
*************** generate_subscripts(PG_FUNCTION_ARGS)
*** 5271,5281 ****
funcctx = SRF_FIRSTCALL_INIT();
/* Sanity check: does it look like an array at all? */
! if (ARR_NDIM(v) <= 0 || ARR_NDIM(v) > MAXDIM)
SRF_RETURN_DONE(funcctx);
/* Sanity check: was the requested dim valid */
! if (reqdim <= 0 || reqdim > ARR_NDIM(v))
SRF_RETURN_DONE(funcctx);
/*
--- 5493,5503 ----
funcctx = SRF_FIRSTCALL_INIT();
/* Sanity check: does it look like an array at all? */
! if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
SRF_RETURN_DONE(funcctx);
/* Sanity check: was the requested dim valid */
! if (reqdim <= 0 || reqdim > AARR_NDIM(v))
SRF_RETURN_DONE(funcctx);
/*
*************** generate_subscripts(PG_FUNCTION_ARGS)
*** 5284,5291 ****
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
fctx = (generate_subscripts_fctx *) palloc(sizeof(generate_subscripts_fctx));
! lb = ARR_LBOUND(v);
! dimv = ARR_DIMS(v);
fctx->lower = lb[reqdim - 1];
fctx->upper = dimv[reqdim - 1] + lb[reqdim - 1] - 1;
--- 5506,5513 ----
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
fctx = (generate_subscripts_fctx *) palloc(sizeof(generate_subscripts_fctx));
! lb = AARR_LBOUND(v);
! dimv = AARR_DIMS(v);
fctx->lower = lb[reqdim - 1];
fctx->upper = dimv[reqdim - 1] + lb[reqdim - 1] - 1;
*************** array_unnest(PG_FUNCTION_ARGS)
*** 5604,5614 ****
{
typedef struct
{
! ArrayType *arr;
int nextelem;
int numelems;
- char *elemdataptr; /* this moves with nextelem */
- bits8 *arraynullsptr; /* this does not */
int16 elmlen;
bool elmbyval;
char elmalign;
--- 5826,5834 ----
{
typedef struct
{
! ARRAY_ITER ARRAY_ITER_VARS(iter);
int nextelem;
int numelems;
int16 elmlen;
bool elmbyval;
char elmalign;
*************** array_unnest(PG_FUNCTION_ARGS)
*** 5621,5627 ****
/* stuff done only on the first call of the function */
if (SRF_IS_FIRSTCALL())
{
! ArrayType *arr;
/* create a function context for cross-call persistence */
funcctx = SRF_FIRSTCALL_INIT();
--- 5841,5847 ----
/* stuff done only on the first call of the function */
if (SRF_IS_FIRSTCALL())
{
! AnyArrayType *arr;
/* create a function context for cross-call persistence */
funcctx = SRF_FIRSTCALL_INIT();
*************** array_unnest(PG_FUNCTION_ARGS)
*** 5638,5660 ****
* and not before. (If no detoast happens, we assume the originally
* passed array will stick around till then.)
*/
! arr = PG_GETARG_ARRAYTYPE_P(0);
/* allocate memory for user context */
fctx = (array_unnest_fctx *) palloc(sizeof(array_unnest_fctx));
/* initialize state */
! fctx->arr = arr;
fctx->nextelem = 0;
! fctx->numelems = ArrayGetNItems(ARR_NDIM(arr), ARR_DIMS(arr));
!
! fctx->elemdataptr = ARR_DATA_PTR(arr);
! fctx->arraynullsptr = ARR_NULLBITMAP(arr);
! get_typlenbyvalalign(ARR_ELEMTYPE(arr),
! &fctx->elmlen,
! &fctx->elmbyval,
! &fctx->elmalign);
funcctx->user_fctx = fctx;
MemoryContextSwitchTo(oldcontext);
--- 5858,5885 ----
* and not before. (If no detoast happens, we assume the originally
* passed array will stick around till then.)
*/
! arr = PG_GETARG_ANY_ARRAY(0);
/* allocate memory for user context */
fctx = (array_unnest_fctx *) palloc(sizeof(array_unnest_fctx));
/* initialize state */
! ARRAY_ITER_SETUP(fctx->iter, arr);
fctx->nextelem = 0;
! fctx->numelems = ArrayGetNItems(AARR_NDIM(arr), AARR_DIMS(arr));
! if (VARATT_IS_EXPANDED_HEADER(arr))
! {
! /* we can just grab the type data from expanded array */
! fctx->elmlen = arr->xpn.typlen;
! fctx->elmbyval = arr->xpn.typbyval;
! fctx->elmalign = arr->xpn.typalign;
! }
! else
! get_typlenbyvalalign(AARR_ELEMTYPE(arr),
! &fctx->elmlen,
! &fctx->elmbyval,
! &fctx->elmalign);
funcctx->user_fctx = fctx;
MemoryContextSwitchTo(oldcontext);
*************** array_unnest(PG_FUNCTION_ARGS)
*** 5669,5700 ****
int offset = fctx->nextelem++;
Datum elem;
! /*
! * Check for NULL array element
! */
! if (array_get_isnull(fctx->arraynullsptr, offset))
! {
! fcinfo->isnull = true;
! elem = (Datum) 0;
! /* elemdataptr does not move */
! }
! else
! {
! /*
! * OK, get the element
! */
! char *ptr = fctx->elemdataptr;
!
! fcinfo->isnull = false;
! elem = ArrayCast(ptr, fctx->elmbyval, fctx->elmlen);
!
! /*
! * Advance elemdataptr over it
! */
! ptr = att_addlength_pointer(ptr, fctx->elmlen, ptr);
! ptr = (char *) att_align_nominal(ptr, fctx->elmalign);
! fctx->elemdataptr = ptr;
! }
SRF_RETURN_NEXT(funcctx, elem);
}
--- 5894,5901 ----
int offset = fctx->nextelem++;
Datum elem;
! ARRAY_ITER_NEXT(fctx->iter, offset, elem, fcinfo->isnull,
! fctx->elmlen, fctx->elmbyval, fctx->elmalign);
SRF_RETURN_NEXT(funcctx, elem);
}
*************** array_replace_internal(ArrayType *array,
*** 5946,5952 ****
result->ndim = ndim;
result->dataoffset = dataoffset;
result->elemtype = element_type;
! memcpy(ARR_DIMS(result), ARR_DIMS(array), 2 * ndim * sizeof(int));
if (remove)
{
--- 6147,6154 ----
result->ndim = ndim;
result->dataoffset = dataoffset;
result->elemtype = element_type;
! memcpy(ARR_DIMS(result), ARR_DIMS(array), ndim * sizeof(int));
! memcpy(ARR_LBOUND(result), ARR_LBOUND(array), ndim * sizeof(int));
if (remove)
{
diff --git a/src/backend/utils/adt/datum.c b/src/backend/utils/adt/datum.c
index 014eca5..e8af030 100644
*** a/src/backend/utils/adt/datum.c
--- b/src/backend/utils/adt/datum.c
***************
*** 12,19 ****
*
*-------------------------------------------------------------------------
*/
/*
! * In the implementation of the next routines we assume the following:
*
* A) if a type is "byVal" then all the information is stored in the
* Datum itself (i.e. no pointers involved!). In this case the
--- 12,20 ----
*
*-------------------------------------------------------------------------
*/
+
/*
! * In the implementation of these routines we assume the following:
*
* A) if a type is "byVal" then all the information is stored in the
* Datum itself (i.e. no pointers involved!). In this case the
***************
*** 34,44 ****
--- 35,49 ----
*
* Note that we do not treat "toasted" datums specially; therefore what
* will be copied or compared is the compressed data or toast reference.
+ * An exception is made for datumCopy() of an expanded object, however,
+ * because most callers expect to get a simple contiguous (and pfree'able)
+ * result from datumCopy(). See also datumTransfer().
*/
#include "postgres.h"
#include "utils/datum.h"
+ #include "utils/expandeddatum.h"
/*-------------------------------------------------------------------------
***************
*** 46,51 ****
--- 51,57 ----
*
* Find the "real" size of a datum, given the datum value,
* whether it is a "by value", and the declared type length.
+ * (For TOAST pointer datums, this is the size of the pointer datum.)
*
* This is essentially an out-of-line version of the att_addlength_datum()
* macro in access/tupmacs.h. We do a tad more error checking though.
*************** datumGetSize(Datum value, bool typByVal,
*** 106,114 ****
/*-------------------------------------------------------------------------
* datumCopy
*
! * make a copy of a datum
*
* If the datatype is pass-by-reference, memory is obtained with palloc().
*-------------------------------------------------------------------------
*/
Datum
--- 112,127 ----
/*-------------------------------------------------------------------------
* datumCopy
*
! * Make a copy of a non-NULL datum.
*
* If the datatype is pass-by-reference, memory is obtained with palloc().
+ *
+ * If the value is a reference to an expanded object, we flatten into memory
+ * obtained with palloc(). We need to copy because one of the main uses of
+ * this function is to copy a datum out of a transient memory context that's
+ * about to be destroyed, and the expanded object is probably in a child
+ * context that will also go away. Moreover, many callers assume that the
+ * result is a single pfree-able chunk.
*-------------------------------------------------------------------------
*/
Datum
*************** datumCopy(Datum value, bool typByVal, in
*** 118,161 ****
if (typByVal)
res = value;
else
{
Size realSize;
! char *s;
!
! if (DatumGetPointer(value) == NULL)
! return PointerGetDatum(NULL);
realSize = datumGetSize(value, typByVal, typLen);
! s = (char *) palloc(realSize);
! memcpy(s, DatumGetPointer(value), realSize);
! res = PointerGetDatum(s);
}
return res;
}
/*-------------------------------------------------------------------------
! * datumFree
*
! * Free the space occupied by a datum CREATED BY "datumCopy"
*
! * NOTE: DO NOT USE THIS ROUTINE with datums returned by heap_getattr() etc.
! * ONLY datums created by "datumCopy" can be freed!
*-------------------------------------------------------------------------
*/
! #ifdef NOT_USED
! void
! datumFree(Datum value, bool typByVal, int typLen)
{
! if (!typByVal)
! {
! Pointer s = DatumGetPointer(value);
!
! pfree(s);
! }
}
- #endif
/*-------------------------------------------------------------------------
* datumIsEqual
--- 131,201 ----
if (typByVal)
res = value;
+ else if (typLen == -1)
+ {
+ /* It is a varlena datatype */
+ struct varlena *vl = (struct varlena *) DatumGetPointer(value);
+
+ if (VARATT_IS_EXTERNAL_EXPANDED(vl))
+ {
+ /* Flatten into the caller's memory context */
+ ExpandedObjectHeader *eoh = DatumGetEOHP(value);
+ Size resultsize;
+ char *resultptr;
+
+ resultsize = EOH_get_flat_size(eoh);
+ resultptr = (char *) palloc(resultsize);
+ EOH_flatten_into(eoh, (void *) resultptr, resultsize);
+ res = PointerGetDatum(resultptr);
+ }
+ else
+ {
+ /* Otherwise, just copy the varlena datum verbatim */
+ Size realSize;
+ char *resultptr;
+
+ realSize = (Size) VARSIZE_ANY(vl);
+ resultptr = (char *) palloc(realSize);
+ memcpy(resultptr, vl, realSize);
+ res = PointerGetDatum(resultptr);
+ }
+ }
else
{
+ /* Pass by reference, but not varlena, so not toasted */
Size realSize;
! char *resultptr;
realSize = datumGetSize(value, typByVal, typLen);
! resultptr = (char *) palloc(realSize);
! memcpy(resultptr, DatumGetPointer(value), realSize);
! res = PointerGetDatum(resultptr);
}
return res;
}
/*-------------------------------------------------------------------------
! * datumTransfer
*
! * Transfer a non-NULL datum into the current memory context.
*
! * This is equivalent to datumCopy() except when the datum is a read-write
! * pointer to an expanded object. In that case we merely reparent the object
! * into the current context, and return its standard R/W pointer (in case the
! * given one is a transient pointer of shorter lifespan).
*-------------------------------------------------------------------------
*/
! Datum
! datumTransfer(Datum value, bool typByVal, int typLen)
{
! if (!typByVal && typLen == -1 &&
! VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(value)))
! value = TransferExpandedObject(value, CurrentMemoryContext);
! else
! value = datumCopy(value, typByVal, typLen);
! return value;
}
/*-------------------------------------------------------------------------
* datumIsEqual
diff --git a/src/backend/utils/adt/expandeddatum.c b/src/backend/utils/adt/expandeddatum.c
index ...039671b .
*** a/src/backend/utils/adt/expandeddatum.c
--- b/src/backend/utils/adt/expandeddatum.c
***************
*** 0 ****
--- 1,163 ----
+ /*-------------------------------------------------------------------------
+ *
+ * expandeddatum.c
+ * Support functions for "expanded" value representations.
+ *
+ * Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/utils/adt/expandeddatum.c
+ *
+ *-------------------------------------------------------------------------
+ */
+ #include "postgres.h"
+
+ #include "utils/expandeddatum.h"
+ #include "utils/memutils.h"
+
+ /*
+ * DatumGetEOHP
+ *
+ * Given a Datum that is an expanded-object reference, extract the pointer.
+ *
+ * This is a bit tedious since the pointer may not be properly aligned;
+ * compare VARATT_EXTERNAL_GET_POINTER().
+ */
+ ExpandedObjectHeader *
+ DatumGetEOHP(Datum d)
+ {
+ varattrib_1b_e *datum = (varattrib_1b_e *) DatumGetPointer(d);
+ varatt_expanded ptr;
+
+ Assert(VARATT_IS_EXTERNAL_EXPANDED(datum));
+ memcpy(&ptr, VARDATA_EXTERNAL(datum), sizeof(ptr));
+ Assert(VARATT_IS_EXPANDED_HEADER(ptr.eohptr));
+ return ptr.eohptr;
+ }
+
+ /*
+ * EOH_init_header
+ *
+ * Initialize the common header of an expanded object.
+ *
+ * The main thing this encapsulates is initializing the TOAST pointers.
+ */
+ void
+ EOH_init_header(ExpandedObjectHeader *eohptr,
+ const ExpandedObjectMethods *methods,
+ MemoryContext obj_context)
+ {
+ varatt_expanded ptr;
+
+ eohptr->vl_len_ = EOH_HEADER_MAGIC;
+ eohptr->eoh_methods = methods;
+ eohptr->eoh_context = obj_context;
+
+ ptr.eohptr = eohptr;
+
+ SET_VARTAG_EXTERNAL(eohptr->eoh_rw_ptr, VARTAG_EXPANDED_RW);
+ memcpy(VARDATA_EXTERNAL(eohptr->eoh_rw_ptr), &ptr, sizeof(ptr));
+
+ SET_VARTAG_EXTERNAL(eohptr->eoh_ro_ptr, VARTAG_EXPANDED_RO);
+ memcpy(VARDATA_EXTERNAL(eohptr->eoh_ro_ptr), &ptr, sizeof(ptr));
+ }
+
+ /*
+ * EOH_get_flat_size
+ * EOH_flatten_into
+ *
+ * Convenience functions for invoking the "methods" of an expanded object.
+ */
+
+ Size
+ EOH_get_flat_size(ExpandedObjectHeader *eohptr)
+ {
+ return (*eohptr->eoh_methods->get_flat_size) (eohptr);
+ }
+
+ void
+ EOH_flatten_into(ExpandedObjectHeader *eohptr,
+ void *result, Size allocated_size)
+ {
+ (*eohptr->eoh_methods->flatten_into) (eohptr, result, allocated_size);
+ }
+
+ /*
+ * Does the Datum represent a writable expanded object?
+ */
+ bool
+ DatumIsReadWriteExpandedObject(Datum d, bool isnull, int16 typlen)
+ {
+ /* Reject if it's NULL or not a varlena type */
+ if (isnull || typlen != -1)
+ return false;
+
+ /* Reject if not a read-write expanded-object pointer */
+ if (!VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(d)))
+ return false;
+
+ return true;
+ }
+
+ /*
+ * If the Datum represents a R/W expanded object, change it to R/O.
+ * Otherwise return the original Datum.
+ */
+ Datum
+ MakeExpandedObjectReadOnly(Datum d, bool isnull, int16 typlen)
+ {
+ ExpandedObjectHeader *eohptr;
+
+ /* Nothing to do if it's NULL or not a varlena type */
+ if (isnull || typlen != -1)
+ return d;
+
+ /* Nothing to do if not a read-write expanded-object pointer */
+ if (!VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(d)))
+ return d;
+
+ /* Now safe to extract the object pointer */
+ eohptr = DatumGetEOHP(d);
+
+ /* Return the built-in read-only pointer instead of given pointer */
+ return EOHPGetRODatum(eohptr);
+ }
+
+ /*
+ * Transfer ownership of an expanded object to a new parent memory context.
+ * The object must be referenced by a R/W pointer, and what we return is
+ * always its "standard" R/W pointer, which is certain to have the same
+ * lifespan as the object itself. (The passed-in pointer might not, and
+ * in any case wouldn't provide a unique identifier if it's not that one.)
+ */
+ Datum
+ TransferExpandedObject(Datum d, MemoryContext new_parent)
+ {
+ ExpandedObjectHeader *eohptr = DatumGetEOHP(d);
+
+ /* Assert caller gave a R/W pointer */
+ Assert(VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(d)));
+
+ /* Transfer ownership */
+ MemoryContextSetParent(eohptr->eoh_context, new_parent);
+
+ /* Return the object's standard read-write pointer */
+ return EOHPGetRWDatum(eohptr);
+ }
+
+ /*
+ * Delete an expanded object (must be referenced by a R/W pointer).
+ */
+ void
+ DeleteExpandedObject(Datum d)
+ {
+ ExpandedObjectHeader *eohptr = DatumGetEOHP(d);
+
+ /* Assert caller gave a R/W pointer */
+ Assert(VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(d)));
+
+ /* Kill it */
+ MemoryContextDelete(eohptr->eoh_context);
+ }
diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c
index 202bc78..4b24066 100644
*** a/src/backend/utils/mmgr/mcxt.c
--- b/src/backend/utils/mmgr/mcxt.c
*************** MemoryContextSetParent(MemoryContext con
*** 266,271 ****
--- 266,275 ----
AssertArg(MemoryContextIsValid(context));
AssertArg(context != new_parent);
+ /* Fast path if it's got correct parent already */
+ if (new_parent == context->parent)
+ return;
+
/* Delink from existing parent, if any */
if (context->parent)
{
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 40fde83..a98a7af 100644
*** a/src/include/executor/executor.h
--- b/src/include/executor/executor.h
*************** extern void FreeExprContext(ExprContext
*** 312,318 ****
extern void ReScanExprContext(ExprContext *econtext);
#define ResetExprContext(econtext) \
! MemoryContextReset((econtext)->ecxt_per_tuple_memory)
extern ExprContext *MakePerTupleExprContext(EState *estate);
--- 312,318 ----
extern void ReScanExprContext(ExprContext *econtext);
#define ResetExprContext(econtext) \
! MemoryContextResetAndDeleteChildren((econtext)->ecxt_per_tuple_memory)
extern ExprContext *MakePerTupleExprContext(EState *estate);
diff --git a/src/include/executor/spi.h b/src/include/executor/spi.h
index 9e912ba..fbcae0c 100644
*** a/src/include/executor/spi.h
--- b/src/include/executor/spi.h
*************** extern char *SPI_getnspname(Relation rel
*** 124,129 ****
--- 124,130 ----
extern void *SPI_palloc(Size size);
extern void *SPI_repalloc(void *pointer, Size size);
extern void SPI_pfree(void *pointer);
+ extern Datum SPI_datumTransfer(Datum value, bool typByVal, int typLen);
extern void SPI_freetuple(HeapTuple pointer);
extern void SPI_freetuptable(SPITupleTable *tuptable);
diff --git a/src/include/executor/tuptable.h b/src/include/executor/tuptable.h
index 48f84bf..00686b0 100644
*** a/src/include/executor/tuptable.h
--- b/src/include/executor/tuptable.h
*************** extern Datum ExecFetchSlotTupleDatum(Tup
*** 163,168 ****
--- 163,169 ----
extern HeapTuple ExecMaterializeSlot(TupleTableSlot *slot);
extern TupleTableSlot *ExecCopySlot(TupleTableSlot *dstslot,
TupleTableSlot *srcslot);
+ extern TupleTableSlot *ExecMakeSlotContentsReadOnly(TupleTableSlot *slot);
/* in access/common/heaptuple.c */
extern Datum slot_getattr(TupleTableSlot *slot, int attnum, bool *isnull);
diff --git a/src/include/nodes/primnodes.h b/src/include/nodes/primnodes.h
index 1d06f42..932a96b 100644
*** a/src/include/nodes/primnodes.h
--- b/src/include/nodes/primnodes.h
*************** typedef struct WindowFunc
*** 310,315 ****
--- 310,319 ----
* Note: the result datatype is the element type when fetching a single
* element; but it is the array type when doing subarray fetch or either
* type of store.
+ *
+ * Note: for the cases where an array is returned, if refexpr yields a R/W
+ * expanded array, then the implementation is allowed to modify that object
+ * in-place and return the same object.)
* ----------------
*/
typedef struct ArrayRef
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 082c75b..5dd897a 100644
*** a/src/include/postgres.h
--- b/src/include/postgres.h
*************** typedef struct varatt_indirect
*** 88,93 ****
--- 88,110 ----
} varatt_indirect;
/*
+ * struct varatt_expanded is a "TOAST pointer" representing an out-of-line
+ * Datum that is stored in memory, in some type-specific, not necessarily
+ * physically contiguous format that is convenient for computation not
+ * storage. APIs for this, in particular the definition of struct
+ * ExpandedObjectHeader, are in src/include/utils/expandeddatum.h.
+ *
+ * Note that just as for struct varatt_external, this struct is stored
+ * unaligned within any containing tuple.
+ */
+ typedef struct ExpandedObjectHeader ExpandedObjectHeader;
+
+ typedef struct varatt_expanded
+ {
+ ExpandedObjectHeader *eohptr;
+ } varatt_expanded;
+
+ /*
* Type tag for the various sorts of "TOAST pointer" datums. The peculiar
* value for VARTAG_ONDISK comes from a requirement for on-disk compatibility
* with a previous notion that the tag field was the pointer datum's length.
*************** typedef struct varatt_indirect
*** 95,105 ****
--- 112,129 ----
typedef enum vartag_external
{
VARTAG_INDIRECT = 1,
+ VARTAG_EXPANDED_RO = 2,
+ VARTAG_EXPANDED_RW = 3,
VARTAG_ONDISK = 18
} vartag_external;
+ /* this test relies on the specific tag values above */
+ #define VARTAG_IS_EXPANDED(tag) \
+ (((tag) & ~1) == VARTAG_EXPANDED_RO)
+
#define VARTAG_SIZE(tag) \
((tag) == VARTAG_INDIRECT ? sizeof(varatt_indirect) : \
+ VARTAG_IS_EXPANDED(tag) ? sizeof(varatt_expanded) : \
(tag) == VARTAG_ONDISK ? sizeof(varatt_external) : \
TrapMacro(true, "unrecognized TOAST vartag"))
*************** typedef struct
*** 294,299 ****
--- 318,329 ----
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
#define VARATT_IS_EXTERNAL_INDIRECT(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_INDIRECT)
+ #define VARATT_IS_EXTERNAL_EXPANDED_RO(PTR) \
+ (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_EXPANDED_RO)
+ #define VARATT_IS_EXTERNAL_EXPANDED_RW(PTR) \
+ (VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_EXPANDED_RW)
+ #define VARATT_IS_EXTERNAL_EXPANDED(PTR) \
+ (VARATT_IS_EXTERNAL(PTR) && VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
#define VARATT_IS_SHORT(PTR) VARATT_IS_1B(PTR)
#define VARATT_IS_EXTENDED(PTR) (!VARATT_IS_4B_U(PTR))
diff --git a/src/include/utils/array.h b/src/include/utils/array.h
index d9fac80..93bee4d 100644
*** a/src/include/utils/array.h
--- b/src/include/utils/array.h
***************
*** 45,50 ****
--- 45,55 ----
* We support subscripting on these types, but array_in() and array_out()
* only work with varlena arrays.
*
+ * In addition, arrays are a major user of the "expanded object" TOAST
+ * infrastructure. This allows a varlena array to be converted to a
+ * separate representation that may include "deconstructed" Datum/isnull
+ * arrays holding the elements.
+ *
*
* Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
***************
*** 57,62 ****
--- 62,69 ----
#define ARRAY_H
#include "fmgr.h"
+ #include "utils/expandeddatum.h"
+
/*
* Arrays are varlena objects, so must meet the varlena convention that
*************** typedef struct
*** 75,80 ****
--- 82,167 ----
} ArrayType;
/*
+ * An expanded array is contained within a private memory context (as
+ * all expanded objects must be) and has a control structure as below.
+ *
+ * The expanded array might contain a regular "flat" array if that was the
+ * original input and we've not modified it significantly. Otherwise, the
+ * contents are represented by Datum/isnull arrays plus dimensionality and
+ * type information. We could also have both forms, if we've deconstructed
+ * the original array for access purposes but not yet changed it. For pass-
+ * by-reference element types, the Datums would point into the flat array in
+ * this situation. Once we start modifying array elements, new pass-by-ref
+ * elements are separately palloc'd within the memory context.
+ */
+ #define EA_MAGIC 689375833 /* ID for debugging crosschecks */
+
+ typedef struct ExpandedArrayHeader
+ {
+ /* Standard header for expanded objects */
+ ExpandedObjectHeader hdr;
+
+ /* Magic value identifying an expanded array (for debugging only) */
+ int ea_magic;
+
+ /* Dimensionality info (always valid) */
+ int ndims; /* # of dimensions */
+ int *dims; /* array dimensions */
+ int *lbound; /* index lower bounds for each dimension */
+
+ /* Element type info (always valid) */
+ Oid element_type; /* element type OID */
+ int16 typlen; /* needed info about element datatype */
+ bool typbyval;
+ char typalign;
+
+ /*
+ * If we have a Datum-array representation of the array, it's kept here;
+ * else dvalues/dnulls are NULL. The dvalues and dnulls arrays are always
+ * palloc'd within the object private context, but may change size from
+ * time to time. For pass-by-ref element types, dvalues entries might
+ * point either into the fstartptr..fendptr area, or to separately
+ * palloc'd chunks. Elements should always be fully detoasted, as they
+ * are in the standard flat representation.
+ *
+ * Even when dvalues is valid, dnulls can be NULL if there are no null
+ * elements.
+ */
+ Datum *dvalues; /* array of Datums */
+ bool *dnulls; /* array of is-null flags for Datums */
+ int dvalueslen; /* allocated length of above arrays */
+ int nelems; /* number of valid entries in above arrays */
+
+ /*
+ * flat_size is the current space requirement for the flat equivalent of
+ * the expanded array, if known; otherwise it's 0. We store this to make
+ * consecutive calls of get_flat_size cheap.
+ */
+ Size flat_size;
+
+ /*
+ * fvalue points to the flat representation if it is valid, else it is
+ * NULL. If we have or ever had a flat representation then
+ * fstartptr/fendptr point to the start and end+1 of its data area; this
+ * is so that we can tell which Datum pointers point into the flat
+ * representation rather than being pointers to separately palloc'd data.
+ */
+ ArrayType *fvalue; /* must be a fully detoasted array */
+ char *fstartptr; /* start of its data area */
+ char *fendptr; /* end+1 of its data area */
+ } ExpandedArrayHeader;
+
+ /*
+ * Functions that can handle either a "flat" varlena array or an expanded
+ * array use this union to work with their input.
+ */
+ typedef union AnyArrayType
+ {
+ ArrayType flt;
+ ExpandedArrayHeader xpn;
+ } AnyArrayType;
+
+ /*
* working state for accumArrayResult() and friends
* note that the input must be scalars (legal array elements)
*/
*************** typedef struct ArrayMapState
*** 149,165 ****
/* ArrayIteratorData is private in arrayfuncs.c */
typedef struct ArrayIteratorData *ArrayIterator;
! /*
! * fmgr macros for array objects
! */
#define DatumGetArrayTypeP(X) ((ArrayType *) PG_DETOAST_DATUM(X))
#define DatumGetArrayTypePCopy(X) ((ArrayType *) PG_DETOAST_DATUM_COPY(X))
#define PG_GETARG_ARRAYTYPE_P(n) DatumGetArrayTypeP(PG_GETARG_DATUM(n))
#define PG_GETARG_ARRAYTYPE_P_COPY(n) DatumGetArrayTypePCopy(PG_GETARG_DATUM(n))
#define PG_RETURN_ARRAYTYPE_P(x) PG_RETURN_POINTER(x)
/*
! * Access macros for array header fields.
*
* ARR_DIMS returns a pointer to an array of array dimensions (number of
* elements along the various array axes).
--- 236,259 ----
/* ArrayIteratorData is private in arrayfuncs.c */
typedef struct ArrayIteratorData *ArrayIterator;
! /* fmgr macros for regular varlena array objects */
#define DatumGetArrayTypeP(X) ((ArrayType *) PG_DETOAST_DATUM(X))
#define DatumGetArrayTypePCopy(X) ((ArrayType *) PG_DETOAST_DATUM_COPY(X))
#define PG_GETARG_ARRAYTYPE_P(n) DatumGetArrayTypeP(PG_GETARG_DATUM(n))
#define PG_GETARG_ARRAYTYPE_P_COPY(n) DatumGetArrayTypePCopy(PG_GETARG_DATUM(n))
#define PG_RETURN_ARRAYTYPE_P(x) PG_RETURN_POINTER(x)
+ /* fmgr macros for expanded array objects */
+ #define PG_GETARG_EXPANDED_ARRAY(n) DatumGetExpandedArray(PG_GETARG_DATUM(n))
+ #define PG_GETARG_EXPANDED_ARRAYX(n, elmlen, elmbyval, elmalign) \
+ DatumGetExpandedArrayX(PG_GETARG_DATUM(n), elmlen, elmbyval, elmalign)
+ #define PG_RETURN_EXPANDED_ARRAY(x) PG_RETURN_DATUM(EOHPGetRWDatum(&(x)->hdr))
+
+ /* fmgr macros for AnyArrayType (ie, get either varlena or expanded form) */
+ #define PG_GETARG_ANY_ARRAY(n) DatumGetAnyArray(PG_GETARG_DATUM(n))
+
/*
! * Access macros for varlena array header fields.
*
* ARR_DIMS returns a pointer to an array of array dimensions (number of
* elements along the various array axes).
*************** typedef struct ArrayIteratorData *ArrayI
*** 207,212 ****
--- 301,402 ----
#define ARR_DATA_PTR(a) \
(((char *) (a)) + ARR_DATA_OFFSET(a))
+ /*
+ * Macros for working with AnyArrayType inputs. Beware multiple references!
+ */
+ #define AARR_NDIM(a) \
+ (VARATT_IS_EXPANDED_HEADER(a) ? (a)->xpn.ndims : ARR_NDIM(&(a)->flt))
+ #define AARR_HASNULL(a) \
+ (VARATT_IS_EXPANDED_HEADER(a) ? \
+ ((a)->xpn.dvalues != NULL ? (a)->xpn.dnulls != NULL : ARR_HASNULL((a)->xpn.fvalue)) : \
+ ARR_HASNULL(&(a)->flt))
+ #define AARR_ELEMTYPE(a) \
+ (VARATT_IS_EXPANDED_HEADER(a) ? (a)->xpn.element_type : ARR_ELEMTYPE(&(a)->flt))
+ #define AARR_DIMS(a) \
+ (VARATT_IS_EXPANDED_HEADER(a) ? (a)->xpn.dims : ARR_DIMS(&(a)->flt))
+ #define AARR_LBOUND(a) \
+ (VARATT_IS_EXPANDED_HEADER(a) ? (a)->xpn.lbound : ARR_LBOUND(&(a)->flt))
+
+ /*
+ * Macros for iterating through elements of a flat or expanded array.
+ * Use "ARRAY_ITER ARRAY_ITER_VARS(name);" to declare the local variables
+ * needed for an iterator (more than one set can be used in the same function,
+ * if they have different names).
+ * Use "ARRAY_ITER_SETUP(name, arrayptr);" to prepare to iterate, and
+ * "ARRAY_ITER_NEXT(name, index, datumvar, isnullvar, ...);" to fetch the
+ * next element into datumvar/isnullvar. "index" must be the zero-origin
+ * element number; we make caller provide this since caller is generally
+ * counting the elements anyway.
+ */
+ #define ARRAY_ITER /* dummy type name to keep pgindent happy */
+
+ #define ARRAY_ITER_VARS(iter) \
+ Datum *iter##datumptr; \
+ bool *iter##isnullptr; \
+ char *iter##dataptr; \
+ bits8 *iter##bitmapptr; \
+ int iter##bitmask
+
+ #define ARRAY_ITER_SETUP(iter, arrayptr) \
+ do { \
+ if (VARATT_IS_EXPANDED_HEADER(arrayptr)) \
+ { \
+ if ((arrayptr)->xpn.dvalues) \
+ { \
+ (iter##datumptr) = (arrayptr)->xpn.dvalues; \
+ (iter##isnullptr) = (arrayptr)->xpn.dnulls; \
+ (iter##dataptr) = NULL; \
+ (iter##bitmapptr) = NULL; \
+ } \
+ else \
+ { \
+ (iter##datumptr) = NULL; \
+ (iter##isnullptr) = NULL; \
+ (iter##dataptr) = ARR_DATA_PTR((arrayptr)->xpn.fvalue); \
+ (iter##bitmapptr) = ARR_NULLBITMAP((arrayptr)->xpn.fvalue); \
+ } \
+ } \
+ else \
+ { \
+ (iter##datumptr) = NULL; \
+ (iter##isnullptr) = NULL; \
+ (iter##dataptr) = ARR_DATA_PTR(&(arrayptr)->flt); \
+ (iter##bitmapptr) = ARR_NULLBITMAP(&(arrayptr)->flt); \
+ } \
+ (iter##bitmask) = 1; \
+ } while (0)
+
+ #define ARRAY_ITER_NEXT(iter,i, datumvar,isnullvar, elmlen,elmbyval,elmalign) \
+ do { \
+ if (iter##datumptr) \
+ { \
+ (datumvar) = (iter##datumptr)[i]; \
+ (isnullvar) = (iter##isnullptr) ? (iter##isnullptr)[i] : false; \
+ } \
+ else \
+ { \
+ if ((iter##bitmapptr) && (*(iter##bitmapptr) & (iter##bitmask)) == 0) \
+ { \
+ (isnullvar) = true; \
+ (datumvar) = (Datum) 0; \
+ } \
+ else \
+ { \
+ (isnullvar) = false; \
+ (datumvar) = fetch_att(iter##dataptr, elmbyval, elmlen); \
+ (iter##dataptr) = att_addlength_pointer(iter##dataptr, elmlen, iter##dataptr); \
+ (iter##dataptr) = (char *) att_align_nominal(iter##dataptr, elmalign); \
+ } \
+ (iter##bitmask) <<= 1; \
+ if ((iter##bitmask) == 0x100) \
+ { \
+ if (iter##bitmapptr) \
+ (iter##bitmapptr)++; \
+ (iter##bitmask) = 1; \
+ } \
+ } \
+ } while (0)
+
/*
* GUC parameter
*************** extern Datum array_remove(PG_FUNCTION_AR
*** 248,253 ****
--- 438,452 ----
extern Datum array_replace(PG_FUNCTION_ARGS);
extern Datum width_bucket_array(PG_FUNCTION_ARGS);
+ extern void CopyArrayEls(ArrayType *array,
+ Datum *values,
+ bool *nulls,
+ int nitems,
+ int typlen,
+ bool typbyval,
+ char typalign,
+ bool freedata);
+
extern Datum array_get_element(Datum arraydatum, int nSubscripts, int *indx,
int arraytyplen, int elmlen, bool elmbyval, char elmalign,
bool *isNull);
*************** extern ArrayType *array_set(ArrayType *a
*** 269,275 ****
Datum dataValue, bool isNull,
int arraytyplen, int elmlen, bool elmbyval, char elmalign);
! extern Datum array_map(FunctionCallInfo fcinfo, Oid inpType, Oid retType,
ArrayMapState *amstate);
extern void array_bitmap_copy(bits8 *destbitmap, int destoffset,
--- 468,474 ----
Datum dataValue, bool isNull,
int arraytyplen, int elmlen, bool elmbyval, char elmalign);
! extern Datum array_map(FunctionCallInfo fcinfo, Oid retType,
ArrayMapState *amstate);
extern void array_bitmap_copy(bits8 *destbitmap, int destoffset,
*************** extern ArrayType *construct_md_array(Dat
*** 286,291 ****
--- 485,493 ----
int *lbs,
Oid elmtype, int elmlen, bool elmbyval, char elmalign);
extern ArrayType *construct_empty_array(Oid elmtype);
+ extern ExpandedArrayHeader *construct_empty_expanded_array(Oid element_type,
+ MemoryContext parentcontext,
+ int elmlen, bool elmbyval, char elmalign);
extern void deconstruct_array(ArrayType *array,
Oid elmtype,
int elmlen, bool elmbyval, char elmalign,
*************** extern int mda_next_tuple(int n, int *cu
*** 339,344 ****
--- 541,557 ----
extern int32 *ArrayGetIntegerTypmods(ArrayType *arr, int *n);
/*
+ * prototypes for functions defined in array_expanded.c
+ */
+ extern Datum expand_array(Datum arraydatum, MemoryContext parentcontext,
+ int elmlen, bool elmbyval, char elmalign);
+ extern ExpandedArrayHeader *DatumGetExpandedArray(Datum d);
+ extern ExpandedArrayHeader *DatumGetExpandedArrayX(Datum d,
+ int elmlen, bool elmbyval, char elmalign);
+ extern AnyArrayType *DatumGetAnyArray(Datum d);
+ extern void deconstruct_expanded_array(ExpandedArrayHeader *eah);
+
+ /*
* prototypes for functions defined in array_userfuncs.c
*/
extern Datum array_append(PG_FUNCTION_ARGS);
diff --git a/src/include/utils/datum.h b/src/include/utils/datum.h
index 663414b..c572f79 100644
*** a/src/include/utils/datum.h
--- b/src/include/utils/datum.h
***************
*** 24,41 ****
extern Size datumGetSize(Datum value, bool typByVal, int typLen);
/*
! * datumCopy - make a copy of a datum.
*
* If the datatype is pass-by-reference, memory is obtained with palloc().
*/
extern Datum datumCopy(Datum value, bool typByVal, int typLen);
/*
! * datumFree - free a datum previously allocated by datumCopy, if any.
*
! * Does nothing if datatype is pass-by-value.
*/
! extern void datumFree(Datum value, bool typByVal, int typLen);
/*
* datumIsEqual
--- 24,41 ----
extern Size datumGetSize(Datum value, bool typByVal, int typLen);
/*
! * datumCopy - make a copy of a non-NULL datum.
*
* If the datatype is pass-by-reference, memory is obtained with palloc().
*/
extern Datum datumCopy(Datum value, bool typByVal, int typLen);
/*
! * datumTransfer - transfer a non-NULL datum into the current memory context.
*
! * Differs from datumCopy() in its handling of read-write expanded objects.
*/
! extern Datum datumTransfer(Datum value, bool typByVal, int typLen);
/*
* datumIsEqual
diff --git a/src/include/utils/expandeddatum.h b/src/include/utils/expandeddatum.h
index ...3a8336e .
*** a/src/include/utils/expandeddatum.h
--- b/src/include/utils/expandeddatum.h
***************
*** 0 ****
--- 1,148 ----
+ /*-------------------------------------------------------------------------
+ *
+ * expandeddatum.h
+ * Declarations for access to "expanded" value representations.
+ *
+ * Complex data types, particularly container types such as arrays and
+ * records, usually have on-disk representations that are compact but not
+ * especially convenient to modify. What's more, when we do modify them,
+ * having to recopy all the rest of the value can be extremely inefficient.
+ * Therefore, we provide a notion of an "expanded" representation that is used
+ * only in memory and is optimized more for computation than storage.
+ * The format appearing on disk is called the data type's "flattened"
+ * representation, since it is required to be a contiguous blob of bytes --
+ * but the type can have an expanded representation that is not. Data types
+ * must provide means to translate an expanded representation back to
+ * flattened form.
+ *
+ * An expanded object is meant to survive across multiple operations, but
+ * not to be enormously long-lived; for example it might be a local variable
+ * in a PL/pgSQL procedure. So its extra bulk compared to the on-disk format
+ * is a worthwhile trade-off.
+ *
+ * References to expanded objects are a type of TOAST pointer.
+ * Because of longstanding conventions in Postgres, this means that the
+ * flattened form of such an object must always be a varlena object.
+ * Fortunately that's no restriction in practice.
+ *
+ * There are actually two kinds of TOAST pointers for expanded objects:
+ * read-only and read-write pointers. Possession of one of the latter
+ * authorizes a function to modify the value in-place rather than copying it
+ * as would normally be required. Functions should always return a read-write
+ * pointer to any new expanded object they create. Functions that modify an
+ * argument value in-place must take care that they do not corrupt the old
+ * value if they fail partway through.
+ *
+ *
+ * Portions Copyright (c) 1996-2015, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/utils/expandeddatum.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef EXPANDEDDATUM_H
+ #define EXPANDEDDATUM_H
+
+ /* Size of an EXTERNAL datum that contains a pointer to an expanded object */
+ #define EXPANDED_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_expanded))
+
+ /*
+ * "Methods" that must be provided for any expanded object.
+ *
+ * get_flat_size: compute space needed for flattened representation (which
+ * must be a valid in-line, non-compressed, 4-byte-header varlena object).
+ *
+ * flatten_into: construct flattened representation in the caller-allocated
+ * space at *result, of size allocated_size (which will always be the result
+ * of a preceding get_flat_size call; it's passed for cross-checking).
+ *
+ * Note: construction of a heap tuple from an expanded datum calls
+ * get_flat_size twice, so it's worthwhile to make sure that that doesn't
+ * incur too much overhead.
+ */
+ typedef Size (*EOM_get_flat_size_method) (ExpandedObjectHeader *eohptr);
+ typedef void (*EOM_flatten_into_method) (ExpandedObjectHeader *eohptr,
+ void *result, Size allocated_size);
+
+ /* Struct of function pointers for an expanded object's methods */
+ typedef struct ExpandedObjectMethods
+ {
+ EOM_get_flat_size_method get_flat_size;
+ EOM_flatten_into_method flatten_into;
+ } ExpandedObjectMethods;
+
+ /*
+ * Every expanded object must contain this header; typically the header
+ * is embedded in some larger struct that adds type-specific fields.
+ *
+ * It is presumed that the header object and all subsidiary data are stored
+ * in eoh_context, so that the object can be freed by deleting that context,
+ * or its storage lifespan can be altered by reparenting the context.
+ * (In principle the object could own additional resources, such as malloc'd
+ * storage, and use a memory context reset callback to free them upon reset or
+ * deletion of eoh_context.)
+ *
+ * We set up two TOAST pointers within the standard header, one read-write
+ * and one read-only. This allows functions to return either kind of pointer
+ * without making an additional allocation, and in particular without worrying
+ * whether a separately palloc'd object would have sufficient lifespan.
+ * But note that these pointers are just a convenience; a pointer object
+ * appearing somewhere else would still be legal.
+ *
+ * The typedef declaration for this appears in postgres.h.
+ */
+ struct ExpandedObjectHeader
+ {
+ /* Phony varlena header */
+ int32 vl_len_; /* always EOH_HEADER_MAGIC, see below */
+
+ /* Pointer to methods required for object type */
+ const ExpandedObjectMethods *eoh_methods;
+
+ /* Memory context containing this header and subsidiary data */
+ MemoryContext eoh_context;
+
+ /* Standard R/W TOAST pointer for this object is kept here */
+ char eoh_rw_ptr[EXPANDED_POINTER_SIZE];
+
+ /* Standard R/O TOAST pointer for this object is kept here */
+ char eoh_ro_ptr[EXPANDED_POINTER_SIZE];
+ };
+
+ /*
+ * Particularly for read-only functions, it is handy to be able to work with
+ * either regular "flat" varlena inputs or expanded inputs of the same data
+ * type. To allow determining which case an argument-fetching function has
+ * returned, the first int32 of an ExpandedObjectHeader always contains -1
+ * (EOH_HEADER_MAGIC to the code). This works since no 4-byte-header varlena
+ * could have that as its first 4 bytes. Caution: we could not reliably tell
+ * the difference between an ExpandedObjectHeader and a short-header object
+ * with this trick. However, it works fine if the argument fetching code
+ * always returns either a 4-byte-header flat object or an expanded object.
+ */
+ #define EOH_HEADER_MAGIC (-1)
+ #define VARATT_IS_EXPANDED_HEADER(PTR) \
+ (((ExpandedObjectHeader *) (PTR))->vl_len_ == EOH_HEADER_MAGIC)
+
+ /*
+ * Generic support functions for expanded objects.
+ * (More of these might be worth inlining later.)
+ */
+
+ #define EOHPGetRWDatum(eohptr) PointerGetDatum((eohptr)->eoh_rw_ptr)
+ #define EOHPGetRODatum(eohptr) PointerGetDatum((eohptr)->eoh_ro_ptr)
+
+ extern ExpandedObjectHeader *DatumGetEOHP(Datum d);
+ extern void EOH_init_header(ExpandedObjectHeader *eohptr,
+ const ExpandedObjectMethods *methods,
+ MemoryContext obj_context);
+ extern Size EOH_get_flat_size(ExpandedObjectHeader *eohptr);
+ extern void EOH_flatten_into(ExpandedObjectHeader *eohptr,
+ void *result, Size allocated_size);
+ extern bool DatumIsReadWriteExpandedObject(Datum d, bool isnull, int16 typlen);
+ extern Datum MakeExpandedObjectReadOnly(Datum d, bool isnull, int16 typlen);
+ extern Datum TransferExpandedObject(Datum d, MemoryContext new_parent);
+ extern void DeleteExpandedObject(Datum d);
+
+ #endif /* EXPANDEDDATUM_H */
diff --git a/src/pl/plpgsql/src/pl_comp.c b/src/pl/plpgsql/src/pl_comp.c
index f364ce4..d021145 100644
*** a/src/pl/plpgsql/src/pl_comp.c
--- b/src/pl/plpgsql/src/pl_comp.c
*************** build_datatype(HeapTuple typeTup, int32
*** 2202,2207 ****
--- 2202,2223 ----
typ->typbyval = typeStruct->typbyval;
typ->typrelid = typeStruct->typrelid;
typ->typioparam = getTypeIOParam(typeTup);
+ /* Detect if type is true array, or domain thereof */
+ /* NB: this is only used to decide whether to apply expand_array */
+ if (typeStruct->typtype == TYPTYPE_BASE)
+ {
+ /* this test should match what get_element_type() checks */
+ typ->typisarray = (typeStruct->typlen == -1 &&
+ OidIsValid(typeStruct->typelem));
+ }
+ else if (typeStruct->typtype == TYPTYPE_DOMAIN)
+ {
+ /* we can short-circuit looking up base types if it's not varlena */
+ typ->typisarray = (typeStruct->typlen == -1 &&
+ OidIsValid(get_base_element_type(typeStruct->typbasetype)));
+ }
+ else
+ typ->typisarray = false;
typ->collation = typeStruct->typcollation;
if (OidIsValid(collation) && OidIsValid(typ->collation))
typ->collation = collation;
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b7e3bc4..7d98ce1 100644
*** a/src/pl/plpgsql/src/pl_exec.c
--- b/src/pl/plpgsql/src/pl_exec.c
*************** static void exec_assign_value(PLpgSQL_ex
*** 171,176 ****
--- 171,177 ----
Datum value, Oid valtype, bool *isNull);
static void exec_eval_datum(PLpgSQL_execstate *estate,
PLpgSQL_datum *datum,
+ bool getrwpointer,
Oid *typeid,
int32 *typetypmod,
Datum *value,
*************** plpgsql_exec_function(PLpgSQL_function *
*** 295,300 ****
--- 296,339 ----
var->value = fcinfo->arg[i];
var->isnull = fcinfo->argnull[i];
var->freeval = false;
+
+ /*
+ * Force any array-valued parameter to be stored in
+ * expanded form in our local variable, in hopes of
+ * improving efficiency of uses of the variable. (This is
+ * a hack, really: why only arrays? Need more thought
+ * about which cases are likely to win. See also
+ * typisarray-specific heuristic in exec_assign_value.)
+ *
+ * Special cases: If passed a R/W expanded pointer, assume
+ * we can commandeer the object rather than having to copy
+ * it. If passed a R/O expanded pointer, just keep it as
+ * the value of the variable for the moment. (We'll force
+ * it to R/W if the variable gets modified, but that may
+ * very well never happen.)
+ */
+ if (!var->isnull && var->datatype->typisarray)
+ {
+ if (VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(var->value)))
+ {
+ /* take ownership of R/W object */
+ var->value = TransferExpandedObject(var->value,
+ CurrentMemoryContext);
+ var->freeval = true;
+ }
+ else if (VARATT_IS_EXTERNAL_EXPANDED_RO(DatumGetPointer(var->value)))
+ {
+ /* R/O pointer, keep it as-is until assigned to */
+ }
+ else
+ {
+ /* flat array, so force to expanded form */
+ var->value = expand_array(var->value,
+ CurrentMemoryContext,
+ 0, 0, 0);
+ var->freeval = true;
+ }
+ }
}
break;
*************** plpgsql_exec_function(PLpgSQL_function *
*** 461,478 ****
/*
* If the function's return type isn't by value, copy the value
! * into upper executor memory context.
*/
if (!fcinfo->isnull && !func->fn_retbyval)
! {
! Size len;
! void *tmp;
!
! len = datumGetSize(estate.retval, false, func->fn_rettyplen);
! tmp = SPI_palloc(len);
! memcpy(tmp, DatumGetPointer(estate.retval), len);
! estate.retval = PointerGetDatum(tmp);
! }
}
}
--- 500,513 ----
/*
* If the function's return type isn't by value, copy the value
! * into upper executor memory context. However, if we have a R/W
! * expanded datum, we can just transfer its ownership out to the
! * upper executor context.
*/
if (!fcinfo->isnull && !func->fn_retbyval)
! estate.retval = SPI_datumTransfer(estate.retval,
! false,
! func->fn_rettyplen);
}
}
*************** exec_stmt_return(PLpgSQL_execstate *esta
*** 2455,2460 ****
--- 2490,2502 ----
* Special case path when the RETURN expression is a simple variable
* reference; in particular, this path is always taken in functions with
* one or more OUT parameters.
+ *
+ * This special case is especially efficient for returning variables that
+ * have R/W expanded values: we can put the R/W pointer directly into
+ * estate->retval, leading to transferring the value to the caller's
+ * context cheaply. If we went through exec_eval_expr we'd end up with a
+ * R/O pointer. It's okay to skip MakeExpandedObjectReadOnly here since
+ * we know we won't need the variable's value within the function anymore.
*/
if (stmt->retvarno >= 0)
{
*************** exec_stmt_return_next(PLpgSQL_execstate
*** 2580,2585 ****
--- 2622,2632 ----
* Special case path when the RETURN NEXT expression is a simple variable
* reference; in particular, this path is always taken in functions with
* one or more OUT parameters.
+ *
+ * Unlike exec_statement_return, there's no special win here for R/W
+ * expanded values, since they'll have to get flattened to go into the
+ * tuplestore. Indeed, we'd better make them R/O to avoid any risk of the
+ * casting step changing them in-place.
*/
if (stmt->retvarno >= 0)
{
*************** exec_stmt_return_next(PLpgSQL_execstate
*** 2598,2603 ****
--- 2645,2655 ----
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("wrong result type supplied in RETURN NEXT")));
+ /* let's be very paranoid about the cast step */
+ retval = MakeExpandedObjectReadOnly(retval,
+ isNull,
+ var->datatype->typlen);
+
/* coerce type if needed */
retval = exec_simple_cast_value(estate,
retval,
*************** exec_assign_value(PLpgSQL_execstate *est
*** 4061,4086 ****
/*
* If type is by-reference, copy the new value (which is
* probably in the eval_econtext) into the procedure's memory
! * context.
*/
if (!var->datatype->typbyval && !*isNull)
! newvalue = datumCopy(newvalue,
! false,
! var->datatype->typlen);
/*
! * Now free the old value. (We can't do this any earlier
! * because of the possibility that we are assigning the var's
! * old value to it, eg "foo := foo". We could optimize out
! * the assignment altogether in such cases, but it's too
! * infrequent to be worth testing for.)
*/
! free_var(var);
var->value = newvalue;
var->isnull = *isNull;
! if (!var->datatype->typbyval && !*isNull)
! var->freeval = true;
break;
}
--- 4113,4162 ----
/*
* If type is by-reference, copy the new value (which is
* probably in the eval_econtext) into the procedure's memory
! * context. But if it's a read/write reference to an expanded
! * object, no physical copy needs to happen; at most we need
! * to reparent the object's memory context.
! *
! * If it's an array, we force the value to be stored in R/W
! * expanded form. This wins if the function later does, say,
! * a lot of array subscripting operations on the variable, and
! * otherwise might lose. We might need to use a different
! * heuristic, but it's too soon to tell. Also, are there
! * cases where it'd be useful to force non-array values into
! * expanded form?
*/
if (!var->datatype->typbyval && !*isNull)
! {
! if (var->datatype->typisarray &&
! !VARATT_IS_EXTERNAL_EXPANDED_RW(DatumGetPointer(newvalue)))
! {
! /* array and not already R/W, so apply expand_array */
! newvalue = expand_array(newvalue, CurrentMemoryContext,
! 0, 0, 0);
! }
! else
! {
! /* else transfer value if R/W, else just datumCopy */
! newvalue = datumTransfer(newvalue,
! false,
! var->datatype->typlen);
! }
! }
/*
! * Now free the old value, unless it's the same as the new
! * value (ie, we're doing "foo := foo"). Note that for
! * expanded objects, this test is necessary and cannot
! * reliably be made any earlier; we have to be looking at the
! * object's standard R/W pointer to be sure pointer equality
! * is meaningful.
*/
! if (var->value != newvalue || var->isnull || *isNull)
! free_var(var);
var->value = newvalue;
var->isnull = *isNull;
! var->freeval = (!var->datatype->typbyval && !*isNull);
break;
}
*************** exec_assign_value(PLpgSQL_execstate *est
*** 4277,4283 ****
} while (target->dtype == PLPGSQL_DTYPE_ARRAYELEM);
/* Fetch current value of array datum */
! exec_eval_datum(estate, target,
&parenttypoid, &parenttypmod,
&oldarraydatum, &oldarrayisnull);
--- 4353,4359 ----
} while (target->dtype == PLPGSQL_DTYPE_ARRAYELEM);
/* Fetch current value of array datum */
! exec_eval_datum(estate, target, true,
&parenttypoid, &parenttypmod,
&oldarraydatum, &oldarrayisnull);
*************** exec_assign_value(PLpgSQL_execstate *est
*** 4423,4438 ****
*
* The type oid, typmod, value in Datum format, and null flag are returned.
*
* At present this doesn't handle PLpgSQL_expr or PLpgSQL_arrayelem datums.
*
! * NOTE: caller must not modify the returned value, since it points right
! * at the stored value in the case of pass-by-reference datatypes. In some
! * cases we have to palloc a return value, and in such cases we put it into
! * the estate's short-term memory context.
*/
static void
exec_eval_datum(PLpgSQL_execstate *estate,
PLpgSQL_datum *datum,
Oid *typeid,
int32 *typetypmod,
Datum *value,
--- 4499,4522 ----
*
* The type oid, typmod, value in Datum format, and null flag are returned.
*
+ * If getrwpointer is TRUE, we'll return a R/W pointer to any variable that is
+ * an expanded object; otherwise we return a R/O pointer to such variables.
+ *
* At present this doesn't handle PLpgSQL_expr or PLpgSQL_arrayelem datums.
*
! * NOTE: the returned Datum points right at the stored value in the case of
! * pass-by-reference datatypes. Generally callers should take care not to
! * modify the stored value. Some callers intentionally manipulate variables
! * referenced by R/W expanded pointers, though; it is those callers'
! * responsibility that the results are semantically OK.
! *
! * In some cases we have to palloc a return value, and in such cases we put
! * it into the estate's short-term memory context.
*/
static void
exec_eval_datum(PLpgSQL_execstate *estate,
PLpgSQL_datum *datum,
+ bool getrwpointer,
Oid *typeid,
int32 *typetypmod,
Datum *value,
*************** exec_eval_datum(PLpgSQL_execstate *estat
*** 4448,4454 ****
*typeid = var->datatype->typoid;
*typetypmod = var->datatype->atttypmod;
! *value = var->value;
*isnull = var->isnull;
break;
}
--- 4532,4543 ----
*typeid = var->datatype->typoid;
*typetypmod = var->datatype->atttypmod;
! if (getrwpointer)
! *value = var->value;
! else
! *value = MakeExpandedObjectReadOnly(var->value,
! var->isnull,
! var->datatype->typlen);
*isnull = var->isnull;
break;
}
*************** setup_param_list(PLpgSQL_execstate *esta
*** 5284,5290 ****
PLpgSQL_var *var = (PLpgSQL_var *) datum;
ParamExternData *prm = ¶mLI->params[dno];
! prm->value = var->value;
prm->isnull = var->isnull;
prm->pflags = PARAM_FLAG_CONST;
prm->ptype = var->datatype->typoid;
--- 5373,5381 ----
PLpgSQL_var *var = (PLpgSQL_var *) datum;
ParamExternData *prm = ¶mLI->params[dno];
! prm->value = MakeExpandedObjectReadOnly(var->value,
! var->isnull,
! var->datatype->typlen);
prm->isnull = var->isnull;
prm->pflags = PARAM_FLAG_CONST;
prm->ptype = var->datatype->typoid;
*************** plpgsql_param_fetch(ParamListInfo params
*** 5350,5356 ****
/* OK, evaluate the value and store into the appropriate paramlist slot */
datum = estate->datums[dno];
prm = ¶ms->params[dno];
! exec_eval_datum(estate, datum,
&prm->ptype, &prmtypmod,
&prm->value, &prm->isnull);
}
--- 5441,5447 ----
/* OK, evaluate the value and store into the appropriate paramlist slot */
datum = estate->datums[dno];
prm = ¶ms->params[dno];
! exec_eval_datum(estate, datum, false,
&prm->ptype, &prmtypmod,
&prm->value, &prm->isnull);
}
*************** make_tuple_from_row(PLpgSQL_execstate *e
*** 5542,5548 ****
if (row->varnos[i] < 0) /* should not happen */
elog(ERROR, "dropped rowtype entry for non-dropped column");
! exec_eval_datum(estate, estate->datums[row->varnos[i]],
&fieldtypeid, &fieldtypmod,
&dvalues[i], &nulls[i]);
if (fieldtypeid != tupdesc->attrs[i]->atttypid)
--- 5633,5639 ----
if (row->varnos[i] < 0) /* should not happen */
elog(ERROR, "dropped rowtype entry for non-dropped column");
! exec_eval_datum(estate, estate->datums[row->varnos[i]], false,
&fieldtypeid, &fieldtypmod,
&dvalues[i], &nulls[i]);
if (fieldtypeid != tupdesc->attrs[i]->atttypid)
*************** free_var(PLpgSQL_var *var)
*** 6335,6341 ****
{
if (var->freeval)
{
! pfree(DatumGetPointer(var->value));
var->freeval = false;
}
}
--- 6426,6437 ----
{
if (var->freeval)
{
! if (DatumIsReadWriteExpandedObject(var->value,
! var->isnull,
! var->datatype->typlen))
! DeleteExpandedObject(var->value);
! else
! pfree(DatumGetPointer(var->value));
var->freeval = false;
}
}
*************** format_expr_params(PLpgSQL_execstate *es
*** 6542,6549 ****
curvar = (PLpgSQL_var *) estate->datums[dno];
! exec_eval_datum(estate, (PLpgSQL_datum *) curvar, ¶mtypeid,
! ¶mtypmod, ¶mdatum, ¶misnull);
appendStringInfo(¶mstr, "%s%s = ",
paramno > 0 ? ", " : "",
--- 6638,6646 ----
curvar = (PLpgSQL_var *) estate->datums[dno];
! exec_eval_datum(estate, (PLpgSQL_datum *) curvar, false,
! ¶mtypeid, ¶mtypmod,
! ¶mdatum, ¶misnull);
appendStringInfo(¶mstr, "%s%s = ",
paramno > 0 ? ", " : "",
diff --git a/src/pl/plpgsql/src/plpgsql.h b/src/pl/plpgsql/src/plpgsql.h
index 00f2f77..c95087c 100644
*** a/src/pl/plpgsql/src/plpgsql.h
--- b/src/pl/plpgsql/src/plpgsql.h
*************** typedef struct
*** 180,185 ****
--- 180,186 ----
bool typbyval;
Oid typrelid;
Oid typioparam;
+ bool typisarray; /* is "true" array, or domain over one */
Oid collation; /* from pg_type, but can be overridden */
FmgrInfo typinput; /* lookup info for typinput function */
int32 atttypmod; /* typmod (taken from someplace else) */
pgsql-hackers by date: