Thread: Writing SRF
Hello, I'm writing a SRF following the example from http://www.postgresql.org/docs/8.4/interactive/xfunc-c.html (section 34.9.10). In the example, in the code executed in first call, we get the number of tuples to be returned, and no more. Then, in each call, a new tuple is allocated and returned, until reach max_calls. My problem is I want to return an array of structs, and that array is returned (as a pointer) by an external function. I can call that funcion in first call, and then store a the returned array in funcctx->user_fctx. In successive calls, I can get one new element of the array pointed by user_fctx, but I'd need the number of elements (the classical problem "how to get the number of elements of a dinamically allocated array"). Would the best way to modify the external function to get, somehow, the number of structs returned? Is there any better way to do what I need? Thanks in advance, and best regards Jorge ---- http://www.gis4free.org/blog
Jorge Arevalo <jorgearevalo@gis4free.org> writes: > I'm writing a SRF following the example from > http://www.postgresql.org/docs/8.4/interactive/xfunc-c.html (section > 34.9.10). In the example, in the code executed in first call, we get > the number of tuples to be returned, and no more. Then, in each call, > a new tuple is allocated and returned, until reach max_calls. You don't have to do it that way, by any means. max_calls is just a field you can use if you feel like it --- it's not going to be looked at by anything outside your SRF. If you don't want to determine the number of result rows at the start, just ignore max_calls, and use whatever method is convenient to decide that you're done returning rows. regards, tom lane
On Wed, Apr 28, 2010 at 7:00 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Jorge Arevalo <jorgearevalo@gis4free.org> writes: >> I'm writing a SRF following the example from >> http://www.postgresql.org/docs/8.4/interactive/xfunc-c.html (section >> 34.9.10). In the example, in the code executed in first call, we get >> the number of tuples to be returned, and no more. Then, in each call, >> a new tuple is allocated and returned, until reach max_calls. > > You don't have to do it that way, by any means. max_calls is just > a field you can use if you feel like it --- it's not going to be > looked at by anything outside your SRF. If you don't want to determine > the number of result rows at the start, just ignore max_calls, and > use whatever method is convenient to decide that you're done > returning rows. > > regards, tom lane > Yes, actually, I'm not using max_calls. I have my own stop condition. Basically, I check if myStructsArray + call_cntr == NULL. I don't know if it's the best way. I'll test it. My doubt is if I'm doing things right getting all the stuff I need (an array) in the first call, pointing user_fctx to this array and accessing myStructsArray[call_cntr] in each successive call, until myStructsArray + call_cntr == NULL (last array element?). In the example, the row returned is generated from scratch in each call. Many thanks, Jorge
Jorge Arevalo <jorgearevalo@gis4free.org> writes:
> My doubt is if I'm doing things right getting all the stuff I need (an
> array) in the first call, pointing user_fctx to this array and
> accessing myStructsArray[call_cntr] in each successive call, until
> myStructsArray + call_cntr == NULL (last array element?).
Sounds reasonable enough. Is it not working for you? Maybe you need
to be careful about which memory context the array is created in.
regards, tom lane
On Wed, Apr 28, 2010 at 10:43 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Jorge Arevalo <jorgearevalo@gis4free.org> writes: >> My doubt is if I'm doing things right getting all the stuff I need (an >> array) in the first call, pointing user_fctx to this array and >> accessing myStructsArray[call_cntr] in each successive call, until >> myStructsArray + call_cntr == NULL (last array element?). > > Sounds reasonable enough. Is it not working for you? Maybe you need > to be careful about which memory context the array is created in. > > regards, tom lane > Yes. For example, the function expects 2 arguments, and it's called with 2 arguments: 1 composite type (following this format https://svn.osgeo.org/postgis/spike/wktraster/doc/RFC1-SerializedFormat) and one integer. But PG_NARGS() returns a really big value (16297) when I first check the number of arguments at the beginning of the function. Has sense? The array, if I'm doing things right, is created in the context pointed by fcinfo->flinfo->fn_mcxt. But I'd like to solve "silly" things like the previous before, and to be sure I'm doing things right, in general. Now, I know at least sounds reasonable :-) Many thanks!
Jorge Arevalo <jorgearevalo@gis4free.org> writes: > Yes. For example, the function expects 2 arguments, and it's called > with 2 arguments: 1 composite type (following this format > https://svn.osgeo.org/postgis/spike/wktraster/doc/RFC1-SerializedFormat) > and one integer. But PG_NARGS() returns a really big value (16297) > when I first check the number of arguments at the beginning of the > function. Has sense? Given only that data point, I would guess that you forgot to mark the function as being called with V1 protocol (PG_FUNCTION_INFO_V1). regards, tom lane
On Thu, Apr 29, 2010 at 3:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Jorge Arevalo <jorgearevalo@gis4free.org> writes: >> Yes. For example, the function expects 2 arguments, and it's called >> with 2 arguments: 1 composite type (following this format >> https://svn.osgeo.org/postgis/spike/wktraster/doc/RFC1-SerializedFormat) >> and one integer. But PG_NARGS() returns a really big value (16297) >> when I first check the number of arguments at the beginning of the >> function. Has sense? > > Given only that data point, I would guess that you forgot to mark the > function as being called with V1 protocol (PG_FUNCTION_INFO_V1). > > regards, tom lane > Many thanks! That was one of my errors. Another one was this: char szDataPointer[10]; sprintf(szDataPointer, "%p", a_pointer); These lines caused a memory error. I changed them for: char * pszDataPointer; pszDataPointer = (char *)allocator(10 * sizeof(char)); sprintf(pszDataPointer, "%p", a_pointer); Meaning "allocator" a memory allocator in a valid memory context for PostgreSQL. And seems to work :-). Is the static memory "dangerous" in a PostgreSQL memory context? Thanks again! Jorge
Jorge Arevalo <jorgearevalo@gis4free.org> writes:
> Many thanks! That was one of my errors. Another one was this:
> char szDataPointer[10];
> sprintf(szDataPointer, "%p", a_pointer);
> These lines caused a memory error.
That looks all right in itself (unless you're on a 64-bit machine, in
which case you need a bigger array to hold %p output). However the
array would only live as long as the function it's in. What were you
doing with the data afterwards, returning it maybe?
regards, tom lane
it has been years since i've mucked in the C++ swamp but
that means your (near) heap is ok but you're stack is hosed..
probably specific to compiler (version) and Operating System(version) and environment settings..ping back if you are still experiencing those problems with those configuration settings
Saludos Cordiales desde EEUU!
Martin Gainty
______________________________________________
Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
> From: jorgearevalo@gis4free.org
> Date: Thu, 29 Apr 2010 19:45:41 +0200
> Subject: Re: [GENERAL] Writing SRF
> To: tgl@sss.pgh.pa.us
> CC: pgsql-general@postgresql.org
>
> On Thu, Apr 29, 2010 at 3:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Jorge Arevalo <jorgearevalo@gis4free.org> writes:
> >> Yes. For example, the function expects 2 arguments, and it's called
> >> with 2 arguments: 1 composite type (following this format
> >> https://svn.osgeo.org/postgis/spike/wktraster/doc/RFC1-SerializedFormat)
> >> and one integer. But PG_NARGS() returns a really big value (16297)
> >> when I first check the number of arguments at the beginning of the
> >> function. Has sense?
> >
> > Given only that data point, I would guess that you forgot to mark the
> > function as being called with V1 protocol (PG_FUNCTION_INFO_V1).
> >
> > regards, tom lane
> >
>
> Many thanks! That was one of my errors. Another one was this:
>
> char szDataPointer[10];
> sprintf(szDataPointer, "%p", a_pointer);
>
> These lines caused a memory error. I changed them for:
>
> char * pszDataPointer;
> pszDataPointer = (char *)allocator(10 * sizeof(char));
> sprintf(pszDataPointer, "%p", a_pointer);
>
> Meaning "allocator" a memory allocator in a valid memory context for PostgreSQL.
>
> And seems to work :-). Is the static memory "dangerous" in a
> PostgreSQL memory context?
>
> Thanks again!
> Jorge
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. Get busy.
that means your (near) heap is ok but you're stack is hosed..
probably specific to compiler (version) and Operating System(version) and environment settings..ping back if you are still experiencing those problems with those configuration settings
Saludos Cordiales desde EEUU!
Martin Gainty
______________________________________________
Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité
Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht dient lediglich dem Austausch von Informationen und entfaltet keine rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen.
Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le destinataire prévu, nous te demandons avec bonté que pour satisfaire informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie de ceci est interdite. Ce message sert à l'information seulement et n'aura pas n'importe quel effet légalement obligatoire. Étant donné que les email peuvent facilement être sujets à la manipulation, nous ne pouvons accepter aucune responsabilité pour le contenu fourni.
> From: jorgearevalo@gis4free.org
> Date: Thu, 29 Apr 2010 19:45:41 +0200
> Subject: Re: [GENERAL] Writing SRF
> To: tgl@sss.pgh.pa.us
> CC: pgsql-general@postgresql.org
>
> On Thu, Apr 29, 2010 at 3:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Jorge Arevalo <jorgearevalo@gis4free.org> writes:
> >> Yes. For example, the function expects 2 arguments, and it's called
> >> with 2 arguments: 1 composite type (following this format
> >> https://svn.osgeo.org/postgis/spike/wktraster/doc/RFC1-SerializedFormat)
> >> and one integer. But PG_NARGS() returns a really big value (16297)
> >> when I first check the number of arguments at the beginning of the
> >> function. Has sense?
> >
> > Given only that data point, I would guess that you forgot to mark the
> > function as being called with V1 protocol (PG_FUNCTION_INFO_V1).
> >
> > regards, tom lane
> >
>
> Many thanks! That was one of my errors. Another one was this:
>
> char szDataPointer[10];
> sprintf(szDataPointer, "%p", a_pointer);
>
> These lines caused a memory error. I changed them for:
>
> char * pszDataPointer;
> pszDataPointer = (char *)allocator(10 * sizeof(char));
> sprintf(pszDataPointer, "%p", a_pointer);
>
> Meaning "allocator" a memory allocator in a valid memory context for PostgreSQL.
>
> And seems to work :-). Is the static memory "dangerous" in a
> PostgreSQL memory context?
>
> Thanks again!
> Jorge
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with Hotmail. Get busy.
On Thu, Apr 29, 2010 at 8:03 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: > Jorge Arevalo <jorgearevalo@gis4free.org> writes: >> Many thanks! That was one of my errors. Another one was this: > >> char szDataPointer[10]; >> sprintf(szDataPointer, "%p", a_pointer); > >> These lines caused a memory error. > > That looks all right in itself (unless you're on a 64-bit machine, in > which case you need a bigger array to hold %p output). However the > array would only live as long as the function it's in. What were you > doing with the data afterwards, returning it maybe? > > regards, tom lane > Thanks for the tip. And about the data pointed by this address, is copied in a safe place (I hope...) before using it to construct the data that will be returned. Just now, it's working, but I'll be careful. Many thanks again! Best regards, Jorge
On Thu, Apr 29, 2010 at 8:08 PM, Martin Gainty <mgainty@hotmail.com> wrote: > it has been years since i've mucked in the C++ swamp but > that means your (near) heap is ok but you're stack is hosed.. > > probably specific to compiler (version) and Operating System(version) and > environment settings..ping back if you are still experiencing those problems > with those configuration settings > Ok, now it's working. In GNU/Linux with gcc 4.4.1, and I hope in Windows XP too (it will be tested). Many thanks! > Saludos Cordiales desde EEUU! > Martin Gainty ¡Saludos desde España también! Jorge > ______________________________________________ > Verzicht und Vertraulichkeitanmerkung/Note de déni et de confidentialité > > Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene > Empfaenger sein, so bitten wir hoeflich um eine Mitteilung. Jede unbefugte > Weiterleitung oder Fertigung einer Kopie ist unzulaessig. Diese Nachricht > dient lediglich dem Austausch von Informationen und entfaltet keine > rechtliche Bindungswirkung. Aufgrund der leichten Manipulierbarkeit von > E-Mails koennen wir keine Haftung fuer den Inhalt uebernehmen. > > Ce message est confidentiel et peut être privilégié. Si vous n'êtes pas le > destinataire prévu, nous te demandons avec bonté que pour satisfaire > informez l'expéditeur. N'importe quelle diffusion non autorisée ou la copie > de ceci est interdite. Ce message sert à l'information seulement et n'aura > pas n'importe quel effet légalement obligatoire. Étant donné que les email > peuvent facilement être sujets à la manipulation, nous ne pouvons accepter > aucune responsabilité pour le contenu fourni. > > > > >> From: jorgearevalo@gis4free.org >> Date: Thu, 29 Apr 2010 19:45:41 +0200 >> Subject: Re: [GENERAL] Writing SRF >> To: tgl@sss.pgh.pa.us >> CC: pgsql-general@postgresql.org >> >> On Thu, Apr 29, 2010 at 3:56 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote: >> > Jorge Arevalo <jorgearevalo@gis4free.org> writes: >> >> Yes. For example, the function expects 2 arguments, and it's called >> >> with 2 arguments: 1 composite type (following this format >> >> >> >> https://svn.osgeo.org/postgis/spike/wktraster/doc/RFC1-SerializedFormat) >> >> and one integer. But PG_NARGS() returns a really big value (16297) >> >> when I first check the number of arguments at the beginning of the >> >> function. Has sense? >> > >> > Given only that data point, I would guess that you forgot to mark the >> > function as being called with V1 protocol (PG_FUNCTION_INFO_V1). >> > >> > regards, tom lane >> > >> >> Many thanks! That was one of my errors. Another one was this: >> >> char szDataPointer[10]; >> sprintf(szDataPointer, "%p", a_pointer); >> >> These lines caused a memory error. I changed them for: >> >> char * pszDataPointer; >> pszDataPointer = (char *)allocator(10 * sizeof(char)); >> sprintf(pszDataPointer, "%p", a_pointer); >> >> Meaning "allocator" a memory allocator in a valid memory context for >> PostgreSQL. >> >> And seems to work :-). Is the static memory "dangerous" in a >> PostgreSQL memory context? >> >> Thanks again! >> Jorge >> >> -- >> Sent via pgsql-general mailing list (pgsql-general@postgresql.org) >> To make changes to your subscription: >> http://www.postgresql.org/mailpref/pgsql-general > > ________________________________ > The New Busy think 9 to 5 is a cute idea. Combine multiple calendars with > Hotmail. Get busy.