Re: fork/exec patch - Mailing list pgsql-patches

> > > Why not use an anonymous pipe to send data from the parent to the
> > > child process?
> >
> > Doesn't that require the postmaster to stay around to feed that
> > information into the pipe or can the postmaster just shove the data
> > and continue on, and how do the old pipes get cleaned up?
>
> I think that one can just output the data and close that end
> of the pipe.
> But i've not looked at win32 the last 5 years or so, I could be wrong.

For anonymous pipes, large writes will block (pipes created with
CreatePipe()).

For named pipes (which can be given unique names - see
CreateNamedPipe()), you can use Overlapped IO (MS speak for async IO),
and then forget about it. Or rather, register a callback that will do a
CloseHandle() on the pipe, so you don't leak handles.
(Of course, the child process has to do CloseHandle() on the other end
of the pipe).


A way to do this using named pipes would be:
a) Have the postmaster listen on a named pipe always (along with general
connections).
b) Have the clients use CallNamedPipe() to hit the postmasters known
pipe name (the actual pipe name can be passed on the commandline). The
postmaster then sends everything down the pipe.
c) The postmaster only closes the pipe when it shuts down. The same pipe
endpoint is reused all the time.


> Does windows have a temp filesystem where the temp files are
> not actually
> written out on disk? It's still ugly but better then hitting
> a disk all
> the time.

You can specify the FILE_ATTRIBUTE_TEMPORARY parameter to CreateFile().
This does not *guarantee* it will not go to disk, but it allows the
system to store it in RAM. Small files will never hit disk. Large ones
will when the memory manager figures it needs the space for something
else.


> >  Also has to work on Unix too for testing.
>
> Everything can not work in unix, CreateProcess() and fork()
> are different.
> However, the pipe solution can be mimiced in unix, but it
> will not be the
> same code since the api's are different. So that does not give much.

If you want to use any of the ways that "windows were made to use", they
probably won't be compatible with Unix. Might be better off starting
with something simple and once everything else works move to something
more Windows specific (which can then be tested isolated from all the
other changes).




FWIW, the most common way to do this on windows is the "create anonymous
memory mapped region, duplicate with INHERITABLE flag, and use it as
shared memory". You then call DuplicateHandle() specifying read-only
access to make sure the child process cannot write in the parent
process' memory.
If you want to use unique memory regions for each process (everytime you
fork), do:
CreateFileMapping(ALL_ACCESS, ANONYMOUS)
MapViewOfFile()
--> write all data to mapped region
UnmapViewOfFile()
DuplicateHandle(DUPLICATE_CLOSE_SOURCE, READ_ACCESS)
--> exec and specify handle

Child process does MapViewOfFile() to read. When done, UnmapViewOfFile()
and CloseHandle(). On that CloseHandle(), the memory will be freed.


If you want shared memory, just don't specify DUPLICATE_CLOSE_SOURCE,
instead have the postmaster shut it down.


Another option is to used named shared memory (specifying a name and
security attribute, still mapping the pagefile so you don't need an
actual file), in which case the child processes just use
OpenFileMapping() (replaces DuplicateHandle()). DuplicateHandle() is the
better way unelss you need to access it from a non-child process,
though.


//Magnus