Review : Add hooks for pre- and post-processor executables for COPY and \copy - Mailing list pgsql-hackers

<div class="WordSection1"><p class="MsoNormal"><tt><b><span
style="font-size:10.0pt;font-family:"Arial","sans-serif"">Basicstuff:</span></b></tt><span
style="font-size:10.0pt;font-family:"Arial","sans-serif""><br/><tt><span
style="font-family:"Arial","sans-serif"">------------</span></tt><br/><tt><span
style="font-family:"Arial","sans-serif"">       - Rebase of Patch is required.</span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">       - Compiles cleanly without any errors/warnings</span></tt><br
/><tt><spanstyle="font-family:"Arial","sans-serif"">        - Regression tests pass.</span></tt><br /><br
/><tt><b><spanstyle="font-family:"Arial","sans-serif"">What it does:</span></b></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">---------------------</span></tt><br/><tt><span
style="font-family:"Arial","sans-serif"">       This patch is useful when COPY command input/output are stored in
compressionformat or in any command/script uses these output/input in any means; without generating intermediate
temporaryfiles.</span></tt><br /><tt><span style="font-family:"Arial","sans-serif"">        This feature can be used in
serverside using "COPY statement" by administrator. Or can be used in psql internal "\copy" command by any
user.</span></tt><br/><br /><br /><tt><b><span style="font-family:"Arial","sans-serif"">Code Review
comments:</span></b></tt><br/><tt><span style="font-family:"Arial","sans-serif"">---------------------</span></tt><br
/>    <br/><tt><span style="font-family:"Arial","sans-serif"">1. Modify the comment in function header of:
parse_slash_copy</span></tt>(needs to modify for new syntax)<br /><tt><span style="font-family:"Arial","sans-serif"">2.
Commentsfor functions OpenPipeStream & ClosePipeStream are missing. <b>        </b></span></tt></span><p
class="MsoNormal"><tt><spanstyle="font-size:10.0pt;font-family:"Arial","sans-serif"">3. Any Script errors are not
directlyvisible to user; If there problems in script no way to cleanup.</span></tt><span
style="font-size:10.0pt;font-family:"Arial","sans-serif""><tt><span
style="font-family:"Arial","sans-serif"">  </span></tt></span><pclass="MsoNormal"><tt><span
style="font-size:10.0pt;font-family:"Arial","sans-serif"">  Shouldn’t this be mentioned in User
Manual.</span></tt><spanstyle="font-size:10.0pt;font-family:"Arial","sans-serif""><br /><br /><tt><b><span
style="font-family:"Arial","sans-serif"">Testcase issues:</span></b></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">------------------</span></tt><br/><tt><span
style="font-family:"Arial","sans-serif"">1."Broken pipe" is not handled in case of psql "\copy" command;</span></tt><br
/><tt><spanstyle="font-family:"Arial","sans-serif"">    Issue are as follows:</span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">       Following are verified on SuSE-Linux 10.2.</span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">       1) psql is exiting when "\COPY xxx TO" command is issued and
command/scriptis not found</span></tt> </span><p class="MsoNormal"><tt><span
style="font-size:10.0pt;font-family:"Arial","sans-serif"">                When popen is called in write mode it is
creatingvalid file descriptor and when it tries to write to file "Broken pipe" error is coming which is not handled.
</span></tt><spanstyle="font-size:10.0pt;font-family:"Arial","sans-serif""><br /><tt><span
style="font-family:"Arial","sans-serif"">                       psql# \copy pgbench_accounts TO PROGRAM '../compress.sh
pgbench_accounts4.txt'</span></tt><br/><tt><span style="font-family:"Arial","sans-serif"">        2) When "\copy"
commandis in progress then program/command is killed/"crashed due to any problem" </span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">          psql is exiting.</span></tt><br /><br /><tt><b><span
style="font-family:"Arial","sans-serif"">Scriptused in testcases:</span></b></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">------------------</span></tt><br/><tt><span
style="font-family:"Arial","sans-serif"">1.compress.sh</span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">       echo 'cat > $1' > compress.sh</span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">       echo 'bzip2 -z $1' >> compress.sh</span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">       chmod +x compress.sh</span></tt><br /><br /><tt><span
style="font-family:"Arial","sans-serif"">2.decompress.sh</span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">       echo 'bzip2 -d -c -k $*' > decompress.sh</span></tt><br /><tt><span
style="font-family:"Arial","sans-serif"">       chmod +x decompress.sh</span></tt><br /><br /></span><p
class="MsoNormal"><spanstyle="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif"">Testcasesexecuted are attached with this mail.</span><p
class="MsoNormal"><spanstyle="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif"">WithRegards,</span><p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif"">AmitKapila.</span><p class="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span><pclass="MsoNormal"><span
style="font-size:10.0pt;font-family:"Arial","sans-serif""> </span></div>