Re: pg_dump -Fd and compression level - Mailing list pgsql-hackers

>On Sat, Jul 25, 2015 at 11:09 PM, Marc Mamin <M.Mamin@intershop.de> wrote:
>>
>>>On Sat, Jul 25, 2015 at 9:56 PM, Andrew Dunstan <andrew@dunslane.net> wrote:
>>>>
>>>> On 07/25/2015 03:20 AM, Andrew Dunstan wrote:
>>>>>
>>>>>
>>>>> On 07/25/2015 02:34 AM, Marc Mamin wrote:
>>>>>>
>>>>>>
>>>>>>> Andrew Dunstan <andrew@dunslane.net> writes:
>>>>>>>>
>>>>>>>> Hmm. Yeah. It looks like commit
>>>>>>>> a7ad5cf0cfcfab8418000d652fa4f0c6ad6c8911
>>>>>>>> changed from using the default compression for libz to using the
>>>>>>>> compression set in pg_dump options, which defaults to 0. This actually
>>>>>>>> seems like the right thing to do, but it certainly should have been
>>>>>>>> called out much more forcefully in release notes, and arguably should
>>>>>>>> not have been changed in stable releases. Not sure what we do about it
>>>>>>>> now.
>>>>>>
>>>>>> really 0? wouldn't that mean no compression at all?
>>>>>
>>>>>
>>>>> No, that's not right either. The default should be Z_DEFAULT_COMPRESSION,
>>>>> so we shouldn't actually see a difference in the default case. And it is
>>>>> definitely compressing some. So I'm now puzzled by what you're seeing.
>>>>>
>>>>>
>>>>
>>>>
>>>> OK, I have got this worked out. I'll have a bug fix shortly.
>>>
>>>So you are basically planning to switch to Z_BEST_SPEED instead of
>>>Z_DEFAULT_COMPRESSION where needed for the default code path?
>>
>> It looks like Z_DEFAULT_COMPRESSION was the used value before, and from the source code I guess this should still be
thecase. 
>>
>> From a quick testing, it now behaves as if the minus sign from Z_DEFAULT_COMPRESSION is lost on the way, resulting
in-Z1. this might indicate that Z_DEFAULT_COMPRESSION is parsed like the command line argument. 
>
>I think I understand what is happening... With a quick test using the
>default compression level in directory mode, gzopen is called with
>w-1. By decrypting the docs of zlib (http://www.zlib.net/manual.html),
>section File Access Functions, it seems to me that calling gzopen like
>that will cause the file to not be compressed at all, which is
>actually why you are seeing an increase in your dump files,

But it does compress. Seems to be the same as -Z1

>while we
>should call it with a compression mode of 6 actually based on what
>Z_DEFAULT_COMPRESSION means for zlib now, or simply without a
>compression level specified such as the default is used by zlib. Prior
>to a7ad5cf0, what we did was actually gzopen with always "w" or "wb"
>that caused the default compression level of 6 to be used. Hence I
>think that we should definitely use "w" or "wb" without specifying a
>level number when the compression level is Z_DEFAULT_COMPRESSION, and
>"wbN" (0<=N<=9) when a compression level is given by the user. A patch
>is attached for this purpose. Marc, does it work for you?

I'm on vacation and won't be able to test patches for the 2 next weeks.
sorry

>
>> e.g. pg_dump -Z-2 is like pg_dump -Z2
>
>The compression needs to be included in range [0,9].
Yes, but pg_dump is not such strict.
e.g. pg_dump -Zx is accepted and leads to no compression

These 2 calls result in the same compression, which is how I came to the idea
that the minus sign of Z_DEFAULT_COMPRESSION might get lost on the way:
 pg_dump -Z9 ... pg_dump -Z-9 ...

Marc

>--
>Michael
>