Hi John,
Thanks for your summary and here are responses:
> #1 - The choice of AVX-512. There is no such thing as a "CRC instruction operating
> on 8 bytes", and the proposed algorithm is a multistep process using carryless
> multiplication and requiring at least 256 bytes of input. The Chromium sources
> cited as the source for this patch also contain an implementation using 128-bit
> instructions, and which only requires at least 64 bytes of input. Is there a reason
> that not tested or proposed as well? That would be much easier to read/maintain,
> work on more systems, and might give a speed boost on smaller inputs. These are
> useful properties to have.
>
> https://github.com/chromium/chromium/blob/main/third_party/zlib/crc32_simd
> .c#L215
Agreed. postgres already has the SSE42 version pg_comp_crc32c_sse42, but I didn’t
realize it uses the crc32 instruction which processes only 8 bytes at a time. This can
certainly be upgraded to process 64bytes at a time and should be faster. Since most
of the AVX-512 stuff is almost ready, I propose to do this in a follow up patch immediately.
Let me know if you disagree. The AVX512 version processes 256 bytes at a time and will
most certainly be faster than the improved SSE42 version, which is why the chromium
library has both AVX512 and SSE42.
>
> #2 - The legal status of the algorithm from following Intel white paper, which is
> missing from its original location, archived here:
>
> https://web.archive.org/web/20220802143127/https://www.intel.com/content/
> dam/www/public/us/en/documents/white-papers/crc-iscsi-polynomial-crc32-
> instruction-paper.pdf
>
> https://github.com/torvalds/linux/blob/master/arch/x86/crypto/crc32c-pcl-intel-
> asm_64.S
>
> ...so I'm unclear if these patents are applicable to software implementations.
> They also seem to be expired, but I am not a lawyer.
> Could you look into this please? Even if we do end up with AVX-512, this would be
> a good fallback.
Given that SSE42 is pretty much available in all x86 processors at this point, do we need a
fallback C version specially after we improve the SSE42 version.
Raghuveer
