Checksum calculation takes ~0.5% of query time, it is not bottleneck in PostgreSQL.
I consider checksums=on to be a mandatory setting. Often, these types of things are not bugs in postgres itself, but bugs in storage, the underlying operating system, or extensions. Checksums can and will protect you, and may even bring you close to the thing causing the corruption. Given that your replica is ok, this very much smells like a similar type of issue.
In a prior case, I was using pl/sh to load data to the database with 'copy', and for what I believe to be o/s issues, was getting corruption. Enabling checksums completely addressed the source of the damage. Turn them on!
