Our Application is using Postgres Version (Image- postgres:9.6.13) and we are running it as Docker Container with Volume mounted to a NFS storage. We came across a peculiar issue.
We had enabled unique key constraint on a particular table, it was violated and had duplicate data. When this table was queried in the application, it was resulting in chunk errors. This had major outage in the application.
ERROR: missing chunk number 0 for toast value 241012 in pg_toast_2619 STATEMENT: SELECT distinct role_name FROM tapplication_roles INNER JOIN troles ON tapplication_roles.role_id = troles.role_id where user_id in ('user1','value1')
We followed several posts and somehow managed to remove the duplicate rows and reindexed and vacuumed the table. We also had to manually remove duplicate entries in pg_statistic table and reindex and vaccum it.
There are couple of other tables in which few rows are corrupted. It is because of a corrupt value in one of the columns.
We did not see any error in the container logs to isolate this issue when it first occurred.
Can you please throw some light what might have caused the corruption and how can we prevent it?
