Thread: Fwd: persistent read cache
Posted to the wrong alias. Just wonder if anyone else feels such a use case. Thanks. ---------- Forwarded message ---------- From: Sand Stone <sand.m.stone@gmail.com> Date: Sun, Feb 11, 2018 at 11:34 AM Subject: persistent read cache To: pgsql-hackers@lists.postgresql.org Hi. I wonder if there is such a thing or extension in the PG world. Here is my use case. I am using PG (PG10 to be more specific) in a cloud VM environment. The tables are stored in RAID0 managed SSD backed attached storage. Depending on the VM I am using, I usually have 256GB local SSD unused. I wonder if PG could leverage this local SSD as a read (page/block) cache, to complement/extend the DRAM by used by shared_buffer today. Thanks.
> On Feb 11, 2018, at 5:14 PM, Sand Stone <sand.m.stone@gmail.com> wrote: > > > Hi. I wonder if there is such a thing or extension in the PG world. > > Here is my use case. I am using PG (PG10 to be more specific) in a > cloud VM environment. The tables are stored in RAID0 managed SSD > backed attached storage. Depending on the VM I am using, I usually > have 256GB local SSD unused. > > I wonder if PG could leverage this local SSD as a read (page/block) > cache, to complement/extend the DRAM by used by shared_buffer today. It seems something that PostgreSQL could take advantage of, but it's probably the wrong layer to implement it. If your VM infrastructure doesn't have any way to use it directly, maybe you could do it at the drive / filesystem level with something like bcache, lvmcache or enhanceio? Adding that sort of complexity to something that needs solid data integrity makes me nervous, but those solutions have been in the field for years. Cheers, Steve
Thanks for the info. on bcache etc., first time heard about it. But it seems quite coarse grained for the needs of a database. The closest thing might be this implemented by the RocksDB engine: https://github.com/facebook/rocksdb/wiki/Persistent-Read-Cache On Mon, Feb 12, 2018 at 9:18 AM, Steve Atkins <steve@blighty.com> wrote: > >> On Feb 11, 2018, at 5:14 PM, Sand Stone <sand.m.stone@gmail.com> wrote: >> >> >> Hi. I wonder if there is such a thing or extension in the PG world. >> >> Here is my use case. I am using PG (PG10 to be more specific) in a >> cloud VM environment. The tables are stored in RAID0 managed SSD >> backed attached storage. Depending on the VM I am using, I usually >> have 256GB local SSD unused. >> >> I wonder if PG could leverage this local SSD as a read (page/block) >> cache, to complement/extend the DRAM by used by shared_buffer today. > > It seems something that PostgreSQL could take advantage of, but > it's probably the wrong layer to implement it. If your VM infrastructure > doesn't have any way to use it directly, maybe you could do it at the > drive / filesystem level with something like bcache, lvmcache or > enhanceio? > > Adding that sort of complexity to something that needs solid data > integrity makes me nervous, but those solutions have been in the > field for years. > > Cheers, > Steve > > >