Re: Add the ability to limit the amount of memory that can be allocated to backends. - Mailing list pgsql-hackers
| From | reid.thompson@crunchydata.com |
|---|---|
| Subject | Re: Add the ability to limit the amount of memory that can be allocated to backends. |
| Date | |
| Msg-id | 457dd07b21810463d8dd282e48b99cd90c24a440.camel@crunchydata.com Whole thread Raw |
| In response to | Re: Add the ability to limit the amount of memory that can be allocated to backends. (Andres Freund <andres@anarazel.de>) |
| Responses |
Re: Add the ability to limit the amount of memory that can be allocated to backends.
|
| List | pgsql-hackers |
Updated patches attached.
====================================================================
pg-stat-activity-backend-memory-allocated
====================================================================
DSM allocations created by a process and not destroyed prior to it's exit are
considered long lived and are tracked in global_dsm_allocated_bytes.
created 2 new system views (see below):
pg_stat_global_memory_allocation view displays datid, shared_memory_size,
shared_memory_size_in_huge_pages, global_dsm_allocated_bytes. shared_memory_size
and shared_memory_size_in_huge_pages display the calculated read only values for
these GUCs.
pg_stat_memory_allocation view
Migrated allocated_bytes out of pg_stat_activity view into this view.
pg_stat_memory_allocation also contains a breakdown of allocation by allocator
type (aset, dsm, generation, slab). View displays datid, pid, allocated_bytes,
aset_allocated_bytes, dsm_allocated_bytes, generation_allocated_bytes,
slab_allocated_bytes by process.
Reduced calls to initialize allocation counters by moving
intialization call into InitPostmasterChild.
postgres=# select * from pg_stat_global_memory_allocation;
datid | shared_memory_size | shared_memory_size_in_huge_pages | global_dsm_allocated_bytes
-------+--------------------+----------------------------------+----------------------------
5 | 192MB | 96 | 1048576
(1 row)
postgres=# select * from pg_stat_memory_allocation;
datid | pid | allocated_bytes | aset_allocated_bytes | dsm_allocated_bytes | generation_allocated_bytes |
slab_allocated_bytes
-------+--------+-----------------+----------------------+---------------------+----------------------------+----------------------
| 981842 | 771512 | 771512 | 0 | 0 |
0
| 981843 | 736696 | 736696 | 0 | 0 |
0
5 | 981913 | 4274792 | 4274792 | 0 | 0 |
0
| 981838 | 107216 | 107216 | 0 | 0 |
0
| 981837 | 123600 | 123600 | 0 | 0 |
0
| 981841 | 107216 | 107216 | 0 | 0 |
0
(6 rows)
postgres=# select ps.datid, ps.pid, state,application_name,backend_type, pa.* from pg_stat_activity ps join
pg_stat_memory_allocationpa on (pa.pid = ps.pid) order by dsm_allocated_bytes, pa.pid;
datid | pid | state | application_name | backend_type | datid | pid | allocated_bytes |
aset_allocated_bytes| dsm_allocated_bytes | generation_allocated_bytes | slab_allocated_bytes
-------+--------+--------+------------------+------------------------------+-------+--------+-----------------+----------------------+---------------------+----------------------------+----------------------
| 981837 | | | checkpointer | | 981837 | 123600 |
123600 | 0 | 0 | 0
| 981838 | | | background writer | | 981838 | 107216 |
107216 | 0 | 0 | 0
| 981841 | | | walwriter | | 981841 | 107216 |
107216 | 0 | 0 | 0
| 981842 | | | autovacuum launcher | | 981842 | 771512 |
771512 | 0 | 0 | 0
| 981843 | | | logical replication launcher | | 981843 | 736696 |
736696 | 0 | 0 | 0
5 | 981913 | active | psql | client backend | 5 | 981913 | 5390864 |
5382824 | 0 | 8040 | 0
(6 rows)
====================================================================
dev-max-memory
====================================================================
Include shared_memory_size in max_total_backend_memory calculations.
max_total_backend_memory is reduced by shared_memory_size at startup.
Local allowance is refilled when consumed from global
max_total_bkend_mem_bytes_available.
pg_stat_global_memory_allocation view
add columns max_total_backend_memory_bytes, max_total_bkend_mem_bytes_available.
max_total_backend_memory_bytes displays a byte representation of
max_total_backend_memory. max_total_bkend_mem_bytes_available tracks the balance
of max_total_backend_memory_bytes available to backend processes.
postgres=# select * from pg_stat_global_memory_allocation;
datid | shared_memory_size | shared_memory_size_in_huge_pages | max_total_backend_memory_bytes |
max_total_bkend_mem_bytes_available| global_dsm_allocated_bytes
-------+--------------------+----------------------------------+--------------------------------+-------------------------------------+----------------------------
5 | 192MB | 96 | 2147483648 |
1874633712 | 5242880
(1 row)
postgres=# select * from pg_stat_memory_allocation ;
datid | pid | allocated_bytes | aset_allocated_bytes | dsm_allocated_bytes | generation_allocated_bytes |
slab_allocated_bytes
-------+--------+-----------------+----------------------+---------------------+----------------------------+----------------------
| 534528 | 812472 | 812472 | 0 | 0 |
0
| 534529 | 736696 | 736696 | 0 | 0 |
0
5 | 556271 | 4458088 | 4458088 | 0 | 0 |
0
5 | 534942 | 1298680 | 1298680 | 0 | 0 |
0
5 | 709283 | 7985464 | 7985464 | 0 | 0 |
0
5 | 718693 | 8809240 | 8612504 | 196736 | 0 |
0
5 | 752113 | 25803192 | 25803192 | 0 | 0 |
0
5 | 659886 | 9042232 | 9042232 | 0 | 0 |
0
| 534525 | 2491088 | 2491088 | 0 | 0 |
0
| 534524 | 4465360 | 4465360 | 0 | 0 |
0
| 534527 | 107216 | 107216 | 0 | 0 |
0
(11 rows)
postgres=# select ps.datid, ps.pid, state,application_name,backend_type, pa.* from pg_stat_activity ps join
pg_stat_memory_allocationpa on (pa.pid = ps.pid) order by dsm_allocated_bytes, pa.pid;
datid | pid | state | application_name | backend_type | datid | pid | allocated_bytes |
aset_allocated_bytes| dsm_allocated_bytes | generation_allocated_bytes | slab_allocated_bytes
-------+--------+--------+------------------+------------------------------+-------+--------+-----------------+----------------------+---------------------+----------------------------+----------------------
| 534524 | | | checkpointer | | 534524 | 4465360 |
4465360 | 0 | 0 | 0
| 534525 | | | background writer | | 534525 | 2491088 |
2491088 | 0 | 0 | 0
| 534527 | | | walwriter | | 534527 | 107216 |
107216 | 0 | 0 | 0
| 534528 | | | autovacuum launcher | | 534528 | 812472 |
812472 | 0 | 0 | 0
| 534529 | | | logical replication launcher | | 534529 | 736696 |
736696 | 0 | 0 | 0
5 | 534942 | idle | psql | client backend | 5 | 534942 | 1298680 |
1298680 | 0 | 0 | 0
5 | 556271 | active | psql | client backend | 5 | 556271 | 4866576 |
4858536 | 0 | 8040 | 0
5 | 659886 | active | | autovacuum worker | 5 | 659886 | 8993080 |
8993080 | 0 | 0 | 0
5 | 709283 | active | | autovacuum worker | 5 | 709283 | 7928120 |
7928120 | 0 | 0 | 0
5 | 752113 | active | | autovacuum worker | 5 | 752113 | 27935608 |
27935608 | 0 | 0 | 0
5 | 718693 | active | psql | client backend | 5 | 718693 | 8669976 |
8473240 | 196736 | 0 | 0
(11 rows)
Attachment
pgsql-hackers by date: