> On 7/8/25 03:55, Cédric Villemain wrote:
>> Hi Andres,
>>
>>> Hi,
>>>
>>> On 2025-07-05 07:09:00 +0000, Cédric Villemain wrote:
>>>> In my work on more careful PostgreSQL resource management, I've come
>>>> to the
>>>> conclusion that we should avoid pushing policy too deeply into the
>>>> PostgreSQL core itself. Therefore, I'm quite skeptical about integrating
>>>> NUMA-specific management directly into core PostgreSQL in such a way.
>>>
>>> I think it's actually the opposite - whenever we pushed stuff like this
>>> outside of core it has hurt postgres substantially. Not having
>>> replication in
>>> core was a huge mistake. Not having HA management in core is probably the
>>> biggest current adoption hurdle for postgres.
>>>
>>> To deal better with NUMA we need to improve memory placement and various
>>> algorithms, in an interrelated way - that's pretty much impossible to do
>>> outside of core.
>>
>> Except the backend pinning which is easy to achieve, thus my comment on
>> the related patch.
>> I'm not claiming NUMA memory and all should be managed outside of core
>> (though I didn't read other patches yet).
>>
>
> But an "optimal backend placement" seems to very much depend on where we
> placed the various pieces of shared memory. Which the external module
> will have trouble following, I suspect.
>
> I still don't have any idea what exactly would the external module do,
> how would it decide where to place the backend. Can you describe some
> use case with an example?
>
> Assuming we want to actually pin tasks from within Postgres, what I
> think might work is allowing modules to "advise" on where to place the
> task. But the decision would still be done by core.
Possibly exactly what you're doing in proc.c when managing allocation of
process, but not hardcoded in postgresql (patches 02, 05 and 06 are good
candidates), I didn't get that they require information not available to
any process executing code from a module.
Parts of your code where you assign/define policy could be in one or
more relevant routines of a "numa profile manager", like in an
initProcessRoutine(), and registered in pmroutine struct:
pmroutine = GetPmRoutineForInitProcess();
if (pmroutine != NULL &&
pmroutine->init_process != NULL)
pmroutine->init_process(MyProc);
This way it's easier to manage alternative policies, and also to be able
to adjust when hardware and linux kernel changes.
--
Cédric Villemain +33 6 20 30 22 52
https://www.Data-Bene.io
PostgreSQL Support, Expertise, Training, R&D
