Thread: ExecForceStoreMinimalTuple leaks memory like there's no tomorrow

Using HEAD,

create table t1 as select generate_series(1,40000000) id;
vacuum analyze t1;
explain select * from t1, t1 t1b where t1.id = t1b.id;
-- should indicate a hash join
explain analyze select * from t1, t1 t1b where t1.id = t1b.id;

... watch the process's memory consumption bloat.  (It runs for
awhile before that starts to happen, but eventually it goes to
a couple of GB.)

It looks to me like the problem is that ExecHashJoinGetSavedTuple
calls ExecForceStoreMinimalTuple with shouldFree = true, and
ExecForceStoreMinimalTuple's second code branch simply ignores
the requirement to free the supplied tuple.

            regards, tom lane

On Mon, Apr 15, 2019 at 10:46:56PM -0400, Tom Lane wrote:
> create table t1 as select generate_series(1,40000000) id;
> vacuum analyze t1;
> explain select * from t1, t1 t1b where t1.id = t1b.id;
> -- should indicate a hash join
> explain analyze select * from t1, t1 t1b where t1.id = t1b.id;
>
> ... watch the process's memory consumption bloat.  (It runs for
> awhile before that starts to happen, but eventually it goes to
> a couple of GB.)
>
> It looks to me like the problem is that ExecHashJoinGetSavedTuple
> calls ExecForceStoreMinimalTuple with shouldFree = true, and
> ExecForceStoreMinimalTuple's second code branch simply ignores
> the requirement to free the supplied tuple.

Open item added, as the root comes from 4da597ed.
--
Michael

Hi,

On 2019-04-15 22:46:56 -0400, Tom Lane wrote:
> Using HEAD,
> 
> create table t1 as select generate_series(1,40000000) id;
> vacuum analyze t1;
> explain select * from t1, t1 t1b where t1.id = t1b.id;
> -- should indicate a hash join
> explain analyze select * from t1, t1 t1b where t1.id = t1b.id;
> 
> ... watch the process's memory consumption bloat.  (It runs for
> awhile before that starts to happen, but eventually it goes to
> a couple of GB.)
> 
> It looks to me like the problem is that ExecHashJoinGetSavedTuple
> calls ExecForceStoreMinimalTuple with shouldFree = true, and
> ExecForceStoreMinimalTuple's second code branch simply ignores
> the requirement to free the supplied tuple.

Thanks for finding. The fix is obviously easy - but looking through the
code I think I found another similar issue. I'll fix both in one go
tomorrow.

Greetings,

Andres Freund

Hi,

On 2019-04-18 19:04:09 -0700, Andres Freund wrote:
> On 2019-04-15 22:46:56 -0400, Tom Lane wrote:
> > Using HEAD,
> > 
> > create table t1 as select generate_series(1,40000000) id;
> > vacuum analyze t1;
> > explain select * from t1, t1 t1b where t1.id = t1b.id;
> > -- should indicate a hash join
> > explain analyze select * from t1, t1 t1b where t1.id = t1b.id;
> > 
> > ... watch the process's memory consumption bloat.  (It runs for
> > awhile before that starts to happen, but eventually it goes to
> > a couple of GB.)
> > 
> > It looks to me like the problem is that ExecHashJoinGetSavedTuple
> > calls ExecForceStoreMinimalTuple with shouldFree = true, and
> > ExecForceStoreMinimalTuple's second code branch simply ignores
> > the requirement to free the supplied tuple.
> 
> Thanks for finding. The fix is obviously easy - but looking through the
> code I think I found another similar issue. I'll fix both in one go
> tomorrow.

Pushed the combined fix for that. Thanks!

Greetings,

Andres Freund