Re: Remove usage of finalizers ? - Mailing list pgsql-jdbc

Heiko W. Rupp wrote
> the other day we ran into a situation where our app ran into a OOME
> situation on heavy
> load. It turned out that we had around 290k objects on the Finalizer
> queue, that
> were Statements.

It would help to have an idea of the kind of load that would cause 290k
statement objects to sit waiting in the finalizer queue when each one's
finalizer routine is a no-op (assumes you previously closed these statements
properly).  Any idea of the pace of accumulation involved?


> There has been a discussion in the past about finalizers (
> see e.g. around  http://www.postgresql.org/message-id/

> BCD42993-CB7B-453F-95B4-09E84A956AB0@

>  ).

That thread references an alternative driver and does so with respect to a
different use-case.  The programmer also came to the conclusion that using a
finalizer was a good idea though as applied to Statements no comment was
rendered.


> Other JDBC objects like Statements are extremely short lived and the
> creation rate can be
> on a busy application much higher than the finalization rate (which is
> what we were seeing).

I am suspicious about this claim as made though my limited experience is
part of the reason.  I do not doubt the queue can get backlogged but a
significant number of large companies make use of this driver and the
encounter rate of this problem does not seem that large.  The first thing to
check is to try and see if it is truly the rate of object
creation/destruction that is the driver or is it that the time needed to
destroy a single object is simply too long (i.e., you are not closing your
statements properly).


> So I wonder if the driver could be rewritten in a way that either
> - uses no finalizers for the short lived objects
> or
> - exist in 2 flavors: a debug version that does excessive logging in the
>  finalizer if the objects were not yet closed (and stays as is wrt the
> extra work)
>  and a production version where
>  the finalize() methods are removed, so that the objects do not end up
>  in the finalizer queue and can't pile up under high load.

This seems to be a straight-forward change to make and the project is
open-source.  Before it becomes part of a production release someone would
have to try it and report back on their findings anyway - it is too large a
change to just go ahead and make blindly.  Ideally bench-marking would be
involved as well but even just putting it into production and seeing if/when
the system crashes (if ever) would be a worthwhile activity.  Is this
something you'd be interested in doing?

Is there some kind of pathologically bad application out there where it is
possible to test how quickly a poorly implemented application would fail
under various algorithms but using the same hardware?

Having separate drivers is undesirable but it is possible to run-time decide
on an implementation.  Simply no-op'ing the finalizer method, to my
knowledge and as implied here, does not solve the problem if indeed it is
the act of being added to the queue that is to blame.  I do not know enough
of the JVM to know if this is indeed the case.  It is possible that a
constant no-op would cause the JVM (at least some of them) to optimize away
the finalizer call during run-time but that is probably better not relied
upon.

David J.





--
View this message in context: http://postgresql.1045698.n5.nabble.com/Remove-usage-of-finalizers-tp5775186p5775195.html
Sent from the PostgreSQL - jdbc mailing list archive at Nabble.com.