about numerical accuracy of statistical functions - Mailing list pgsql-general

From Yuanyuan Tian
Subject about numerical accuracy of statistical functions
Date
Msg-id CALndwgELDid0-Y+VyvWAG_fznvZG9DSp57UdwteaQoQWMHAVxQ@mail.gmail.com
Whole thread Raw
List pgsql-general
Hi,

I want to use statistical functions, such as variance(), stddev() and
covar_samp(), to compute some basic statistics of my data. I have
strict requirement for the numerical accuracy of the results. So, I
want to know whether the implementation of these functions are
numerically stable. For example, variance can be computed as
sum(x^2)/n - sum(x)^2/n^2. But this computation can lead to serious
accuracy problems with double precision. Can anybody shed lights on
what algorithms are used to implement these functions and whether they
are numerically stable?

Cheers,
Yuanyuan

pgsql-general by date:

Previous
From: Colin Taylor
Date:
Subject: broken xlog - recovery plan check
Next
From: Arvind Singh
Date:
Subject: pgcon 2012