Re: Performance monitor signal handler - Mailing list pgsql-hackers

Tom Lane wrote:
> Jan Wieck <JanWieck@Yahoo.com> writes:
> >     Uh - not much time to spend if the statistics should at least
> >     be  half  accurate. And it would become worse in SMP systems.
> >     So that was a nifty idea, but I think it'd  cause  much  more
> >     statistic losses than I assumed at first.
>
> >     Back to drawing board. Maybe a SYS-V message queue can serve?
>
> That would be the same as a pipe: backends would block if the collector
> stopped accepting data.  I do like the "auto discard" aspect of this
> UDP-socket approach.
   Does  a pipe guarantee that a buffer, written with one atomic   write(2), never can get intermixed with  other  data
on  the   readers  end?   I know that you know what I mean, but for the   broader audience: Let's define a message to
thecollector  to   be  4byte-len,len-bytes.   Now  hundreds  of  backends hammer   messages into the (shared) writing
endof the pipe, all  with   different     sizes.     Is     it    GUARANTEED    that    a   read(4bytes),read(nbytes)
sequencewill  allways  return  one   complete  message  and  never  intermixed  parts of different   write(2)s?
 
   With message queues, this is guaranteed. Also, message queues   would  make  it  easy  to query the collected
statistics(see   below).
 

> I think Philip had the right idea: each backend should send totals,
> not deltas, in its messages.  Then, it doesn't matter (much) if the
> collector loses some messages --- that just means that sometimes it
> has a slightly out-of-date idea about how much work some backends have
> done.  It should be easy to design the software so that that just makes
> a small, transient error in the currently displayed statistics.
   If we use two message queues (IPC_PRIVATE  is  enough  here),   one  into collector and one into backend direction,
this'dbe   an easy way to collect and query statistics.
 
   The backends send delta stats messages to  the  collector  on   one  queue. Message queues block, by default, but
thebackend   could use IPC_NOWAIT and just go on and collect up,  as  long   as  it finally will use a blocking call
beforeexiting. We'll   loose  statistics  for  backends  that  go  down  in   flames   (coredump), but who cares for
statisticsthen?
 
   To  query statistics, we have a set of new builtin functions.   All functions share  a  global  statistics  snapshot
in  the   backend.  If  on  function call the snapshot doesn't exist or   was generated by  another
XACT/commandcounter, the  backend   sends  a  statistics  request  for  his  database  ID  to the   collector and waits
forthe messages to arrive on the  second   message  queue. It can pick up the messages meant for him via   message
type,which's equal to his backend number +1, because   the  collector will send 'em as such.  For table access stats
forexample, the snapshot will have slots identified  by  the   tables  OID,  so  a function
pg_get_tables_seqscan_count(oid)  should be easy  to  implement.  And  setting  up  views  that   present access stats
inreadable format is a nobrainer.
 
   Now  we  have communication only between the backends and the   collector.  And we're  certain  that  only  someone
able to   SELECT from a system view will ever see this information.
 


Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck@Yahoo.com #


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com