Thread: [HACKERS] Generate wait event list and docs from text file
Hi all, As $subject has been touched on two threads recently (https://www.postgresql.org/message-id/CAB7nPqTbHLcHFn6m11tfpwAdgz8BmnBza2jjN9AK=SdX_kBJZQ@mail.gmail.com and https://www.postgresql.org/message-id/20170808213537.wkmmagf2a6i3hjyi@alvherre.pgsql), the list of wait event and their category are becoming harder to maintain because of their number and the fact that: 1) It is easy to add a wait event and... 2) It is easy to not update its documentation. 3) the documentation tables get easily broken. As I have participated in the situation present now, I propose to write a patch to make the maintenance of the whole thing easier, because wait events are important, and because I see more of them added in the future. So here is a thread dedicated to this problem, so let's discuss a solution to make things more maintainable. So, like errcodes.txt, let's use two small perl scripts that generate the set of SGML summarizing wait events and an include file with all the wait events listed (enum list and sections). One wait event is associated with the following set of data: - A class, like Activity, IPC, BufferPin, LWLock, etc. - A user-visible event name, like AutoVacuumMain, etc. - A enum-like name, like WAIT_EVENT_AUTOVACUUM_MAIN. - A description, like "Waiting in main loop of autovacuum launcher process." Note as well that a class has a user-visible name, and a symbol name. Using this set of meta-data, it is possible to generate the SGML tables and the set of include tables. While thinking about that, I have been wondering about what could be the shape of the text file holding this meta-data, which needs to be readable. So here is an idea: # Comment, which gets ignored when processing. # First set a section, followed by all the events in it # A section is name with a class name, and an object name. Perhaps the object name could be dynamically generated? Section: ClassName PG_CLASS_NAME # Each event is listed: user-visible event name, object name, description within quotes EventName WAIT_EVENT_NAME "Description of this event name" EventName2 WAIT_EVENT_NAME_2 "Description of this event name 2" There is a link between the class name and its variable name, as well as between the event name and its variable name, so both could be generated dynamically: - the class name is upper-cased, and prefixed with PG_. - the event name is upper-cased, and prefixed with WAIT_EVENT. Maintaining the description of the object within quotes makes parsing with a regex easier. So, thoughts? -- Michael
On Mon, Aug 14, 2017 at 11:58 PM, Michael Paquier <michael.paquier@gmail.com> wrote: > 1) It is easy to add a wait event and... > 2) It is easy to not update its documentation. > 3) the documentation tables get easily broken. > > As I have participated in the situation present now, I propose to > write a patch to make the maintenance of the whole thing easier, > because wait events are important, and because I see more of them > added in the future. Seems an apt occasion to point out that 10rc1 is missing documentation for a couple events. From src/backend/storage/lmgr/lwlocknames.txt: CLogTruncationLock was added to the docs but BackendRandomLock and LogicalRepWorkerLock are missing. (Maybe there's a reason for this... I just checked the diffs from 9.6 to 10 then checked the docs for completeness.) > Using this set of meta-data, it is possible to generate the SGMO > tables and the set of include tables. The lock-related events might present a challenge here since they come from a different place. In particular, there is no single location to find the descriptions of tranche locks - you have to search through the source code and find the LWLockRegisterTranche() call to see what text it used for the lock name! (Consolidating that seems like a great candidate for a patch...) -Jeremy P.S. The number of wait events has gone from 69 in 9.6 up to 184 in 10rc1. IMHO this is very much worth mentioning as a new feature for postgresql 10!I counted 67 new I/O related events, 31 new latch-related events and 8 new replication-related events! Seems like a big deal to me. :) -- http://about.me/jeremy_schneider -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
On Fri, Sep 22, 2017 at 1:52 PM, Schneider <schneider@ardentperf.com> wrote: > Seems an apt occasion to point out that 10rc1 is missing documentation > for a couple events. Meh. Thanks for pointing this out. One more. > From src/backend/storage/lmgr/lwlocknames.txt: CLogTruncationLock was > added to the > docs but BackendRandomLock and LogicalRepWorkerLock are missing. (Maybe there's > a reason for this... I just checked the diffs from 9.6 to 10 then > checked the docs for > completeness.) Indeed, I can see as well that those are the three new ones between 9.6 and 10. Attached is a patch to correct the documentation, I am adding an open item so as this does not fall into oblivion. >> Using this set of meta-data, it is possible to generate the SGMO >> tables and the set of include tables. > > The lock-related events might present a challenge here since they come > from a different > place. In particular, there is no single location to find the > descriptions of tranche locks - you > have to search through the source code and find the > LWLockRegisterTranche() call to see > what text it used for the lock name! (Consolidating that seems like a > great candidate for a > patch...) Perhaps... I did not dig much into the details of this area. I don't recall that it was particularly challenging either to get a base generation in place. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Attachment
Michael Paquier wrote: > On Fri, Sep 22, 2017 at 1:52 PM, Schneider <schneider@ardentperf.com> wrote: > > Seems an apt occasion to point out that 10rc1 is missing documentation > > for a couple events. > Indeed, I can see as well that those are the three new ones between > 9.6 and 10. Attached is a patch to correct the documentation, I am > adding an open item so as this does not fall into oblivion. Pushed, thanks. -- Álvaro Herrera https://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers