Re: Log rotation - Mailing list pgsql-hackers

Lamar Owen wrote:
> On Saturday 13 March 2004 10:36 am, Fernando Nasser wrote:
> 
>>The problem is that sysloging has more overhead than a plain append to a
>>file.  There are some very strict response time AppServer applications
>>where we want to keep this things out of the picture.
> 
> 
> Well, I have a couple of ideas on that.  First, a better syslog.  SDSC secure 
> syslog has been available for some time and is BSD licensed, and is 
> specifically designed for high-performance RFC3195 compliant secure syslog.
> 

I hope someone in the OS group is watching it.  I will ask.


> Second, if the syslog server is separate, then you might get better 
> performance as the size of the logs grow, since appending a very large file 
> is slower than generating an IP datagram.
> 

It may be.  It takes some effort to convince customers to change their 
practices though.  I will try to set up something like this next time we 
run benchmarks and see if there is any noticeable change in the results.


> Third, it seems that you don't have enough profiling data to support a 'syslog 
> is bad' position. 

That is true.  It is from hearsay, from people who run production 
environments.  It may be a belief based on old experiences though.  If 
someone had real up-to-date reliable numbers maybe we could use it as an 
argument to the users.


> Java is bad too, from a performance standpoint (at this 
> time, at least, you always get a performance hit of some kind using any 
> portable code). 

Actually, this is increasingly less noticeable.  With just in time 
compilation or pre-compiled code (like with GNU gcj) it is a one time 
hit or not even that.


> But if this AppServer is based on the ArsDigita Java 
> codebase, you have other performance issues already (the Tcl codebase, OTOH, 
> is very fast, partly because AOLserver is a faster appserver than most Java 
> appservers).).
>

No, it is based on ObjectWeb JOnAS (Java Open Source Application 
Server).  The Servelet Container is Tomcat and the database is 
PostgreSQL (it also works with Oracle, DB2, MySQL, etc.).


> 
>>The other problem is that we have some nice graphical tools for
>>configuring logging but the /etc/syslog.conf is a very hard one to
>>automate dur to the pattern matching.  We can add some lines there, like
>>one to get local0 to a specific file, but it will probably be guesswork
>>and require human inspection anyway.
> 
> 
> Control Center looks promising.  But I much prefer having a central syslog 
> server than having each server in a cluster having separate logs.
> 

There are certain advantages with doing it that way and I guess some 
customers will indeed do it.  I believe you can very well set up logging 
to a centralized syslog with Control Center (if not please open a buf 
report).


> 
>>It may be desirable to logrotate them at different times as well, so
>>they would have to be in different files.
> 
> 
> Different facilities can then go to different files.  The problem, of course, 
> is syslog's limited number of facilities.  
>

If not used for other things, maybe 8 would be a good enough number.

> 
>>>Or you'll have to include some other log rotator.
> 
> 
>>That is what Tom is proposing.
> 
> 
> I am not opposed to including a small logrotator for stderr logging.  I just 
> think it is redundant when a good highly configurable logging facility 
> already exists.  But, if Red Hat wants to pay Tom to do it... :-)

Maybe it is a question of preference or what is more convenient depends 
on the specific circunstances of the installation.  I would prefer to 
give the option to the users, if possible.


Cheers,
Fernando