Thread: wiki.postgresql.org is awfully slow this evening
Is anyone else seeing verrrry slllooowww response from the PG wiki?
regards, tom lane
Tom Lane wrote: > Is anyone else seeing verrrry slllooowww response from the PG wiki? Yep. I've given up on assigning reviewers for the evening. Keeps timing out. --Josh
Josh Berkus wrote: > Tom Lane wrote: >> Is anyone else seeing verrrry slllooowww response from the PG wiki? > > Yep. I've given up on assigning reviewers for the evening. Keeps > timing out. I have received Nagios timeouts today on the French boxen. I believe the wiki is hosted over there. I am sure they will report on the status shortly; they are pretty good about that. Sincerely, Joshua D. Drake > > --Josh >
Joshua D. Drake wrote: > Josh Berkus wrote: >> Tom Lane wrote: >>> Is anyone else seeing verrrry slllooowww response from the PG wiki? >> >> Yep. I've given up on assigning reviewers for the evening. Keeps >> timing out. > > I have received Nagios timeouts today on the French boxen. I believe the > wiki is hosted over there. I am sure they will report on the status > shortly; they are pretty good about that. while I cannot confirm an actual network related issue (latency tracking from remus which is in austria shows no problem for the last 24h) we had two runaway cvsps processes in the git jail(which is on the same physical host as the wiki) that more or less hogged all the CPU on the box. Stefan
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 > from remus which is in austria shows no problem for the last 24h) we had > two runaway cvsps processes in the git jail(which is on the same > physical host as the wiki) that more or less hogged all the CPU on the box. Oh for Pete's sake, can we please get away from jails and just use dedicated servers? We've had enough people volunteer hardware and time to make this happen. - -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200809041007 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iEYEAREDAAYFAki/66gACgkQvJuQZxSWSsgFbACg38bPRwFMKFoixPg80QVtiVpO sgsAnRu2S3lKJ3udfQTKl0gFujVKA/sD =UT0M -----END PGP SIGNATURE-----
On Thu, Sep 4, 2008 at 3:08 PM, Greg Sabino Mullane <greg@turnstep.com> wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: RIPEMD160 > > >> from remus which is in austria shows no problem for the last 24h) we had >> two runaway cvsps processes in the git jail(which is on the same >> physical host as the wiki) that more or less hogged all the CPU on the box. > > Oh for Pete's sake, can we please get away from jails and just use > dedicated servers? We've had enough people volunteer hardware and > time to make this happen. No we haven't - not even remotely. At a rough count we'd need at least another 17 servers (based on the current number of VMs), and we'd lose the ability to move services between hardware quickly and easily. Oh, and we'd have the management headache of dealing with a bunch more hosting providers, as I doubt the current ones will give us that many boxes. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com
Dave Page wrote: > On Thu, Sep 4, 2008 at 3:08 PM, Greg Sabino Mullane <greg@turnstep.com> wrote: >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: RIPEMD160 >> >> >>> from remus which is in austria shows no problem for the last 24h) we had >>> two runaway cvsps processes in the git jail(which is on the same >>> physical host as the wiki) that more or less hogged all the CPU on the box. >> Oh for Pete's sake, can we please get away from jails and just use >> dedicated servers? We've had enough people volunteer hardware and >> time to make this happen. > > No we haven't - not even remotely. At a rough count we'd need at least > another 17 servers (based on the current number of VMs), and we'd lose > the ability to move services between hardware quickly and easily. Oh, > and we'd have the management headache of dealing with a bunch more > hosting providers, as I doubt the current ones will give us that many > boxes. *cough* Yes I think they would. That being said :) I think its a mistake. It would be a complete waste of resources to go to dedicated machines. Some of the machines we have we hardly use at this point. Sincerely, Joshua D. Drake
On Thu, Sep 4, 2008 at 3:32 PM, Joshua D. Drake <jd@commandprompt.com> wrote: >> No we haven't - not even remotely. At a rough count we'd need at least >> another 17 servers (based on the current number of VMs), and we'd lose >> the ability to move services between hardware quickly and easily. Oh, >> and we'd have the management headache of dealing with a bunch more >> hosting providers, as I doubt the current ones will give us that many >> boxes. > > *cough* > > Yes I think they would. Ya think? I'm struggling to see which of our 6 providers would pony up more than a couple more machines. > That being said :) I think its a mistake. It would be a complete waste of > resources to go to dedicated machines. Some of the machines we have we > hardly use at this point. I think most are certainly used, but some require much fewer resources than others. -- Dave Page EnterpriseDB UK: http://www.enterprisedb.com
-----BEGIN PGP SIGNED MESSAGE----- Hash: RIPEMD160 Dave Page wrote: > No we haven't - not even remotely. At a rough count we'd need at least > another 17 servers (based on the current number of VMs), and we'd lose > the ability to move services between hardware quickly and easily. Oh, > and we'd have the management headache of dealing with a bunch more > hosting providers, as I doubt the current ones will give us that many > boxes. Well, maybe we don't need to replace all 17, just some of the more active ones. Joshua points out: > That being said :) I think its a mistake. It would be a complete waste > of resources to go to dedicated machines. Some of the machines we have > we hardly use at this point. Fair enough, I withdraw the dedicated box request. Can we perhaps separate the wiki then, so we don't have a repeat of yesterday? Maybe put wiki or git onto one of the more lightly loaded physical boxes? - -- Greg Sabino Mullane greg@turnstep.com PGP Key: 0x14964AC8 200809041059 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -----BEGIN PGP SIGNATURE----- iEYEAREDAAYFAki/+DoACgkQvJuQZxSWSsj22QCfZlZQ4XoxYxo+UOxXAjNeKdio UecAoOPRoRN8EPKhgTGRINzjnAxXOJNd =MNYh -----END PGP SIGNATURE-----
"Greg Sabino Mullane" <greg@turnstep.com> writes:
> Fair enough, I withdraw the dedicated box request. Can we perhaps separate
> the wiki then, so we don't have a repeat of yesterday? Maybe put wiki
> or git onto one of the more lightly loaded physical boxes?
I think the real bottom line here is that our (meaning the developers')
use of and dependence on the wiki has increased enormously over the past
few months, meaning that it's time to put it on beefier hardware. The
VM/jail setup was intended to make exactly that sort of thing easy, no?
(I'm not sure if the above statement is actually true, or if this was
just a one-time occurrence. But I'd certainly vote for moving the wiki
if it's getting at all resource-starved.)
regards, tom lane
Tom Lane wrote: > "Greg Sabino Mullane" <greg@turnstep.com> writes: >> Fair enough, I withdraw the dedicated box request. Can we perhaps separate >> the wiki then, so we don't have a repeat of yesterday? Maybe put wiki >> or git onto one of the more lightly loaded physical boxes? > > I think the real bottom line here is that our (meaning the developers') > use of and dependence on the wiki has increased enormously over the past > few months, meaning that it's time to put it on beefier hardware. The > VM/jail setup was intended to make exactly that sort of thing easy, no? > > (I'm not sure if the above statement is actually true, or if this was > just a one-time occurrence. But I'd certainly vote for moving the wiki > if it's getting at all resource-starved.) this was clearly a one-time incident(and I'm fairly sure there was a network issue too) - we only have two jails on alderaan at all and it is more than fast enough to handle the wiki and git. http://www.kaltenbrunner.cc/files/alderaan.postgresql.org-cpu-day.png shows pretty nicely that after I killed the two rough processes load diminished - what is more interesting here actually why our current monitoring failed to detect that situation which is what I'm current investigating ... Stefan
Stefan, all, > while I cannot confirm an actual network related issue (latency tracking > from remus which is in austria shows no problem for the last 24h) we had > two runaway cvsps processes in the git jail(which is on the same > physical host as the wiki) that more or less hogged all the CPU on the > box. Hmmmph. Doesn't FreeBSD have some way to cap CPU/RAM usage per jail yet? I thought we got that with 7. No? -- --Josh Josh Berkus PostgreSQL San Francisco