On 07/14/2012 09:26 AM, B Sreejith wrote:
Dear Robert,
We need to scale up both size and load.
Could you please provide steps I need to follow.
For load, first you need to build a representative sample of your application's querying patterns by logging queries and analysing the logs. Produce a load generator based on that data, set up a test copy of your database, and start pushing the query rate up to see what happens.
For simpler loads you can write a transaction script for pgbench based on your queries.
For size: Copy your data set, then start duplicating it with munged copies. Repeat, then use the load generator you wrote for the first part to see how scaling the data up affects your queries. See if anything is unacceptably slow (the "auto_explain" module is useful here) and examine it.
The truth is that predicting how complex database driven apps will scale is insanely hard, because access patterns change as data sizes and user counts grow. You're likely to land up tuning for a scenario that's quite different to the one that you actually face when you start hitting scaling limitations. This doesn't mean you should not investigate, it just means your trials don't prove anything and the optimisations you make based on what you learn may not gain you much.
--
Craig Ringer
