Thread: Any examples of companies using PG in high volume and/or large scale environs...

I've been looking for examples of high volume (in terms of throughput)
and large scale (500GB-terabtype+) examples of Postgres
impllmenetations.

I keep seeing the same few examples (BASF, Los Alamos Labs, Futijsu
and .info administrator). Impossible to tell anything from the names.
.info administrator might be dealing with 100 transactions per
hour...or per second. Not seen anything on that. And a few hundred
.info records may add up to less than 1GB.

I've seen nothing showing real world numbers in terms of hourly/daily
transactions activity and nothing showing how large the databases
really are.

From experience I've come to seek clarification of the meaning  of
statements such as "we have a hugh database". One persons hugh is 1GB
while another's is 100GB while another's is 1TB+. And as one who has
dealt with TB+ in Oracle the issues are way different.

Can any of you suggest some companies I can point to as I try to
convince my company to move to it. We will be storing 100's GB of
data.

tia
David

you'll probably find some answers in the comments of a similar Slashdot post:

http://it.slashdot.org/article.pl?sid=05/03/21/1635210

Check out http://advocacy.postgresql.org
Two other examples are Whitepages.com (375GB) and the .org registry as well.


On Wed, March 23, 2005 17:07, David B said:
> I've been looking for examples of high volume (in terms of throughput)
> and large scale (500GB-terabtype+) examples of Postgres
> impllmenetations.
>
> I keep seeing the same few examples (BASF, Los Alamos Labs, Futijsu
> and .info administrator). Impossible to tell anything from the names.
> .info administrator might be dealing with 100 transactions per
> hour...or per second. Not seen anything on that. And a few hundred
> .info records may add up to less than 1GB.
>
> I've seen nothing showing real world numbers in terms of hourly/daily
> transactions activity and nothing showing how large the databases
> really are.
>
>From experience I've come to seek clarification of the meaning  of
> statements such as "we have a hugh database". One persons hugh is 1GB
> while another's is 100GB while another's is 1TB+. And as one who has
> dealt with TB+ in Oracle the issues are way different.
>
> Can any of you suggest some companies I can point to as I try to
> convince my company to move to it. We will be storing 100's GB of
> data.
>
> tia
> David
>
> ---------------------------(end of broadcast)---------------------------
> TIP 4: Don't 'kill -9' the postmaster
>

On Wed, Mar 23, 2005 at 02:07:03PM -0800, David B wrote:
> .info administrator might be dealing with 100 transactions per
> hour...or per second. Not seen anything on that. And a few hundred

The short answer to this is "both" (I am one of those .info people.
Also .org, .in, and a bunch of others).

Our databases are several gig on disk (for comparison, < 10G after
pg_dump is done).  They are by no means very large databases these
days -- the entire top-level domain system of the Internet isn't even
a very large database these days, given that .com is under 40 million
name records.  Also, our transaction volume is by no means huge as
compared to, say, eBay or Amazon.

What we do have is extremely "spikey" load, with some hours of
practically nothing followed by occasional periods of hundreds of
transactions per second.  We have definitely had 10 million
transaction days, but I'm pretty sure we've never had a 100 million
transaction day in production (we've done tests to those levels,
though).  We also have not-five-nines SLAs.  I will tell you that the
users _expectation_ does not conform to the SLAs, and is instead
rather closer to five nines than I am happy with.  I think this will
come as no surprise to anyone who's worked on the operations side of
databases.

You can find out more about our PostgreSQL use by looking for Afilias
on the ICANN web site (icann.org) -- look particularly for bid
materials for TLD delegations.  We successfully backed the ISOC
proposal to take over .org from VeriSign in 2002.  If you need more
detail, it'd be good to contact me at work (I'm andrew at ca dot
afilias dot info there).

> statements such as "we have a hugh database". One persons hugh is 1GB
> while another's is 100GB while another's is 1TB+. And as one who has
> dealt with TB+ in Oracle the issues are way different.

Yes.  We also have some rather unpleasantly large off-line databases,
but nothing yet in the TB-in-one-db range.  We are scraping that
range, however.  In case it helps, we do have one system with
absolutely critical financial data in it, which we anticipate will be
a TB in not very long.  Postgres has performed well for us so far,
thanks partly to the team of great DBAs I've been able to assemble.

A

--
Andrew Sullivan  | ajs@crankycanuck.ca
This work was visionary and imaginative, and goes to show that visionary
and imaginative work need not end up well.
        --Dennis Ritchie