On Tue, Oct 26, 2010 at 6:18 PM, Ivan Voras <ivoras@freebsd.org> wrote:
> FWIW, yes - once the IO is fast enough or not necessary (e.g. the
> read-mostly database fits in RAM), RAM bandwidth *is* the next bottleneck
> and it really, really can be observed in actual loads. Buying a QPI-based
> CPU instead of the cheaper DMI-based ones (if talking about Intel chips),
> and faster memory modules (DDR3-1333+) really makes a difference in this
> case.
>
> (QPI and DMI are basically the evolution the front side bus; AMD had HT -
> HyperTransport for years now. Wikipedia of course has more information for
> the interested.)
Note that there are greatly different speeds in HyperTransport from
one AMD chipset to the next. The newest ones, currently Magny Cours
are VERY fast with 1333MHz memory in 64 banks on my 4 cpu x 12 core
machine. And it does scale with each thread I throw at it through
right at 48. Note that those CPUs have 12Megs L3 cache, which makes a
big difference if a lot can fit in cache, but even if it can't the
speed to main memory is very good. There was an earlier thread with
Greg and I in it where we posted the memory bandwidth numbers for that
machine and it was insane how much data all 48 cores could pump into /
out of memory at the same time.
