FiringSquad and [H]ard|OCP Sqaure Off

Over the past few days editors from FiringSquad.Com and [H]ard|OCP have squared off and somewhat rekindled the debate regarding the validity of traditional benchmarks / timedemos versus supposed "real-world" performance evaluation.  Comments in [H]ard|OCP's Core 2 Duo game-play performance article spurred FirinqSquad to write this editorial.  Of course, [H]ard|OCP then responded with an editorial of their own, rebutting some of FiringSquad's claims. Interesting reading at both sites, and both sites make some good points.  We recommend you read them both and form an opinion of your own on the subject.

Here at HotHardware, we use a combination of synthetic and real-world applications to gauge performance, using some publicly available benchmarks in addition to some custom benchmarks that we do not distribute.  In our opinion, using a variety of applications and testing at a variety of settings is a perfectly valid and valuable way to assess performance.

It's about having a balanced approach to testing and presenting the resulting data in a way that you hope will be useful to your readers. The problem is, what one person considers a balanced and useful approach, may not be interpreted in the same way by another.  This alone virtually guarantees that debates like this one will rage every so often.  The bottom line is that both sites make a good arguments because both sites are right in some of their opinions.

As a consumer trying to make an informed buying decision, do yourself a favor and read multiple articles evaluating the products you're interested in.  And when you feel you've gleaned enough information from the opinions of others, formulate your own.