Pro/Forums - View Single Post

Cathar · 08-28-2003, 06:42 PM

My thoughts from a financial and long term viability thing.

In my "real job" we make a product that sells fairly well (i.e. makes hundreds of millions of dollars). Most of you are all probably accessing the internet in some way or form through a product's software that I co-designed and co-wrote.

Now this product/software has a measurable performance, not terribly unlike the TPC database benchmarking that goes on.

For a while, there was a benchmarking company for the types of products that we and other companies in the industry made. Every 6 months to 1 year they would put out requests for all major industry players to participate, and then we'd all get together in a big hall somewhere with our boxes (some with many hundreds of thousands of dollars worth of equipment) and have a big "bake-off" to see who came out on top.

The entry fee for this bake-off was fairly large, mostly due to the administration costs of the independent group of people who ran it. We're talking many thousands of dollars. There were maybe 10-15 major players. We'd run the tests and everyone would go home and around 3 months later the results were published, typically around about the same time as most players were about to release a new update/speed refresh.

Over time various players dropped out of the bakeoffs over disagreements with the testing methodology, over anger at the delays in posting results that essentially compared an older product instead of their newer product, and generally getting pissed off with their products maybe not matching up to the higher performance players, but the market-place didn't really care as long as seemed to be "okay", until competitors could now use the testing information in their own marketing against the lower performing players, and suddenly these players found their market share shrinking dramatically when they were happy before (and generally their customers were too).

i.e. the idea of a high-quality test-bed is a noble one, but over time people won't want to use it if the results are to be made public and those results can be used against them. It doesn't matter how fair or unfair that sounds, that's just what will happen, at least from my experience.

I think there's a few lessons to be learned from this poorly written little anecdote, and a few more lessons to be learned again when one considers just how small the water-cooling market is, in comparison to a market that buys many tens of thousands of very expensive items.

08-28-2003, 06:42 PM	#58
Cathar Thermophile Join Date: Sep 2002 Location: Melbourne, Australia Posts: 2,538	My thoughts from a financial and long term viability thing. In my "real job" we make a product that sells fairly well (i.e. makes hundreds of millions of dollars). Most of you are all probably accessing the internet in some way or form through a product's software that I co-designed and co-wrote. Now this product/software has a measurable performance, not terribly unlike the TPC database benchmarking that goes on. For a while, there was a benchmarking company for the types of products that we and other companies in the industry made. Every 6 months to 1 year they would put out requests for all major industry players to participate, and then we'd all get together in a big hall somewhere with our boxes (some with many hundreds of thousands of dollars worth of equipment) and have a big "bake-off" to see who came out on top. The entry fee for this bake-off was fairly large, mostly due to the administration costs of the independent group of people who ran it. We're talking many thousands of dollars. There were maybe 10-15 major players. We'd run the tests and everyone would go home and around 3 months later the results were published, typically around about the same time as most players were about to release a new update/speed refresh. Over time various players dropped out of the bakeoffs over disagreements with the testing methodology, over anger at the delays in posting results that essentially compared an older product instead of their newer product, and generally getting pissed off with their products maybe not matching up to the higher performance players, but the market-place didn't really care as long as seemed to be "okay", until competitors could now use the testing information in their own marketing against the lower performing players, and suddenly these players found their market share shrinking dramatically when they were happy before (and generally their customers were too). i.e. the idea of a high-quality test-bed is a noble one, but over time people won't want to use it if the results are to be made public and those results can be used against them. It doesn't matter how fair or unfair that sounds, that's just what will happen, at least from my experience. I think there's a few lessons to be learned from this poorly written little anecdote, and a few more lessons to be learned again when one considers just how small the water-cooling market is, in comparison to a market that buys many tens of thousands of very expensive items.