Pro/Forums

Pro/Forums (http://forums.procooling.com/vbb/index.php)
-   Testing and Benchmarking (http://forums.procooling.com/vbb/forumdisplay.php?f=61)
-   -   define the relevant topics (http://forums.procooling.com/vbb/showthread.php?t=7777)

bigben2k 09-02-2003 05:30 PM

Quote:

Originally posted by Blackeagle
I would suggest that a fan area is also needed here.

Not only would fan testing aid those who are making use of air cooling thier CPU, but it would aid in choosing the best fan(s) for use on rads, to get them to perform thier best.

Well, the WBTA isn't going to get into fan testing, but if you want to open it up, go ahead.

WE will need a "cooling solution" thread though, wether it's a chiller or some rad/fan combo.

bigben2k 09-02-2003 05:32 PM

Quote:

Originally posted by jaydee116
Then we MUST do anyalitical testing. What if that Honda had a $20,000 engine with NOS and twin turbos and was capable of 0-60 in 8 seconds. Hard to tell from just walking by.
Exactly.

But it still doesn't stop you from comparing blocks. If you smoked the poor fool, let it be known. ;)

gone_fishin 09-02-2003 06:00 PM

Quote:

Originally posted by bigben2k
Exactly.

But it still doesn't stop you from comparing blocks. If you smoked the poor fool, let it be known. ;)

Is this the goal of the clubhouse?

Your mission statement could read...

Does your block smoke, we do.:D

On a further thought, are members in your club who actually make blocks going to compare theirs to interested folk who make submissions? I can hear the cries of unfair now in the distance should their performance not come out favorably. They may even want their donations back if any were made.
Have you thought this through yet?

Impartiality is a high priority in comparative testing of any kind.

RoboTech 09-02-2003 06:05 PM

Quote:

Originally posted by bigben2k
What Robotech is referring to, is wether the testing that we're trying to discuss is "analytical" or "comparative".
Yes, I assumed the WBTA is all about analytical testing - higher resolution, more accuracy, and a LOT more skill, money and time than even good comparative testing requires.

By clarifying objectives and goals I mean to better define (put into words?) what we expect to accomplish. Is the goal just to be able to accurately measure a block's C/W? - how accurately? Heck, I'm no expert so just thinking through the goals I thought might help us get on (or confirm we are all on) the same path... :)

At the end of next year (?) how will we measure our success?

jaydee 09-02-2003 06:07 PM

Quote:

Originally posted by bigben2k
Exactly.

But it still doesn't stop you from comparing blocks. If you smoked the poor fool, let it be known. ;)

But what if you smoked him because the USER doesn't know how to drive? Proves nothing on the cars performance. ;) Which leads us back to the problem we have today with a bunch of ignorant fools doing half assed reviews.


gone_fishin has a point. It would be pretty easy for people to start shouting bias if the WBTA members blocks are showing to be better than others. :shrug:

Groth 09-02-2003 06:52 PM

Quote:

Originally posted by jaydee116
gone_fishin has a point. It would be pretty easy for people to start shouting bias if the WBTA members blocks are showing to be better than others.
Credibility. This is the big reason the standards and procedures need to be open and the development process transparent. If someone doesn't believe my results, they can reproduce them themselves.

jaydee 09-02-2003 07:41 PM

Quote:

Originally posted by Groth
Credibility. This is the big reason the standards and procedures need to be open and the development process transparent. If someone doesn't believe my results, they can reproduce them themselves.
Sure if you have the equipment and knowledge to try and reproduce the results, but I think we were talking about the 99% of the rest of the people out there that don;t know jack about anything. They see WBTA blocks better than other big name companies and they will think the WBTA is BS and just a marketing scam to promote our own blocks.

Groth 09-02-2003 08:05 PM

Well, I don't think anything we do can help the people that don't want to educate themselves. They will always be prey for marketing types and will alway be prone to weird conspiracy beliefs. Screw 'em. The whole idea is accurate information for those who care enough to learn.

gone_fishin 09-02-2003 08:58 PM

Quote:

Originally posted by jaydee116
They see WBTA blocks better than other big name companies and they will think the WBTA is BS and just a marketing scam to promote our own blocks.
Bingo;)

jaydee 09-02-2003 09:03 PM

Quote:

Originally posted by Groth
Well, I don't think anything we do can help the people that don't want to educate themselves. They will always be prey for marketing types and will alway be prone to weird conspiracy beliefs. Screw 'em. The whole idea is accurate information for those who care enough to learn.
That pretty much rules out any reason whatsoever to make a WBTA. I thought this was for showing the untrained people that a review has been done under a respected standard. Those who care enough already know better and do not need this WBTA!

Groth 09-03-2003 08:53 AM

Quote:

Originally posted by jaydee166
That pretty much rules out any reason whatsoever to make a WBTA. I thought this was for showing the untrained people that a review has been done under a respected standard. Those who care enough already know better and do not need this WBTA!
Jeez, man, we don't need to obtain god-like status, where everyone bows before the might and power of a certified test. As it stands now, block reviews range from half-assed to fully-assed. I buy stuff and I want accurate reviews.

jaydee 09-03-2003 09:25 AM

Quote:

Originally posted by Groth
Jeez, man, we don't need to obtain god-like status, where everyone bows before the might and power of a certified test. As it stands now, block reviews range from half-assed to fully-assed. I buy stuff and I want accurate reviews.
Well if we are all going to spend all this time and money each on a test bed then it damn well better be for more than a few people! I don't understand your thinking here?

bigben2k 09-03-2003 10:23 AM

Wow, so much talk, in just 24 hrs!

Gone_Fishin: The WBTA is not a "law enforcement" agency, nor is it a judge of characters. The honesty and integrity of each member is assumed.

However, to earn a WBTA label, a test bench must be presented to all members, and that's going to include the whole testing procedure. This is no small effort. It's a lot of very detailed work.

So if a tester really, really wants to do all of those things, then cheat and make false claims, that's not our problem, at least until it becomes apparent that it's going on.

As with anything else, you should always keep in mind that even if you get one person's opinion, it doesn't necessarily make it a fact. If you want to make sure that a particular block performs as well as it has been claimed, then you can submit it for comparative testing, or submit it for another round of analytical testing by another member, or someone else entirely. That's just common sense, really. Of course, having a test bench with a label, should mean that you won't have to ask a second source to verify the results. No testing should happen without you knowing the testing procedure.

To clarify: the WBTA is going to set standards, and approve testbenches. What a member ends up doing with it, is up to that member. We don't differentiate between testers and blockmakers.

So if you're going to make claims of inappropriateness, I'd suggest that you get your information together, and send it to the WBTA, so that we can review it. (We'll have a process for that).

Thank you.



Robotech: I'm polishing up the mission statement, as well as objectives.


Jaydee116: Yes, driving errors are possible. That's why comparative testing also includes making sure that such variations are accounted for. Did you really think that you'd only get to run a single test, and call it a day? ;)

jaydee 09-03-2003 10:31 AM

Quote:

Originally posted by bigben2k


Did you really think that you'd only get to run a single test, and call it a day? ;)

Happens everyday....;)

gone_fishin 09-03-2003 04:11 PM

Quote:

Originally posted by bigben2k
Wow, so much talk, in just 24 hrs!

Gone_Fishin: The WBTA is not a "law enforcement" agency, nor is it a judge of characters. The honesty and integrity of each member is assumed.

However, to earn a WBTA label, a test bench must be presented to all members, and that's going to include the whole testing procedure. This is no small effort. It's a lot of very detailed work.

So if a tester really, really wants to do all of those things, then cheat and make false claims, that's not our problem, at least until it becomes apparent that it's going on.

As with anything else, you should always keep in mind that even if you get one person's opinion, it doesn't necessarily make it a fact. If you want to make sure that a particular block performs as well as it has been claimed, then you can submit it for comparative testing, or submit it for another round of analytical testing by another member, or someone else entirely. That's just common sense, really. Of course, having a test bench with a label, should mean that you won't have to ask a second source to verify the results. No testing should happen without you knowing the testing procedure.

To clarify: the WBTA is going to set standards, and approve testbenches. What a member ends up doing with it, is up to that member. We don't differentiate between testers and blockmakers.

So if you're going to make claims of inappropriateness, I'd suggest that you get your information together, and send it to the WBTA, so that we can review it. (We'll have a process for that).

Thank you.



Robotech: I'm polishing up the mission statement, as well as objectives.


Jaydee116: Yes, driving errors are possible. That's why comparative testing also includes making sure that such variations are accounted for. Did you really think that you'd only get to run a single test, and call it a day? ;)

If I were to spend my hard earned money and time on a test bench, why would it need a label from some club. The equiptment needed is already certified. All that would be needed for anyone to verify its potential accuracy would be a full disclosure of each componant and the procedures for its use along with results to be scrutinized. Which could easily be posted here on this forum.

Is it your hope that blockmakers will use your group as their R&D department? How about anyone trusting you guys with nondisclosure agreements?

bigben2k 09-04-2003 08:49 AM

Quote:

Originally posted by gone_fishin
If I were to spend my hard earned money and time on a test bench, why would it need a label from some club. The equiptment needed is already certified. All that would be needed for anyone to verify its potential accuracy would be a full disclosure of each componant and the procedures for its use along with results to be scrutinized. Which could easily be posted here on this forum.

Is it your hope that blockmakers will use your group as their R&D department? How about anyone trusting you guys with nondisclosure agreements?

As I've stated before, one of the WBTA's purpose is to set a minimum standard. The benefit to you, if you build a test bench, is that you'll know ahead of time, what kind of accuracy you'll need to achieve. You're free to exceed it.

You may use "certified equipment" (do you mean "calibrated", or spec'ed?), but it doesn't mean that the application is correct. So you submit your test bench details, along with the procedure, to the WBTA, and members will vote on wether or not it can hold the WBTA label. The review of the test bench is by good people, with the purpose of making sure that it meets WBTA standards.

Once you have the WBTA label, what you do with your test bench is up to you. Wether you run testing for a website review, or sell the service to blockmakers, you've got the WBTA backing that your testing is sufficiently accurate, and properly done.

Of course no one is twisting your arm to join the WBTA. If you'd rather have your test bench reviewed in a Forum, you're free to do so. There are more than one means to an end.


As for blockmakers, the WBTA doesn't offer testing services, but individual members may offer it. NDAs are common for new and upcoming products, but that's between the tester and the blockmaker.

bigben2k 09-04-2003 11:51 AM

Just for kicks...

I was just going over Bill Adam's article, titled "Waterblock Bench Testing" (here), and I thought I'd quote this paragraph, from page 1:

Quote:

It is hoped that the test method outlined here might be a 'talking point' for some future collaboration between interested testing parties. After a general agreement on test methods, the real issue of cross-calibration could be addressed to enable the comparison of test results from different sources. And farther yet down the road, standardized test procedures could be drafted and validated with conventional round-robin testing.

gone_fishin 09-05-2003 10:44 AM

Quote:

Originally posted by bigben2k
Just for kicks...

I was just going over Bill Adam's article, titled "Waterblock Bench Testing" (here), and I thought I'd quote this paragraph, from page 1:

Don't ignore the rest of the article.

He has answered all of the relevent questions in that one article. The limitations of his setup. Why cross calibration is required for comparative data. How to cross calibrate with multiple test setups, even a suggested set of standard test conditions for determining a cross calibration offset curve for each unique test die package. That should have been the very first place to start with this whole new forum instead of rehashing and trying to cut corners.

The best thing a so called WBTA could offer is to make and loan out a single standard test die with insulation, probes and heat elements for other testers to cross calibrate with. That is the way I see the model BillA laid out. If this WBTA is to lead the way it must do so by example (do it right).

Also needed is to win the lotto, mortgage the house or wait for that rich uncle to kick off.

pHaestus 09-05-2003 12:39 PM

i do collaborative research for a living, and we don't usually do it by cobbling together a bunch of half assed equipment for every lab involved in the project. Instead, we decide upon the project's needs and grant a single top notch solution. Does a single website or forum need 10 testers all cross calibrating with one another at all? I don't see why. Would the hardware community as a whole benefit from this kind of collaboration? Sure. This exercise is basically preaching to the choir at least in its current form.

What I see in these threads for the most part is the following thought process:

Hmm I wanna test some waterblocks and not get flamed by Jaydee and/or BillA on the internet when I'm done. How do I test them? Hey let's look at Bill's bench testing article; I bet it explains it! Ok this seems reasonable; how much will it cost? Holy mother of god HOW MUCH? Ok I better e-mail Bill and find out if there's a cheaper way. Shit there's NOT? I better make an alliance and a bunch of posts on a forum then. Maybe with enough people involved we can talk this whole cost thing out of existance. Or at least get rid of the error bars.

I want to make a waterblock testing setup just like Bill's except with hookers and blackjack. On second thought never mind the blackjack.....and the testing setup.

jaydee 09-05-2003 01:28 PM

Quote:

Originally posted by pHaestus


What I see in these threads for the most part is the following thought process:

Hmm I wanna test some waterblocks and not get flamed by Jaydee and/or BillA on the internet when I'm done.

Might be easier just to Ban me and Bill eh? :D I will tell you now though, my flaming (I never really seen it as flaming but whatever) days are going to be winding down. As will my posts here and everywhere. Newbs can relax a little now! Got more important things going on now and coming up soon....

Joe 09-05-2003 01:57 PM

Quote:

Originally posted by pHaestus
What I see in these threads for the most part is the following thought process:

Hmm I wanna test some waterblocks and not get flamed by Jaydee and/or BillA on the internet when I'm done. How do I test them? Hey let's look at Bill's bench testing article; I bet it explains it! Ok this seems reasonable; how much will it cost? Holy mother of god HOW MUCH? Ok I better e-mail Bill and find out if there's a cheaper way. Shit there's NOT? I better make an alliance and a bunch of posts on a forum then. Maybe with enough people involved we can talk this whole cost thing out of existance. Or at least get rid of the error bars.

Hmmm seems the nail on the head has been hit :D

Groth 09-05-2003 04:15 PM

<begin rant>

Good god, no wonder so few results and useful articles get posted! Is the point of these forums to mock people and revel in your supposed superior knowledge?

Yes, Bill had a wonderful test bench, but do we need to deify him? Is worshipful replication of his testing temple the 'one true path'? Is a test suite with a margin of error larger that of The Perfect One completely useless, or would you prefer to read more of the currently available reviews based on socket thermistor readings?

I read all the frickin' posts with no useful content, and it pisses me off. If something is wrong or inadequate, then explain why! Constructive criticism, analysis, numbers! If you ain't got 'em, then STFU!

And stop throw your damned preconceived notions on everything. Who the hell claimed a test bench could be talked down to free? Why do you think error margins will be glossed over? There'll always be trade-offs between money and accuracy. If you don't like the direction it's headed, say something specific, like "if you spend an extra hundreds buck on XX, you can reduced the error by YY percent."

Either post your own testing results, or help out here, or go away.

<end rant>

I need a drink....

gone_fishin 09-05-2003 04:58 PM

Quote:

Originally posted by Groth
<begin rant>


Either post your own testing results, or help out here, or go away.

<end rant>

I need a drink....

Everyone is trying to help out. What's wrong with do it right or don't do it at all?

A work group never sets out to establish standards that are poorer than what came before. That's not progress it's called regression and a waste of resources.

satanicoo 09-05-2003 05:28 PM

Are the normal thermometers that we use to measure our own temperature (to check fever and stuff) reliable?

I know they have a low resolution, but if they measure 37.2 degrees, is it 37.2 degrees or they have an error?

Zhentar 09-05-2003 05:30 PM

since one degree fahrenheit is considerably important for our bodies, I would assume they are pretty accurate; but I don't think they'd be suitable here, because of they're designed range; 38C plus or minus 2C is all they need.

bigben2k 09-05-2003 10:03 PM

Quote:

Originally posted by pHaestus
...

What I see in these threads for the most part is the following thought process:

Hmm I wanna test some waterblocks and not get flamed by Jaydee and/or BillA on the internet when I'm done. How do I test them? Hey let's look at Bill's bench testing article; I bet it explains it! Ok this seems reasonable; how much will it cost? Holy mother of god HOW MUCH? Ok I better e-mail Bill and find out if there's a cheaper way. Shit there's NOT? I better make an alliance and a bunch of posts on a forum then. Maybe with enough people involved we can talk this whole cost thing out of existance. Or at least get rid of the error bars.

...

I definitely understand that you may be used to doing things a certain way, but it doesn't exclude the possibility of doing it another way now does it?

As I stated earlier, even if we got 20 members, there is no way in h*ll that we could reasonably expect each member to fork out $1'000, so that one member can have a test bench of Bill's caliber.

As I also stated, the idea of going with multiple test benches is one of many possible directions, and the one that I envisioned for the WBTA. If you believe that you could start gathering $20'000 in contributions/donations, etc, then go right ahead, but we both know the odds of that happening.

I certainly never made any claims that we would reach Bill's level of testing, but all of us will certainly give it our best shot. It's one of those "ideal goals", a target to shoot for, sort of speak.

As for the cost issue, that's very simply an every day thing. Accuracy still remains the top priority, but cost is definitely close behind. If there is any way that I can trade "putting some time and effort", instead of buying a ready made solution, you can bet that I'm going to consider it: that's just common sense.

[edit]As I also stated (in another thread here), there may be a need for some higher accuracy, strictly for building. From a cost perspective, that leaves me with a possible option of borrowing or renting a tool.[/edit]


Satanicoo: what your real question is, is "Is the resolution related to the accuracy?" and the answer is "no". A meter could have a resolution of +/- 0.1 (i.e. display), but an accuracy of +/- 0.5. That's where it gets interesting, because you have to go over the specifications of the device. Then it extends into "repeatability" and then into "calibration".

gone_fishin 09-06-2003 04:52 AM

This is a slippery fish indeed.

pHaestus 09-06-2003 01:10 PM

Accuracy is basically the product of resolution and calibration.

Stated accuracy on a product spec sheet is what should be possible sans any calibration (out of the box).

Some medical thermometers are presumably pretty accurate: keeping track of a woman's temperature as part of a way to monitor fertility would require a decent accuracy.

Groth: I was quite honestly in the mood to make smart ass comments the other day. Is Bill's data the holy grail? Nope. Does his approach make it harder for testers? Yep. He has assuredly stopped me in my tracks for publishing any test data over the last year or so as I improve my data collection and measurement at the house. Is this a bad thing? To me, no. I would feel bad if myths were perpetuated or if waterblocks were bought and sold based on erroneous data from me. Is this a bad thing to the WBTA? Not so sure. I get the vibe that this is a "good enough" type group.

Don't think that cross calibration will get you where you want to go with cheap gear. Think about this logically. If the error bars are large, and we include them as we correlate the blocks, then what have we gained? Statistics is the one true friend that tells you that your fiance is a whore.


All times are GMT -5. The time now is 05:58 PM.

Powered by vBulletin® Version 3.7.4
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
(C) 2005 ProCooling.com
If we in some way offend you, insult you or your people, screw your mom, beat up your dad, or poop on your porch... we're sorry... we were probably really drunk...
Oh and dont steal our content bitches! Don't give us a reason to pee in your open car window this summer...