Go Back   Pro/Forums > ProCooling Technical Discussions > Testing and Benchmarking
Password
Register FAQ Members List Calendar Chat

Testing and Benchmarking Discuss, design, and debate ways to evaluate the performace of he goods out there.

Reply
Thread Tools
Unread 09-10-2003, 02:21 PM   #51
pHaestus
Big Player
Making Big Money
 
pHaestus's Avatar
 
Join Date: Aug 2001
Location: irc.lostgeek.com #procooling.com
Posts: 4,782
Default

I don't think I meet the stated requirements of G_F: My 2190C setup is 0.01C res but no better than 0.3C absolute accuracy. That's what I read die temp and wb baseplate temp from btw.

I don't think the raw accuracy numbers are as important as moving your error along in your calcs and being careful to reproduce each test as close as possible. If I find a rare block that I can get a reproducible mount upon and the std deviation for the first 3 or 4 mounts is extremely small then I probably will stop there. If I have trouble getting a good mount I might do more. Be honest! I can't distinguish 0.1C differences in a block's performance. I can't control flowrate batter than 0.05 GPM. I am not confident that these sets of numbers are statistically different. It's ok! Everyone who has tried to run wb tests will understand

I have seen testing very carefully done by people with a minimum of expense (look at Hoot's stuff at overclockers and Cathar's block testing for examples. Probably wrong numbers in absolute terms but very useful and as controlled as they could make them with their stuff). There was a guy at a British website who did some nice testing on flowrate restriction using nothing more than a bucket and a stopwatch The key is to be honest about the confidence in the results and to repeat tests and become experienced. For all Ben's talk of analytical testing, it is extremely unlikely you'll all come up with a "true" number for a waterblock's performance even if everyone uses exactly the same testing gear. Practice is really the best teacher on this stuff.
pHaestus is offline   Reply With Quote
Unread 09-10-2003, 02:52 PM   #52
gone_fishin
Cooling Savant
 
gone_fishin's Avatar
 
Join Date: May 2002
Location: Da UP
Posts: 517
Default

I've seen no absolute proof that Cathar ever quantified a difference in performance (other than a rough .5C, well within his margin of error) with his new block over his ww block. If there is a performance increase the source of the increase has not been pinpointed (witness Les's argument on bp flatness from machining differences recently). Cathar's two blocks are a perfect example of why resolution and accuracy are needed for any answers. I seriously doubt he could quantify a difference with his homestyle test equiptment. I believe that a difference has been noted through higher overclocks, but chips sometimes do overclock differently over time (not diffinitive). His block has so many variables to it that I do not see how he could have optimized them all together without SEEING a changed variables effect. That being said there is nothing wrong with applying ones theories through invention but even trial and error has its limits when you cannot identify an error.
gone_fishin is offline   Reply With Quote
Unread 09-10-2003, 02:57 PM   #53
jaydee
Put up or Shut Up
 
jaydee's Avatar
 
Join Date: Dec 2001
Location: Spokane WA
Posts: 6,506
Default

Quote:
Originally posted by gone_fishin
I've seen no absolute proof that Cathar ever quantified a difference in performance (other than a rough .5C, well within his margin of error) with his new block over his ww block. If there is a performance increase the source of the increase has not been pinpointed (witness Les's argument on bp flatness from machining differences recently). Cathar's two blocks are a perfect example of why resolution and accuracy are needed for any answers. I seriously doubt he could quantify a difference with his homestyle test equiptment. I believe that a difference has been noted through higher overclocks, but chips sometimes do overclock differently over time (not diffinitive). His block has so many variables to it that I do not see how he could have optimized them all together without SEEING a changed variables effect. That being said there is nothing wrong with applying ones theories through invention but even trial and error has its limits when you cannot identify an error.
He said the Cascade is 2.5C better than the White Water last I heard.

So your basically saying give up on testing? As from everything I have read from you there is no right way to do things no matter what equipment is used....
jaydee is offline   Reply With Quote
Unread 09-10-2003, 03:07 PM   #54
pHaestus
Big Player
Making Big Money
 
pHaestus's Avatar
 
Join Date: Aug 2001
Location: irc.lostgeek.com #procooling.com
Posts: 4,782
Default

I was referring to the original design process of the whitewater block. The absolute numbers were irrelevant but the changes that he made through that process were validated through testing and could be seen in the final product.
pHaestus is offline   Reply With Quote
Unread 09-10-2003, 03:15 PM   #55
gone_fishin
Cooling Savant
 
gone_fishin's Avatar
 
Join Date: May 2002
Location: Da UP
Posts: 517
Default

Quote:
Originally posted by jaydee116
He said the Cascade is 2.5C better than the White Water last I heard.

So your basically saying give up on testing? As from everything I have read from you there is no right way to do things no matter what equipment is used....
No, I'm saying doing testing right could give some answers that are helpful in designing a block. I stand corrected if it is 2.5C (is this verifiable?). Do it right not halfassed is all I was ever saying in these discussions.
gone_fishin is offline   Reply With Quote
Unread 09-10-2003, 03:22 PM   #56
gone_fishin
Cooling Savant
 
gone_fishin's Avatar
 
Join Date: May 2002
Location: Da UP
Posts: 517
Default

Quote:
Originally posted by pHaestus
I was referring to the original design process of the whitewater block. The absolute numbers were irrelevant but the changes that he made through that process were validated through testing and could be seen in the final product.
I believe that his main decisions came through the use of a mathematical simulator that he put together. I was of the belief that this influenced his design decisions not the testing. His testing showed promise but true validation came when BillA slapped it on his test die.
gone_fishin is offline   Reply With Quote
Unread 09-10-2003, 03:33 PM   #57
bigben2k
Responsible for 2%
of all the posts here.
 
bigben2k's Avatar
 
Join Date: May 2002
Location: Texas, U.S.A.
Posts: 8,302
Default

[edit: rambling removed]

Last edited by bigben2k; 09-12-2003 at 08:35 AM.
bigben2k is offline   Reply With Quote
Unread 09-10-2003, 03:40 PM   #58
jaydee
Put up or Shut Up
 
jaydee's Avatar
 
Join Date: Dec 2001
Location: Spokane WA
Posts: 6,506
Default

Quote:
Originally posted by gone_fishin
No, I'm saying doing testing right could give some answers that are helpful in designing a block. I stand corrected if it is 2.5C (is this verifiable?). Do it right not halfassed is all I was ever saying in these discussions.
But doing it right, by what you have stated to be right, is not going to be possible by anyone here, not even pH. So we should just all give up? or do our best to eliminate as many errors as possible and move forward?

Also I guess it was 2C, was sure I seen 2.5C somewhere but stand corrected:
Quote:
originally posted by Cathar
Straight away on the first remount I'm seeing a full 2.0C clear lead over the best result I had previously gotten with the White Water over 12 mounts of that block. This is using a T'Bred B at 2400MHz/1.85v running BurnK7. This is significant because it was my design goal to gain a full 2C, and secondly, it's just the first time I've mounted the block. History tells me that I rarely get it right first time, so there may be a little bit more hiding there as I remount to test again.
http://forums.procooling.com/vbb/sho...5&pagenumber=6
jaydee is offline   Reply With Quote
Unread 09-10-2003, 05:47 PM   #59
Cathar
Thermophile
 
Cathar's Avatar
 
Join Date: Sep 2002
Location: Melbourne, Australia
Posts: 2,538
Default

I never state that my testing is accurate. It is true that until BillA tested the White Water, then my results weren't really validated.

My Cascade results ranged from 1.5-2.0C better than my reference White Water block here in my testing. That's of course with the disclaimer about the (gross) inaccuracies of my testbed.

The Cascade results have been partially independently validated by the German water-cooling site who tested the Cascade against the DTek White Water, and found the Cascade to be 1.5C in front. The aluinium topped DTek White Water is an exact copy of the original White Water for the cooling specific bits (middle and base plates).

The German site used an Eheim 1046 with a large number of added restrictions, so admittedly this was an extremely low flow test. In my testing the Cascade does pick up a bit of performance separation over the White Water as the pumping pressure is increased.

Now what that all tells me is that despite the inaccuracies of my testbed, I can still get a somewhat decent indication of what's going on. It's not accurate in any way, but it's at least good enough for me to develop the block designs and pick a difference and really that's all that I was after.

True validation comes from independent testing, because let's face it, it is perhaps never wise to take a block maker's word on how their product performs?
Cathar is offline   Reply With Quote
Unread 09-10-2003, 06:10 PM   #60
pHaestus
Big Player
Making Big Money
 
pHaestus's Avatar
 
Join Date: Aug 2001
Location: irc.lostgeek.com #procooling.com
Posts: 4,782
Default

I wasn't trying to put you on the spot Cathar or hold your testing methods up as the ideal. I merely meant that it was a good example of you you can get good results from "typical" equipment with being careful and with a bit of thought. You can throw thousands of dollars at the problem as well, but without attention to detail and careful experiment design it still wont be reliable. That was my point.
pHaestus is offline   Reply With Quote
Unread 09-10-2003, 06:49 PM   #61
jaydee
Put up or Shut Up
 
jaydee's Avatar
 
Join Date: Dec 2001
Location: Spokane WA
Posts: 6,506
Default

God damn, where is that banging head off wall smilie?

We got GF saying "do it right or don't do it at all" with his strict list of minimum required testing which will take very expensive equipment.

And we got pH who says it can be done with hardly any money in the equipment as long as the method it good..... i.e. Cathars and his own testing methods.

We got Ben.....well.....we got Ben.....

Cathars way doesn't require base plate or inlet/outlet temps yet he is still able to come up with fairly accurate "relative" numbers and has built 2 of the best blocks on the market using those methods.

So what is "good enough". :shrug:

Bah....
jaydee is offline   Reply With Quote
Unread 09-10-2003, 07:16 PM   #62
Cathar
Thermophile
 
Cathar's Avatar
 
Join Date: Sep 2002
Location: Melbourne, Australia
Posts: 2,538
Default

It's okay Ph, you didn't put me "on the spot", well maybe a little, but I did understand where you were coming from.

I was just wanting to highlight my personal views on the matter as really being somewhere in-between the sides being taken in this thread.

Call me a fence-sitter if you will. I perfectly understand the respective needs for both approaches.
Cathar is offline   Reply With Quote
Unread 09-10-2003, 08:38 PM   #63
gone_fishin
Cooling Savant
 
gone_fishin's Avatar
 
Join Date: May 2002
Location: Da UP
Posts: 517
Default

Using a test method such as Cathar has done to help give indications in his developmental process is far different then stating block A is x amount better than block B and publishing it for public consumption (doing so will effect the going rate in the free market and open yourself up to lawsuit so you better be able to back up that data). Different horses for different courses.

What are the goals of this discussion, gettin kinda hazy in here.
gone_fishin is offline   Reply With Quote
Unread 09-10-2003, 09:31 PM   #64
pHaestus
Big Player
Making Big Money
 
pHaestus's Avatar
 
Join Date: Aug 2001
Location: irc.lostgeek.com #procooling.com
Posts: 4,782
Default

Depends on what you are doing JayDee. I assume what you care about are (a) bitching and (b) improving the blocks you make yourself (in no particular order).

For me, I care about building a testbed where I can get the same results for a block on day one as on day 500 with good enough accuracy to write reviews for this site. I haven't exactly spent "hardly any money" on my own testing setup (I think it is still measured in hundreds of dollars spent rather than the thousands thrown around in this thread though). Can I get that last bit of accuracy needed to separate every single good waterblock from one another? Probably not. I can certainly give some good evaluation and buying advice though. If a $100 block and a $30 block are too close together for my testbed to separate then I'd go with the $30 one, for example.

I also care about temperature measurement and monitoring in my own personal PCs that I run daily. That's where all my messing with Maxim ICs and CF633s and gameport thermistors comes in. If you calibrate those types of setups and use a bit of thought in where you place sensors then you can really enhance your ability to troubleshoot your cooling and your system. Would I trust those kinds of methods enough to do a Procooling waterblock roundup with them? Nope. But I trust them with my $1000 PC (go figure).

As for Ben he cares about googling and blue sky and his post per day count. I wouldn't make big purchases based upon that.

I guess what I was trying to say (and missed the mark) with the earlier examples of good work done by testers using cheap equipment is that a testbed is an evolving thing. I started off with an interest and a minimum of equipment. I assumed it would be easy; I am a bright fellow after all. As I ran tests I realized there were some flaws with how I was doing things. I'd correct the human errors as well as I could, and then when I was still bumping up against problems I would upgrade a part here and there. Add the ability to measure water temps to 0.01C res with decent reproducibility. Add a bigger pump to offset that. Fiddle with how to control water temps. Add a die simulator and related hardware. Etc etc.

To me it is irresponsible to start some "Alliance" and make these recommendations that tons of money must be invested to get some "magical" accuracy. The errors in mounting coming from a tester who has never installed a socketA waterblock before and never worked in a lab are going to dwarf the accuracy of the equipment for months. Give things a try and see if you enjoy it. See if you can spare the time. If so, and if it's important to you, eventually a testbed will emerge. Do the best with what you have/can afford, be honest about your testbed's limitations, and upgrade as needed/as available. "I'm waiting to find that used with traceable certs" is a valid excuse for not going platinum RTD in my mind.
pHaestus is offline   Reply With Quote
Unread 09-10-2003, 10:14 PM   #65
jaydee
Put up or Shut Up
 
jaydee's Avatar
 
Join Date: Dec 2001
Location: Spokane WA
Posts: 6,506
Default

GF I think your thinking in the lines of a manufacturers point of view when testing. I gave that up long ago. I have no desire to sell blocks and rapidly loosing desire to keep designing them for my own use.

I am more interested in things from a reviewers point of view at this point. I pretty much ruled out myself doing this over the last year though.

pH you hit it in your first sentance on your last post. I am more tired of people claiming ambient CPU temps with 500mhz OC at 1.95Vcore on AIR and would like a well layed out article (or even a website) devoted to measuring temps in computers that we can point these people to and tell them not to say another word about their temps untill they do the things in the article/site. I like your current articles but they fall short of the reasoning for doing what your doing.

I have been thinking about putting up a website devoted to just this subject. Problem is I need some more experienced people to assist in the information put on it. Something I am considering as opposed to this WBTA deal. I like procoling and all but the subject matter is just to broad to really cover temp monitoring well IMO. I would like the site to just be about temp monitoring computers and maybe even other things.....
jaydee is offline   Reply With Quote
Unread 09-10-2003, 11:27 PM   #66
bigben2k
Responsible for 2%
of all the posts here.
 
bigben2k's Avatar
 
Join Date: May 2002
Location: Texas, U.S.A.
Posts: 8,302
Default

Quote:
Originally posted by pHaestus
...

As for Ben he cares about googling and blue sky and his post per day count. I wouldn't make big purchases based upon that.

...

Add the ability to measure water temps to 0.01C res with decent reproducibility. Add a bigger pump to offset that. Fiddle with how to control water temps. Add a die simulator and related hardware. Etc etc.

...

To me it is irresponsible to start some "Alliance" and make these recommendations that tons of money must be invested to get some "magical" accuracy.

...

Do the best with what you have/can afford, be honest about your testbed's limitations, and upgrade as needed/as available. "I'm waiting to find that used with traceable certs" is a valid excuse for not going platinum RTD in my mind.
I couldn't care less about my post count, actually.

[edit: rambling removed]

I'm sorry for my ramblings about that ultra high accuracy bit. I was just trying to figure out a (relatively) cheap way of measuring that secondary loss. Obviously, as Bill himself pointed out, it's beyond our capabilities/means. I certainly didn't mean to imply that we would need to spend huge amounts of money (used or new).


Going over some theoretical figures, I'm trying to figure out how each individual error applies to the result: the C/W. One thing that's stomping me right now (a brain fart), is, if I have a temp differential (die to water) error of say, +/- 0.2 deg C, given say, a 70 Watt source (measured at +/- 2%), then what's my C/W's error margin?

Last edited by bigben2k; 09-12-2003 at 08:38 AM.
bigben2k is offline   Reply With Quote
Unread 09-10-2003, 11:55 PM   #67
pHaestus
Big Player
Making Big Money
 
pHaestus's Avatar
 
Join Date: Aug 2001
Location: irc.lostgeek.com #procooling.com
Posts: 4,782
Default

http://www.rit.edu/~vwlsps/uncertain...rtainties.html
http://www.rit.edu/~vwlsps/uncertain...tiespart1.html
http://www.rit.edu/~vwlsps/uncertain...tiespart2.html

Fairly general overview. I have a set of old handwritten analytical chemistry lecture notes I actually refer to though for most basic stats.

Quote:
One thing that's stomping me right now (a brain fart), is, if I have a temp differential (die to water) error of say, +/- 0.2 deg C, given say, a 70 Watt source (measured at +/- 2%), then what's my C/W's error margin?
Think about it logically. The upper limit on the calculated C/W will be when you overestimate the temperature differential by the largest amount and underestimate the wattage my the maximum amount. The lower limit would be the smallest temp differential divided by the largest wattage. So let's say you have a real 0.2C/W; that's a 14C delta T. The range of C/W you can get from your error is 13.8/71.4 =0.19 to 14.2/68.6 = 0.21 or your C/W value is +/- 0.01. That's actually pretty good; you'll see bigger differences than that from wb mount to wb mount.

Digitec and my YSI probes are guaranteed to 0.3C accuracy out of box. My Fluke is probably a little worse. This number is a worst case scenario statement from mfgr that the temperature will be within this range of the true temp; it is not a measure of how tightly the instrument responds to a change in temperature. I would expect the digitec to be offset by as much as 0.3C, but when using it to monitor changes in temperature the error in the delta T is much less. Meaning the linearity in response is quite good. I can't necessarily say the same about my TCs and so I don't care for them as much.

Um what I am saying I guess is I have bigger error bars if I want to compare my numbers with your numbers than I do if I want to compare a block I test on today with a block I test next month on the same setup.
pHaestus is offline   Reply With Quote
Unread 09-11-2003, 08:49 PM   #68
jaydee
Put up or Shut Up
 
jaydee's Avatar
 
Join Date: Dec 2001
Location: Spokane WA
Posts: 6,506
Default

Quote:
Originally posted by bigben2k
I couldn't care less about my post count, actually.

Seems as if post count has been visually turned of anyway?
jaydee is offline   Reply With Quote
Unread 09-11-2003, 11:45 PM   #69
gone_fishin
Cooling Savant
 
gone_fishin's Avatar
 
Join Date: May 2002
Location: Da UP
Posts: 517
Default

On the topic of temperature measurement, this PDF should be interesting. If nothing else it gives a good description of resolution, precision, accuracy, calibration, sensitivity, stability(system drift).

http://www.luxtron.com/product/produ...iescatalog.pdf
gone_fishin is offline   Reply With Quote
Unread 09-12-2003, 08:19 AM   #70
Blackeagle
Thermophile
 
Join Date: Oct 2002
Location: U.S.A = Michigan
Posts: 1,243
Default

Quote:
Originally posted by gone_fishin
Using a test method such as Cathar has done to help give indications in his developmental process is far different then stating block A is x amount better than block B and publishing it for public consumption (doing so will effect the going rate in the free market and open yourself up to lawsuit so you better be able to back up that data).
It would seem that there are massive numbers of reviewing sites that are not to worried about law suits. Hell some have 15 block "shootouts" or "roundups", yet claim to be able to tell which is best.

I'm not sure who here posted it but, ".........the hardest part of a roundup, as they are usually done, is rounding up all the blocks."

Just pointing out that if legal issues were a restraint, we wouldn't see all those off the wall articles being posted.:shrug:
Blackeagle is offline   Reply With Quote
Unread 09-12-2003, 09:51 AM   #71
Joe
The Pro/Life Support System
 
Joe's Avatar
 
Join Date: Dec 1999
Location: Denver, CO
Posts: 4,041
Default

Quote:
Originally posted by jaydee116
Seems as if post count has been visually turned of anyway?
Yep I removed it, I didnt want that to be a factor in discussions anymore. Would rather people gain credibility based off what they say not how many times they say it.
__________________
Joe - I only take this hat off for one thing...

ProCooling archive curator and dusty skeleton.
Joe is offline   Reply With Quote
Unread 09-12-2003, 11:44 AM   #72
gone_fishin
Cooling Savant
 
gone_fishin's Avatar
 
Join Date: May 2002
Location: Da UP
Posts: 517
Default

Quote:
Originally posted by Blackeagle
It would seem that there are massive numbers of reviewing sites that are not to worried about law suits. Hell some have 15 block "shootouts" or "roundups", yet claim to be able to tell which is best.

I'm not sure who here posted it but, ".........the hardest part of a roundup, as they are usually done, is rounding up all the blocks."

Just pointing out that if legal issues were a restraint, we wouldn't see all those off the wall articles being posted.:shrug:
That is true and evident by non action in the past. If watercooling were to get bigger and more big money was involved, you can bet there will be lawyers along for the ride. I agree not much of an issue now but something to keep in the dark recesses of the mind.
gone_fishin is offline   Reply With Quote
Unread 09-12-2003, 02:03 PM   #73
jaydee
Put up or Shut Up
 
jaydee's Avatar
 
Join Date: Dec 2001
Location: Spokane WA
Posts: 6,506
Default

Quote:
Originally posted by gone_fishin
That is true and evident by non action in the past. If watercooling were to get bigger and more big money was involved, you can bet there will be lawyers along for the ride. I agree not much of an issue now but something to keep in the dark recesses of the mind.
This is an interesting point. I don't see how it can really be inforced though. Then again if the manufacturer had their own evidence of how well their blocks worked then we wouldn't need reviews in the first place. I think you could only get in trouble if the manufacturer had it's own results which are severly different than yours and yours were negative to theirs. Then they might be able to put some sort of image damaging law suit together. This might be a larger issue in the future.
jaydee is offline   Reply With Quote
Unread 09-16-2003, 05:59 PM   #74
bigben2k
Responsible for 2%
of all the posts here.
 
bigben2k's Avatar
 
Join Date: May 2002
Location: Texas, U.S.A.
Posts: 8,302
Default

Was Googling around this aft, and came across this little gem:
http://www.itl.nist.gov/div898/handbook/index.htm

Enjoy: I will!
bigben2k is offline   Reply With Quote
Unread 09-29-2003, 10:49 AM   #75
bigben2k
Responsible for 2%
of all the posts here.
 
bigben2k's Avatar
 
Join Date: May 2002
Location: Texas, U.S.A.
Posts: 8,302
Default

Time to stir things up a bit!

I'm looking into a mercury thermometer from Cole Parmer.

The purpose is to calibrate my temp probes, but firstly the temp readings on my chiller. I want to get as close to 25 deg C as possible, as the coolant test temp.

Link to thermometer:
http://www.coleparmer.com/catalog/pr...=1&sel=0800130

That thermometer has a range of 19 deg C to 27 deg C, and is accurate to +/- 0.1 deg C. (ASTM# 17C)

The thing is, I don't know if the "ASTM method" is relevant: in this case, it's "Saybolt viscosity". Does anyone know?


The other problem I have is that this thermometer appears to be meant to be used "fully submerged". Although Cole Parmer provides a tutorial on how to compensate (calculated) for that, as far as I can tell, if I used it only partly submerged, I would be throwing off the accuracy of a reading by a maximum of 0.002 deg C. Is this of importance?


My third question would be: do I really need to get the NIST certificate (yearly?), and why?


Lastly, could I actually use this thermometer to measure a temp, while the pump is running, or would it be preferable to briefly stop the pump for a reading?
bigben2k is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump


All times are GMT -5. The time now is 12:27 PM.


Powered by vBulletin® Version 3.7.4
Copyright ©2000 - 2025, Jelsoft Enterprises Ltd.
(C) 2005 ProCooling.com
If we in some way offend you, insult you or your people, screw your mom, beat up your dad, or poop on your porch... we're sorry... we were probably really drunk...
Oh and dont steal our content bitches! Don't give us a reason to pee in your open car window this summer...