It does depend on how into it you are and what your intents are, I recall reading Cathar's statement (see below) regarding his extesnsive initial testing, including statements about lapping the Maze 3 and running 10 tests, etc.
I think a standard setup would be difficult, but if someone truly feels the need to discuss their results what I said in the Tuff thread I still believe applies:
Quote:
Originally Posted by Khledar
I mean this has all been discussed before - the more thorough a set of results (testing methods, etc.) - the more acurately they can be interpreted by the readers... Maybe wanna indulge us with your setup Tuff?
|
If you give a full description of how you tested it, maybe a few pictures and a short paragraph - I doubt anyone will challenge you unless there is something serious that needs to be improved. I think it's generally understood that we're not talking about professional testing, but a gauge of one block to another. If your methods are well thought out and lack any serious problems they will be taken for what they are. But necessary explanation of what has gone on during testing is required.
I mean when BillA was testing blocks he didn't just post the nice C/W vs flow graphs etc and say glad you tuned in. There was a huge explanation of his setup, how he tested, what he did to eliminate error. I'm not saying we all need to do this, but in a short few paragraphs (getting back to the beginning of this post) Cathar was able to give some sort of credibility to his testing with respect to the Maze 3.
[EDIT: Below]
In case you're not tuning into the Tuff thread:
Quote:
Originally Posted by Cathar
My Maze 3 was lapped properly by me after I received it. The base wasn't that flat to begin with and I picked up an easy 2C from that lapping of it. Since I'm comparing against a well known product, I want that product to be operating in the best possible way that it can be made to, and to remove any possible inherent performance variances that may exist as it comes from the factory. That way I'm comparing against a well defined "high point" for a design, rather than a possible "middle or low point" in which performing "better" can suddenly be absorbed by merely picking/receiving a better example of the competing block.
All tests were run to equilibrium and the environment was very carefully controlled. Each block was mounted at least 10 times to minimise mounting variations which could, on occasion amount to 3C differences, and the various blocks were alternated between mounts to ensure that results were repeatable and comparable with earlier results and to provide additional control and detection for test setup and environment variances.
Since I'm developing a block, I feel that I absolutely owe it to myself to do these things, since it costs me money to make blocks and I don't want to be fooling myself as to what direction I'm heading at any point in time, or worse, that should anyone buy a block from me and perceive that I have fooled them as well.
I don't know how any of that applies to what is going on here. I just thought I'd share what I do to determine how well a block that I make performs relative to any other block. If I'm going to claim it's better than another block, it must be better than the best possible representation of that other block.
|