As long as we're talking magazine reviews…

I read and then re-read the Bench Chisel Review in the December, 1999 Fine Woodworking (issue #139). I gotta tell you, between the router bit episode and this submission, somebody needs to get involved at that magazine and learn something about devising tests.

Now I am not an expert in testing, but after many years of reading product reviews in a variety of areas of interest, I can tell you that it is always possible to build a really impressive testing apparatus, and then carefully work it to produce results, and then tabulate those results into a very compelling synopsis.

The trouble is, the test that was developed may not mean anything. Thus, the results obtained are equally meaningless. Tabulating those meaningless results into some sort of array does not lend them any more credence.

When you read a test that flies in the face of every experience, every bit of knowledge, and every intuition that you have, it's time to question the testing paradigm.

I'm sorry; when two Sorby and a Marples chisel don't score half as good as a Craftsman, the test is flawed. I have Sorbys and I have Craftsmans. There is absolutely no comparison. Well, actually there wasn't in this test, either, but theirs was in the opposite direction.

Oh, and this isn't parochialism about my brand, either. Just look at the test. Talk about separating fly specks from pepper (there's a better version of this that shouldn't be printed here)! The whole range of chisels from 4 to 16 probably will still shave arm hair after the test (one of my friends from Badger Pond pointed out that he quit reading when he learned they didn't even micro-bevel the edges). And yet there will be people making buying decisions based on this nonsense.

(I was really frustrated at this point because I knew that there was a perfect analogy to this thing floating around in my head that I couldn't bring to mind but I knew I would remember the instant I hit the Send key.)

Oh, yeah. I remember. Back in the '60s when I first starting getting involved in stereo equipment, my parents popped for a Garrard Lab 80 record changer (records!?!). I went through all the specs of the Shures and the Empires and the Pickerings trying to decide what stylus cartridge to get. I settled on a Shure because it had .060 x 10^-9 ergs of dynamic compliance while the Pickering only had .058 x 10^-9, and the Empire even less. It wasn't until years later that the silliness of that buying decision struck me. The quantity being measured was a billionth of an erg, and I was making a buying decision on 2/1000ths of them! This is the sort of thing that was tabulated in the chisel test.

Rod Peterson

Subscriber Hating Idiotic Tests

Last updated: 27 January 2009