Home  Camera Articles   FOR SALE    Orders     I Buy / Wants     Repairs     Books   Adapters

Lens Tester's Anonymous: It's a lot easier being a Great Consumer than a Great Photographer!

Some people love to compare published lens tests,  as if  someone else's tests  have  anything to do with the real world lens performance they will get.    A new  test will appear and  Lens Tester's  Anonymous  members will dutifully sell   their old "not as good" lens to buy the new wonderful "better" lens.  This is photography and consumerism at its worst.    

Have you ever noticed that the worst photographers usually have newest and  best equipment? They are so busy checking the latest lens tests, they don't have time to learn how to use the lenses  they already have.     You can spot them because they are excited about the new lenses or camera they just bought, instead of the wonderful picture they just made.

The Problems with Someone Else's Lens Test:

• How accurately were the tests done? How repeatable with what standard methods?  How can you compare one test result with another if they used different methods and criteria?

• How many lens variables were tested?   Two notable factors seldom discussed are flare variations and lens performance at different distances.  Good or bad flare performance can vary with point light intensity,  position from the lens axis, and  f/stop.   Superb performance at infinity can easily change to  so-so close up performance, and vice versa. 

• Lens tests can vary from body to body:  WHY? 

1. Focusing accuracy -- whether rangefinder or single lens reflex.  Bad focus equals lower resolution

2. Vibration, shutter or mirror vibration means lower resolution.  Yes, your lens testing results can vary camera to camera because different cameras have different vibration levels.     This means if you have five different bodies of the same mount, the results from the same lens may vary due to body vibration.

3. Flare within the film chamber lowers resolution.  Contrary to popular belief, cameras can vary considerably with this little discussed variable.  In the mid 70's Fujica marketed a line of 35 SLRs with the mirror box flocked in black matte material.  Modern Photography tested it, and confirmed that Fujica has the lowest flare level of any 35 tested.   Want to improve  lens performance? Eliminate flare in your film chamber.

4. Film flatness.  The only way to really test the lens and not the camera is to shoot glass plates....which of course no one does.   That is, no one except Kilfitt which shipped a test glass plate with each lens!!!!   Realizing the problem, both Rollei and the CIA use special glass plates or a special film pressure plate to really get accurate images.   A few cameras have addressed the problem, notably the Contax RTS with its vacuum back and the Contax ST with its extra large ceramic film plate.   Some photogs report their believe that  the first couple of exposures on each roll have lower resolution than the rest ... because the of film flatness.

5. Shutter speed accuracy.  If one camera over exposes half a stop, that means more negative density and less resolution.

6. Camera support. Different tripods are vastly different in their ability to stabilize the camera and lens, as well as dampening all vibrations.   Kilfitt used to ship a glass plate with each of their lenses showing its resolution.  Guess what kind of tripod they used for testing......a tripod head mounted in concrete.!

7. What does this add up to?   STANDARDIZE your testing with the best camera and tripod you can.  Test your lenses with the SAME camera body, to make sure you are testing the lenses, and not the camera.   Testing your lenses on different tripods and bodies will reduce the validity of your test results by introducing more uncontrollable variables.  

8. THIS MEANS:  test with the same camera body that is  calibrated for shutter accuracy and minimum vibration, with a film chamber black flocked for maximum flare reduction, with the same tripod or camera mount which is incredibly solid, with the finest grain sharpest film you can find.   Switch camera bodies, tripods, and films between lens tests, and you are throwing away the ability to compare the lens' sharpness ... the new variables become a test of the camera too!!    Of course this also means if you have a group of lens tests, and then improve your testing procedure using the items above, your old test results  are no longer directly comparable to your new test results.   oh well.

• What is the test sample size ??  Sometimes over a 100,000 lenses of a certain type can be made over more than a decade.  Guess what?  You can bet the longer a lens is in production, the more unannounced changes have been made to it.  These changes can vary from improvements in coating or optical design, to "improvements" in the lens barrel (aka cheapening the lens to lower production cost).   If you test ONE lens out of a large number, all you are getting is a general idea of what that lens will.    Manufacturer's have tolerances.  How do you know how your test lens fits into those tolerances ?  You don't.     On the other hand, if someone tests 50 or more lenses throughout a production range, I would tend to believe the average results.  

• The older the lens, the higher chance you have of lens problems:  decentering from a drop, coating problems due to bad lens cleaning,  bad re-assembly after a repair, or fogging which will lower contrast.  That's right Sherlock, just because that lens was among the world's best 30 years ago, doesn't mean it still is.   Don't believe someone else's test as predicting its performance, test it yourself.

• Published lens tests may be "optimistic" by feeding the lens tester or magazine that depends upon the manufacturer's revenue  the very best off the assembly line.  BEWARE any lens test which is not a random sample, bought off the shelf, without the lens tester identifying that the lens will be tested.

• Great lens tests BY SOMEONE ELSE  do not mean that YOUR  lens  will equal the tested lens.   Lenses can vary  right off the production line.   In used lenses decades old,  the more chances you have for someone dropping it just hard enough to knock an element out of alignment.

• Only one test distance?  Most lens test  ONLY tests lenses at ONE focused distance.  The inherent problem here is that lenses can easily perform differently between close up, medium distance, and infinity.   A great lens at infinity may give poor results at 10 feet, and vice versa.  So....how valid is a lens test only done at ten feet to 20 feet?    Have you ever seen a lens test at infinity?  There are not many.

• Lens tests are done with very fine grain films on a tripod.  If you shoot hand held with medium speed or high speed films,  the differences between a good lens and a great lens may be almost impossible to see.   The unsteadiness of the human tripod combined with lower resolution films will go a long way to destroy lens performance.

• What does all of these variables mean?   That the lens tests so many photogs slavishly refer to  and spend their cash on are really only vague generalizations of what to expect with YOUR lens, YOUR camera body, YOUR film, YOUR shooting conditions.  In other words, someone else's lens tests are about as accurate as weather reports.   If you want to be sure of what results you are getting, TEST YOUR OWN LENSES.

• If you want to be sure,  TEST YOUR OWN LENSES.   Don't worry about someone else's test results.   Your lens could be much bettor, or worse, than theirs.  So test your own.  Sure it's strange to actually take YOUR own pictures instead of checking  someone else's test reports, but how often do those other guys check your pictures?     Of course, the biggest problem with this approach is that it eats into valuable time you could be reading published lens tests and shopping ---  HA HA HA.   I've recently been surprised to find a line of extremely sharp lenses which I have never heard praised by anyone ....  how many good shots have you taken by reading a test report??

• Surprise!  It's the photographer, not the lens, that takes the picture.   If you take terrible shots now with your old lens, switching to a new lens will just get you the same terrible shots and a thinner wallet.    If you want to improve your photography,  1) shoot a lot and print your own  2) attend seminars by master photogs   3) take many photography classes. 

• "New, Improved, Sharper" published lens tests are often an excuse to trade in a lens the photog can't push to its limits,  for an even "better lens"  they won't be able to do justice to either.   But still, it's a godsend for fragile egos if they don't have to take responsibility for their own mediocre images. 

• Are the Lens Testing Experts really testing Expertly?  Recently it turned out that a much quoted Net lens testing guru was foolishly testing older lenses WITHOUT having them cleaned first, TOTALLY INVALIDATING his "scientific test."  Wow, what a surprise that 40 or more years of accumulated haze would affect resolution.    Again, if you want accurate lens tests,  TEST YOUR OWN LENSES.

Home  Camera Articles   FOR SALE    Orders     I Buy / Wants     Repairs     Books   Adapters

Revised: November 25, 2003 .  Copyright � 1998-2002  Stephen Gandy. All rights reserved.    This means you may NOT copy and re-use the text or the pictures in ANY other internet or printed publication of ANY kind.  Information in this document is subject to change without notice.  Other products and companies referred to herein are trademarks or registered trademarks of their respective companies or mark holders.