I think I have a masochistic streak in me somewhere...

Given all the interest and input I got from last weekend's set of lens tests. I redid them this weekend.

I tried to address the most common objections I heard last week. The test methodology has been substantially improved, and I included tests at f/2.0 as well as 4.0 and 8.0. Additional Nikkor lenses were added (a 35, another 50 and an 85 - all AF-D). The test objectives are still sharply limited.

Abandon Hope All Ye Who Enter Here

Have fun.

Paul Chefurka

-- Paul Chefurka (chefurka@home.com), August 05, 2001

Answers

"The duplicate images made by the same lens at the same aperture showed significant sharpness differences, especially at wide apertures" Paul, I have experienced this testing lenses as well, and I am afraid another culprit besides accurate focusing and vibration is film flatness. The slight curl of the film, (even with the small size of 35mm film), can be a big factor at wide apertures. Some of the resolution chart tests I did with my 50 Summicron showed higher edge sharpness than center sharpness at all the widest apertures (2.0, 2.8, 4.0)and I am certain this odd anomoly was lack of film flatness.

You are a glutton for punishment--what a thankless job doing this and posting it! It would have been interesting to have put 4 or 5 images side by side just marked "a, b, c, d, e" and telling the f stop and let people guess which was Nikon glass and what was Leica. By the way, who kicked the tripod on the Voigtlander APO 90 tests?

Seems to me what you get for the extra money on the Leica glass mainly is better wide open performance--and I think your tests confirm this. Happy wide aperture shooting.

-- Andrew Schank (aschank@flash.net), August 05, 2001.

Paul: Nice job! Given the amount of greif you got over the last test you did, I think it's admirable that you took the task on a second time! Brass cohones, mi amigo :-)

Your tests do seem to indicate that Leica glass is noticably better than Nikon glass 'till f8. I was especially interested in the Nikkor 85 results... This had been one of my sharpest Nikkor lenses, and it appers to have paled when up against the 90AA! (BTW, who kicked the tripod during the Voigtlander test? :-)) ) I also agree with the earlier post that film flatness plays a big role in sharpness results from lens tests. Perhaps all the extra inconvenience we suffer when loading our M's with film is worth the effort after all?

One last comment I know you'll hate. For the record, Fuji Provia F (RDP III) is one of the sharpest films you can buy due to it having better (smaller) granularity than most other films -- including even Velvia (RVP). I know it may not make sense being it is faster, but it is none-the-less true.

-- Jack Flesher (jbflesher@msn.com), August 05, 2001.

About the sharpness/granularity issue, I remember an early magazine test of Provia 100F vs Velvia that indicated that Provia was less grainy, but Velvia was still sharper. The smaller dye clouds of Provia didn't result in higher acutance, or something like that. Given that the results of this test were scanned, I don't really think it matters much.

I put the Voigtlander results in there in the interests of honesty - I did the test, those are the results. What they mean is something for another day. I suspect a close focussing error, but it will take more shooting to figure out what's going on. For the time being, I'll certainly be using the 90AA for anything except happy snaps.

-- Paul Chefurka (chefurka@home.com), August 05, 2001.

Paul,

I really enjoyed and learned from this last round of tests! There clearly are differences. My conclusions:

The color on the Leica glass generally seems better.

The 75 summilux is an outstanding lens. I wish I had one.

The Nikkor 85 AF is also outstanding.

The 50 summicron beats the Nikon lenses hands down.

The 35 summilux is good but not great.

Thanks for doing this.

I would be interested in learning if you perceive meaningful differences when you view the chromes side by side through a loupe. Also, please share with us your personal conclusions. We'll be kind :).

David

PS The lower left image for normal and telephoto did not appear on my monitor. I don't know why.

-- David Enzel (dhenzel@vei.net), August 05, 2001.

The two images that appeared as gray boxes were the f/2.0 positions for lenses that don't have maximum apertures of 2.0 - the Micro-Nikkor and the APO-Lanthar. Rather than mess up the sublime symmetry I had going, I decided to put in the dummy boxes.

-- Paul Chefurka (chefurka@home.com), August 05, 2001.

Good job, Paul!

The Summilux 35 looks great wide open....

-- Steve Hoffman (shoffman2@socal.rr.com), August 05, 2001.

Paul, just wanna thank you for the lens test--really enjoyed and learned much from it, and it helped to settle an argument with a friend about the sharpness between the Nikkor 50mm f/1.4 and f/1.8 when shot wide open (he won! :-( ).

-- Hoyin Lee (leehoyin@hutchcity.com), August 05, 2001.

Hoyan, I have both the 50mm f1.8 AIS Nikkor and the new plastic 50mm f1.8 AF lens. The AF lens is noticeably sharper wide open than the better built AIS manual focus lens.

-- Andrew Schank (aschank@flash.net), August 05, 2001.

Paul, I forgot to mention it looks like something is amiss on the 85mm f2.0 Nikkor test. All the shots appear to have the same depth of field which is also much greater than any of the other short teles at the wider settings. I know how hard it is to keep all this stuff straight when testing a bag full of stuff. When I do testing on a chart, I make up post-it notes of what lens and f stop I am using and stick it right on the chart so I can see it right on the negative/slide after the film is developed.

-- Andrew Schank (aschank@flash.net), August 05, 2001.

I just scratched the Voightlander 90mm off my list of possible lenses.

-- Dan Brown (brpatnet@swbell.net), August 05, 2001.

Viewing lens tests on scanned images on a monitor screen to me is like the story of the guy who calls up the doctor and says "Doc, I fell down the basement stairs and my arm really hurts. Do you think it's broken?" and the doctor replies "I can't tell...why don't you hold it a little closer to the 'phone?"

-- Jay (infinitydt@aol.com), August 05, 2001.

Dan, about the quality of the CV 90 - I wouldn't be so quick to dismiss it. It looks a lot better in normal shooting than that test implies. I'm not sure what's up with it - I need to use it some more to get a feel for it. FWIW, in the first set of test pics I posted, it did pretty well. A couple of magazine reviews have been pretty laudatory, too.

Remember - one test is just one test. There is no statistical validity, and you shouldn't ever base your decisions on somebody's internet posting. I'll shoot some more real pictures with it and get post some words - they're always safer :-/

-- Paul Chefurka (chefurka@home.com), August 05, 2001.

Thank you Andrew for the info. I wish I'd known it earlier, as my friend and I were arguing about the AIS version of the f/1.8 and the f/1.4 50mm Nikkor, and I had based my argument on the assumption that the optical quality of both the AF and AIS version of the f/1.8 lens is the same. Never mind, I'll let my friend enjoy his chest-pounding triumphant ritual.

-- Hoyin Lee (leehoyin@hutchcity.com), August 05, 2001.

Oh, and Andrew - I put a card with the lens and f-stop in each shot, so it's on the original (you can see an example of it on the full- size reference shot). I'll go back and check the 85/2.0.

-- Paul Chefurka (chefurka@home.com), August 05, 2001.

Paul: My thanks also for the test. No complaints about technique from ME!

I was interested that the 90 'Cron and 75 'Lux seem to beat the vaunted 50 'cron. Maybe it's a function of shooting distance, or (dare we say it?) maybe the long lenses are just better.

I was also surprised (again) by the Nikon 85 f/2 wide open - I would have expected it to be like the 85 f/1.8 (at least that's how MY images always looked). But a very nice lens stopped down a little (assuming you can put up with the weight and mirror bounce of the camera that goes with it ;-).)

As a long time Nikon and occasional Canon user in the past, I've never held that Leica lenses would be sharper at all apertures - in fact I did tests once upon a time with Nikon AIS 85/35/20 vs. 1988-vintage Leica 90/35/21 (f/2 except the 20/21) at middle apertures that persuaded me to stick with Nikon another 10 years. They were B&W tests. The 21 was a little sharper wide-open, but I couldn't see any difference between the 90/85 and 35s at that time to justify the cost.

What I am paying for is not visibly better performance at f/8 per se, but better color, better tonality, better wide-open performance,and better performance at f/8 when shooting at 1/30th second.

On Velvia vs. Provia - Velvia incorporates chemical edge sharpening and extra contrast (D-Max) to give it (as closely as possible) acutance similar to Kodachrome. For a test like this Provia may have been the better choice. I've always had a pet theory that Nikon and Canon specifically commissioned the creation of Velvia in order to make their lenses look as much like Leica/Zeiss as possible, so it might have tweaked the results here like a PhotoShop 'unsharp mask'.

-- Andy Piper (apidens@denver.infi.net), August 06, 2001.

Paul I have been doing very similar tests for the last five years (everytime I buy a new body and/or lens). My tests are done at 1.8m (each time the same row of antique cans in my kitchen - these are covered in all sort and size of intricate typography and lines), and exactly at 50m (always the same building with signs on it and in open shadow). I scan a 5mm square portion of the film (at centre and at edge) at 2700dpi (Nikon LS-1000). At pixel size, as you have pointed out it indeed corresponds to about x40 magnification producing 950000K to 1mb files - saved uncompressed.

Having analysed your test results thoroughly and compared them with my results, may I suggest the following:

1)If you haven't already done so please check both your Leica and Nikon F3 to see whether they focus correctly or not. I find the best method for this is to first shoot a frame at the best focus point as indicated by the camera rangefinder, then take further 2-3 shots at TINY increments turning the focusing ring first clockwise, then anti-clockwise from the camera indicated point. With Leica you will have to do this test with every lens at 1.5m and 50m. Don't be shocked with the results. With an SLR I would recommend a wide and a tele lens again at 1.5m and 50m. ( I have owned 2 F3 bodies - both bought brand new, and although technically within factory tolerances, both needed personal tweeking to focus spot on). Then scan and analyse the film as below.

2)I would recommend using Fuji Reale 100 Neg film for this type of testing. I have used Velvia, Ektachrome E100VS, Kodak neg films and find that neg films scan better and Reale scans best.

3)For scanning test negs set the scanner gamma to 1.5. Do all colour and density corrections on the scanner and then - MOST IMPORTANT - focus the scanner manually at the exact spot you are going to scan for EACH SCAN.

4)Once the scan is in Photoshop, I would recommend 2x 100 (not 1x 200) unsharp mask. I apply 3x100 but you might find the result too gritty. The result will be similar to what one would get if one was using a good enlarger/lens combination with a CONDENSER, for the tests. I save the file uncompressed. Compression makes a bit of difference but only if you look at the compressed and uncompressed files side by side. To evaluate the scans when comparing, look at them at both at 100% and 50%.

I am hoping that you will find the above suggestions, made with the best intentions, constructive and try them. Please do not hesitate to contact me directly if you wish.

All the best

-- sait (akkirman@clear.net.nz), August 06, 2001.

Andy - I checked the Nikkor 85/2.0 slides again, and I'm satisfied I got them right.

Sait - thanks for the suggestions. In fact, this is the first time in 25 years that I've tested lenses in any kind of formal way. I really don't like doing it, and this exercise has taught me just how hard it is to get results that will give you any kind of confidence that they reflect reality. In fact, most of what attracts me to a lens has nothing to do with sharpness per se. Tonal qualities, colour rendition, flare resistance and the lens' physical properties all factor in along with absolute sharpness.

This test was mainly done to provide some kind of reasonably controlled demonstration I can point to when the usual nethead Leica/Nikkor lens arguments break out. That plus I had all this amazing glass sitting around, and I started to wonder...

I hope this will be the last time I feel compelled to flagellate myself in public :-)

-- Paul Chefurka (chefurka@home.com), August 06, 2001.

I hope you don't feel that you have flagellated yourself in public. Despite the disclaimers about problems with the tests, I think that everyone who has posted is in agreement that you made a significant effort, and is pleased that you took the time and effort to post this stuff right in front of God and everybody. Thanks for doing it. Bill

-- Bill Mitchell (bmitch@home.com), August 06, 2001.

Paul,

Thank you for your effort. I have a question about the lay out of the images, and I wish to make sure I am reading them correctly.

The three images, two on top, one on the bottom correspond to three tested apertures... while the three apertures are listed on the bottom in a linier fashion. Is the single frame on the bottom the f/8.0 or the f/4.0 shot? Logic would say f/4.0, since it is in the middle right over the f/4.0 narrative, but in many examples the top right shot seems sharper. So... what exactly is the sequence: Top left = f/2.0, bottom middle = f/4.0 and top right = f/8.0... or bottom middle = f/8.0 (as the last shot in the sequence), top right = f/4.0 (as the second shot in the sequence)?

Again, thank you for your effort. I think this would be of value for many people, and might inspire others to do the test themselves with their own equipment. Some of you results surprised me, while others confirmed things that I have experienced, and I am sure some sample variation is at fault... my Nikkor f/1.4 AF was never that good, while my AIS 50mm f/1.8 is outstanding, every bit the equal of my Summicron M at the middle apertures. In the end, the only thing that counts is the lenses in our own bag... the ones we will use.

-- Al Smith (smith58@msn.com), August 06, 2001.

Hmm. It looks like I've been bitten by browser layout and sizing issues again. On my screen, each set of three shots is in a single row, above the aperture designations. Each row is centered on the screen. It looks like what happens is that when the browser window is too small, the line with the pictures "wraps", and the third shot drops to a new line. So the single shot you're seeing is the f/8.0 frame.

If anyone has a suggestion about how to keep the pics from doing this (i.e. keep all the images on one line and force a "scroll" rather than wrapping), I'd love to hear it. I did the pages using Netscape Composer, BTW.

-- Paul Chefurka (chefurka@home.com), August 06, 2001.

Thanks Paul, that explain some things that were confusing me. I didn't think diffraction would be setting in at f/8.

-- Al Smith (smith58@msn.com), August 06, 2001.

So Mr. Mandler seems to be the king so long, Sloms needed an aspheric element in their 90/2 to reach the 75/1.4 quality, and the 28/2 really disapointed me, but that 35 lux is a marvell, thank´s for your test Paul, it seems very objective, will analise it for many days to come.

-- r watson (al1231234@hotmail.com), August 06, 2001.

I may be looking in the wrong place but I cant seem to find any pics from the Voigtlander 90 in your second test that people on here are saying are poor?

-- James Cooke (james.c@mis.net.au), August 11, 2001.

On further reflection I decided that the CV 90 shots weren't representative of the the results I'm seeing with that lens in everyday shooting. Rather than cast possibly unfair aspersions on the lens, I decided to pull those shots.

-- Paul Chefurka (chefurka@home.com), August 11, 2001.

I must have a serious problem. Everything looks like it was taken with an Olympus XA on outdated Ektachrome 400. I can't tell much of anything from the images I'm displaying.

I'm using 24 bit color, a 17" Sony monitor, both Netscape 4.78 and IE 5.01.

I've displayed images on the Internet that will blow your socks off. I can't sort these out.

Maybe I need an Apple computer:-)

-- Bud (budcook@attglobal.net), August 11, 2001.

Bud:

I have Mac, Windows and Unix. I do most of my image analysis [at least 8 h/day] on a G4 with about 1.5 gigs. I have great respect for anyone who does it on a Windows machine. Photoshop is extremely clunky there. Like doing great photography with a Kodak Brownie. It can be done, but what a job. Unix does graphics well too, but it is hard to get some of the software for it; actually the same is true for windows; much of my software isn't available for windows.

Art

-- Art (AKarr90975@aol.com), August 11, 2001.

Bud:

The images you're looking at are fairly extreme enargements of tiny sections of the original images. Take a look at the introductory page, and you'll see the small section that was cropped out of each image for final display. The point was not to display nice pictures, but to take a look at the level of ultimate resolution available from each lens, to the extent that this can be displayed on the web.

Colour quality doesn't count, and the grain and incoherence of the image are the result of looking at what is in effect a 40x enlargement with additional scanning artifacts thrown in. Think of making a 40x60 print, copping out a 3"x5" section, scanning that at 72dpi and displaying it. That's more or less what you're looking at here.

-- Paul Chefurka (chefurka@home.com), August 11, 2001.

Thanks for the explaination Paul. My system isn't good enough to see the differences among the various lenses (or maybe I need glasses).

-- Bud (budcook@attglobal.net), August 11, 2001.

You may just be seeing that, given good lenses of different makes, differences are very hard to demonstrate on the net. This in itself is a valuable insight, and should be applied religiously whenever someone puts up web demonstrations claiming to show the quality of this or that lens. Even Leica lenses... and even my demonstration.

-- Paul Chefurka (chefurka@home.com), August 11, 2001.

My lens testing procedure is pretty simple. I shoot Kodachrome film and mix the slides in a Prodovit tray. If the slides are as neutral and as brilliant as those produced by my old Rigid Summicron, I'm satisfied. Note that none of my dozen or more Nikon AI-S lenses measured up so that's why I switched to Leicaflexes.

Incidently, I've been going back over kodachrome slides using a 5X magnifier and found that my M slides are sharper than even my R slides. Looks like I should be using a tripod more often.

-- Bud (budcook@attglobal.net), August 12, 2001.

Well, I've gone back and looked at those test images again after letting them sit for a while, hoping to give them a more objective analysis.

The following things struck me:

1. Most of the shots (especially at 4.0 and 8.0 are essentially indistinguishable.

2. Even the shots at 2.0 are pretty similar with some notable exceptions. The Nikkors 35/2.0, 50/1.8, 85/1.8 and the Leica 28/2.0 all look worse than any of the other lenses at 2.0. I find it hard to believe that the Leica is as bad as it looks :-/

3. Most of the small differences seem to be caused by focus variations. It could easily be the case focus problems contributed to some degree to the big differences as well.

My final conclusion is that it is virtually impossible to conclusively demonstrate minor resolution differences between lenses using images on the net.

The biggest limiting factor for this (IMO) is the lowered sensitivity of the test due to the scanning step. The LS-4000 I used has an effective resolution (measured by Pop Photo) of 60 lp/mm. Given the additional processing required (sharpening and reduction to JPEG) and monitor resolution limitations, the final resolution is inevitably less than that (<50?). This means that pretty much any lens will meet or exceed the resolution of the test, except when the lens is at its most severely compromised.

One other obvious limiting factor is focus accuracy, especially with closeup shots.

So there you go. It was an interesting exercise. It taught me a lot about what can and can't be shown on the net, as well as what the scanner is doing to my slides. Luckily some other aspects of Leica quality (flare resistance etc.) will come through on a scanner, and there are enough other reasons to pursue the digital darkroom approach. Ultimate resolution just isn't one of them.

-- Paul Chefurka (paul_chefurka@pmc-sierra.com), August 17, 2001.

Paul:

You have done a lot of work and it is appreciated. You have concluded what I have without so much work. Too many variables to actually differentiate resolution, total range and color reproduction on the web. Besides the overall difference [greater that the errors you discuss] is the viewers moniter quality and set-up. I can tell that the the resolution quality from my [now] $200 p&s is better than that from my $499 5 digital camera.

Art

-- Art (AKarr90975@aol.com), August 17, 2001.