The Holding Tank

Home Page Photography

Photography RSS 2.0 Feed

| Photos & Themes RSS | Hardware | Technical | Links | Books and Magazines  | Downloads |

Technical

| Flash Tests | RAW Converter Tests | Depth of Field, Image Quality and Sensor Size |

| MTF | Resolution | Cropping and Printing | Image Stabilization and Magnification |

| Lens Image Simulation | Anti-Alias Filter and Sharpening | | Tilt and Shift Lenses |

Lens Distortion Correction and Image Detail | Digital Darkroom Workflow | Optical Coatings |

Lens Computer Simulations

Table of Contents

Introduction  

Examining the Results

Pattern Results Wide Angle 5 mm  

Image Results Wide Angle 5 mm

Pattern Results Wide Angle 13 mm  

Image Results Wide Angle 13 mm

Pattern Results Telephoto 13 mm  

Image Results Telephoto 13 mm

Conclusions

Sagittal vs Meridonal MTF

Appendix A (Algorithm and Implementation)

 Appendix B Lens Test Method IMATEST

Appendix C Lens Test Method DxO

Introduction

The purpose of this exercise is to connect the information in the MTF chart with a real world indication of the effect upon actual images. To accomplish this, image filtering is required to simulate the MTF characteristics of a lens, how this is achieved is detailed in the appendix.

There are two test images used in this operation that are intended to represent a small cropped section of the whole image sensor, the sensor sample rate (pixel spacing) is representative of an 8.2Mpixel 1.6X crop factor sensor such as the Canon 20D:

i)                    ti.jpg, this is an image with good contrast and detail that has been down sampled so most of the effects of the lens and capture mechanism are insignificant compared to the simulated lens characteristic, this image is about 600X400 pixels, this represents about 20% of the image frame of a typical DLSR. Alternatively this would be an area of about 2X1.3 inch if the imaginary full image was printed to a size of 10X6 inch.

ii)                  tp.jpg, this is a computer generated test pattern consisting of two sets of orthogonal lines spaced at 10 and 30 lp/mm with an image size of about 400X400 pixels. No dithering is applied to the pattern so the exact line width varies somewhat.

As explained in the MTF page, the MTF data applies to lines in the sagittal and meriodonal directions. So below the filtered images are presented with the filter in two orientations, 0 deg and 90 degrees. This corresponds to the line direction and test image crop locations as explained in the table below:

To further clarify see the below sketch:

The simplifying assumption here is that the lens MTF characteristics and meridonal and sagittal line direction is assumed constant over the test crop image. All MTF data is at f/8.  See the note on simulation accuracy.

Note that the MTF of the actual sensor and anti-alias filter are neglected as I have little solid information. However I have looked at this point somewhat in Anti-Alias Filter and Sharpening. In any event digital camera resolution will improve and be subject to upgrade cycles as are computers for the time being so what is important is the lens performance. Also note that the simulation for the corner points is a little fudged as the lens coordinate system is aligned with the pixel coordinate system. In practice this does not matter.

NB: The Tokina simulation is based on photodo measured data, all other lenses simulation is based upon Canon published MTF charts which are engineering simulation rather than actual measurement. So comparatively the Tokina will look worse than it really is.

Back to TOC

Examining the Results  

In the below tables the resulting images are presented sized to be 150X100 and 100X100 pixels respectively. If your monitor has the typical ~75 ppi resolution they will appear about the same size as they would in a 10X6 inch print. However, you will have trouble critically comparing sharpness at this size.

Some of the filtered images may show darkening around the edge, this is a side effect of applying a strong filter near the edge of the image so ignore this.

To compare the images at 100% right click the image and select “open link in new window”. Alternatively, probably the best way to compare results, is to download all the images and step through them in a viewing application. All the images may be downloaded in a zip archive here. These include some additional simulations of wide angle zooms interpolated to 20mm and 35mm along with the two 35mm prime lenses and more on 85, 135 and 200mm focal lengths.

If you are using Photoshop open a few related images at 100% and place the windows directly over each other. Use CTRL-TAB to step through the image windows. Make any small adjustments you need to bring the windows into exact alignment. It is best to work on a few images and add more when you have eliminated some, keep to one filter orientation at a time.

You can now step through the windows fixing your eyes on an area of detail to see how it is affected by the lens MTF characteristic. Close the least sharp images one at a time until you are left with ones you can't tell which is the sharpest. Now repeat with the other filter orientation.

Back to TOC  

 Test Pattern Wide Angle at 5mm off Optical Axis

  Back to TOC

Test Image Wide Angle at 5mm off Optical Axis

   Back to TOC

Test Pattern Wide Angle at 13mm off Optical Axis

  Back to TOC

Test Image Wide Angle at 13mm off Optical Axis

   Back to TOC

Test Pattern Telephoto at 13mm off Optical Axis

  Back to TOC

Test Image Telephoto at 13mm off Optical Axis

  Back to TOC

Conclusions

Looking through the results conclusions are that:

i)                    it is not as easy to see small differences in lens performance with a generalised image, as compared to a clear test pattern. This suggests that lens tests need to be conducted with a pattern rather than just taking a picture of your back garden as often is done.

ii)                  Lens quality differences are most marked at the corners, but then we knew that.

iii)                When photographing a real image the reduction of quality for lenses with larger differences between sagittal and meridional sharpness are not as pronounced as expected. It seems that the eye sees the detail that is there rather than that which is not. This suggests that using the minimum MTF, as has been done in the lens selection page, may be rather pessimistic. Landscape and architecture are obvious subjects that have dominant line directions.

iv)                Comparing the images by sequencing through them in a viewing programme is the easiest way to observe the differences in sharpness. In general perceived sharpness seems to follow the sharpness expected from the MTF charts as detailed in lens selection page. With this method differences in performance between even the best lenses are observable with the pixel resolution of 8.2 Mpixel APS-C.

Back to TOC

Sagittal vs Meridonal MTF

The issue of if the best, worst or average of sagittal and meridional resolving power is the most relevant, can be understood easier by examining the images in the tables below. Here three hypothetical lenses are simulated:

i)                    A good Lens with the same MTF performance in the sagittal and meriodonial directions of 0.99 and 0.75 for 10 and 30 lp/mm respectively.

ii)                  A bad lens with the same MTF performance in the sagittal and meriodonial directions of 0.75 and 0.50 for 10 and 30 lp/mm respectively.

iii)                A mixed lens with a combination of the above MTF performance in the sagittal and meriodonial directions of 0.75 & 0.99 and 0.50 & 0.85 for sagital & meridional and the 10 and 30 lp/mm line frequencies respectively.

iv)                An average lens with the same MTF performance in the sagittal and meriodonial directions of 0.87 and 0.625 for 10 and 30 lp/mm respectively.

Examination of these results suggests that for typical photographic subjects the average of the MTF performance is representative (perhaps slightly optomistic) of a lens with mixed sagittal and meriodonal resolution in terms of subjective sharpness. This is not the case for test targets that well separate the resolution in the two directions.

This explains the anecdotal difference often encountered between MTF specification and test results and tests using real world subjects.

This of course does still leave the argument of how to interpret MTF data. One thing to keep in mind is that a lens with significantly differing sagittal and meriodonal resolution is likely to have inferior bokeh.

It seems that the relative sharpness conclusions made using the worst case of the sagittal and meriodonal still apply if the average of the two are used, it is just that the differences between lenses are not so great.

Back to TOC

Appendix A (Algorithm and Implementation)

This section explains the method for filtering the above images; this requires some familiarity with some maths terms that might be encountered in the first year of a science or engineering degree.

Algorithm

i)                    The lens MTF data for 10 lp/mm and 30 lp/mm are fitted to the line frequency response equation model resulting in estimates of the order and MTF50% for each of the two directions.

ii)                  A 2D sampled surface is created by interpolating between the two line direction models based on sample orientation resulting in a 2D magnitude spatial frequency response. 

            Update 4th May 05, the new angle interpolation gives an ellipse shaped MTF contour in the frequency domain instead of the linear interpolation by angle in the earlier version. This is probably more representative of real lenses.

iii)                Quadrant symmetry is applied to provide a double sided 2D spectrum.

iv)                The appropriate phase gradient is applied across the surface to ensure central placement of a real valued impulse response. This results in the frequency domain surface being complex valued.

v)                  The frequency domain surface is converted to the time domain by a 2D inverse Fourier transform.

vi)                This transform results in a 2D filter impulse response with the desired frequency characteristics.

vii)              The filtering process is then accomplished using 2D convolution of the subject image and the impulse response.

Assumptions

i)     It is assumed the lens has a MTF of 1.0 at 0 lp/mm, this is a good assumption for multi-coated lenses in low flare conditions.

ii)    It is assumed that the lens follows the assumed roll-off law asymptotically. This should be a reasonable assumption to 50 lp/mm after which the MTF may in reality roll-off faster.

iii)   It is assumed the MTF changes in a linear monotonic fashion with angle between the sagittal and meriodional directions. This may be OK but as long as the two input directions are right the perceptive effect should be similar.

Implementation

The above algorithm was implemented using the industry standard technical computing package Matlab. The Matlab code is available for download here. The simulations presented here will be produced by the function file run_all.m. A filter impulse response of 41X41 pixels was used in all cases.  

Accuracy

The accuracy is reasonable based on simulated test results. The 10 lp/mm MTF comes out high but is consistent across various relative MTF specifications and so is good for comparisons. The 30 lp/mm seems very accurate. The reason for the 10 lp/mm error is unknown, it could be the frequency to time domain conversion or something in the MTF evaluation test. Be aware that lens contrast may be a little optimistic.

Example wire frame plots of a typical frequency response surface and impulse response surface are included below for illustration.

References

[1] There is some useful overview in Mathematica Documention.

[2] Matlab on line documentation.

Back to TOC

Appendix B Lens Test Method IMATEST

The IMATEST method uses a slanted slope target which is used to create an oversampled edge. The sharpness is then estimated by 1-D Fourier transform. This has the advantage of being shift invariant and not being limited by the sensor resolution, however the sensor anti-alias filter will limit the usefulness of the oversampled feature somewhat.

A disadvantage of this method compared to the DxO method is the test target is not well localized resulting in an average estimate of sharpness over a more extended area.

The method does not measure absolute contrast and so can not report MTF, mathematically it assumes unity MTF at zero spatial frequency. Instead SFR (spatial frequency response) is measured. Most tests being reported in the web report the MTF50 (in fact this is the 50% point of the SFR as the test can not measure MTF). 

The IMATEST method seems to be derived from sfrmat developed by Peter Burns for 13A ISO. The original matlab code is available for free download at I3A ISO tools download page. I have experimented with this code and found that the high frequency results are very sensitive to exposure level. However, as long as the techniques used are consistent the MTF50 (sic) results should be reliable.

PhotoZone amongst others are starting to report results using IMATEST, unfortunately only with a 1.6 crop camera at present. Here MTF50 is reported in line widths per picture height (LW/PH). This is relatively easy to convert to cycles/mm (equivalent to lp/mm).

This will give an approximate indication of MTF roll-off although the order of the characteristic is not revealed by this test.

Back to TOC  

Appendix C Lens Test Method DxO

The Dxo Analyzer is a software lens test system for digital photography. The test target consists of an array of spots across the image, because of this the results may not be entirely shift invariant but in any event can only be comparable with results taken on the same camera. 

Unlike the IMATEST method the results are not oversampled and so give no indication of performance beyond the sensors alias frequency.

Results for lenses are starting to appear on web sites, most notably SLRGear who are publishing results for mostly 20D but also some with both 20D and the 5D. Additionally results for Nikon and some independent manufactures lenses are being added.

DxO's sharpness measure is called the Blur Experience Unit or BxU. DxO state one blur factor is equivalent to one Photoshop Blur More operation. This is illustrated in the table below using the same test pattern and downsampled test image as for the lens simulations in the main section of this page.

The test pictures are intended to represent a small cropped section of the whole image sensor, the sensor sample rate (pixel spacing) is representative of an 8.2Mpixel 1.6X crop factor sensor such as the Canon 20D.

In the below tables the resulting images are presented sized to be 150X100 and 100X100 pixels respectively. If your monitor has the typical ~75 ppi resolution they will appear about the same size as they would in a 10X6 inch print. However, you will have trouble critically comparing sharpness at this size.

To compare the images at 100% right click the image and select “open link in new window”. Alternatively, probably the best way to compare results, is to download all the images and step through them in a viewing application.

The following comments are a guide for SLRGear results with a 20D.

The sharpest lens tested (100mm f2.8 Macro USM) seems to have a BxU of about 1, suggesting this is the limit of the sensor system. The worst lens tested (at the time of writing) seems to have a maximum aperture corner sharpness of about 7-8 BxU. 

A cheap consumer zoom may have a stopped down performance of 2-4 BxU from centre to edge. 

A good quality f2.8 Canon L series zoom may have 1.5-3 BxU wide open across the frame and zoom range to 1.2 to 2 stopped down.

So a sub BxU unit is very significant. Unfortunately SLRGear only display results on a fixed graphics scale of 12 BxU making fine comparisons difficult.

Analysis by IMATEST however suggests that the calibration of BxU against PS blur more operations is flawed.

Back to TOC  

Last Updated 05/06/2008

All Content © 2005-09 Lester Wareham All Rights Reserved     

All material is supplied  as is and without warranty, use at your own risk.  

All opinions stated are the authors own.

All quoted information remains the copyright of the respective authors.

Contact: (anti spam remove underscores)

Note this email gets spammed a lot so if you want to get your message through please prefix "Holding Tank" in the title

f_ w_ a_ r_ e_ 9_ 0_ @_l_i_n_e_o_n_e_._n_e_t