Lenses
Lens choice depends on camera. Check with camera manufacturer for the best lenses for your cameras.
Here are links to some frequently used lenses:
Fujinon c-mount high definition
Fujinon fixed focal length
Vignetting occurs when the lens size is smaller than the chip on the camera. See the example of vignetting below.
The photogrammetric relationships used to convert between image and world coordinates or into other image configurations like rectifications are based on the perfect pin-hole camera model. However, all camera-lens combinations have some amount of lens distortion that must be compensated for. This is done through lens calibration. Figure 1 shows an example of an image with extreme barrel distortion collected using a DJI Phantom 2 (Vision Plus), typical also of many GoPro images.
Figure 1. Example snap from a Phantom 2 with a great deal of barrel distortion. Note how curved the horizon is. Distortion scales with distance from the image center so the pier looks straighter than the horizon because it is mostly closer to the image center.
An advanced and free software package for dealing with lens distortion has been created by Jean-Yves Bouguet of Caltech and can be found by googling ‘Caltech lens distortion’ or using the link https://www.vision.caltech.edu/bouguetj/calib_doc/index.html.
The package has extensive documentation. However we will repeat the basis of the algorithm here.
The Caltech toolbox performs the calibration of a camera/lens system based on viewing a checkerboard target from a suite of viewing angles (for example, Figure 2). Corners across the entire target are automatically detected based on user-clicked locations of four outside corners, always in the same order from the top-left determined in the first image. Chose these enclosing corners such that full squares lie on the outside, not just small fractional parts of squares.
Figure 2. Checkerboard target for camera calibration. Roughly 20 images should be collected from a wide range of viewing angles. The bolts in the center helped keep the target flat and did not interfere with the analysis.
The algorithm solves for both the distortion coefficients of the lens and the intrinsic parameters of the lens/camera combination. The latter includes: the focal length of the lens, fc in their terms, expressed in pixel units and having components in the x and y direction of the image; the image skewness, alpha_c, which characterizes the degree to which the image x and y axes are not quite orthogonal; and the coordinates of the principal point of the sensor, the effective location of the pinhole in the pinhole camera model.
While we normally measure image coordinates in pixels from the top left image corner, [U, V], distortion is based on normalized measures equivalent to tangent of the angle away from the focal point, i.e. x = (U-U0)/fU and y = (V-V0)/fV, where [U0,V0] are the pixel coordinates of the image center and [fU, fV] are the two components of the focal length, in pixels. If we denote undistorted (pinhole) coordinates as Xu = [xu; yu] and the distorted coordinates as Xd = [xd; yd], and we measure the radial distance of any point from the principal point as r, where r2 = xu^2+yu^2, then the distorted coordinates could be found as
where
Equation (2) represents the tangential correction, usually a negligible correction. Equation (1) adds this to the radial correction, the main part of lens distortion.
The five coefficients in the kc vector are listed as returned by the Caltech toolbox. Note that the three radial coefficients are the first, second and fifth components of the distortion vector. Often the tangential and the sixth order radial term (kc(5)) are neglected.
Once the Caltech toolbox has been downloaded it is run in matlab by typing calib_gui. The steps generally follow the flow in the calibration toolbox window and are well described in the toolbox documentation. We have loaded a demo set of 30 images here (https://github.com/Coastal-Imaging-Research-Network/Forum-Wiki/blob/master/lensDistortionTestImages) that you can use for testing the method. These are for the 4000 x 2250 (9 MPixels) snapshot mode for a Phantom 3 Professional quadcopter. The target checkerboard has 6 cm squares (from the toolbox, printed locally and mounted on stiff cardboard). The center squares have a pattern of three circles that allow you to identify target orientation in snapshots. For comparison, I found the following answers:
Fc = [2309.1, 2306.8]; % U and V focal length, in pixels
Cc = [2006.0, 1156.7]; % principal point, in pixels
Alpha_c = 0; % skewness of axes – set to 0 by inputs
Kc = [-0.0204, 0.0124, 0.0075, 0.0012, 0] % last element forced to zero
In the CIL, lens distortion data are stored in a database and are easily called. For our initial CIRN toolboxes, we have manually entered the information into an expected structure called a Lens Calibration Profile (name taken from structure from motion work). An example, makeLCPP3 is included in the UAV toolbox and corresponds to an example DJI Phantom Professional 3 (P3P) quadcopter (the same images as are included in this lens distortion tutorial). The P3P allows image collection in five different modes, each with a different image size, hence covering different areas of the imaging chip. Thus, an LCP file is needed for any imaging type that you plan to analyze (i.e. you need to run the Caltech analysis for each imaging type and keep track of the different results). The routine makeLCPP3 allows for two different imaging modes that I have used and makes the decision of which is used based on the size of the input image, NU (number of columns) and NV (number of rows). An example of the structure is shown below (the 4K video mode from the P3P).
lcp.NU = 3840; % number of pixel columns
lcp.NV = 2160; % number of pixel rows
lcp.c0U = 1957.13; % two components of principal point
lcp.c0V = 1088.21;
lcp.fx = 2298.59; % two components of focal length (in pixels)
lcp.fy = 2310.87;
lcp.d1 = -0.14185; % radial distortion coefficients. Same as kc(1, 3 and 5)
lcp.d2 = 0.11168;
lcp.d3 = 0.0;
lcp.t1 = 0.00369; % tangential distortion coefficients. Same as kc(2, 4)
lcp.t2 = 0.002314;
lcp.r = 0:0.001:1.5; % the rest is needed for processing distortion
lcp = makeRadDist(lcp);
lcp = makeTangDist(lcp); % add tangential dist template
The final three lines compute functional forms of the distortion as a function of radius, lcp.r, to simplify the later computation of distortion.
Implementation of distortion removal (for geometry purposes) or un-distortion (to determine where computed pixel location should lie in a sampled image) are carried out using routines DJIDistort.m and DJIUndistort.m, located in the UAV toolbox. At this stage these names are overly specific to UAV work. This is because we need to better clarify distortion strategies in the CIL and in CIRN.
There is sometimes confusion about whether to use distort or undistort. For example, to create an undistorted rectification from an image, you end up using distort, not undistort. The reason is simple. The ideal rectification is designed as a set of world xyz points (usually a matrix of xy points with z set to 0 or tide level). The image intensities in this designed array are found by interpolation into an oblique image by first finding the corresponding location of the xyz point in terms of distorted UV coordinates.