Calibration with an over-exposed colour checker?

[danieladlz]

I am capturing images with an Arducam of a mini ColorChecker from X-Rite in a 3D scanner. The issue I'm facing is that the calibration is not working, which I assume is due to overexposure in some of the images. For example:

However, I have also taken other pictures that are not as overexposed. I have compared the values from the color swatches after the calibration with the official values and often found significant variations in the values.

These pictures were taken under different lighting conditions than D65. I have measured the spectrum xy coordinates of the LED used in the 3D scanner to input the illuminant, but I am not sure if it is really necessary or helpful. I'm stuck and do not know how to make it work. I would greatly appreciate some tips on how to operate the code correctly or what to improve in the pictures."

[KelSolaar]

Hello,

A few notes an questions:

• Lighting seems to be a bit blown out indeed.
• The LED light primaries seem a bit narrowband, e.g. the red really pops.
• The illuminant xy will not really be helpful here.
• Are you sure that your images are linearised properly?

Cheers,

Thomas

[danieladlz]

Hello, thank you for the quick response to my question. I've conducted a thorough investigation into all the points you mentioned, and it turns out that the light primaries of the LEDs used in the second picture were narrowband. We've since switched to different LEDs that have a stronger blue component, and this has significantly improved calibration accuracy. However, some challenges persist, which I believe are still related to the LEDs.

I captured several pictures in the same setup, only varying the voltage supplied to the LEDs, which in turn adjusts the intensity of the light, ranging from 1V to 6V. Notably, the best calibration results were achieved with the lights operating below 5V.

this is the before:

this is the after:

I have compiled a data table comprising colors with variations greater than 0.015 from the desired values. Many of these colors reappear as I adjust the voltage, but I have not observed a discernible pattern in how changes in voltage relate to alterations in the differences between the desired and obtained values post-calibration

At least, when it comes to whether the images are properly linearized, I don't have a clear answer. The calibrated images are all taken in JPG format. I colud probably take PNG images if it should make a diference. I plotted the histogram of the image above for the RGB channels, and here are the results.

It's possible that the images are in the sRGB color space rather than the RGB color space. Could you please clarify the specific function of this part of your code?

COLOUR_CHECKER_IMAGES = [
colour.cctf_decoding(colour.io.read_image(path)[..., 0:3])
for path in COLOUR_CHECKER_IMAGE_PATHS
]

I only used the [... 0:3] for the PNG images, as there aperar the lines mentioned in another question. Could you provide a method for me to determine if the images are linear or to linearize them?

If you have any further suggestions on how to improve, I would be greatly appreciative.

[KelSolaar]

The issue is that your image is not linear, if you look at the bottom neutral gray row, you will see that it does not match at all:

The Neutral 6.5 patch is relatively good but all the other ones are darker which implies a non-linearity issue. You could try a higher degree polynomial, e.g. Finlayson et al. (2015):

    swatches_f = colour.colour_correction(swatches, swatches, REFERENCE_SWATCHES, method="Finlayson 2015", degree=4)

but it is dangerous with only a ColorChecker Classic with 24 samples as it is bound to explosions for colours outside the 24 ones.

Ideally, you would want to output a linear image (or linearise it) and then compute the colour correction matrix for it.

[danieladlz]

Hi, I will try that. I also attempted to linearize the images using this code:
import cv2
import numpy as np

Load the sRGB image

sRGB_image = cv2.imread('MyImage.jpg')

Function to apply the sRGB to linear RGB transformation to a single pixel

def sRGB_to_linear(pixel):
linear_pixel = np.zeros(3, dtype=np.float32)
for channel in range(3):
linear_pixel[channel] = (pixel[channel] / 255.0) ** 2.2
return linear_pixel

Apply the transformation to each pixel in the sRGB image

height, width, _ = sRGB_image.shape
linear_image = np.zeros((height, width, 3), dtype=np.float32)

for y in range(height):
for x in range(width):
sRGB_pixel = sRGB_image[y, x]
linear_pixel = sRGB_to_linear(sRGB_pixel)
linear_image[y, x] = linear_pixel

Scale the linear image back to the 0-255 range (if needed)

linear_image_scaled = (linear_image * 255).astype(np.uint8)

But it did not help. The code could not detect the ColorChecker in the image. I also used a function provided by MATLAB called:

B = rgb2lin(A,OutputType="double");

You can find more information about this function here. However, even though the ColorChecker was recognized, the results were not good.

So I still need to figure out how to obtain a linearized image as the output.

[KelSolaar]

The decoding itself using colour.cctf_decoding(colour.io.read_image(path)[..., 0:3]) seems correct, or at least a good start. The issue is really in the camera processing that seems to be applying a S-Curve or something along those lines to embellish the image rendering. We happen to have some Arducams at work and they definitely render the image for display. I never looked into figuring out their camera response functions but it is something you could do with the colour_hdri.camera_response_functions_Debevec1997 definition.

An alternative is to use the AMPAS IDT Calculator for Prosumer Camera: https://github.com/ampas/idt-calculator, the entry point here is helpful to show you how to generate a 1D LUT + 3x3 Matrix: https://github.com/ampas/idt-calculator/blob/master/aces/idt/prosumer_camera.py#L1612. The corrected image will be in ACES2065-1 colourspace.

[danieladlz]

Hello, thank you very much for all the tips. I am currently conducting the research on the topic of linearization. However, I have two more questions outside from that topic.

Firstly, how does your code determine the RGB values of the ColorChecker patches?Does it calculate an average from the RGB values inside the defined field?

Secondly, I am attempting to assess the effectiveness of the calibration process for the color patches. To achieve this, I am calculating the color difference between the calibrated values and the provided values from X-Rite. I'm using the following code for this purpose:

#CIELab
#Data Image ColorChecker

for i, swatches in enumerate(SWATCHES):
    swatches_lab = colour.XYZ_to_Lab(colour.RGB_to_XYZ(
        swatches, D65, D65, colour.RGB_COLOURSPACES['sRGB'].matrix_RGB_to_XYZ))
    print('lab')
    print(swatches_lab)

#Data des ColorCheckers
cc_lab= colour.XYZ_to_Lab(colour.xyY_to_XYZ(list(REFERENCE_COLOUR_CHECKER.data.values())),ICCD50)
print("ColorCheckerLab")
print(cc_lab)

#giben Target Lab values (soll_lab_matrix) and Lab values after calibration (calibrated_lab_matrix). Each matrix should have the shape (n, 3), where n 
# represents the number of color patches, and each row contains L*, a*, and b* values.

# Define a function to calculate ΔE between two Lab values.
def delta_e_lab(lab1, lab2):
    delta_l = lab1[0] - lab2[0]
    delta_a = lab1[1] - lab2[1]
    delta_b = lab1[2] - lab2[2]
    return np.sqrt(delta_l**2 + delta_a**2 + delta_b**2)

# Calculate ΔE for all pairwise combinations of color patches between the Soll-Lab-Werte 
# und kalibrierten Lab-Werte.
n = cc_lab.shape[0]
delta_e_values = np.zeros(n)  # Initialize an array to store ΔE values.

for i in range(n):
    delta_e_values[i] = delta_e_lab(cc_lab[i], swatches_lab[i])

# Now delta_e_values contains the ΔE values for the pairwise comparisons.

However, I must say I'm not completely sure if I've determined the correct swatches_lab values because the RGB illuminant is D65, while the illuminant defined for the ideal ColorChecker is ICC D50.

CCS_ILLUMINANT_COLORCHECKER24_AFTER_NOV2014: NDArray = CCS_ILLUMINANTS[
    "CIE 1931 2 Degree Standard Observer"
]["ICC D50"]

Which means I should transform the illuminant from D65 to ICC D50 to get the accurate difference in colors.

for i, swatches in enumerate(SWATCHES):
    swatches_lab = colour.XYZ_to_Lab(colour.RGB_to_XYZ(
        swatches, D65, ICCD50, colour.RGB_COLOURSPACES['sRGB'].matrix_RGB_to_XYZ), ICCD50)

Would you say this is accurate?