Image Acquisition

Capturing images is straight-forward on digital cameras with auto-setting or mobile phone. DSLR cameras with more control during photography allows us to manually change parameters on the hardware. The cameras we use in robotics are generally machine vision cameras which don’t have any controllers on the hardware itself. Such cameras come with a graphical interface and software development kit (SDK) to capture images.

If your image sensor has a Bayer Format, it is also advisable to record the image in Bayer Format to understand the influence of each channel during imaging. On the other hand if your camera is monochrome, a single channel can still store useful information without introducing artefacts in the later stage. These images can also be saved as a single frame, continuous frames and/or a video format depending on the robotic applications.

There are other imaging parameters such as brightness, contrast, gamma, white-balance and black level that need attention while imaging to enhance image quality. Brightness and contrast allows us to adjust the tonal values of your images while gamma optimise the brightness of acquired images for visualisation in a computer. As we know, robots don’t require visually pleasing results and can work in a lower SNR regime than humans. So during imaging we keep these parameters at default. White-balance allows us to compensate for the colour temperature between cooler and warmer temperatures and black level has a control over the overall brightness of an image. Plotting a histogram of an image and finding the clipping point allows us to compensate for this value. Since there are libraries inbuilt for C and C++ within most of the SDK, calling these functions and setting imaging parameters are only a line of code. The following shows how these parameters are set for a Basler dart camera.

C (generic) language

#define CHECK(errc) if (GENAPI_E_OK != errc) printErrorAndExit(errc)
GENAPIC_RESULT errRes = GENAPI_E_OK;
errRes = PylonDeviceSetIntegerFeature(hdev, "BlackLevel", 32);
errRes = PylonDeviceSetFloatFeature(hdev, "BslBrightness", 0.5);
errRes = PylonDeviceFeatureFromString(hdev, "BslContrastMode", "Linear"); 
errRes = PylonDeviceSetFloatFeature(hdev, "BslContrast", 1.2);
errRes = PylonDeviceSetFloatFeature(hdev, "Gamma", 1.2);
errRes = PylonDeviceFeatureFromString(hdev, "PixelFormat", "Mono12");
CHECK(errRes);

C++ (native) Language

camera.BlackLevel.SetValue(32);
camera.BslBrightness.SetValue(0.5);
camera.BslContrastMode.SetValue(BslContrastMode_Linear);
camera.BslContrast.SetValue(1.2);
camera.Gamma.SetValue(1.2);
camera.PixelFormat.SetValue(PixelFormat_Mono12);

While these parameters are mostly in default during imaging, we can adjust the aperture size as the first step towards handling imaging-trade-offs. This is done manually on the camera where there is generally a range of f-number from 1.2-16. For example, if we compare between f/2 and f/16, f/16 focal ratio will have a wider aperture than f/2 allowing more signal to enter the lens at the expense of depth of field. In robotic imaging, we generally require information regarding the background for feature-matching and triangulation, so we will select a narrow aperture option. While this is true in general, there are applications that focuses only on foreground-objects like object detection and/or face detection for which widening the aperture is the best choice.

While it is possible in some of the context in robotic applications to capture long exposure images by having higher exposure time, there are either dynamic scenes and/or moving camera involved in general applications in robotics including structure-from-motion (SfM) and simultaneous localization and mapping (SLAM). Thus, it is desirable to keep the exposure minimum as possible to avoid getting blur in the image. If that is our choice of action, we have to increase the gain in an image such that we amplify the signal at the expense of noise. On the contrary, we can capture images with minimum gain and higher exposure time, trying to have an acceptable amount of blur in the robotics context as possible in each frame. For example, SIFT feature extraction has the ability to perform accurately at moderate noise and moderate blur but fails after. So, selection of tuning parameters such as exposure time and gain are design choices based on applications. Once these design choices are made for an application, we can capture images using the SDK. Here’s an example using the pylon SDK and python programming language.

from pypylon import pylon

camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())
camera.Open()

new_width = camera.Width.GetValue() - camera.Width.GetInc()
if new_width >= camera.Width.GetMin():
    camera.Width.SetValue(new_width)

numberOfImagesToGrab = 10
camera.StartGrabbingMax(numberOfImagesToGrab)

while camera.IsGrabbing():
    grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)

    if grabResult.GrabSucceeded():
        print("SizeX: ", grabResult.Width)
        print("SizeY: ", grabResult.Height)
        img = grabResult.Array
        print("Gray value of first pixel: ", img[0, 0])

    grabResult.Release()
camera.Close()

While the discussed settings are enough to explore robotic imaging in air, underwater imaging requires additional calibration and tuning to handle back scattering, contrast, light attenuation, scattering, shadows, colour shading and suspended particles. While we all are excited about exploring space and knowing unknowns of the universe, a massive part of the sea, right here remains unexplored.

References:

  1. Flash applets on some technical aspects of photography . 2022.
  2. GitHub. 2022. GitHub – basler/pypylon: The official python wrapper for the pylon Camera Software Suite.
  3. Basler Camera Support from Image Acquisition Toolbox – Hardware Support – MATLAB & Simulink. 2022.
  4. FLIR Camera Support from Image Acquisition Toolbox – Hardware Support – MATLAB & Simulink. 2022.
%d bloggers like this: