Debayering & Bayer2RGB

CvK · January 3, 2018, 8:35pm

The driver itself offers little control over the type of debayering performed on images, but there apparantly are several debayering options present in CVB. What are the differences in the debayering settings?

RGB_INTERPOLATE = 1, RGB_NEIGHBOUR = 2, RGB_HALF = 3, RGB_EDGEINTERPOLATE = 4

The enumeration type documentation gives some more info, but what exactly happens when using RGB_EDGEINTERPOLATE?

I currently have to store images, which will be analyzed by human sight, so the pesky artifacts typical in debayered images are proving troublesome. I’m hoping that perhaps edge interpolate can reduce this.

illusive · December 12, 2017, 2:39pm

Hi @CvK

You are right, the documentation currently does not go into too much detail when it comes to the different interpolation modes… Generally the source of the problem is of course, that thanks to the Bayer Pattern the intensity information for all three channels is sub sampled (with Green still having twice as much information available as Red and Blue), so for building dense RGB information for the whole image, interpolation is necessary and this will inevitably affect the perceived quality of the RGB image.

The quality (or lack thereof) of the interpolation is most evident when looking at black/white transitions in an image, which is why I cut out a section of the Rubic’s Cube test image, de-bayered with one of the available interpolation modes:

RGB_INTERPOLATE
This mode basically does a linear interpolation using the neighboring pixels in horizontal and/or vertical direction (depending on the pattern position to be interpolated) to establish RGB values for all pixels. The result not surprisingly looks somewhat blurry along a black/white transition and may even show some blocky artifacts (seen on the right vertical edge) stemming from the alternating selection of vertical and horizontal neighbors for the interpolation of the red and blue channel (on the left vertical edge this effect is barely visible - this edge apparently was not sharp enough for the artifact to have a significant impact on the quality):

image.png639×863 66.9 KB
RGB_NEIGHBOUR
This page is an interpolation only in the looses possible sense of the word. It only interpolates the green channel (taking into account only one first order diagonal neighbor) and simply reproduces the red and blue channels from the most recent pixel where this data was present. This approach gives rise to very pronounced color artifacts, nicely visible on the diagonal lines in the image below:

image.png634×877 49.9 KB
RGB_HALF
The RGB_HALF mode evades the question of interpolation almost entirely. It gathers the RGB information from a tile of 2x2 pixels by interpolating the green channel from the two green pixels in that tile and taking the red and blue channel as they come. As a result the result image size will be reduced by a factor of 2 in every direction, making it slightly unfair to compare the output to the other modes. But - notwithstanding the reduced resolution - the artifacts here are much less pronounced than in the other cases…

image.png622×875 25.1 KB
RGB_EDGEINTERPOLATE
This mode tries make good on the shortcomings of RGB_INTERPOLATE by being more selective about which neighboring pixels are accounted for in the interpolation. Rather than simply alternating between the horizontal and the vertical neighbors it’ll look at the orientation of the intensity gradient in the green channel. Depending on the orientation of the gradient it’ll favor the horizontal or the vertical neighbors, effectively suppressing the alternating-pixel-artifacts seen in the first image on the right vertical edge:

image.png636×876 67.4 KB

CvK · December 14, 2017, 3:36pm

Ah, thanks, good info!

What is the one used in the driver? So, if i tell cvb to display my images debayered, without using cvb bayer2rgb, which of those is displayed?

parsd · January 3, 2018, 8:34pm

This is not standardized, but it normally is a fast and simple 2x2 RGB_NEIGHBOUR.

CvK · January 3, 2018, 1:48pm

As this might be interesting for others to read, a brief description of my attempts at reducing the color artifacts caused by debayering. All debayering methods in cvb bayer2rgb still have some troubles with the color artifacts to some degree, at least too much for my particular application.

Since the pixel artifacts are usually sharp color changes, I reasoned that perhaps in HSV space these errors could be detected. This did not work, as hue is circular, and using lowpass filters on hue chanels do not give the desired results.

But, as I am optimizing for human perception, the LAB color space proved to be much better for this. Using lowpass filters on the A and B color chanels, and a sharpening filter on the L chanel greatly reduced those pesky color artifacts. And all within CVB foundation too

New artifacts where of course introduced by this approach, but light “glow” of certain sharp color edges was not as bad as the bayer color artifacts.
Plain debayered image;