Acquiring 10/12 bit RGB images from a JAI AP-3200T-PGE

Hi,

I am trying to acquire images from a JAI AP-3200T-PGE over GiGe. I am able to do this (both in genicambrowser, using CVBPy AND using harvester and the CVB .cti file), but only for 8-bit RGB images. For 10/12 bit, the image becomes garbled:
image

I have been in contact with Stemmer support who confirms that proper conversion of the color formats RGB10V1Packed and RGB12V1Packed are not currently supported by the CVB software.

Is it somehow possible to access the “raw” image data and perform a manual conversion ?

Kind Regards,

Jesper

Hi Jesper,

is there any possibility to use a different color format? RGB10V1Packed is not really a sensible one, that’s why there is no support for it. It is also a legacy, not PFNC compliant format.

If you set the format to RAW you should get the buffer as sent from the camera. You can then do the conversion yourself if so desired. The format is described in the JAI manual: https://www.manualslib.com/manual/78955/Jai-At-200ge.html?page=26

Best regards,
usernv

1 Like

Hi usernv,

Thanks for the quick reply!

The camera seems to support RGB10V1Packed, RGB10p32 and RGB12V1Packed (besides RGB8). Of these, the CVB genicam browser reports unsupported format for RGB10p32, which is why i continued looking into the other formats.

Very interesting link - the latest manual for the camera does not provide color format descriptions, but i guess they can be found in the Genicam PFNC: https://www.emva.org/wp-content/uploads/GenICam_PFNC_2_4.pdf (page 34). Does this mean that if i can somehow access the raw buffer with camera set to RGB10p32, i would be able to interpret it as a 32-bit unsigned integer array and decode it pixel by pixel?

Do you know how to access the raw buffer using CVBpy ?

Kind regards,

Jesper

Hi Jesper,

setting the format to RAW as described by @usernv and this example ImagePixelAccess for getting the pixel values should work for you.

Best regards,
Diverl

Hi Nico,

I managed to decode the raw image in the format RGB10p32 into an image using the genicam harvester library:

with ia.fetch_buffer() as buffer: 
    if len(buffer.payload.components) < 1:
        continue

    elif buffer.payload.components[0] is None:
        buffer_raw = buffer._buffer.raw_buffer


        if pixelformat == "RGB10p32":
            # buffer is a byte array of <H>x<W>x4 8-bit bytes, where each 4 bytes represent 3 10-bit RGB pixels
            # bit 0:9 is red, 10:19 green and 20:29 blue, the last two bits being spare and always 0.
            bytes_pr_pixel = 4
            clearing_byte = np.uint16(0b0000001111111111)
            buffer_int = np.frombuffer(buffer_raw, dtype=np.uint32)
            R = np.uint16(buffer_int) & clearing_byte
            G = np.uint16(buffer_int >> 10) & clearing_byte
            B = np.uint16(buffer_int >> 20) & clearing_byte

            # print binary using: print(f"{r:016b}")

        img_data = np.c_[R, G, B]
    else:
        img_data = buffer.payload.components[0].data

img = np.uint16((2** 16 - 1) * (img_data.reshape(height, width, 3) / 1023)

This works but takes a few seconds to compute (on my ryzen 5800HS), any suggestions for optimization is very welcome EDIT: vectorizing the code reduced the computation time to 0.015 sec.

The manual you provided @usernv described the format for RGB10V1Packed, but i haven’t been able to find a description for RGB12V1Packed - it is also unclear to me whether or not the first byte contains the first or last bits of the RGB pixels:

Kind regards,

Jesper

Sorry to say that but I also did not find anything on the topic. Maybe JAI knows more about the format. After all they decided to use it. You can also inquire our support staff if they can get the information from JAI.

That makes sense - i will contact JAI support. Thanks for the help!

Hi again - i managed to decode the image data with help from JAI support, sharing the code for anyone interested here.
The RGB12V1Packed stores the three 12 bits color components in 4.5 bytes:

The following python code seems to do the trick:

import cvb # to prevent watermark in image
from harvesters.core import Harvester
import numpy as np 
import cv2


h = Harvester()
h.add_file("C:/Program Files/STEMMER IMAGING/Common Vision Blox/GenICam/bin/win64_x64/TLIs/GEVTL.cti")
h.update()

ia = h.create_image_acquirer(0)
pixelformat = "RGB12V1Packed"
width = 1280
height = 800
ia.remote_device.node_map.PixelFormat.value = pixelformat
ia.remote_device.node_map.Width.value = width
ia.remote_device.node_map.Height.value = height
ia.start_acquisition()


with ia.fetch_buffer() as buffer: 
    if buffer.payload.components[0] is None:
        buffer_raw = buffer._buffer.raw_buffer
        if pixelformat == "RGB10p32":
            # buffer is a byte array of <H>x<W>x4 8-bit bytes, where each 4 bytes represent 3 10-bit RGB pixels
            # bit 0:9 is red, 10:19 green and 20:29 blue, the last two bits being spare and always 0.
            bit_depth = 10
            clearing_byte = np.uint16(0b0000001111111111)
            buffer_int = np.frombuffer(buffer_raw, dtype=np.uint32)
            R = np.uint16(buffer_int) & clearing_byte
            G = np.uint16(buffer_int >> 10) & clearing_byte
            B = np.uint16(buffer_int >> 20) & clearing_byte

        elif pixelformat == "RGB12V1Packed":
            # information stored in 12 x 3 = 36 or 4.5 bytes. 
            bit_depth = 12
            clearing_byte = np.uint16(0b0000111111111111)
            clearing_byte_4 = np.uint16(0b0000000000001111)
            
            # inpack into 8-bit integer array
            buffer_int = np.uint16(np.frombuffer(buffer_raw, dtype=np.uint8))

            # unpack red, green and blue channels. Bit pattern repeats every 9 bytes, which contain two pixels.
            unpack_1 = lambda buff, b1, b2: ((buff[b1:buff.shape[0]:9] << 4) | (buff[b2:buff.shape[0]:9] & clearing_byte_4)) & clearing_byte
            unpack_2 = lambda buff, b1, b2: ((buff[b1:buff.shape[0]:9] >> 4) | (buff[b2:buff.shape[0]:9] << 4)) & clearing_byte

            R1 = unpack_1(buffer_int, 0, 1) # byte 0 has last 8 bits of red1, byte 1 first 4.
            R2 = unpack_2(buffer_int, 4, 5) # byte 4 has first 4 bits of red2, byte 5 last 8
            R = np.c_[R1, R2].reshape(R1.shape[0] + R2.shape[0])

            G1 = unpack_2(buffer_int, 1, 2) # byte 1 has first 4 bits of green2, byte 2 last 8
            G2 = unpack_1(buffer_int, 6, 7) # byte 6 has last 8 bits of red1, byte 7 first 4.
            G = np.c_[G1, G2].reshape(G1.shape[0] + G2.shape[0])

            B1 = unpack_1(buffer_int, 3, 4) # byte 3 has last 8 bits of blue1, byte 1 first 4.
            B2 = unpack_2(buffer_int, 7, 8) # byte 4 has first 4 bits of red2, byte 5 last 8
            B = np.c_[B1, B2].reshape(B1.shape[0] + B2.shape[0])


        img_data = np.c_[R, G, B]
    else:
        img_data = buffer.payload.components[0].data

# reshape and rescale as 16 bit integer array
img = np.uint16((2 ** 16 - 1) * (img_data.reshape(height, width, 3) / (2 ** bit_depth - 1)))
cv2.imshow("image", img)
cv2.waitKey()

ia.destroy()
h.reset()

Kind regards,

Jesper

3 Likes

Hi Jesper,

thank you! Much appreciated! :slight_smile:

Best regards,
Nico