Acquiring images > 8-bit using CVBpy

Hello,

I want to acquire images in 10 or 12 bit from a jai Fusion camera. I can acquire 8 bit images without a problem but as soon as I change the PixelFormat.values to “BayerRG10” or “BayerRG12” for the color sensor or “Mono10 or 12” for the monochromatic sensors the images show a plain image of zeros if I plot the MultiPartImage using matplotlib.pyplot.imshow(). If I use cvb.as_array on the MultiPartImage I receive the following error message:

File "...\cvb\__init__.py", line 368, in as_array
    planeCount = len(image.planes)
UnboundLocalError: local variable 'image' referenced before assignment

I’m aware that the camera only allows 12bit images being captured when VideoBypassMode is enabled, but enabling doesn’t change anything in Python. I can however display data in >8bit when I use the GeniCamBrowser.

I’m using cvb v13.04.005, python 3.8.3 and a jai FS-3200-10GE-NNC on a Windows 10 mashine.

Thank you for your help!

Hi @JRo85 !
Please provide a minimum working example of the task you want to accomplish.
The error slightly indicates faulty use of Python and/or CVBpy.

Hi,

sure! Here is a bit of code that should ilustrate the issue:

import matplotlib.pyplot as plt
import cvb

discover = cvb.DeviceFactory.discover_from_root(flags=cvb.DiscoverFlags.IgnoreVins,)
device_info = discover[0]
device_info.set_parameter("PacketSize", "8192")
dev = cvb.DeviceFactory.open(device_info.access_token, cvb.AcquisitionStack.GenTL)
dev_node_map = dev.node_maps["Device"]

stream = []
PixelFormatlist = ["BayerRG8","Mono8","Mono8"]
for n in range(3):
    stream.append(dev.stream(cvb.ImageStream, n))
    dev_node_map["SourceSelector"].value = "Source%s"%n
    dev_node_map["PixelFormat"].value = PixelFormatlist[n]
    dev_node_map["GevSCPSPacketSize"].value = 8192

img = [[],[],[]]
for n in range(3):
    stream[n].start()
for n in range(3):
    img[n],_,_ = stream[n].wait()

fig, ax = plt.subplots(1,3, figsize=(18,4))
for n in range(3):
    ax[n].imshow(img[n])
plt.show()

np_img = cvb.as_array(img[0])

When changing the elements in PixelFormatlist to [“BayerRG10”, “Mono10”, “Mono10”], matplotlib shows a plain image of zeros and the beforementioned error in cvb.as_array occurs.

First: I would recommend to put your acquisition into a context in order to guarantee the correct lifetime of acquisition related objects:

with cvb.DeviceFactory.open(device_info.access_token, cvb.AcquisitionStack.GenTL) as dev:
  dev_node_map = dev.node_maps["Device"]
  [...]

Check whether the set of the pixel format is successful (by debugging). Also let the driver react to the change of pixel format, which obviously changes the data size:

[...]
for n in range(3):
    stream.append(dev.stream(cvb.ImageStream, n))
    dev_node_map["SourceSelector"].value = "Source%s"%n
    dev_node_map["PixelFormat"].value = PixelFormatlist[n]
    stream[n].deregister_flow_set_pool() # this helps to update the flowsetpool accordingly
[...]

You may have a look at the DeviceConfiguration sample program (included starting from CVB 14.00.000) in %CVB%\Tutorial\Image Manager\CvbPy\DeviceConfiguration

Thank you! I changed my code according to your suggestions with no effect on the outcome. Setting the PixelFormat seems to be successfull as the img[n].planes[0].data_type.bits_per_pixel returns the value 10 and max_val 1023.0 for all three MultiPartImages in img, when I use “BayerRG10” and so on and bits_per_pixel = 12 and max_val 4095 if using “BayerRG12” and so on.
The waitStatus after img[n] return the Value 0. So this should be fine as well.
The data are still missing… is it possible that this is a bug of my cvb version?
On your homepage my version of cvb 13.04.005 seems to be the most recent one.

The most recent version is 14.00.000 (here for Linux, here for Windows).

Just to be clear, the changes should be:

import matplotlib.pyplot as plt
import cvb

discover = cvb.DeviceFactory.discover_from_root(flags=cvb.DiscoverFlags.IgnoreVins,)
device_info = discover[0]
device_info.set_parameter("PacketSize", "8192")
with cvb.DeviceFactory.open(device_info.access_token, cvb.AcquisitionStack.GenTL) as dev:
    dev_node_map = dev.node_maps["Device"]
    
    stream = []
    PixelFormatlist = ["BayerRG8","Mono8","Mono8"]
    for n in range(3):
        stream.append(dev.stream(cvb.ImageStream, n))
        dev_node_map["SourceSelector"].value = "Source%s"%n
        dev_node_map["PixelFormat"].value = PixelFormatlist[n]
        dev_node_map["GevSCPSPacketSize"].value = 8192
        stream[n].deregister_flow_set_pool() # this helps to update the flowsetpool accordingly
    
    img = [[],[],[]]
    for n in range(3):
        stream[n].start()
    for n in range(3):
          img[n],_,_ = stream[n].wait()
      
    fig, ax = plt.subplots(1,3, figsize=(18,4))
    for n in range(3):
        ax[n].imshow(img[n])
    plt.show()
  
    np_img = cvb.as_array(img[0])

As a last suggestion, you could try to copy the image out of the array:
Change the last line to:

np_img = cvb.as_array(img[0].copy())

If that doesn’t help, you may get some more detailed support at our support, by mentionning this thread here.

Hi s-woe,

it seems indeed to be a bug of version 13.04.005. After updating to version 14.00.000 it sort of works. However, it works only for stream[1] and stream[2]. Weirdly stream[0] still shows the plain image filled with 0s and isn’t able to return an array via cvb.as_array(). Shall I go to your support with this?

Okay, great. Which camera do you use and what are your remaining setup details (OS, arch, python version)?

I’m using a jai FS-3200T-10GE-NNC on a Windows 10 (10.0.19044) and Python 3.8.3. Let me know if you need anything else.

Hi @JRo85 ,

your code should work when cloning the image directly after the wait. The cvb.as_array works as a view on buffers which may not exist anymore lateron. I would suggest to write something like:

(...)
    stream = []
    PixelFormatlist = ["BayerRG8","Mono8","Mono8"]
    for n in range(3):
        stream.append(dev.stream(cvb.ImageStream, n))
        dev_node_map["SourceSelector"].value = "Source%s"%n
        dev_node_map["PixelFormat"].value = PixelFormatlist[n]
        stream[n].deregister_flow_set_pool() # this helps to update the flowsetpool accordingly
        
    img = []
    for n in range(3):
        stream[n].start()
        image,state,nodemaps = stream[n].wait()
        np_image = cvb.as_array(image.clone(), copy=True)
        img.append(np_image)

    fig, ax = plt.subplots(1,3, figsize=(18,4))
    for n in range(3):
        print(img[n].dtype)
        ax[n].imshow(img[n])
    plt.show()

Hi s-woe,

thanks for your answer. Indeed, I get images from all three sensors using this method. But something still goes wrong with Sensor0 in case of bitdepth >8bit. While sensor1(middle) and sensor2(right) show the expected image, I get some weird response from sensor0(left).


In the 8bit case everything looks normal.
Also, in case of >8bit a simple cvb.as_array(image) does the trick to go from MultiPartImage to numpy for sensors 1 and 2, while doing the same for sensor0 triggers the error message:

  File "...\cvb\__init__.py", line 386, in as_array
    plane_count = len(image.planes)
UnboundLocalError: local variable 'image' referenced before assignment

At least for your second problem

should be the trick…?

This is how the picture in my last post was generated. I used cvb.as_array(image.clone(), copy=True) to extract the images directly after stream[n].wait(), but the image from sensor0 (the left one of the 3 images) shows this weird tiled image. I guess the tiles have something to do with the Bayer-Pattern of the sensor, but still only the upper two show the actual scene, the lower tiles are just vertical lines. If I use 8-bit for all 3 channels (using cvb.as_array(img),not cvb.as_array(image.clone(), copy=True), the images look as expected:


To make things more complicated, if I set sensor0 to 8bit and sensors1 and 2 to 12 bit. I get the result using cvb.as_array(image.clone(), copy=True):

Okay, I guess, we might have to dive a little deeper into that. As soon as we found the problem, we will return to you.
Cheers.

Ok! Thank you! Let me know if I can do something to support you.

Hi @JRo85 , you spotted a bug, which will be fixed in the next service release. Thanks for that!