CVB Device Stream Stop crashing undeterministically

Hi,

We are using cvb-13.04.000 version of the SDK, mainly the C++ API. Based on certain events, our application is starting and stopping the camera streams.

We are facing a problem with our application crashing when we try to stop the cameras through the API call:

Cvb::V_1_5::Driver::StreamPtr->Stream()->Stop();

Note that this is happening undeterministically. For instance, sometimes that crash is happening after 50 cycles of Start/Stop, sometimes 100, sometimes 20.

Here is the gdb stack trace when the crash is occuring:

Thread 47 (Thread 0x7f6de0ff9000 (LWP 106489)):
#0  futex_wait_cancelable (private=<optimized out>, expected=0, futex_word=0x7f6dc4003f38) at ../sysdeps/nptl/futex-internal.h:183
        __ret = -512
        oldtype = 0
        err = <optimized out>
        spin = 0
        buffer = {__routine = 0x7f6f1cc83050 <__condvar_cleanup_waiting>, __arg = 0x7f6de0fd6ee0, __canceltype = -520261920, __prev = 0x0}
        cbuffer = {wseq = 0, cond = 0x7f6dc4003f10, mutex = 0x7f6dc4003ee8, private = 0}
        err = <optimized out>
        g = 0
        flags = <optimized out>
        g1_start = <optimized out>
        signals = <optimized out>
        result = 0
        wseq = 0
        seq = 0
#1  __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x7f6dc4003ee8, cond=0x7f6dc4003f10) at pthread_cond_wait.c:508
        spin = 0
        buffer = {__routine = 0x7f6f1cc83050 <__condvar_cleanup_waiting>, __arg = 0x7f6de0fd6ee0, __canceltype = -520261920, __prev = 0x0}
        cbuffer = {wseq = 0, cond = 0x7f6dc4003f10, mutex = 0x7f6dc4003ee8, private = 0}
        err = <optimized out>
        g = 0
        flags = <optimized out>
        g1_start = <optimized out>
        signals = <optimized out>
        result = 0
        wseq = 0
        seq = 0
#2  __pthread_cond_wait (cond=0x7f6dc4003f10, mutex=0x7f6dc4003ee8) at pthread_cond_wait.c:638
#3  0x00007f6f390ca613 in  () at /opt/cvb-13.04.000/lib/libCVCDriver.so.4
#4  0x00007f6f39157c0c in cvboost159::thread::join_noexcept() () at /opt/cvb-13.04.000/lib/libCVCDriver.so.4
#5  0x00007f6ea3815cd9 in  () at /opt/cvb/drivers/GenICam.vin
#6  0x00007f6ea3809c9c in  () at /opt/cvb/drivers/GenICam.vin
#7  0x00007f6ea3d21a19 in  () at /opt/cvb/drivers/GenICam.vin
#8  0x00007f6ea3d13719 in  () at /opt/cvb/drivers/GenICam.vin
#9  0x00007f6ea3d12fb1 in  () at /opt/cvb/drivers/GenICam.vin
#10 0x00007f6f390f6517 in G2Freeze () at /opt/cvb-13.04.000/lib/libCVCDriver.so.4
#11 0x000055876ac418e7 in Cvb::V_1_5::Driver::Stream::Grab2::TryStop() (this=<optimized out>) at /usr/include/c++/9/bits/unique_ptr.h:360
        result = <optimized out>
#12 0x000055876ac3ed7a in Cvb::V_1_5::Driver::Stream::Stop() (this=<optimized out>) at /usr/include/c++/9/bits/unique_ptr.h:360

Any help is appreciated,
Best regards
Goksan

Hi @goksan23 and welcome to our forum.

Some ideas from my side how to avoid issues with starting/stopping acquisitions.

Use software trigger
It seems that you are starting and stopping the acquisition very frequently. And this is not a good idea in general. If there is enough time between it, this might be OK. But then this should be more than just several seconds.

The better approach would be to leave the acquisition active and use a software trigger to get an image every time you want.

Give the device enough time
You should give the driver and the device enough time to stop the acquisition before you start the next acquisition. This time depends also on the device itself and can’t be specified by us.
Which cameras do you use?

Use newest CVB Version
Additionally, you are not using the newest CVB Version. It might be an idea to try the newest version, which you can download from this forum.

Hi Sebastian,
Thanks a lot for your valuable inputs.

For this test, I was starting stream, capturing for 10 seconds, stopping stream, waiting for 5 seconds and repeating.

Is there a sample code in the CVB SDK that you could point me for the software trigger, just to make sure I am looking at the right thing?

I am using Nano-C2340 cameras.

And will try the newest CVB ASAP.

Cheers

I have a code snippet for you:

#include <cvb/device_factory.hpp> 
#include <cvb/utilities/system_info.hpp> 
#include <cvb/driver/stream.hpp> 

int main(int argc, char* argv[]) 
{ 
    auto path = Cvb::InstallPath(); 
    path += CVB_LIT("drivers/GenICam.vin"); 

    // Configure the TriggerMode of a camera and execute a SoftwareTrigger using GenApi 
    // Tested With Dalsa Genie Nano, names of Nodes may be different on other devices 

    // Load driver 
    auto device = Cvb::DeviceFactory::Open(path); 
    auto stream = device->Stream(); 

    // Load NodeMap for the device 
    auto nodeMap = device->NodeMap("Device"); 

    // First set the TriggerMode 
    auto triggerMode = nodeMap->Node<Cvb::EnumerationNode>("TriggerMode"); 
    triggerMode->SetValue("On"); 

   // Then choose TriggerSource (not available if TriggerMode not set) 
    auto triggerSource = nodeMap->Node<Cvb::EnumerationNode>("TriggerSource"); 
    triggerSource->SetValue("Software");   

    // Start stream to be able to get an image 
    stream->Start(); 

     // Then execute a SoftwareTrigger 
    auto softWareTrigger = nodeMap->Node<Cvb::CommandNode>("TriggerSoftware"); 
    softWareTrigger->Execute(); 

    // Actually wait for the image 
    auto waitResult = stream->WaitFor(std::chrono::seconds(10)); 
    stream->Stop(); 
} 

Hi @Sebastian and @goksan23.

We are observing the same problems in our setup. The stream abortion / stopping does not work as reliable as expected. If it takes too much time to close the stream, every access over the api (in our case: a user set change) leads to a crash of our laser sensor. Following, the sensor is not accessible anymore and a hard reset is the only solution.

Since we need to apply a frequent change of the user sets (different dimension of the test objects), the stream needs to be stopped in less than a second, and started again after the new user set was applied…

@Sebastian, all of your suggestions make sense, but in our industrial use case, we need to adapt to the production flow, but the only reliable solution seems to be: “Give the device enough time”. As a consequence of this, we need to implement a complicated communication in our production facility, that ensures, that the test object is not inserted in the laser measurement unit untill the stream was stopped, the user set was changed, and the stream was started again.

In fact we had expected, beeing able to use CVB and the laser sensor without any interruption, but continuously.

Hope, our feedback was useful for you @goksan23.

Abort/Stop
As you wrote abortion / stopping I first want to clearify that it is recommended to do abort the acquisition instead of stopping it.

Stop means that the call waits for the current image to be acquired. Which takes time and depending on the status of the camera can take until a defined timeout.

Abort directly aborts the acquisition and returns when the camera signals us that the acquisition is stopped.
Then you would be able to directly set the user sets.

Setting the User Sets
Keep in mind that setting the user sets can take time as well which we can´t influence from the software side.
You write that after setting user sets and starting the acquisition the camera crashes.

This leads me to the assumption that the camera handles its own commands not correctly.

If the camera was purchased from us, you can contact your local Stemmer Imaging support like de.support@stemmer-imaging.com in the DACH region.
If we can reproduce it we can contact the supplier.

Aquisition Start
Restarting the acquisition takes some time as all the acquisition buffers need to be reallocated. Especially using the old acquisition stack (VIN) it is slower than using the GenTL acquisition stack. With a low number of buffers the difference is very low. with a higher number this could make a difference.

Acquisition Stacks and Flow Set Pool
If you want to speed up the start of the acquisition even further you can give CVB an external memory (External Flow Set Pool) which is then used as the image queue.
If this is interesting for you take a look in this document and look for “External Flow Set Pool”:
Common Vision Blox: Migration Guide for the Acquisition Stacks
This document also describes the differences between the old and new acquisition stack and how to use it.