AT and stack 3 cause acquisition problems

Programming Questions C++ API
,
1

Hello CVB team,

Hope you are doing well.

Since more than one year now, I use the stack 3 to take pictures on our main vision products. It use two AT (mostly C5 SCT) cameras that reconstruct huge carts to control components on it.

We set up the AT to take fixed size images with a maximum of profile (5400). We also open all Ethernet board parameters like Jumbo packets to there maximum. We set also the scsp packet size to it’s maximum.

With the stack two, no problem at all. When we use stack 3 we can see this :

when we should have this :

We also understand that when this happen, we got verry small pikes (in red in the image) contrary to the huge pikes that we have normally (in green) with the task manager :

image
image920×820 95.9 KB

When we switched from stack 2 to stack 3 or the inverse, no parameters are changed.

The stack 3 used a composite image to get data. Both camera are on the same Ethernet board, without any other element on it.

We suspect that either we get a improper network settings or a problem with stack 3 like bad uses or missing a parameters that need to set up ? Or have you any ideas ?

2

Hi @AxelDosSantosDoAlto,

With our experience of the new GenTL stack (3rd Gen stack) I expect no difference in performance or stability.
The first image clearly shows lost packets, which you also should see when checking for corrupt frames or Incomplete frames. if you want to do it programmatically.

I strongly suspect a setting which is different in these two setups. And the one which is set from CVB is the PacketSize. So this would be the first thing to check. Has the packet size the expected value?
There is a difference how the packet size can be changed during the startup of the VIN-Stack and the GenTL-Stack.

With the VIN-Stack and loading the GenICam.vin file, you use the setting from the INI-File. Mostly configured over the GenICam Browser.

If you use the GenTL-Stack you have to set the feature programmatically. e.g. directly via the “GevSCPSPacketSize” feature.

p.s.
Please check if you are allowed to post these images publicly. If not, you should delete them.

3

Hello @Sebastian

Thanks for your quick answer.

As I wrote on my message, we do set up the scps packet size at is maximum like this, directly after connection to the device and get the nodes we need :
m_node_scpsp_packet_size->SetValue(8192);

Unless we need to do it at another point, normally we are ok on this.

4

Yes, this should work. Additionally you can check for errors and if the value was set correctly.

5

@Sebastian
We already put this value on our code. And we have the problem.
We will check the value for each packet we will received.
How can we check for errors ?

6

What I meant with checking for errors in my last post was using try/catch if something goes wrong during access to the node or value of the node.

7

Hi @AxelDosSantosDoAlto

any news on this?
Did you find what caused the differences?

Cheers
Chris

8

Hi @Chris
Sorry for my late answer, I was on sick live.
We planned test with our client during August, so I cannot give you news for now.
I will give you that when I can.

9

Hello @Sebastian , @Chris ,

I was able to do some test today about this problem directly with our client system. It take some time to plan that for different reasons, that is why I write about it now.

So, we check that the node scpsp packet size is always the full value, and if there is a problem accessing it, to have an alarm about it. So far, no problem, we can set it each time we are connected to the camera.

Also, I’m coding now (after this message) the cheking of missing packet value each time we received a image from the camera. I will tell you what happen with that as soon as possible.

For now, as far as I am in my test, I always have the problem (the images can be display here the time of the resolution of the problem, so no worries about it) sometime :

image
image731×858 78.1 KB

image
image755×883 76.3 KB

Beside the scsp packet size, on the ethernet board, we set up the jumbo packet to the maximum, the reception and transmission data to the maximum too.

As a reminder, we received N small images (20 profils or pixel in height) and we reconstruct the all images of 4900 profiles/ pixel in height. We set the CVB buffer to N = 4900 / 20 = 245 small images, for a production time of 8 secondes (each 8 secondes we get a trigger that inform us a new product is here to see). The internal CVB buffer are for now 245 * 3 =~750 buffer of 20 profiles for one camera. We use two AT C5 SCT cameras.

Hope those information will help you tell me what the problem is.

10

Hi @AxelDosSantosDoAlto ,

first of all thank you for the detailed description, I will check on this with @Sebastian in more depth today.
Allow me some additional questions for now:

  • Do you see this behaviour with each Image or only occasionally?

  • There are 2 Packet Sizes that have to be taken into consideration, one that you set on the "Device Nodenmap, which is very likely the “GevSCPSPacketSize”.
    Then there is another PacketSize in the “DeviceTL” Nodemap. This one is to configure the transportlayer. Usually CVB does the configuration of the TL for you and only setting the correct value in the Device Nodemap should be enough. However I have seen cases where this was not the case.
    So just to finally rule out the packet size, after setting it on the device, you could read out the setting from the DeviceTL Nodemap as well just to be sure.

  • PacketTimeout
    In the DataStream Nodemap (yet another Nodemap available through the NodeMaps on the discovered device) there is a PacketTimeout value which is usually set to 200. In the past we had customers with similar behaviour as you have where setting this value either to 0 or a very high value (to pretty much have an endless value) this behaviour was fixed.
    What happened was, when the device did not get any new triggers but the image was not yet ready with being taken, the already taken packets were sent over to the PC containing old image data from the buffer that was not yet overwritten due to the interrupt because of the missing trigger along with the PacketTimeout pushing the camera to send the data.

    This happens usually if you have a conveyor belt or a camera scanning along an object being triggered by a rotary encoder or PLC. Now at some point you might have to stop your belt or scan to avoid collision in the following process along the belt. In these cases, once the triggering stopped for a short time, the camera immediately sent the data to the processing machine.

    If I interprete your images correctly, this happens not only once. At a certain point in the scan process (you seem to be scanning horizontally with the camera moving from left to right) the trigger might start stuttering, meaning we have a gate high which is interrupted for a short time exceeding the packet timeout. This happens with the next frames again and again leading to this staircase kind of blocks we see.

    This is just a rough guess but if you change the PacketTimeout to be indefinite OR monitor your trigger signal (accuracy of measurement method < PacketTimeout) we can validate if we see interrupts in the signal which have to be taken care of.

    I am looking forward to hearing from you, the problem seems complex but I am sure we will find a way to solve this.

    Cheers
    Chris

11

Hello all, hello @Chris, hello @Sebastian

Sorry again for the delay, but I had to juggle between tests for this and others subjects !

So, after a few tests, we were able to find a solution to our problem ! Finally !

I have 3 systems that show us the problem I spoke about previously. Two systems use 2 AT C5 SCT cameras, and the third one use 3 AT C5 SCT.

The first test we did was to wait for the problem to come, then to disconnect any ethernet cables connected to the computer, beside the cameras. There we saw that having only the cameras do not create the problem on one system with two cameras and one system with three cameras, and create it on one system with two cameras.

After this test, we try to do what you talk me about, checking or changing node, but nothing happen. We delete those change after this, so that our last test will not interfere with those changed.

Last test comming, we try to disable the network throlling index in the registry base, who is 10 basically for all standard Windows OS, by putting the value to FFFFFFFF.

And… It work ! No more stripped images since more than 24 hours on our three systems.

So, I understand that stack 3 used a more generic way to communicate with the camera (in my memory stack 2 was CVB proprietary way to communicate, with access to one kind of data from the camera at the time), and because stack 3 can give us multiple data in one shot, so that is why we didn’t understand why the problem was on stack 3 and not and stack 2…

Anyways, did you use on some cases this network throlling index ?

It will help me to have more information about difference in stack 2 and 3, and why stack 3 have this problem, if you can tell me. And do you know if we can have a similar solution with CVB setup that did’n’t require to change network throlling index in the registry base ?

Thanks again for your time.

I will wait for your answer, whatever it is, to mark as resolve this discussion.

1 Like
12

Hi @AxelDosSantosDoAlto

thats great news! I am happy, that you found a way to get your system up and running.

About the details in the AcquisitionStack: @s-woe or @Andreas could you provide more information on this topic or generally jump in with some comments about what @AxelDosSantosDoAlto found out here?

Thank you and cheers
Chris

1 Like
13

Hi @AxelDosSantosDoAlto,

we do not use or change the NetworkThrottlingIndex in our software, neither in the old nor in the new stack.
To my knowledge we never saw a system where disabling was required. Nevertheless, I would assume that you system was running on it’s limit and just needed a little extra to regain stability.

Be aware that disabling it may have side effects, depending on what else is running on the system.

Thank you for your feedback.

14

Thanks for your answer @Andreas. The next step for us is to see if by changing the value of this parameter from 10 to something bigger instead of disable it will help or not. I will let you know here if we can find something interesting.

And, if you have a way to manage this problem without changing the network throlling index, I am all ears. :slight_smile:

1 Like