Create image for processing from two ring buffer images

Programming Questions
,
1

Hello CVB,

C# programmer!

I would like to perform some image processing on an image that is a combination of two images created from a GigE camera, using the GenICam ring buffer.

The new image should be made up from: all of image 1 and x number of lines from image 2. When the camera grabs the next image it should do the same. The code snippet posted below seems to work well when I load the images manually.

I would like to include this functionality in a custom user control class, which I have created (with the help of a very helpful person) that contains an AX Display, and uses the image library to manage the image. This class has been created so I can multi-thread on acquisition, image handling and processing which are all functions of the user control class. I have tried to implement the code in an AquistitionThread method below. This has not yielded the results I expect as the image combination appears to be out of sync. I am using the GEVServer example and GenICam driver to feed in my test images. My GenICam driver image buffer is set to 2.

I feel like I am almost there but I am probably miles off.

I hope I have explained this well enough?!

    private Cvb.Image.IMG Combine2BufferImages(Cvb.Image.IMG img1)
    {

        Cvb.SharedImg _image0;
        Cvb.SharedImg _image1;

        _area1.X0 = 0;
        _area1.X1 = Cvb.Image.ImageWidth(_hImg);
        _area1.X2 = 0;
        _area1.Y0 = 0;
        _area1.Y1 = 0;
        _area1.Y2 = Cvb.Image.ImageHeight(_hImg);

        _area2.X0 = 0;
        _area2.X1 = Cvb.Image.ImageWidth(_hImg);
        _area2.X2 = 0;
        _area2.Y0 = 0;
        _area2.Y1 = 0;
        _area2.Y2 = _LinesToAppend;

        Cvb.Driver.IRingBuffer.RBGetBufferImage(img1, 0, out _image0);
        Cvb.Driver.IRingBuffer.RBGetBufferImage(img1, 1, out _image1);

        Cvb.Image.CreateCompatibleImage(img1, (int)Cvb.Image.ImageWidth(img1),
            (int)Cvb.Image.ImageHeight(img1) + _LinesToAppend, false, out _hImgCombine);

        Cvb.Image.CopyImageArea(_image0, _hImgCombine, 0, 0, _area1, 0, 0);
        Cvb.Image.CopyImageArea(_image1, _hImgCombine, 0, 0, _area2, 0,
            (int)Cvb.Image.ImageHeight(img1));

        _image0.Dispose();
        _image1.Dispose();

        return _hImgCombine;


    }


    private void AcquisitionThread()
    {
        if (Cvb.Driver.IGrab2.G2Grab(_hImg) != 0)
            return;

        while (Grab)
        {
            var success = (Cvb.Error.CVC_ERROR_CODES)Cvb.Error.CVC_ERROR_FROM_HRES(Cvb.Driver.IGrab2.G2Wait(_hImg));

            if (success == Cvb.Error.CVC_ERROR_CODES.CVC_E_OK)
            {


  
                    _cvDisplay.Image = combine2bufferimages(_hImg);
                    _cvDisplay.Refresh();
                



            }

        }

        Cvb.Driver.IGrab2.G2Freeze(_hImg, true);

    }
2

Hi @BBQRibs,

I use the CVB.Net API to answer your question. Why? Because you have a complex use case which becomes much more complex with the C-style API :slight_smile:. Without further ado here the combine method:

Image CombineImages(Image imageTop, Image imageBottom, int overlappingLines)
{	
  var result = new Image(imageTop.Width, imageTop.Height + overlappingLines, imageTop.Planes.Count, imageTop.Planes[0].DataType);
  imageTop.CopyTo(result);
  imageBottom.CopyTo
  (
    result, 
    new Rect { Left = 0, Top = 0, Width = imageBottom.Width, Height = overlappingLines }, 
    new Point2D { X = 0, Y = imageTop.Height }
  );
  return result;
}
I assume the following prerequisites (open if you are interested) 
Debug.Assert(imageTop != null);
Debug.Assert(imageBottom != null);
Debug.Assert(overlappingLines >= 0);
Debug.Assert(imageTop.Width == imageBottom.Width);
Debug.Assert(imageBottom.Height >= overlappingLines);
Debug.Assert(imageTop.Planes.Count == imageBottom.Planes.Count);
Debug.Assert(imageTop.Planes.DataTypesIdentical && imageBottom.Planes.DataTypesIdentical);
Debug.Assert(imageTop.Planes[0].DataType == imageBottom.Planes[0].DataType);

The implementation does not assume anything special about the input images. The ringbuffer handling is done in the acquisition method:

using (var device = OpenFirstDevice())
{
  if (device == null)
  {
    Console.WriteLine("Please attach a device");
    return;
  }
	
  var stream = device.Stream;
  var ringBuffer = stream.RingBuffer;

  // we always use two images of the ring buffer, thus double its default
  ringBuffer.ChangeCount(6, DeviceUpdateMode.UpdateDeviceImage);
  // as we always use two buffers at a time we want to handle locking manually
  ringBuffer.LockMode = RingBufferLockMode.On;
	
  stream.Start();
  try
  {
    var predecessorImage = await stream.WaitAsync() as RingBufferImage;
		
    // this can be run in the ui thread
    for (int i = 0; i < 10; i++) // for demo purpose we do 10 frames
    {
      var currentImage = await stream.WaitAsync() as RingBufferImage;
      var combinedImage = CombineImages(predecessorImage, currentImage, overlappingLines: 100);
      predecessorImage.Unlock();
				
      Task.Run(() =>
      {
        // do your parallel processing 
        combinedImage.Dispose();
      });
    
      predecessorImage = currentImage;
    }
  }
  finally
  {
    stream.Abort();
  }
}
OpenFirstDevice uses device discovery.

In this case for loop-back :cvb: GevServer:

Device OpenFirstDevice()
{
  using (var devices = DeviceFactory.Discover(DiscoverFlags.IgnoreGevFD | DiscoverFlags.IgnoreVins))
  {
    if (devices.Count > 0)
      return DeviceFactory.Open(devices[0]);
    else
      return null;
  }
}

Two important notes:

  1. Do not reduce the ringbuffer buffer count below 3:
    You need at least one buffer being able to be filled, one to await processing/being filled and one currently being processed.
    If you lock more than one or have jitter in processing you need to increase the buffer size to not loose images.
  2. Do not use the fixed indices of the ringbuffer:
    The order can deviate from 0,1,2,…,n,0,1… Also it might not start at 0 depending on the driver used.
    The beauty of the new OOP interfaces (CVB.Net, CVB++ and pyCVB) is that the .Wait methods return RingBufferImages if the driver provides them. Thus you have the correct order and can simply .Unlock() them when you are done with processing.
4 Likes
3

Hi,

Thanks for this will give it a try :slight_smile: