Difference between Cvb.SharedImg and Cvb.Image.IMG

Hi,

I made a small application in C#.

I’m using CVB library to grab (Driver.IGrab2.G2Grab(m_Camera)) an image from a camera device.
var result = Driver.IGrab2.G2Wait(m_Camera);

When I received an Image from the device, I would like to process the image.
The point is that I don’t understand the meaning of SharedImg.
Is it the shared image that I grabbed from the camera, stored in Shared Memory object?
The only thing that I understand about SharedImg is that I don’t need to release the image (as mentioned in this thread What is the best way to copy an image)
I cannot find explanation of the SharedImg in the documentation.

In case it is stored in a shared memory object that mean is not safe to do operation on this image.
How can I copy the image that I grabbed from the camera in an other new Image object?

I tried to use Image.CreateDuplicateImageEx(m_Camera, out Cvb.Image.IMG capturedImg) the result of the function is success, but I cannot access linearly to the pixel of the image. That mean the copied image is not working properly.

Thank you in advance.

Hi Andrea,

SharedImg
The SharedImg is not shared in a shared memory object. It is more an object oriented representation of the original IMG Object. The SharedImg Object and all other CVB Shared .NET Objects are meant to be used as a replacement of the original objects like the IMG Object to be able to use typical .NET syntax like .Dispose instead of calling a ReleaseObject on an IMG object.

I don´t see that the SharedImg is mentioned in the thread What is the best way to copy an image
Be careful to think that you do not need to release the image if a SharedImg is used. It is correct that with a SharedImg the .NET garbage collector releases the object when it is not needed any more. But the garbage collector does not have the information how big the object is and therefore how important it is to release it early enough. For the garbage collector the size of the object is only a pointer. And then the garbage collector might release the memory sometime in the future or never as this seems to be not important.

If you process images and you get a new SharedImg as result you should dispose it at the moment you do not need the image any more. Otherwise the memory can rise and rise continuously even with SharedIMG Objects.

Acquisition and Processing
If you get a positive result (zero) from the G2Wait call you can safely process your image over the m_Camera variable until you call G2Wait the next time. The next call to G2Wait unlocks the buffer with the old image in the video interface driver for further acquisition and locks the new buffer with the new image which is then available over your m_Camera variable.

You should not call ReleaseObject or Dispose on your m_Camera Object during your processing. This object was created with LoadImageFile and represents the driver instance and the current image. The m_Camera Object should be released when you do not need access to your camera any more. For Example when closing your application.

1 Like

Hi @andrea.annovi, @Sebastian explained quite nicely the usage of Cvb.SharedImages. I just want to add some theory: :nerd_face:

We use reference counting for a lot of :cvb: object types. This includes all kind of images (we call them IImageVPA internally). The IMG is just a handle (a kind of pointer to a publicly unknown data type object) that references an IImageVPA object. And that object has a reference count. Thus all of our objects live on the free store (like .Net reference types or boxed value types).

This is a shared ownership life-time model. Every component that is interested in using this object calls ShareObject and if the reference is not needed anymore calls ReleaseObject. Also every create (e.g LoadImageFile) must also be paired with a ReleaseObject call – this is the initial share. If this contract is not fulfilled you either leak resources or use destroyed objects.

Theoretically this is a simple means of lifetime management. Our library handles how to create and destroy objects and you just tell it when. The downside of this nice mechanism is that it is a manual one. And in case of .NET it also isn’t the the way you would manage resources.

The name SharedImg is inspired by C++'s std::shared_ptr<T> which lets the compiler handle the reference count management. In .Net one does pure memory management via the garbage collector and more specific resource handling via IDisposable. The SharedImg itself is not the actual IImageVPA object, but just a facade that translates the reference count scheme into .Net’s IDisposable scheme.

Every method creating reference counted objects is overloaded in our C-style .Net API returning either a handle (like IMG) or its shared variant (SharedImg). We encourage you to use the later one. Also the functions (or to be more accurate methods) just using an IImageVPA object still only use the handle type (IMG) as they don’t take part in ownership. The SharedImg implicitly decays to an IMG type for the purpose of calling these methods with a SharedImg.

:warning: Don’t mix manual ShareObject/ReleaseObject with the automatic SharedImg handling!

Sorry for the longish answer, but I hope that makes it more clear. :innocent:

2 Likes

Hi @andrea.annovi

I think @Sebastian and @parsd have covered the topic of why the SharedImg is called SharedImg (sorry for the confusion this name has apparently caused).

I’d like to address the issue you raised in the last paragraph about the output generated by Image.CreateDuplicateImageEx. I am a bit puzzled because - like you - I would have expected the result image to be linearized, in which case Utilities.GetLinearAccess should return true and give you the base pointer and the increments needed to access the pixel data. To further investigate this: Can you call Utilities.AnalyzeXVPAT and Utilities.AnalyzeYVPAT on capturedImg and check the return values (and post them here)? Also a code snippet that shows what you are doing would be good.

Additionally you might want to have a look at this post (specifically the 2nd half), because often the image would not even need to be copied - the image data will remain intact until the next IGrab.G2Wait call returns (and copying can be a bit of a performance hog).

1 Like

Thank you for the answer @Sebastian, @parsd, and ‘illusive’. All your answers were really helpful.

At the end I succeeded in creating a duplication of the original image, captured by the camera, using the function: Image.CreateDuplicateImageEx. and I’m also able to access linearly GetLinearAccess.

In my program I used SharedImg object to load the camera driver and then G2Wait and Grab image.
All works fine, but in a separate task I want to process the image, because in case the processing function takes really long time, I don’t want to block the main thread that is acquiring data.

Therefore I decided to used Image.CreateDuplicateImageEx. to create a duplication of the image, acquired by the camera, and to use the duplicated image to process (eg. GetLinearAccess and other stuff).
In this way the main thread that acquired image from the camera, does not have to wait for the processing function to complete.
To avoid memory leak I also have to release the duplicated image after the processing part it has been completed.

I do not release the camera object during acquisition, but I only dispose the camera object when the acquisition has stopped.

Is it correct what I have implemented?

Thank you in advance.

3 Likes

The way you handle your application looks correct to me. If you dispose the driver IImageVPA you unload the driver, release the camera and free the driver’s ring buffer memory (if nobody else is sharing the ownership :wink:). If you want to acquire again within your app, it would be faster to not unload the driver. As otherwise you would need to reload everything again from scratch.

IRingBuffer for Task-based Processing
Also as @illusive hinted, you can get rid of the costly duplication step. In :cvb: we use the IRingBuffer interface for your use case. This is the companion interface to IGrab2, which you probably used to acquire the images.

For this use case you probably need to increase the number of driver buffers via the driver’s ini-file or programmatically via the RBNumBuffer function with RINGBUFFER_NUMBUFFER_CMD_SET as Action. This will create a new driver IImageVPA with the desired number of buffers if successful. The old IImageVPA object stays intact until you release it. Only all driver interfaces were moved to the new handle. This enables you to e.g. finish processing while switching to a different mode in your app.

After you have your desired number of buffers, you can set the lock mode to RINGBUFFER_LOCKMODE_ON via the RBLockMode function. This is a manual mode in which the acquisition engine only locks the buffers to write-protect it for the engine. You as the user must unlock the buffer via RBUnlock when you don’t need it anymore. Otherwise no buffers are available anymore after the ring buffer is filled and no new images can be acquired.

With RBBufSeq(m_Camera, 0, bufferIndex) you get the buffer index of the last awaited buffer (G2Wait). With RBGetBufferImage and this index you get a view (no copy) on that buffer as a :cvb: image. For more information about RBBufSeq see this post.

Instead of your copy, you can give that ringbuffer image to your processing task(s). After processing is finished you can unlock it (but note, that RBUnlock works on the driver image m_Camera). Whether you release the ringbuffer image every time or cache these and free them after the acquisition is finished is up to you. The ringbuffer images stay valid until released. And as long as you don’t change the number of buffers or their size (IImageRect}), they stay connected to the acquisition engine.

I normally bind together the ringbuffer image, the driver image, the buffer index (the last two for unlocking) and my own ref count initialized to the number of tasks running on the image. Every task decreases the count by one when finished and then unlocks (and maybe releases) the ringbuffer image if that count reaches zero. A Parallel.For for example wouldn’t need the ref count as you have a sync point after the loop finished.

Again a longer post… This is a bit more complicated programming wise, but yields better performance in the short (no copy) and long run (less memory fragmentation). And yes, this scenario including ReleaseObject on the ringbuffer images is thread safe (as long as the driver image stays alive). I hope that helps.

2 Likes

As a postscript: if this sounds too cryptic I can write an example for you next week. (When I have more than my mobile phone with me… :wink:)

Hi @andrea.annovi!

Thanks for the feedback. It’s good to hear you succeeded. The approach you describe sounds reasonable enough. As you are using the SharedImg I suppose that by “… have to release the duplicated image…” you mean calling the Dispose method - which is good.

What @parsd describes is a good approach if you are running low on CPU resources. If that’s not the case, Image.CreateDuplicateImageEx is a valid (albeit slightly wasteful) approach and you can work with it. Just keep in mind that result display from within a worker thread requires Invoke or BeginInvoke… (sorry for bringing up banalities :blush:).

Just to be sure: It is not necessary to dispose the camera handle after acquisition has stopped if you plan to restart acquisition later on. You can stop and restart acquisition on the same handle as often as you like, it’s not necessary to unload (i. e. dispose)/reload the driver every time.

1 Like

Thank you for the nice explanation @parsd. I’m making an application that should be as light as possible in term of Memory and Cpu used. It sounds really great your approach and I wasn’t aware of this kind of solution to solve my problem.

Yest it will be really useful if you can provide me an example.
Thank you in advance.

1 Like

Ok, here is the promised example:

const int NumDriverBuffers = 100;

static void Main()
{	
  var driverPath = Path.Combine(Environment.GetEnvironmentVariable("CVB"), "Drivers", "GenICam.vin");
  using (var camera = DriverExtensions.TryOpenWithBufferCount(driverPath, NumDriverBuffers))
  {
    if (camera == null)
    {
      Console.WriteLine("Error loading driver: " + driverPath);
      return;
    }
		
    camera.SetLockModeToOn();

    camera.StartAcquisition();
    try
    {
      for (int i = 0; i < 2 * NumDriverBuffers; i++)
      {
        var image = camera.WaitForNextImage();
				
        // for a real application the tasks should be stored for proper app-exit handling
        Task.Run(() =>
        {
          Parallel.For(0, image.Height, line =>
          {
            // do something
          });
          image.Unlock(); // and thus dispose...
        });
      }
    }
    finally
    {
      camera.AbortAcquisition();
    }
  }
}

And for this you need these helper classes (and the references to iCVCImg.dll and iCVCDriver.dll):

class RingBufferImage : IDisposable
{
  private Cvb.Image.IMG _driverImage;
  private Cvb.SharedImg _bufferImage;
  private int _bufferIndex;

  public static RingBufferImage FromCurrent(Cvb.SharedImg driver)
  {
    if (driver == null)
      throw new NullReferenceException(nameof(driver));

    return new RingBufferImage(driver, driver.GetCurrentBufferIndex());
  }

  private RingBufferImage(Cvb.SharedImg driverImage, int bufferIndex)
  {
    _driverImage = driverImage;
    _bufferImage = driverImage.GetBufferImage(bufferIndex);
    _bufferIndex = bufferIndex;
  }
	
  ~RingBufferImage()
  {
    Dispose(disposing: false);
  }
	
  // Simply a readable alias for dispose.
  public void Unlock()
  {
    Dispose();
  }
	
  public void Dispose()
  {
    Dispose(disposing: true);
    GC.SuppressFinalize(this);
  }

  protected virtual void Dispose(bool disposing)
  {
    if (disposing)
    {
      _bufferImage?.Dispose();
      Cvb.Driver.IRingBuffer.RBUnlock(_driverImage, _bufferIndex);
    }
  }
	
  public static implicit operator Cvb.Image.IMG(RingBufferImage image)
  {
    return image._bufferImage;
  }

  public int Width
  {
    get { return Cvb.Image.ImageWidth(this); }
  }

  public int Height
  {
    get { return Cvb.Image.ImageHeight(this); }
  }
}
static class DriverExtensions
{
  // Opens a driver with a given number of buffers; null on error.
  public static Cvb.SharedImg TryOpenWithBufferCount(string path, int numBuffers)
  {
    var driver = TryOpen(path);
    if (driver == null)
      return null;

    if (driver.GetBufferCount() == numBuffers)
      return driver;

    try
    {
      return driver.CreateWithNewNumberOfBuffers(numBuffers);
    }
    finally
    {
      driver.Dispose();
    }
  }

  // Opens a driver; null on error.
  public static Cvb.SharedImg TryOpen(string path)
  {
    Cvb.SharedImg driver;
    Cvb.Image.LoadImageFile(path, out driver);
    return driver;
  }

  public static int GetBufferCount(this Cvb.SharedImg driver)
  {
    if (driver == null)
      throw new NullReferenceException(nameof(driver));
    if (!Cvb.Driver.IRingBuffer.CanRingBuffer(driver))
      throw new InvalidOperationException("Driver image does not support the IRingBuffer interface");

    int numBuffers = 0;
    Cvb.Image.IMG dummy;
    int result = Cvb.Driver.IRingBuffer.RBNumBuffer(driver, Cvb.Driver.IRingBuffer.TRingbufferNumbufferCMD.RINGBUFFER_NUMBUFFER_CMD_GET, ref numBuffers, out dummy);
    if (result < 0)
      throw new ExternalException("Error getting number of buffers", result);

    return numBuffers;
  }

  public static Cvb.SharedImg CreateWithNewNumberOfBuffers(this Cvb.SharedImg driver, int numBuffers)
  {
    if (driver == null)
      throw new NullReferenceException(nameof(driver));
    if (!Cvb.Driver.IRingBuffer.CanRingBuffer(driver))
      throw new InvalidOperationException("Driver image does not support the IRingBuffer interface");

    Cvb.SharedImg newDriver;
    int result = Cvb.Driver.IRingBuffer.RBNumBuffer(driver, Cvb.Driver.IRingBuffer.TRingbufferNumbufferCMD.RINGBUFFER_NUMBUFFER_CMD_SET, ref numBuffers, out newDriver);
    if (result < 0)
      throw new ExternalException("Error creating new buffer driver image", result);

    return newDriver;
  }

  public static void SetLockModeToOn(this Cvb.SharedImg driver)
  {
    if (driver == null)
      throw new NullReferenceException(nameof(driver));
    if (!Cvb.Driver.IRingBuffer.CanRingBuffer(driver))
      throw new InvalidOperationException("Driver image does not support the IRingBuffer interface");

    var lockModeOn = Cvb.Driver.IRingBuffer.TRingbufferLockmode.RINGBUFFER_LOCKMODE_ON;
    int result = Cvb.Driver.IRingBuffer.RBLockMode(driver, Cvb.Driver.IRingBuffer.TRingbufferLockmodeCMD.RINGBUFFER_LOCKMODE_CMD_SET, ref lockModeOn);
    if (result < 0)
      throw new ExternalException("Error setting lock-mode to: on", result);
  }

  public static int GetCurrentBufferIndex(this Cvb.SharedImg driver)
  {
    if (driver == null)
      throw new NullReferenceException(nameof(driver));
    if (!Cvb.Driver.IRingBuffer.CanRingBuffer(driver))
      throw new InvalidOperationException("Driver image does not support the IRingBuffer interface");

    int bufferIndex;
    int result = Cvb.Driver.IRingBuffer.RBBufferSeq(driver, 0, out bufferIndex);
    if (result < 0)
      throw new ExternalException("Error getting current buffer index", result);

    return bufferIndex;
  }

  public static Cvb.SharedImg GetBufferImage(this Cvb.SharedImg driver, int bufferIndex)
  {
    if (driver == null)
      throw new NullReferenceException(nameof(driver));
    if (!Cvb.Driver.IRingBuffer.CanRingBuffer(driver))
      throw new InvalidOperationException("Driver image does not support the IRingBuffer interface");

    Cvb.SharedImg bufferImage;
    int result = Cvb.Driver.IRingBuffer.RBGetBufferImage(driver, bufferIndex, out bufferImage);
    if (result < 0)
      throw new ExternalException("Error getting image at buffer index", result);

    return bufferImage;
  }

  public static void StartAcquisition(this Cvb.SharedImg driver)
  {
    int result = Cvb.Driver.IGrab2.G2Grab(driver);
    if (result < 0)
      throw new ExternalException("Error starting acquisition", result);
  }
	
  public static void AbortAcquisition(this Cvb.SharedImg driver)
  {
    int result = Cvb.Driver.IGrab2.G2Freeze(driver, kill: true);
    if (result < 0)
      throw new ExternalException("Error stopping acquisition", result);
  }
	
  public static RingBufferImage WaitForNextImage(this Cvb.SharedImg driver)
  {
    int result = Cvb.Driver.IGrab2.G2Wait(driver);
    if (result < 0)
      throw new ExternalException("Error waiting for next image", result);

    return RingBufferImage.FromCurrent(driver);
  }
}
3 Likes

Thank you for the nice code you have posted.
I’m sorry for the late reply.

I have additional 2 questions related to the code you have posted:

  1. In case in my application I have to acquire data from two different cameras, do I need to load the driver twice for every camera and configure the ring buffer twice? Is it possible to use (share) the same driver for both cameras?
  2. In case I want to extract the Timestamp and the Image ID of the acquired image, do I need to use the _driverImage, correct? Why I’m not able to retrieve these date also from the _bufferImage.

In order to retrieve the timestamp of the acquired image I use the following code:

int DC_BUFFER_INFO_TIMESTAMP = 0x00000800;
int cmd = Driver.IDeviceControl.DeviceCtrlCmd(DC_BUFFER_INFO_TIMESTAMP , Cvb.Driver.IDeviceControl.DC_OPERATION.GET);
int res = Driver.IDeviceControl.DCStrCommand(img, cmd, "", out string pOutHex);

Thank you in advance.

Regards,
Andrea

Hi @andrea.annovi,

Working with two cameras

You can adapt the C code for opening from here:

https://forum.commonvisionblox.com/t/old-image-broken-after-setting-new-camport/66/5?u=parsd

Actually you then use the same driver twice, but have two camera objects open with which you can work in parallel. Each object then has its own, independent ring buffer which needs to be configured.

Ring Buffer Image

You are correct, you need to query that information from the _driverImage despite the fact that this information is stored with the ring buffer image. This is because the returned ring buffer IMGs are only implementing the IImageVPA interface and none of the driver interfaces like IDeviceControl. They are just views on the ring buffer memory. In the C api all is centered around that driver IMG. Dependent types are very simple objects.

Driver IMGs are special. As mentioned above they have all the additional interfaces and the IImageVPA interface points to the one synchronized last via G2Wait. Therefore you can query that information from it.

From an OOP perspective I agree that this is not intuitive. This is why the CVB.Net api has the timestamp property on the RingBufferImage (which derives from StreamImage).

https://forum.commonvisionblox.com/t/getting-started-with-cvb-net/246/6?u=parsd

1 Like

Thank you for the answer.

  • I’m already using the approach you have suggested to acquiring from 2 cameras. Great!
  • I understand the reason to use “_driverImage” in order to query the timestamp of the acquired image.
    If I understand correctly from the code you have sent earlier, the “_driverImage” is bonded together with “_bufferImage” in the ringbuffer. Therefore If I try to retrieve the timestamp from an old image stored in the ringbuffer (as you explain), the function will return the actual timestamp of when the image was acquired and not the timestamp of the last acquired images. Is it correct?

I don’t get the last part of you answer:
"This is why the CVB.Net api has the timestamp property on the RingBufferImage (which derives from StreamImage)."
There is a different way to retrieve the timestamp of the camera using CVB.Net?

I only know two ways:

int DC_BUFFER_INFO_TIMESTAMP = 0x00000800;
int cmd = Driver.IDeviceControl.DeviceCtrlCmd(DC_BUFFER_INFO_TIMESTAMP , Cvb.Driver.IDeviceControl.DC_OPERATION.GET);
int res = Driver.IDeviceControl.DCStrCommand(m_DriverImage, cmd, "", out string pOutHex);

or

Cvb.Driver.IRingBuffer.RBGetRingBufferInfo(m_DriverImage, m_BufferIndex,
Cvb.Driver.IRingBuffer.TRingbufferInfoCMD.RINGBUFFER_INFO_TIMESTAMP, out double val);

I acquired several images, and I noticed that both approaches give me the same results.
What is actually the difference?

I try to clear things up a bit - that is actually why we introduced the new OOP CVB interfaces:

  • CVB.Net for .Net languages
  • CVB++ for C++
  • CVBpy for Python

These are wrappers around our C api (using the C api). You are currently using a direct translation of the C api for .Net. The new .Net wrappers are written like the wrappers I wrote in the earlier post, but using their own types. The goal for these is that the data and operations you need are attached to the objects you expect to hold them (like having an Unlock() method on the RingBufferImage). The link to Getting Started with CVB.Net gives an introduction. It should make your life working with CVB much easier. Switching to it depends heavily on how much CVB code you have written so far.

The Timestamps

Why do we have three ways to read-out a timestamp?

  1. IRingBuffer.RBGetRingBufferInfo:
    To get the timestamp of a specific ring buffer image.
  2. IGrab2.G2GetGrabStatus:
    To get the timestamp of the latest awaited image (via IGrab2.G2Wait). It is shorthand for
    int bufferIndex;
    int res = Driver.IRingBuffer.RBBufferSeq(m_DriverImage, 0, out bufferIndex);
    double timestamp;
    res = Driver.IRingBuffer.RBGetRingBufferInfo(m_DriverImage, bufferIndex, Driver.IRingBuffer.TRingbufferInfoCMD.RINGBUFFER_INFO_TIMESTAMP, out timestamp);
    
  3. IDeviceControl:
    This just gives you the full resolution of the timestamp delivered by the transport technology. A double normally suffices, but in GEV/U3V timestamps are integers with the unit ticks (the tick-frequency can normally be queried via the GenApi).

As I said the driver image (m_DriverImage) is special: it represents the driver, all its controlling interfaces (like IRingBuffer, IImageVPA, IDeviceControl, …). IImageVPA is the interface you use for image data access (width, height, data type, pixels, …). The m_DriverImage always points to the image you synced last via G2Wait. As seen above and in the wrapper I wrote you can query the actual ring buffer image index that has been filled via

int bufferIndex;
int res = Driver.IRingBuffer.RBBufferSeq(m_DriverImage, 0, out bufferIndex);

The query via IDeviceControl and G2GetGrabStatus always accesses the latest awaited image (G2Wait). Thus the timestamp returned by these two ways will change the next time G2Wait returns successfully.

The IRingBuffer timestamp query will return the timestamp of the image at the given buffer index – not the latest awaited one. This value will only change if this specific buffer is overwritten by the acquisition engine.

If you query the timestamp of the ring buffer image of the index returned by RBBufferSeq right after the G2Wait call, the values will be the same for all three methods. They will differ, though, after you called G2Wait again: the buffer index for the newly awaited image will have changed (a new call to RBBufferSeq will give you a new index).

This is also perhaps of interest (regarding RBBufferSeq):

https://forum.commonvisionblox.com/t/where-does-the-image-point-to-using-rgbufferseq-with-numbuffers-1/140?u=parsd