Use image stream in OpenCV

Programming Questions .Net API
,
1

Hello,

I want to access the image data from my camera stream via OpenCV. I have found this guide. Unfortunately this is for C++ and not C#. In addition to that I have found the following post from this forum.
Due to my very limited knowledge from C# I am unfortunately still unable to convert the stream images to OpenCV images for further use.

Any help would be greatly appreciated!!

2

Hi and welcome to the forum!

OpenCV is essentially a C/C++ library (with bindings for other programming languages like Python etc. - but C# is not officially among them) - how are you using it in C#? Do you use DllImport to address OpenCV 1 functions or are you using Emgu (www.emgu.com)? In the latter case what you’d need to do is copy the image bytes into a byte array and assign this to the Emgu.CV.Image's Bytes member. The posts in this thread will help you with that.

1 Like
3

Thank you very much @illusive! The linked post from this forum helped a lot :slight_smile:
I have a couple follow up questions about this.

  1. How do I release the used ressources? In the linked post the use of ReleaseObj is recommended. However this isn’t working for me. The problem seems to be that Stemmer.Cvb.Image does not contain ReleaseObject. What would be the correct way to go about this?
  2. I have a problem with my conversion from the integer list to an actual EmguCV image. My approach was to use this post to convert from an integer list to an byte array and then to utilize this guide to get an actual image. This is however not working as intended as the resulting image looks like the attached picture. I realize that this question does not really fit the scope of this forum but if you happen to know a fix anyways I would be extremely grateful.
  3. The provided example is using a “normal” image. My approach for using a “stream image” would be to simpy use Stemmer.Cvb.Driver.StreamImage instead of Stemmer.Cvb.Image. Is that the correct idea?

Again, thank you very much for your help!!

abc

1 Like
4

Hi!

The answers to your questions in the order asked :slight_smile::

  1. ReleaseObj refers to the old API. You are using CVB.Net, so you can go about resource management the .Net way for objects that implement IDisposable by putting things into a using block:
    // initialize image source and make sure myStream is initialized properly
    ... 
    using (var s = myStream.Wait())
    {
       // process the stream image
       ...
    }

  1. Sorry, my bad: The example that I pointed you to is indeed building a List<int> - I should probably have pointed out that you should omit the conversion to int entirely as you are going to build a byte array. So rather than going through a List<int> (which in that thread was the original poster’s aim for some reason) go through a List<byte>. Or better still: Use a byte array - you do know the image size and can easily allocate it beforehand…
    (btw: Your image actually reflects this: The black columns are the uppermost three bytes of the integers which are of course set to zero whereas the brighter columns are your actual pixel data).

  2. Stemmer.Cvb.Driver.StreamImage inherits from a Stemmer.Cvb.Image and therefore is a Stemmer.Cvb.Image for all intents and purposes, so your general approach makes sense and what has been shown to work with a Stemmer.Cvb.Image has no reason not to work with a Stemmer.Cvb.Driver.StreamImage.

2 Likes
5

Hello again :slight_smile:

I have tried your suggestions and they work just perfect! Thank you so much for your help.

But now I have another problem. Currently I have a console application where I access the camera via stream.Start() to get a picture for EmguCV. The second application is a Windows Forms application for the live view of the camera (again I followed the Getting Started guide). Now I want to combine both applications into a single one. Unfortunately I am not quite sure how I should approach this. Especially how to implement the stream.Wait() function into the GUI.
Any help or links with further information are as always very much appreciated!!

6

Hi @frage12358,

thanks for the feedback - always happy to help :slightly_smiling_face:

For what you plan, I recommend having a look at a tutorial that ships with the CVB.Net wrappers called “StreamDisplay”. You should be able to find it in the folder %CVB%\Tutorial\Image Manager\Cvb.Net\StreamDisplay. As it is, it does nothing else but show a live image from a selectable image source, but it illustrates how to implement a continuous image acquisition with the .Net wrappers. Spoiler: You won’t find any Stream.Wait() calls in it :wink:

While Stream.Wait() is a valid approach to continuous acquisition (and in fact the one that gives you a fairly high degree of control over what is happening when and where), it has the potential to become a bit clunky if you have to match it up with a GUI framework, because in the scenario you describe, Stream.Wait() should be called asynchronously to keep the UI responsive. There are basically two ways to achieve this without resorting to setting up and managing a dedicated acquisition thread:

  1. If you are using at least C# 5 which gives you access to the keywords async and await you can use the Stream.WaitAsync() method in what looks more like a simple loop but in fact runs asynchronously to your UI. And example for this may be found in the WpfGenICam tutorial:
     private async void menuGrab_Checked(object sender, RoutedEventArgs e)
 {
   Device.Stream.Start();
   menuOpen.IsEnabled = false; // keep people from loading a new driver while acquisition is still running
   try
   {
     while (menuGrab.IsChecked)
     {
       using (StreamImage image = await Device.Stream.WaitAsync())
       {
         // add processing here...
       }
     }
   }
   catch (OperationCanceledException)
   {
     // acquisition was aborted
   }
   finally
   {
     menuOpen.IsEnabled = true;
   }
 }
  2. As mentioned previously, async/await is not available for every project, plus traditionally-minded Forms developers feel more at home with event handlers. Therefore, CVB.Net implements the StreamHandler component (Stemmer.Cvb.Forms.dll, namespace Stemmer.Cvb.Forms.Components). This object has a Stream property to which a loaded stream is assigned. With the Start() and Stop() method you can control acquisition and subscription to the component’s NewImage event (and ideally also the Error event) allows you to react to new images and things that might have gone wrong during acquisition.
    Note that these events are fired in the UI thread’s context, which means that you can manipulate the UI in any way you want in the handlers. If, however, processing is expected to take a significant amount of time, doing the processing right inside the handler is not encouraged.
    Have a look at the contents of MainForm.Streaming.cs in the StreamDisplay tutorial for code samples.
2 Likes
7

Thanks you so much. The second approach works perfect :slight_smile:

1 Like