Display most recent image generated by multiple processing threads running in parallel

[VCmonkey]

I want to build up an application for a multiple-camera-system. The application provides multiple asynchronous image processing jobs (one for each camera) that run in parallel. The display in the GUI should always show the processed image of the most recently finished image processing job. I want to use the CVB display class.

Is it save to call the display method refresh() inside one of the asynchronous image processing threads?

Is it possible to refresh the display as soon as a processing thread has finished his job, although the display object is instanced in a separate thread (e.g. a GUI thread)?

[parsd]

Hi @VCmonkey,

do you mean the Display ActiveX Control? If yes, then calling the Refresh() method on it is not safe from non-UI threads as ActiveX controls are COM controls and UI elements and thus life in the Single Threaded Apartment. So correctly calling that function depends on your programming language and used framework. Are you coding in C/C++? Which UI-Framework do you use?

How do you acquire the images? Do you use the DLL functions G2Grab, G2Wait, and G2Freeze in your own thread or the Image ActiveX Control?

[VCmonkey]

Hello @parsd ,

thank you for your fast reply !

Correct, I mean the Display ActiveX Control.
I am coding in C++ and my application is based on the Visual C++ MFC example project that I found in the Programming with CVB section of the CVB documentation. I am using the Visual C++ OCX Wrappers for 64 bit.

The DLL functions G2Grab, G2Wait and G2Freeze are called in separate image processing threads. Each thread loads the CVB driver, changes the port and then starts grabbing and processing in a loop.

In order to preserve thread safety, I will have to find an alternative approach to update the display instead of calling Refresh() inside the processing threads.

Do you have any suggestions?

[parsd]

The good old MFC… Sadly it has no built in way to execute something on the UI thread. If you just want to refresh the simple solution below would work.

Full implementation instructions...

I assume you do all the work in a CWnd derived class, like a dialog.

Register Window Message by putting this in the private section of your class declaratoin or somewhere above the implementation in the cpp-file:

enum
{
  WM_ASYNC_REFRESH = WM_USER
  // other custom messages are entered here...
};

Put the custom message handler method declaration somewhere in the class’ private section:

private:
  afx_msg LRESULT OnAsyncRefresh(WPARAM wParam, LPARAM lParam);

and add the implementation to your cpp-file (exchange CYourClass with your actual class name and m_cvDisp with your actual Display OCX name):

LRESULT CYourClass::OnAsyncRefresh(WPARAM wParam, LPARAM lParam)
{
  UNREFERENCED_PARAMETER(wParam);
  UNREFERENCED_PARAMETER(lParam);

  m_cvDisp.Refresh();
}

Finally add the method to the MFC’s message map (search for BEGIN_MESSAGE_MAP in your class’ cpp file):

BEGIN_MESSAGEMAP(CYourClass, CWnd)
  // many handlers...
  ON_MESSAGE(WM_ASYNC_REFRESH, OnAsyncRefresh)
END_MESSAGE_MAP()

Now everything is prepared and you can either call

SendNotifyMessage(WM_ASYNC_REFRESH, 0, 0);

on your CYourClass object to refresh asynchronously if called from a non-UI-thread or synchronously when called from the UI-thread. If you always want to have it synchronously call

SendMessage(WM_ASYNC_REFRESH);

on your CYourClass object. This will wait for the Refresh() being executed. I you never want to wait, call

PostMessage(WM_ASYNC_REFRESH);

on CYourClass object. This always refreshes asynchronously, even when you are on the UI thread.

Asynchronous messages are fire and forget here (you cannot wait for it).

[VCmonkey]

Thanks parsd!

Your detailed implementation instructions are a good start .

[illusive]

Sorry to be a wise-ass here , but there’s a simpler solution for this particular use-case. If the goal is only to refresh a display that already has the correct image handle set, it would be sufficient to call

SendMessage(m_cvDisp.GethWnd(), WM_PAINT, 0, 0);

or

PostMessage(m_cvDisp.GethWnd(), WM_PAINT, 0, 0);

in your non-UI thread to update the display control’s content.

However, @parsd provided the perfect entry point for a case where a simple m_cvDisp.Refresh() is not sufficient because e.g. you need to call m_cvDisp.SetImage() because your processing generates a new image rather than updating the content of an existing image.

In that case, use the code from @parsd and send the new image handle through the LPARAM of your custom window message and modify the OnAsyncRefresh as follows:

LRESULT CYourClass:OnAsyncRefresh(WPARAM wParam, LPARAM lParam)
{
  UNREFERENCED_PARAMETER(wParam);
  m_cvDisp.SetImage(reinterpret_cast<intptr_t>(lParam));
}

(well, maybe we should rename it to something like OnAsyncSetImage in that case, but I guess you get what I am up to…)

The code from @parsd can also be easily extended to other use cases that are forbidden from within a non-UI thread like adding labels or overlay plugins to a display etc.

Generally speaking, doing anything that alters the state of the user interface from withing a non-UI thread is a no-go (some things might work most of the time, but effectively as soon as you touch your UI from a non-UI thread you enter the territory of undefined behavior).

[parsd]

@illusive: I can also nitpick :

If you just do

m_cvDisp.SetImage(reinterpret_cast<intptr_t>(lparam));

you won’t see a refresh if the image handle did not change. So it should be

auto hNewImage = reinterpret_cast<intptr_t>(lparam);
if(m_cvDisp.GetImage() != hNewImage)
  m_cvDisp.SetImage(hNewImage);
else
  m_cvDisp.Refresh();

Also @illusive: why did you choose the lparam? Both WPARAM and LPARAM are pointer sized and I would prefer the first parameter. So I could write

PostMessage(processedImage);

instead of

PostMessage(0, processedImage);

[illusive]

Getting a bit extreme, are we…

But the point about the unchanged handle is actually valid…

[parsd]

To sum the solution up from @illusive and me

Put this in the private section of your CWnd derived class:

private:
  afx_msg LRESULT OnAsyncRefresh(WPARAM wParam, LPARAM lParam);

and that in the cpp-file:

LRESULT CYourClass::OnAsyncRefresh(WPARAM wParam, LPARAM lParam)
{
  UNREFERENCED_PARAMETER(lParam);
  intptr_t hNewImage = reinterpret_cast<intptr_t>(lparam);
  if(m_cvDisp.GetImage() != hNewImage)
    m_cvDisp.SetImage(hNewImage);
  else
    m_cvDisp.Refresh();
}

Search for BEGIN_MESSAGE_MAP in the class’ cpp and add

enum
{
  WM_ASYNC_REFRESH = WM_USER
  // other custom messages are entered here...
};

before it and

BEGIN_MESSAGEMAP( ... )
  // many handlers...
  ON_MESSAGE(WM_ASYNC_REFRESH, OnAsyncRefresh)
END_MESSAGE_MAP()

into it (the ON_MESSAGE-part).

Then you can call

PostMessage(WM_ASYNC_REFRESH, myProcessImage);

to asynchronously set/refresh the image in the display.

@VCmonkey, if you like the solution and it solved your problem, would you please mark an entry as the solution? Thank you

[illusive]

Hi @VCmonkey! If one of the suggestions turned out to be a viable solution to your problem it might be helpful for future visitors if you could mark it as the solution

[VCmonkey]

Hi @parsd and @illusive. Thank you for your help . I am currently working on another topic. I will give you feedback as soon as I switch back to this topic.

[VCmonkey]

Hi @parsd and @illusive. I went back coding and fortunately with your help it worked out for me.

All images generated by the processing thread have the same dimensions.
So, I will use SetImage in the GUI to assign an IMG to the display.
Then, the processing threads will update the content of that image, but will not change the number of planes neither the dimensions.

The GUI thread calls

      IMG processedImage;
      CreateGenericImage(1, 640, 480, false, processedImage);
      m_cvDisp.SetImage(reinterpret_cast<intptr_t>(processedImage));

once.

Each processing thread overwrites the content of the image and refreshes the display

    Processing(…, processedImage);
    PostMessage(m_cvDisp.GethWnd(), WM_PAINT, 0, 0);