Getting image stream from webcam using Aforge.NET in WPF – Asynchronous UI updating

So, I was working on object tracking a few months back. The basic task I faced while doing this was getting the image stream in a format that is easy to process upon later. I was using Andrew Kirillov’s Aforge.NET API for the processing part. So this library also provides some basic functions to rip the image stream off your cam.

Enter the platform compatibility issues. Aforge.NET was basically developed for the older Winforms. Asking it to work for WPF wouldn’t be that difficult. Problems arise when your ‘stream-fetching’ functions try to access the UI controls. In this case your image control.

Two major issues:

• In WPF, your image holding UI element is the “Image” element (which you might have declared in XAML or code behind). ‘Image’ accepts only System.Windows.Media.Imaging.BitmapImage (or ImageSource) elements as its Source. Since Aforge.NET was made for Winforms, it generally returns the cam stream in System.Drawing.Bitmap format for the PictureBox (which is not available in WPF). You cannot convert implicitly or (even cast explicitly) from Bitmap to BitmapImage.
  
• The second and the major issue is the asynchronous updating of the UI elements. In our case, the Image control. Typically, we get a “Calling thread cannot access the UI component” exception. This is because you can’t update any UI control from a thread that doesn’t own the control.

I’ll exemplify by a code snippet:

Issue 1:

publicFilterInfoCollection CamsCollection;
publicVideoCaptureDeviceCam=null;
voidCam_NewFrame(object sender,NewFrameEventArgs eventArgs){frameholder.Source=(Bitmap)eventArgs.Frame.Clone();/* Here lies issue1, as it cannot convert implicitly*/}

privatevoid startcam_Click(object sender,RoutedEventArgs e){CamsCollection=newFilterInfoCollection(FilterCategory.VideoInputDevice);
Cam=newVideoCaptureDevice(CamsCollection[1].MonikerString);
Cam.NewFrame+=newNewFrameEventHandler(Cam_NewFrame);Cam.Start();}

privatevoid stopcam_Click(object sender,RoutedEventArgs e){Cam.Stop();}

Fixing issue 1:

I made a class library called Compatibility, which you can find here in my Codeplex and Github repositories. Feel free to use it. You can also find the source there. My functions Compatibility.BitmaptoBitmapImage() and BitmapImagetoBitmap() convert the formats. I might just put up another blog post to show how they work.

Issue 2:

publicFilterInfoCollection CamsCollection;
publicVideoCaptureDeviceCam=null;
voidCam_NewFrame(object sender,NewFrameEventArgs eventArgs){System.Drawing.Image imgforms =(Bitmap)eventArgs.Frame.Clone();
BitmapImage bi=newBitmapImage();
bi.BeginInit();
bi = Compatibility.Compatibility.BitmaptoBitmapImage((Bitmap)imgforms); //download the compatibility api below
frameholder.Source= bi;/*issue2 here. bi is occupied by some other thread.*/
bi.EndInit(); }

privatevoid startcam_Click(object sender,RoutedEventArgs e){
CamsCollection=newFilterInfoCollection(FilterCategory.VideoInputDevice);
Cam=newVideoCaptureDevice(CamsCollection[1].MonikerString);
Cam.NewFrame+=newNewFrameEventHandler(Cam_NewFrame);
Cam.Start();}

privatevoid stopcam_Click(object sender,RoutedEventArgs e){Cam.Stop();}

Solving Issue 2 using Dispatcher:

So what is Dispatcher? Every UI control (Button, Textbox, Image, etc.) inherits from DispacterObject. This object is what allows you to get a hold of the UI thread’s Dispatcher. Ironically, this is also the reason why you cannot update the controls from threads that don’t own it. But I think this is a very fluent method of ensuring thread-safety. Still if you want to update some asynchronously, you can use the Dispatcher.BeginInvoke() method, as shown:

Final code snippet:

void Cam_NewFrame(object sender,NewFrameEventArgs eventArgs)
{
System.Drawing.Image imgforms =(Bitmap)eventArgs.Frame.Clone();
BitmapImage bi =newBitmapImage();
bi.BeginInit();
bi = Compatibility.Compatibility.BitmaptoBitmapImage((Bitmap)imgforms);
bi.EndInit();
//Using the freeze function to avoid cross thread operations
bi.Freeze();//Calling the UI thread using the Dispatcher to update the 'Image' WPF control
Dispatcher.BeginInvoke(newThreadStart(delegate{ frameholder.Source= bi; /*frameholder is the name of the 'Image' WPF control*/}));
}

I hope the article was helpful. Please leave a feedback if you have any suggestion or any improvement over the above method.

-Sagar