Getting image stream from webcam using Aforge.NET in WPF – Asynchronous UI updating

So, I was working on object tracking a few months back. The basic task I faced while doing this was getting the image stream in a format that is easy to process upon later. I was using Andrew Kirillov’s Aforge.NET API for the processing part. So this library also provides some basic functions to rip the image stream off your cam.

Enter the platform compatibility issues. Aforge.NET was basically developed for the older Winforms. Asking it to work for WPF wouldn’t be that difficult. Problems arise when your ‘stream-fetching’ functions try to access the UI controls. In this case your image control.

Two major issues:

  • In WPF, your image holding UI element is the “Image” element (which you might have declared in XAML or code behind). ‘Image’ accepts only System.Windows.Media.Imaging.BitmapImage (or ImageSource) elements as its Source. Since Aforge.NET was made for Winforms, it generally returns the cam stream in System.Drawing.Bitmap format for the PictureBox (which is not available in WPF). You cannot convert implicitly or (even cast explicitly) from Bitmap to BitmapImage.
  • The second and the major issue is the asynchronous updating of the UI elements. In our case, the Image control. Typically, we get a “Calling thread cannot access the UI component” exception. This is because you can’t update any UI control from a thread that doesn’t own the control.

I’ll exemplify by a code snippet:

Issue 1:

publicFilterInfoCollection CamsCollection;
publicVideoCaptureDeviceCam=null;
voidCam_NewFrame(object sender,NewFrameEventArgs eventArgs){frameholder.Source=(Bitmap)eventArgs.Frame.Clone();/* Here lies issue1, as it cannot convert implicitly*/}

privatevoid startcam_Click(object sender,RoutedEventArgs e){CamsCollection=newFilterInfoCollection(FilterCategory.VideoInputDevice);
Cam=newVideoCaptureDevice(CamsCollection[1].MonikerString);
Cam.NewFrame+=newNewFrameEventHandler(Cam_NewFrame);Cam.Start();}

privatevoid stopcam_Click(object sender,RoutedEventArgs e){Cam.Stop();}

Fixing issue 1:

I made a class library called Compatibility, which you can find here in my Codeplex and Github repositories. Feel free to use it. You can also find the source there. My functions Compatibility.BitmaptoBitmapImage() and BitmapImagetoBitmap() convert the formats. I might just put up another blog post to show how they work.

Issue 2:

publicFilterInfoCollection CamsCollection;
publicVideoCaptureDeviceCam=null;
voidCam_NewFrame(object sender,NewFrameEventArgs eventArgs){System.Drawing.Image imgforms =(Bitmap)eventArgs.Frame.Clone();
BitmapImage bi=newBitmapImage();
bi.BeginInit();
bi = Compatibility.Compatibility.BitmaptoBitmapImage((Bitmap)imgforms); //download the compatibility api below
frameholder.Source= bi;/*issue2 here. bi is occupied by some other thread.*/
bi.EndInit(); }

privatevoid startcam_Click(object sender,RoutedEventArgs e){
CamsCollection=newFilterInfoCollection(FilterCategory.VideoInputDevice);
Cam=newVideoCaptureDevice(CamsCollection[1].MonikerString);
Cam.NewFrame+=newNewFrameEventHandler(Cam_NewFrame);
Cam.Start();}

privatevoid stopcam_Click(object sender,RoutedEventArgs e){Cam.Stop();}

Solving Issue 2 using Dispatcher:

So what is Dispatcher? Every UI control (Button, Textbox, Image, etc.) inherits from DispacterObject. This object is what allows you to get a hold of the UI thread’s Dispatcher. Ironically, this is also the reason why you cannot update the controls from threads that don’t own it. But I think this is a very fluent method of ensuring thread-safety. Still if you want to update some asynchronously, you can use the Dispatcher.BeginInvoke() method, as shown:

Final code snippet:

void Cam_NewFrame(object sender,NewFrameEventArgs eventArgs)
{
System.Drawing.Image imgforms =(Bitmap)eventArgs.Frame.Clone();
BitmapImage bi =newBitmapImage();
bi.BeginInit();
bi = Compatibility.Compatibility.BitmaptoBitmapImage((Bitmap)imgforms);
bi.EndInit();
//Using the freeze function to avoid cross thread operations
bi.Freeze();//Calling the UI thread using the Dispatcher to update the 'Image' WPF control
Dispatcher.BeginInvoke(newThreadStart(delegate{ frameholder.Source= bi; /*frameholder is the name of the 'Image' WPF control*/}));
}

I hope the article was helpful. Please leave a feedback if you have any suggestion or any improvement over the above method. Smile

-Sagar

About these ads

11 thoughts on “Getting image stream from webcam using Aforge.NET in WPF – Asynchronous UI updating

    1. In the event handler for the new frame event, just add a code snippet to handle the job. Save the bitmap frame to your local file system using File Streams. If you just need temporary storage of the frames, I suggest you use MemoryStream since the access time for memory stream is very low as compared to that of File Stream and so this helps your app maintain a realtime fps.

  1. I have done the job thanks ,a lot mate..
    I am wondering if there is anyway without using Aforge or any other libraries,to connect webcam and to save frames?

  2. great snippet, really. I was wondering if it’s possible to optimize it, instead of creating a BitmapImage, you could just use the MemoryStream array and write it on a WriteableBitmap, using the method WritePixels

  3. yes, i thought something like this:

    MemoryStream ms = new MemoryStream();
    img.Save(ms, ImageFormat.Bmp);
    ms.Seek(0, SeekOrigin.Begin);

    this.colorBitmap.WritePixels(
    new Int32Rect(0, 0,width, this.colorBitmap.PixelHeight),
    ms.ToArray(),
    this.colorBitmap.PixelWidth * sizeof(int),
    0);

    but it doesn’t seem to work i think because of those threading issues you pointed out in your article

    1. Are you binding the WriteableBitmap to the Image control on your UI? If not then add this line at the end –

      Dispatcher.BeginInvoke(newThreadStart(delegate{
      frameholder.Source= colorBitmap; /*frameholder is the name of the ‘Image’ WPF control on ur UI*/}));

      As long as you are updating the UI via the Dispatcher the threading exception shouldn’t fire.

    2. thanks a lot!, pratically i am trying to switch between a video and a kinect stream (using the MS SDK), i tried your solution it works, but it’s slow (because of the creation of big bitmap in memory everytime), so i am trying to optimize it, when i click the button i should switch source for the WriteableBitmap

      http://pastebin.com/C2b5QBTi

  4. Precisely my point. You are updating the “image” control of your UI in the Windows_Loaded method which is called only once (i.e. when the window loads for the first time). You have to update the image with the frame after every frame is ready. So add the following line at the end of your videoSourcePlayer_NewFrame method:

    Dispatcher.BeginInvoke(newThreadStart(delegate{
    this.image.Source= colorBitmap; }));
    

    Please correct me if I have misread anything.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s