I am building an OpenCV application which captures a video from camera and overlays it on another video after removing the background.

I am not able to achieve a reasonable speed as it is playing the output at about 1 fps, whereas my background removal is working at 3fps.

Is there a way to display the background video at its normal speed and overlay the processed video at 3fps ?

I tried commenting out my code and I realized that the problem lies majorly with the Rendering part itself. I tried displaying the video along with my web cam feed and I noticed that there is a drop in the actual fps and the fps of video when displayed with openCV.

here is the sample code:

void main()

{

CvCapture* capture, *Vcap;

capture = cvCaptureFromCAM(0);

if(!capture)

{

printf("Video Load Error");

}

Vcap = cvCaptureFromAVI("bgDemo.mp4");

//printf("\nEntered BGR");

if(!Vcap)

{

printf("Video Load Error");

}

while(1)

{

IplImage* src = cvQueryFrame(Vcap);

if(!src)

{

Vcap = cvCaptureFromAVI("bgDemo.mp4");

continue;

}

IplImage* bck1 = cvCreateImage(cvGetSize(src),8,3);

cvResize(src,bck1,CV_INTER_LINEAR);

cvShowImage("BCK",bck1);

cvWaitKey(1);

}

}

解决方案

The main problem is that you are allocating a new image at every iteration of the loop without releasing it at the end of the loop. In other words, you have a beautiful memory leak.

A better approach is to simply grab a frame of the video before the loop starts. This will let you create bck1 with the right size just once.

There are other problems with your code, I'm sharing a fixed version below, make sure you pay attention to every line of code to see what changed. I haven't had time to test it, but I'm sure you'll figure it out:

int main()

{

// I know what you are doing, just one capture interface is enough

CvCapture* capture = NULL;

capture = cvCaptureFromCAM(0);

if(!capture)

{

printf("Ooops! Camera Error");

}

capture = cvCaptureFromAVI("bgDemo.mp4");

if(!capture)

{

printf("Ooops! Video Error");

// if it failed here, it means both methods for loading a video stream failed.

// It makes no sense to let the application continue, so we return.

return -1;

}

// Retrieve a single frame from the camera

IplImage* src = cvQueryFrame(capture);

if(!src)

{

printf("Ooops! #1 cvQueryFrame Error");

return -1;

}

// Now we can create our backup image with the right dimensions.

IplImage* bck1 = cvCreateImage(cvGetSize(src),src->depth, src->nChannels);

if(!bck1)

{

printf("Ooops! cvCreateImage Error");

return -1;

}

while(1)

{

src = cvQueryFrame(capture);

if(!src)

{

printf("Ooops! #2 cvQueryFrame Error");

break;

}

cvResize(src, bck1, CV_INTER_LINEAR);

cvShowImage("BCK",bck1);

cvWaitKey(10);

}

cvReleaseImage( &bck1 ); // free manually allocated resource

return 0;

}

These fixes should speed up your application considerably.