Webcam on BeagleBoardBlack using OpenCV

I’ve been working with my BBB and Logitech C920 webcam trying to stream video at low latency for some time and have not yet managed to get the latency under 2 seconds.

As a side project I wanted to use the BBB to create a time lapse video, capturing a picture a second, and then later stitching all of the pictures into a video using ffmpeg.

I’m using OpenCV for the first time. I’m really only using it for the capture/save and to draw some text and lines onto the image, which probably makes OpenCV significant overkill.

My C++ code for the process is:

#include <iostream> // for standard I/O
#include <string>   // for strings
#include <iomanip>  // for controlling float print precision
#include <sstream>  // string to number conversion
#include <unistd.h> // for sleep
#include<opencv2/opencv.hpp>
using namespace std;
using namespace cv;

std::string timeToISO8601(const time_t & TheTime)
{
	std::ostringstream ISOTime;
	struct tm * UTC = gmtime(&TheTime);
	ISOTime.fill('0');
	ISOTime << UTC->tm_year+1900 << "-";
	ISOTime.width(2);
	ISOTime << UTC->tm_mon+1 << "-";
	ISOTime.width(2);
	ISOTime << UTC->tm_mday << "T";
	ISOTime.width(2);
	ISOTime << UTC->tm_hour << ":";
	ISOTime.width(2);
	ISOTime << UTC->tm_min << ":";
	ISOTime.width(2);
	ISOTime << UTC->tm_sec;
	ISOTime << "Z";
	return(ISOTime.str());
}
std::string getTimeISO8601(void)
{
	time_t timer;
	time(&timer);
	return(timeToISO8601(timer));
}

int main()
{
    VideoCapture capture(-1);	// Using -1 tells OpenCV to grab whatever camera is available.
    if(!capture.isOpened()){
	    std::cout << "Failed to connect to the camera." << std::endl;
		return(1);
    }
    capture.set(CAP_PROP_FRAME_WIDTH,1920);
    capture.set(CAP_PROP_FRAME_HEIGHT,1080);
    //capture.set(CAP_PROP_FRAME_WIDTH,2304);	// This should be possible for still images, but not for 30fps video.
    //capture.set(CAP_PROP_FRAME_HEIGHT,1536);

	for (int OutputFolderNum = 100;	OutputFolderNum < 1000; OutputFolderNum++)
		for (int OutputImageNum = 1; OutputImageNum < 10000; OutputImageNum++)
		{
			Mat C920Image;
		    capture >> C920Image;
			if(!C920Image.empty())
			{
				std::ostringstream OutputFilename;
				OutputFilename.fill('0');
				OutputFilename << "/media/BONEBOOT/DCIM/";
				OutputFilename.width(3);
				OutputFilename << OutputFolderNum;
				OutputFilename << "WIMBO/img_";
				OutputFilename.width(4);
				OutputFilename << OutputImageNum;
				OutputFilename << ".jpg";

				line(C920Image, Point(0, C920Image.rows/2), Point(C920Image.cols, C920Image.rows/2), Scalar(255, 255, 255, 32)); // Horizontal line at center
				line(C920Image, Point(C920Image.cols/2, 0), Point(C920Image.cols/2, C920Image.rows), Scalar(255, 255, 255, 32)); // Vertical line at center

				circle(C920Image, Point(C920Image.cols/2, C920Image.rows/2), 240, Scalar(255, 255, 255, 32)); // Circles based at center
				putText(C920Image, "10", Point((C920Image.cols/2 + 240), (C920Image.rows/2)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
				circle(C920Image, Point(C920Image.cols/2, C920Image.rows/2), 495, Scalar(255, 255, 255, 32)); // Circles based at center
				putText(C920Image, "20", Point((C920Image.cols/2 + 495), (C920Image.rows/2)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
				circle(C920Image, Point(C920Image.cols/2, C920Image.rows/2), 785, Scalar(255, 255, 255, 32)); // Circles based at center
				putText(C920Image, "30", Point((C920Image.cols/2 + 785), (C920Image.rows/2)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
				circle(C920Image, Point(C920Image.cols/2, C920Image.rows/2), 1141, Scalar(255, 255, 255, 32)); // Circles based at center
				putText(C920Image, "40", Point((C920Image.cols/2 + 1141), (C920Image.rows/2)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));

				string DateTimeText = "WimsWorld.com " + getTimeISO8601();
				int baseline=0;
				Size textSize = getTextSize(DateTimeText, FONT_HERSHEY_SIMPLEX, 1, 1, &baseline);
				putText(C920Image, DateTimeText, Point((C920Image.cols - textSize.width), (C920Image.rows - baseline)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
				imwrite(OutputFilename.str(), C920Image);
				std::cout << DateTimeText << " Wrote File : " << OutputFilename.str() << std::endl;
			}
			std::cout << getTimeISO8601() << "\r" << std::flush;
			sleep(1);
		}
    return 0;
}

I compile it on the BBB with the command:

g++ -O2 `pkg-config --cflags --libs opencv` TimeLapse.cpp -o TimeLapse

I’ve got a bug in that I don’t automatically create the directory structure that I’m saving files into. That’s in the to-do list.

I had been interested in the angle of view on the C920 and found it defined on the Logitech support site that the “Diagonal Field of View (FOV) for the Logitech C920 is 78°”. Unfortunately I was not able to understand if that varied based on the resolution being used. I’m currently using the resolution of 1920×1080, but for stills the camera can capture up to 2304×1536.

I did the geometry math to figure out that 10° off center would be a radius of 240, 20° off center would be a radius of 495, and 30° off center would be a radius of 785. Remembering SOHCAHTOA as Some Old Hags Can’t Always Hide Their Old Age from 9th grade math class came in useful. Using 1920×1080 and 78°angle, my diagonal radius (opposite) works out at 1101 and angle of 39° for tangent, allowing me to calculate my eye height of 1360 = (1101/Tan(39°)). Once I had my eye height I could calculate the radius of circles at any angle by Radius = Tan(Angle) * EyeHeight.

I wanted the circles and angles of vision for my streaming video application and decided that seeing them drawn on the images created here would be helpful, along with both the horizontal and vertical center lines.

The thing I’m not happy with is that the application seems to be running between 30% and 60% of the CPU load on the BBB. When I stream video from the C920 using the native H.264 output the C920 can produce, I was only using about 3% of the BBB CPU. I’ve commented out my drawing code, and verified that the CPU load is primarily related to acquiring the image from the capture device and saving it out to a jpeg file. The lines and text drawing produce minimal incremental CPU. I want to keep the CPU load as low as possible because I’m powering this device from a battery and want it to have as long a runtime as possible.

I believe that the OpenCV library is opening the capture device in a movie streaming mode, and it’s using more CPU interpreting the stream as it’s coming in than the method I was using for streaming to a file. I’ve not yet figured out if there’s a way to define what mode OpenCV acquires the image from the camera.

I was trying to draw the lines and circles with some alpha transparency, but it seems that my underlying image is not the right number of channels and so the lines are being drawn fully opaque.

When the capture opens, it outputs several instances of the same error “VIDIOC_QUERYMENU: Invalid argument” that I’ve not figured out what they mean, or stopped procucing.

I am working on a 32GB flash card, partitioned into two 16GB filesystems. The first is Fat32, has a simple uEnv.txt file in the root allowing the BBB onboard flash to be used, and following the Design rules for Camera File systems standard for the image naming. It allows me to take out the card put it in a PC and it’s recognized just like a normal camera memory card.

Contents of uEnv.txt:

mmcdev=1
bootpart=1:2
mmcroot=/dev/mmcblk1p2
optargs=quiet

The camera seems to be focusing on the building across the street instead of West Seattle.

View from 1200 Western Ave, 13th Floor Elevator Room

1200 Western Ave, 13th Floor Elevator Room

Advertisements

I replaced Google Reader with InoReader

Google reader has been my primary method of keeping up with technological changes over the past few years. I’ve liked the cloud based system that I can read from my laptop, desktop, or tablet, and it will keep track of what I’e read so I don’t have to read the same thing multiple times.

In the early days of the internet, I used usenet news feeds and an nntp reader to keep up with what was going on around the world and with various programming projects. A decade after Usenet died under its own weight Google Reader made RSS a usable replacement for me.

Google reader announced they were shutting down after July 1st, and they followed through with it.

I’ve tried out Feedly, Digg, and looked at several more. I decided to go with InoReader because while it only has a cloud based interface, it is most similar to the quick method I’ve been used to for reading my news.

Feedly has a really nice iPad app that syncs with their system. Digg has an iPad app, but it requires iOS6, which I’ve delayed installing, and I’ve not seen it in use yet.

I’m certain that Google exiting the RSS reader market has increased the innovation in RSS Readers, and may create a revitalization of the technology in general.