I’ve been working with my BBB and Logitech C920 webcam trying to stream video at low latency for some time and have not yet managed to get the latency under 2 seconds.
As a side project I wanted to use the BBB to create a time lapse video, capturing a picture a second, and then later stitching all of the pictures into a video using ffmpeg.
I’m using OpenCV for the first time. I’m really only using it for the capture/save and to draw some text and lines onto the image, which probably makes OpenCV significant overkill.
My C++ code for the process is:
#include <iostream> // for standard I/O
#include <string> // for strings
#include <iomanip> // for controlling float print precision
#include <sstream> // string to number conversion
#include <unistd.h> // for sleep
#include<opencv2/opencv.hpp>
using namespace std;
using namespace cv;
std::string timeToISO8601(const time_t & TheTime)
{
std::ostringstream ISOTime;
struct tm * UTC = gmtime(&TheTime);
ISOTime.fill('0');
ISOTime << UTC->tm_year+1900 << "-";
ISOTime.width(2);
ISOTime << UTC->tm_mon+1 << "-";
ISOTime.width(2);
ISOTime << UTC->tm_mday << "T";
ISOTime.width(2);
ISOTime << UTC->tm_hour << ":";
ISOTime.width(2);
ISOTime << UTC->tm_min << ":";
ISOTime.width(2);
ISOTime << UTC->tm_sec;
ISOTime << "Z";
return(ISOTime.str());
}
std::string getTimeISO8601(void)
{
time_t timer;
time(&timer);
return(timeToISO8601(timer));
}
int main()
{
VideoCapture capture(-1); // Using -1 tells OpenCV to grab whatever camera is available.
if(!capture.isOpened()){
std::cout << "Failed to connect to the camera." << std::endl;
return(1);
}
capture.set(CAP_PROP_FRAME_WIDTH,1920);
capture.set(CAP_PROP_FRAME_HEIGHT,1080);
//capture.set(CAP_PROP_FRAME_WIDTH,2304); // This should be possible for still images, but not for 30fps video.
//capture.set(CAP_PROP_FRAME_HEIGHT,1536);
for (int OutputFolderNum = 100; OutputFolderNum < 1000; OutputFolderNum++)
for (int OutputImageNum = 1; OutputImageNum < 10000; OutputImageNum++)
{
Mat C920Image;
capture >> C920Image;
if(!C920Image.empty())
{
std::ostringstream OutputFilename;
OutputFilename.fill('0');
OutputFilename << "/media/BONEBOOT/DCIM/";
OutputFilename.width(3);
OutputFilename << OutputFolderNum;
OutputFilename << "WIMBO/img_";
OutputFilename.width(4);
OutputFilename << OutputImageNum;
OutputFilename << ".jpg";
line(C920Image, Point(0, C920Image.rows/2), Point(C920Image.cols, C920Image.rows/2), Scalar(255, 255, 255, 32)); // Horizontal line at center
line(C920Image, Point(C920Image.cols/2, 0), Point(C920Image.cols/2, C920Image.rows), Scalar(255, 255, 255, 32)); // Vertical line at center
circle(C920Image, Point(C920Image.cols/2, C920Image.rows/2), 240, Scalar(255, 255, 255, 32)); // Circles based at center
putText(C920Image, "10", Point((C920Image.cols/2 + 240), (C920Image.rows/2)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
circle(C920Image, Point(C920Image.cols/2, C920Image.rows/2), 495, Scalar(255, 255, 255, 32)); // Circles based at center
putText(C920Image, "20", Point((C920Image.cols/2 + 495), (C920Image.rows/2)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
circle(C920Image, Point(C920Image.cols/2, C920Image.rows/2), 785, Scalar(255, 255, 255, 32)); // Circles based at center
putText(C920Image, "30", Point((C920Image.cols/2 + 785), (C920Image.rows/2)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
circle(C920Image, Point(C920Image.cols/2, C920Image.rows/2), 1141, Scalar(255, 255, 255, 32)); // Circles based at center
putText(C920Image, "40", Point((C920Image.cols/2 + 1141), (C920Image.rows/2)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
string DateTimeText = "WimsWorld.com " + getTimeISO8601();
int baseline=0;
Size textSize = getTextSize(DateTimeText, FONT_HERSHEY_SIMPLEX, 1, 1, &baseline);
putText(C920Image, DateTimeText, Point((C920Image.cols - textSize.width), (C920Image.rows - baseline)), FONT_HERSHEY_SIMPLEX, 1.0, Scalar(0, 0, 255));
imwrite(OutputFilename.str(), C920Image);
std::cout << DateTimeText << " Wrote File : " << OutputFilename.str() << std::endl;
}
std::cout << getTimeISO8601() << "\r" << std::flush;
sleep(1);
}
return 0;
}
I compile it on the BBB with the command:
g++ -O2 `pkg-config --cflags --libs opencv` TimeLapse.cpp -o TimeLapse
I’ve got a bug in that I don’t automatically create the directory structure that I’m saving files into. That’s in the to-do list.
I had been interested in the angle of view on the C920 and found it defined on the Logitech support site that the “Diagonal Field of View (FOV) for the Logitech C920 is 78°”. Unfortunately I was not able to understand if that varied based on the resolution being used. I’m currently using the resolution of 1920×1080, but for stills the camera can capture up to 2304×1536.
I did the geometry math to figure out that 10° off center would be a radius of 240, 20° off center would be a radius of 495, and 30° off center would be a radius of 785. Remembering SOHCAHTOA as Some Old Hags Can’t Always Hide Their Old Age from 9th grade math class came in useful. Using 1920×1080 and 78°angle, my diagonal radius (opposite) works out at 1101 and angle of 39° for tangent, allowing me to calculate my eye height of 1360 = (1101/Tan(39°)). Once I had my eye height I could calculate the radius of circles at any angle by Radius = Tan(Angle) * EyeHeight.
I wanted the circles and angles of vision for my streaming video application and decided that seeing them drawn on the images created here would be helpful, along with both the horizontal and vertical center lines.
The thing I’m not happy with is that the application seems to be running between 30% and 60% of the CPU load on the BBB. When I stream video from the C920 using the native H.264 output the C920 can produce, I was only using about 3% of the BBB CPU. I’ve commented out my drawing code, and verified that the CPU load is primarily related to acquiring the image from the capture device and saving it out to a jpeg file. The lines and text drawing produce minimal incremental CPU. I want to keep the CPU load as low as possible because I’m powering this device from a battery and want it to have as long a runtime as possible.
I believe that the OpenCV library is opening the capture device in a movie streaming mode, and it’s using more CPU interpreting the stream as it’s coming in than the method I was using for streaming to a file. I’ve not yet figured out if there’s a way to define what mode OpenCV acquires the image from the camera.
I was trying to draw the lines and circles with some alpha transparency, but it seems that my underlying image is not the right number of channels and so the lines are being drawn fully opaque.
When the capture opens, it outputs several instances of the same error “VIDIOC_QUERYMENU: Invalid argument” that I’ve not figured out what they mean, or stopped procucing.
I am working on a 32GB flash card, partitioned into two 16GB filesystems. The first is Fat32, has a simple uEnv.txt file in the root allowing the BBB onboard flash to be used, and following the Design rules for Camera File systems standard for the image naming. It allows me to take out the card put it in a PC and it’s recognized just like a normal camera memory card.
Contents of uEnv.txt:
mmcdev=1 bootpart=1:2 mmcroot=/dev/mmcblk1p2 optargs=quiet
The camera seems to be focusing on the building across the street instead of West Seattle.
1200 Western Ave, 13th Floor Elevator Room
WimsWorld 