I wanted a quick and dirty method to test my camera module installation on my Raspberry Pi ZeroW installation. I don’t have a monitor connected to the Raspberry, and explicitly did not install the desktop version of the operating system. This is especially important because the camera itself may not be properly focused after installation in the case, and the only way to easily focus the camera is with a video stream allowing you to make small adjustments and see them nearly real time.
I’ve used FFMPEG for years as it handles almost any kind of video or audio I can throw at it. I use VLC on my desktop machine for similar reasons.
I did a quick install of ffmpeg on my Pi with the following command, allowing it to install all the requirements, adding up to almost 126 new packages and 56MB that needed to be downloaded and installed.
sudo apt-get install ffmpeg -y
After it finished installing, I was able to run the following command with the 192.168.0.16 address being my desktop computer.
ffmpeg -f video4linux2 -input_format h264 -video_size 1280x720 -framerate 30 -i /dev/video0 -vcodec copy -an -f mpegts udp://192.168.0.16:5000?pkt_size=1316
On my desktop computer I ran VLC, under the Media menu, selected Open Network Stream, and opened:
udp://@0.0.0.0:5000
What I’m doing is to use FFMPEG to pull video from the device and push it using UDP datagrams at my desktop on port 5000. Then VLC opens a port on the local machine at port 5000 to receive the datagrams and it decodes and displays the video. An interesting thing about this method is that I can stop transmitting from the raspberry, then restart it, and VLC will accept the packets since UDP is a connectionless protocol.
What really surprised me was that when I logged in a second time to my Raspberry Pi to view the CPU usage for streaming, it was only running around 12% of the CPU. I was interested in knowing what native formats the camera supported..
ffmpeg -f v4l2 -list_formats all -i /dev/video0
ffmpeg version 4.1.4-1+rpt1~deb10u1 Copyright (c) 2000-2019 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --prefix=/usr --extra-version='1+rpt1~deb10u1' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --arch=arm --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
[video4linux2,v4l2 @ 0x2367e40] Raw : yuv420p : Planar YUV 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : yuyv422 : YUYV 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : rgb24 : 24-bit RGB 8-8-8 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Compressed: mjpeg : JFIF JPEG : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Compressed: h264 : H.264 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Compressed: mjpeg : Motion-JPEG : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : Unsupported : YVYU 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : Unsupported : VYUY 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : uyvy422 : UYVY 4:2:2 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : nv12 : Y/CbCr 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : bgr24 : 24-bit BGR 8-8-8 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : yuv420p : Planar YVU 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : Unsupported : Y/CrCb 4:2:0 : {32-3280, 2}x{32-2464, 2}
[video4linux2,v4l2 @ 0x2367e40] Raw : bgr0 : 32-bit BGRA/X 8-8-8-8 : {32-3280, 2}x{32-2464, 2}
/dev/video0: Immediate exit requested
That output leads me to believe that the camera module could output either h264 or mjpeg without significant CPU overhead. What it doesn’t do is tell me efficient frame sizes. It seems to say that horizontal and vertical sizes can be anything between 32 to 3280 and 32 to 2464. I know that the specs on the camera say that it will run still frames at the high resolution, but video is significantly less.
Two Video4Linux commands that return interesting and similar results are:
pi@WimPiZeroCamera:~ $ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'YU12' (Planar YUV 4:2:0)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[1]: 'YUYV' (YUYV 4:2:2)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[2]: 'RGB3' (24-bit RGB 8-8-8)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[3]: 'JPEG' (JFIF JPEG, compressed)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[4]: 'H264' (H.264, compressed)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[5]: 'MJPG' (Motion-JPEG, compressed)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[6]: 'YVYU' (YVYU 4:2:2)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[7]: 'VYUY' (VYUY 4:2:2)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[8]: 'UYVY' (UYVY 4:2:2)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[9]: 'NV12' (Y/CbCr 4:2:0)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[10]: 'BGR3' (24-bit BGR 8-8-8)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[11]: 'YV12' (Planar YVU 4:2:0)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[12]: 'NV21' (Y/CrCb 4:2:0)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
[13]: 'BGR4' (32-bit BGRA/X 8-8-8-8)
Size: Stepwise 32x32 - 3280x2464 with step 2/2
pi@WimPiZeroCamera:~ $ v4l2-ctl -L
User Controls
brightness 0x00980900 (int) : min=0 max=100 step=1 default=50 value=50 flags=slider
contrast 0x00980901 (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
saturation 0x00980902 (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
red_balance 0x0098090e (int) : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
blue_balance 0x0098090f (int) : min=1 max=7999 step=1 default=1000 value=1000 flags=slider
horizontal_flip 0x00980914 (bool) : default=0 value=0
vertical_flip 0x00980915 (bool) : default=0 value=0
power_line_frequency 0x00980918 (menu) : min=0 max=3 default=1 value=1
0: Disabled
1: 50 Hz
2: 60 Hz
3: Auto
sharpness 0x0098091b (int) : min=-100 max=100 step=1 default=0 value=0 flags=slider
color_effects 0x0098091f (menu) : min=0 max=15 default=0 value=0
0: None
1: Black & White
2: Sepia
3: Negative
4: Emboss
5: Sketch
6: Sky Blue
7: Grass Green
8: Skin Whiten
9: Vivid
10: Aqua
11: Art Freeze
12: Silhouette
13: Solarization
14: Antique
15: Set Cb/Cr
rotate 0x00980922 (int) : min=0 max=360 step=90 default=0 value=0 flags=modify-layout
color_effects_cbcr 0x0098092a (int) : min=0 max=65535 step=1 default=32896 value=32896
Codec Controls
video_bitrate_mode 0x009909ce (menu) : min=0 max=1 default=0 value=0 flags=update
0: Variable Bitrate
1: Constant Bitrate
video_bitrate 0x009909cf (int) : min=25000 max=25000000 step=25000 default=10000000 value=10000000
repeat_sequence_header 0x009909e2 (bool) : default=0 value=0
h264_i_frame_period 0x00990a66 (int) : min=0 max=2147483647 step=1 default=60 value=60
h264_level 0x00990a67 (menu) : min=0 max=11 default=11 value=11
0: 1
1: 1b
2: 1.1
3: 1.2
4: 1.3
5: 2
6: 2.1
7: 2.2
8: 3
9: 3.1
10: 3.2
11: 4
h264_profile 0x00990a6b (menu) : min=0 max=4 default=4 value=4
0: Baseline
1: Constrained Baseline
2: Main
4: High
Camera Controls
auto_exposure 0x009a0901 (menu) : min=0 max=3 default=0 value=0
0: Auto Mode
1: Manual Mode
exposure_time_absolute 0x009a0902 (int) : min=1 max=10000 step=1 default=1000 value=1000
exposure_dynamic_framerate 0x009a0903 (bool) : default=0 value=0
auto_exposure_bias 0x009a0913 (intmenu): min=0 max=24 default=12 value=12
0: -4000 (0xfffffffffffff060)
1: -3667 (0xfffffffffffff1ad)
2: -3333 (0xfffffffffffff2fb)
3: -3000 (0xfffffffffffff448)
4: -2667 (0xfffffffffffff595)
5: -2333 (0xfffffffffffff6e3)
6: -2000 (0xfffffffffffff830)
7: -1667 (0xfffffffffffff97d)
8: -1333 (0xfffffffffffffacb)
9: -1000 (0xfffffffffffffc18)
10: -667 (0xfffffffffffffd65)
11: -333 (0xfffffffffffffeb3)
12: 0 (0x0)
13: 333 (0x14d)
14: 667 (0x29b)
15: 1000 (0x3e8)
16: 1333 (0x535)
17: 1667 (0x683)
18: 2000 (0x7d0)
19: 2333 (0x91d)
20: 2667 (0xa6b)
21: 3000 (0xbb8)
22: 3333 (0xd05)
23: 3667 (0xe53)
24: 4000 (0xfa0)
white_balance_auto_preset 0x009a0914 (menu) : min=0 max=9 default=1 value=1
0: Manual
1: Auto
2: Incandescent
3: Fluorescent
4: Fluorescent H
5: Horizon
6: Daylight
7: Flash
8: Cloudy
9: Shade
image_stabilization 0x009a0916 (bool) : default=0 value=0
iso_sensitivity 0x009a0917 (intmenu): min=0 max=4 default=0 value=0
0: 0 (0x0)
1: 100000 (0x186a0)
2: 200000 (0x30d40)
3: 400000 (0x61a80)
4: 800000 (0xc3500)
iso_sensitivity_auto 0x009a0918 (menu) : min=0 max=1 default=1 value=1
0: Manual
1: Auto
exposure_metering_mode 0x009a0919 (menu) : min=0 max=2 default=0 value=0
0: Average
1: Center Weighted
2: Spot
scene_mode 0x009a091a (menu) : min=0 max=13 default=0 value=0
0: None
8: Night
11: Sports
JPEG Compression Controls
compression_quality 0x009d0903 (int) : min=1 max=100 step=1 default=30 value=30
