FFMPEG and ROAV Dash Cam C1 Pro

I recently purchased a dedicated dashcam on sale to replace my GoPro setup for trip videos. This gives me a new need to understand a new file format.

2018-05-18

The Roav Dashcam stores sequential mp4 files. When configuring the camera it’s possible to set the loop time, which is the duration of each mp4. There’s also an option to watermark the files. I have it turned on, and the only thing I’ve noticed is the ROAV logo, timestamp, and speed in the bottom right. It does not appear to have a way of adjusting the size of the text.

My initial recordings were set to run at 1080p 60 fps. I wanted to concatenate multiple files, add some text of my own, and speed up the video. This was my first experience using the -filter_complex option of FFMPEG. Here’s what I came up with to put together three files, speed the output up by a factor of 60, and add some text. I’m dropping the audio completely. The ROAV can record audio inside the car, but I configured it not to, as I don’t want to hear what I was listening to on the radio or what I might be saying if I make a phone call..

ffmpeg.exe -hide_banner -i 2018_0512_130537_050A.MP4 -i 2018_0512_131537_051A.MP4 -i 2018_0512_132537_052A.MP4 -filter_complex "[0:v] [1:v] [2:v] concat=n=3:v=1 [v];[v]setpts=0.01666*PTS,drawtext=fontfile=C\\:/WINDOWS/Fonts/consola.ttf:fontcolor=white:fontsize=80:y=main_h-text_h-50:x=50:text=WimsWorld[o]" -map "[o]" -c:v libx265 -crf 23 -preset veryfast -movflags +faststart -bf 2 -g 15 -pix_fmt yuv420p  FirstMixSpeed60Concat.mp4

This first video was recorded at 1080p60. The camera can record at 1440p30 which I will be trying soon to see if things like license plates are more legible. The setpts factor that I’m currently using was 1/60, so that 1 minute of real time was compressed to 1 second of video, and just dropping the extra frames. I expect to need to change the setpts factor to 1/30 because of the decreased frame rate at the higher resolution.

Advertisements

FFMPEG and h.265

I’ve noticed that YouTube transcodes my videos after I upload them and wanted to know more. It turns out that they are internally using a form of h.265 video encoding, which reduces the data size significantly without reducing perceived quality over h.264 video compression.

I decided to run some tests using my GoPro time lapse program to see how much compression I’d get versus how much extra time for encoding.

First I had to read up on the settings for using h.265 in FFmpeg. According to https://trac.ffmpeg.org/wiki/Encode/H.265, If I add -c:v libx265 into my existing FFmpeg command line without changing anything, I’ll get an h.265 output with the defaults of -crf 23 and -preset medium.

The -preset value effects how much work is done in the compression, but shouldn’t affect the perceived quality. It will effect both time to create and output file size.

The -crf value effects the perceived quality. I’ve been using the defaults in my previous h.264 mp4 files, which should be approximately -crf 23 and supposedly -crf 28 in h.265 is equivalent to the lower h.264 value. A -crf 0 would be a completely lossless conversion. For my tests, I left crf at the default 23.

I ran all of these tests on a set of 7,129 photos I’d captured while sailboat racing on March 24th using my GoPro Hero 3+ Black. Each input image was 4000×3000 and I’m creating an output video with resolution 3840×2160 by cropping it at 3/4 of the height and scaling to fit.

The original h.264 conversion took 28:36 minutes to create and was 1,162,079,866 bytes long.

Here’s a trimmed down and sorted file listing. The first column is the time it took to create, second is filesize, and third is filename.

28:36 1,162,079,866 20180324-2160p30-cropped-h264.mp4
30:25 1,006,292,960 20180324-veryfast-2160p30-cropped-h265-crf20.mp4
21:35   328,700,100 20180324-ultrafast-2160p30-cropped-crf28.mp4
21:59   339,438,778 20180324-superfast-2160p30-cropped-crf28.mp4
25:12   350,085,434 20180324-veryfast-2160p30-cropped-crf28.mp4
25:17   349,720,030 20180324-faster-2160p30-cropped-crf28.mp4
27:24   348,582,310 20180324-fast-2160p30-cropped-crf28.mp4
52:07   365,050,252 20180324-medium-2160p30-cropped-crf28.mp4

I created a single test at -crf 20 because I was interested in seeing a higher quality video and how different it would be in size. It took slightly longer than the original h.264 compression and had a slight improvement in size.

Two things became obvious to me from this test. More time spent in compression doesn’t always mean better compression. For my application, running preset medium actually hurts the performance, both in file size and time taken.

I ran all of these tests only a single time on my desktop computer with an Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz, 3501 Mhz, 4 Core(s), 8 Logical Processor(s) running the latest version of Windows Version 10.0.16299 Build 16299. I was running FFmpeg version N-88668-g723b6baaf8 from https://ffmpeg.zeranoe.com/builds/. The variations in time could be related to background tasks running on the machine, and I should have run a more comprehensive battery of tests, averaging time and with more accurate timekeeping.

Bad Account Management

I have been getting my hair buzzed at various Rudys Barbershops around seattle for the past few years. They’ve had a web site that visitors can log into to make appointments.

It looks like they’ve updated the engine since I last logged in, and it’s not recognizing the password that I am pretty sure I last used. They have a password recovery link.

2018-03-11

Well that’s not useful at all is it? How do you use email address as a unique key for the account and have multiple entries?

VLAN Tagging for CenturyLink

I’ve dealt with CenturyLink provided WiFi access points in two locations I’ve lived recently, and not been happy with their performance. My 5 year old Netgear WNDR3800 seemed to provide better speed with both 5GHz and 2.4GHz than the Actiontec C1900A provided by CenturyLink, which only supported 2.4GHz.

Unfortunately it was not as simple as learning the PPoE credentials that the Actiontec was using and putting those details into the Netgear. Centurylink in their infinite wisdom decided that the network packets need to be tagged to run on VLAN 201.

One solution would be to go out and buy a new WiFi router that supports VLAN Tagging. The newer Netgear Nighthawk routers support tagging, following the details at this support page.

The Netgear AC1900 router (also referred to as R7000) would do what I want, but would also cost close to $150.

Instead I spent $33 on a Netgear GS105Ev2 switch and spent a little time configuring its VLANs and am mostly happy with the result. My only disappointment is that this switch doesn’t seem to support SNMP for traffic monitoring.

gs105ev2

I have this configured so that Port1 connects to the Centurylink Fiber Termination Box, Port2 connects to my WNDR3800 WAN Port, and Port3 is connected to one of the LAN ports on the WNDR3800.

Port1 is configured to send Tagged Packets on VLAN 201.
Port2 is configured to send Untagged Packets on VLAN 201.
Ports 3-5 are configured to sent Untagged Packets on VLAN 1, the default for this switch.

The steps to get this working, starting with existing setup of Actiontec connected to Fiber Termination box.

  1. Connect GS105Ev2 Port3 to available LAN port on Actiontec and make sure link connection LEDs appear.
  2. Find what IP address the GS105Ev2 acquired on local network using a network scanning tool. I used  NirSoft Wireless Network Watcher and found that my switch was on 192.168.1.17. Going to http://192.168.1.17/ gave me a login to the new switch with the default password of “password”. gs105ev2-login
  3. You should get a switch information page similar to this.gs105ev2-login-successful
  4. Select the menu item VLAN, then 802.1Q and the radio button Enable. You should get a warning message that it’s about to erase all current VLAN settings. Hit OK.gs105ev2-vlan1
  5. Go under Advanced, VLAN Configuration, there’s a text box on the right that says VLAN ID. Enter 201 and push the Add button above it. gs105ev2-vlan2Now we have a new VLAN with no Port Members assigned.gs105ev2-vlan3
  6. Go to Port PVID on the left menu. Select Port 1. Type 201 in the text box. Hit Apply.gs105ev2-vlan4Select Port 2. Type 201. Hit Apply.gs105ev2-vlan5
  7. Now we go to the VLAN Membership setting. With the VLAN ID dropdown showing 1, click Port 1 and Port 2 through the available options until neither T nor U is showing, leaving Ports 3, 4, and 5 showing U. Then click Apply.gs105ev2-vlan6
  8. Now drop down to select VLAN 201. Click so that Port 1 is T, Port 2 is U, Ports 3, 4, and 5 are blank, and Apply.gs105ev2-vlan7
  9. If you look at the VLAN Configuration, you’ll now see that ports 1 and 2 are assigned to 201, while 3, 4, and 5 are assigned to 1.gs105ev2-vlan8
  10. At this point the GS105Ev2 has been configured as much as it needs to be. I had already configured my WNDR3800 to connect to the ISP using PPoE and given it the correct credentials.
  11. Power off Actiontec and put it in a closet. Connect Fiber Termination device to port 1 on GS105Ev2. Connect WNDR3800 WAN to port 2 on GS105Ev2. Optionally connect port 3 on GS105Ev2 to a lan port on WNDR3800, as it will only gain you one extra gigabit port compared to the four built into the WNDR3800.

Thanks to this post for the same information that I’ve presented here. I’d attempted to do this before with an existing GS108Tv2 switch I had sitting around. What I’d forgotten to do was make the port going to the WNDR3800 send Untagged packets. I’d been properly sending tagged packets to the fiber, but the WNDR3800 didn’t know what to do with the tagged packets. After confirming it worked with the GS108Tv2 I ordered the cheaper 5 port switch just to have something else to play with. My only disappointment with the 5 port switch is that it doesn’t seem to support SNMP to monitor the traffic going over the network.

Odd Wildcard Matching in Windows 10

I recently ran into an odd behavior of more files matching a pattern than I expected. I’d used exiftool to modify the dates on files my GoPro produced. It creates backup files of the original images when it modifies the tags. Here’s the command I ran.

exiftool.exe -r "-AllDates+=4:7:6 17:40:00" -ext jpg f:\GoPro\20170807

Now I had about 4000 files with the .JPG extension and another 4000 files with a .JPG_original extension.

I ran my program that parses the directory structure and turns all those images into a time lapse movie, and it seemed to be including both the file extensions, making a very disjointed movie.

I loaded my source code in the debugger and it seemed to be doing a findfirst / findnext specifically looking for .JPG files, and not some other extension, but it was definitely retrieving files both with .JPG and .JPG_original extensions.

I then ran a couple of commands at the windows command prompt and was surprised to find the same results there.

dir F:\GoPro\20170807\372GOPRO\G*.JPG /p
dir F:\GoPro\20170807\372GOPRO\G???????.JPG /p

Each command returned both the JPG and JPG_original files.

dir F:\GoPro\20170807\372GOPRO\G*.JPG_original /p

returned just the JPG_original files.

dir F:\GoPro\20170807\372GOPRO\G??????.JPG /p

had one less question mark and correctly returned no files.

This is all unexpected behavior, though I’m glad to see that it was consistent with the operating system and not something specific to the C runtime. I’d love an explanation of what’s going on.

exiftool to manage DJI media files

DJI Drones don’t seem to remember the image count between formats of a media card. This creates a problem for me when I’m trying to backup and maintain my images and video.

Because the dates are all correct in the media files, retrieved from GPS data, organizing the files by naming them based on the date works for me.

Using ExifTool by Phil Harvey is a great solution for pulling the metadata from the files and renaming the files.

The command line that I was initially using is:

exiftool "-FileName<${CreateDate}.$filetype" -d %Y%m%d-%H%M%S%%-c -ext mp4 -ext dng -ext jpg dji*

It’s problem is that it orphans the SRT subtitle files from my videos that I’d like to keep matching the video files.

I’ve tried this variation to do it in one step but it doesn’t work, because the SRT files get renamed as MP4 files.

exiftool -verbose "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.%%le -ext mp4 -ext dng -ext jpg dji* -srcfile %f.srt

If anyone has a suggestion for how to rename all the media files in one directory I’d appreciate it. Even running two commands in sequence would be fine.

Update:

I’ve figured out that running these two commands in sequence will get me the results I am looking for:

exiftool "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.srt -ext mp4 -srcfile %f.srt dji*
exiftool "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.%%le -ext mp4 -ext dng -ext jpg dji*

I’m still looking for a way of doing it in a single command that may leave less room for error, but this is working for now.

FFMPEG and drawtext

Several years ago I wrote a program that consolidates time-lapse pictures into a directory and calls FFMPEG to create a video.

I had been wanting the time-code from when each picture was taken printed on the screen while the video was playing but had not figured out how to get it done until this weekend.

Video TimeCode

Frame from video showing the DateTimeOriginal timecode embedded.

I’d gone down multiple paths in an attempt to get this result before finally getting the drawtext feature to work. My program manually pulled the metadata from the images before feeding them to ffmpg. I’d tried creating both text files and image files for overlaying. none of those got the result that I was looking for.

When I finally got everything working, it seems simple, but the underlying problem has to do with the amount of string escaping required to get the command to work.

Here’s an example command I was issuing to ffmpeg that got the result I was looking for.

ffmpeg.exe -hide_banner -r 30 -i Wim%05d.JPG -vf crop=in_w:3/4*in_h,drawtext=fontfile=C\\:/WINDOWS/Fonts/OCRAEXT.ttf:fontcolor=white:fontsize=160:y=main_h-text_h-50:x=main_w-text_w-50:text=WimsWorld,drawtext=fontfile=C\\:/WINDOWS/Fonts/OCRAEXT.ttf:fontcolor=white:fontsize=160:y=main_h-text_h-50:x=50:text=%{metadata\\:DateTimeOriginal} -s 3840x2160 -pix_fmt yuv420p -n Test-2160p30-cropped.mp4

If you look at the -vf option parameter, I’m cropping my input pictures to 3/4 their original height, then using the drawtext feature twice. First I write the static text to the bottom right of the frame, then I extract metadata from the source image and write it to the bottom left of the frame.

Because I’m calling this from a program, I had extra escaping of the \ character in my code. All of the escaping required a lot of trial and error to get things working. I’m using OCRAEXT as my font, but I could be using any fixed spacing font. because of the fact that the time is changing every frame, it’s important that the font not be proportional to make it easy to read.