Using FFMPEG to Concatenate and Embed Subtitles

I recently upgraded my drone to a DJI Mavic Air 2. Among other things, it can create h.265 videos directly. It still uses the MP4 container format and the separate SRT format for storing video subtitles, the flight data. Following most camera standards, it creates video files that are individually smaller than 4 GB, which works out to be about five minutes in 4k video.

If I want to upload a longer raw video to social media, the video files need to be concatenated before uploading.

Concatenating the video with FFMPEG has been something I’ve known how to do for a long time using either of two methods. Today I learned how to properly embed the subtitles in either the MKV or MP4 container format.

The MP4 format is more widely supported than the MKV format, but is less flexible as to what it can contain.  The MKV (Matroska Multimedia Container) container format can hold almost any type of media, and so I’m able to copy the SRT format directly. The MP4 (MPEG-4 Part 14) container format only supports a limited selection of subtitle formats, so I’m required to have FFMPEG convert the SRT stream to a MP4 compatible stream.  If you are interested in video container formats, these tables are very helpful.

I’ll give several examples using the two video files and their associated subtitle files created by the drone named DJI00001.MP4, DJI00002.MP4, DJI00001.SRT, and DJI00002.SRT. The method I’m using should work for any number of files, up to the largest filesize you can store on your filesystem.

To simply concatenate the video files, create a text input file (I’m using mp4files.txt) with the contents as follows

file DJI00001.MP4
file DJI00001.MP4

then use the ffmpeg command to create a new concatenated file.

ffmpeg -f concat -safe 0 -i mp4files.txt -c copy ConcatenatedVideo.MP4

If you want to embed the subtitles, you need to create a second text file, do some stream mapping, and specify what format the subtitles should be. In this case I’m using srtfiles.txt

file DJI00001.SRT
file DJI00002.SRT

My FFMPEG command to create an MP4 file gets a lot more involved because now I’m specifying multiple inputs and have to specify the subtitle format.

ffmpeg -f concat -safe 0 -i mp4files.txt -f concat -safe 0 -i srtfiles.txt -map 0:v -map 1 -c:v copy -c:s mov_text ConcatenatedVideo.MP4

The FFMPEG command to create an MKV command is only a tiny bit different, and the resulting file is only a tiny bit smaller.

ffmpeg -f concat -safe 0 -i mp4files.txt -f concat -safe 0 -i srtfiles.txt -map 0:v -map 1 -c:v copy -c:s copy ConcatenatedVideo.MKV

When playing the ConcatenatedVideo files on my local machine, I can now enable or disable the closed caption track properly in the player for either format. Unfortunately in my initial testing with YouTube, neither format maintains the second stream of subtitles.

This is not all a waste of time and effort, because an advantage of embedding the subtitles into the container format is that the timing has been matched to the video, and can now be extracted in a concatenated form for use with YouTube.

ffmpeg -i ConcatenatedVideo.MKV -c copy ConcatenatedVideo.SRT

You can exclude the “-c copy” when extracting the subtitles and FFMPEG fill run it through its subrip codec and produce nearly identical results. It will only work with the MKV file because the subtitle format stored in the MP4 file is not easily converted to a SRT file.

Using the -f concat option invokes the concat demuxer in FFMPEG, which has the limitation that the format needs to be exactly the same for each file. If there are any changes between files you want to concatenate, you must use a more involved command invoking the concat filter. In a different project I ran into a command issue with the concat filter command when the command got to be much over 900 characters long.

 

FAA Drone Questionnaire

I got an email from the FAA asking me to participate in a questionnaire about recreational drone use and information the FAA can provide.

One of the questions was How long do you typically fly? My answer was 60 minutes.  I usually have three fully charged batteries, each of which will fly for approximately 28 minutes. I like to land and swap batteries with a few minutes to spare, giving me a total flight time per session of close to an hour.

2019-10-04 (2)

The form doesn’t like any answer more than 30 minutes. One more example of how out of touch the FAA is with the rapidly advancing technology.

SD Card Holder

I’d been traveling with several microSD cards for both GoPro and Drone usage for the past year when I finally got around to buying this inexpensive plastic card to organize them.

2018-02-16

Before buying it, I had read several reviews, and worried that cheap plastic might either make it brittle, or let the flash cards fall out of the enclosure. For $6.95 it was worth a try.

What I really found out was that I should have been using it for at least a year before I got it. The fact that I can carry 10 micro SD cards and they are all in order is a complete boon for organization.

20170922_185055016_iOSBefore using this, I’d often have two identical looking SD cards in my hands and forget which one was full and which one was empty. I’d also take a card out of my device and set it down, and nearly lose it.  Now I take a card out of my camera and immediately put it into this holder. I fill this holder from one end with empty cards and from the other with full cards and never have to remember which cards have something on them any more.

It’s essentially the same size as a credit card, but is quite a bit thicker. I wouldn’t carry it in my wallet, nor would I use the hole for a key ring, but having it in a pocket in my drone bag keeps all the memory cards in one place in a known order.

Being a responsible drone pilot

I try to be a responsible drone pilot. I use the FAA B4UFly App on a regular basis to see what it has to say about locations that I’m interested in flying. Unfortunately, it lists every uncontrolled heliport as an airport and reports “Action Required” so often as to be nearly useless.

Yesterday I was sitting on the beach at Golden Gardens Park in Seattle, watching sailboats race offshore. I thought it would be good to check to see what the app would show. I was sitting at sea level, with a hill near me that would be at least 200 feet high. The rules for drones say that you are not allowed to fly over 400 feet above the surface without permission from the controlling authority. Aircraft are not allowed to fly below 700 feet without similar permission.

GoldenGardensVFR

Golden Gardens is located about where the “E” in KENMORE sits on the map above. It’s under the Class B Airspace that starts at 3,000 feet, going up to 10,000 feet.

While understanding VFR charts  may be more than the average drone pilot should be expected to recognize, especially for a dense area like Seattle, the B4UFly App’s tendency to show warnings is similar to the boy who cried wolf. It’s impossible to recognize when an action is really appropriate.

AMA Membership and BestBuy

An interesting thing I found in the drone aisle in BestBuy was a card to join the Academy of Model Aeronautics.


I’ve been a member since I got involved with drones. I had to become a member to join the rc flying club I frequent. It’s good to see that it’s being easy to recognize in consumer stores, as opposed to the complete hobby level word of mouth that I’ve seen in the past.

Best Buy and DJI Accessories

I don’t shop too much locally for technology, because it always seems that what I want is not available locally. Yesterday I was wandering around killing time and went into the local Best Buy.  Instead of just looking at the normal things I might go for in a best buy, I looked around to see what was new. They now have an aisle dedicated to drones. There were complete drones from DJI, Yuneec, and several other manufacturers. More interesting to me was that they carried accessories for those drones as well.  I knew that they sold drones on-line, it just never occurred to me to look at them in person in the store.

I bought a Mavic Pro directly from DJI soon after they were released. I got the Fly-More package that entails several items including spare propellers. I’ve not needed to replace a propeller in my first 7 hours of flight time but it’s nice to know that propellers are available locally. DJI charges slightly less, $9.00, but they charge for shipping, and you have to wait for the item to arrive.

img_1717

DJI Battery Monitoring

DJI charges a significant amount of money for their batteries and calls them Smart Batteries. I’ve seen the statistics reported of how many times the battery has been power cycled, as well as details of how much flight time is available. This was a new and interesting feature to me.

Screenshot_20160502-075201

I had left the drone in the back of my car overnight. The temperature had probably gotten into the high 30s, and was still in the mid 50s with the sun shining. The warning message “Battery Temperature Too Low. Warm battery to at least 15 degrees Celcius before flying” came up on my screen and would not let me initiate a takeoff.

I have used batteries in cold climates in the past. I know how temperature affects both current output from batteries and future usability. I’m impressed that DJI has built in this feature to their firmware.

DJI Phantom 3 Standard

DJI has been the standard drone that people visualize when they think of consumer drones for the past several years. It’s a white quadcopter that comes with a white remote control.

This drone is the camera platform I was hoping for when I got into drones three years ago. The flights are very automated, the camera controls are fully integrated, and I can start flying in a new location with very little setup time. The picture below was taken to demonstrate how close the drone came to returning to its launch location and automatically landing after flying over a thousand feet away from where I was standing. The drone had taken off from on top of the ring, and it landed less than 4 feet from the same location.

WP_20160423_002

In early March DJI dropped the prices of their entire line, convincing me to buy the least expensive unit, now on sale for under $500. I ordered it on March 18th, received it on March 23rd, but didn’t successfully fly it until March 29th.

I live in downtown Seattle. I didn’t want my first flight to be anywhere that would draw excess attention. I’ve been flying drones for three years now, and have grown accustomed to things going wrong. I’d planned on flying at the RC field I regularly visit on the evening of  March 24th. When I got there, first I was not able to get the drone to power on, then when I figured out how to do that, I was forced to do a firmware update before I could fly. By the time I’d got all that out of the way I was running late for a meeting. I decided that the safe thing was to put everything away until my next opportunity.

The Phantom 3 Standard is delivered with two sets of propellers, the transmitter / remote control, a flight battery, an AC battery charger, and a few small extra parts related to the camera gymbal.  It is designed to work with an iPhone or Android phone to both control the unit and see what the camera is doing. I don’t have either of those phones, instead I’m using a Google Nexus 7 tablet. The device connects to the transmitter via WiFi. While my tablet is connected to the transmitter it is not able to connect to the internet.

My first flight with the drone was in a Seattle Park. The battery reports that it can fly up to 25 minutes. None of my previous drones would fly for more than about 13 minutes. The controller app on my tablet has plenty of feedback about the battery condition.

Normal flight mode for me has been that I tell the drone to take off via the app. It takes off and hovers about 4 feet in the air. Then I use the sticks to fly the drone to a location telling it to go up/down or rotate left/right with the left stick, and moving horizontally with the right stick. I usually enable the camera in movie mode before taking off, and only take it out of movie mode if I want to take still photos. When I’m ready to finish, I can either drive it back near myself, or toggle the left switch on the transmitter to cause it to go into return home mode. I’ve got it configured so that it will be at least 100 feet above the ground during return to home mode, which is good enough to clear most trees, but I make sure that’s true early in my flight just in case. If the drone loses contact with the transmitter for more than three seconds it will enter return to home mode.

After owning the drone for one month, I also purchased the DJI backpack specifically designed to carry the drone. It has been what really makes the drone fun for me because I can store the drone in the backpack, knowing it’s fully protected, and easily grab the entire thing and throw it in my car to go somewhere that might have interesting things to photograph.

 

FrSky X4rSB Receiver controlling Eachine Racer 250

I wanted to use S.Bus communication to connect my receiver to my new Eachine 250 
Racer drone. I wanted to be able to remotely control the lights on the drone. I wanted to bring back telemetry from the drone to my transmitter. All of those are possible with the X series receivers from FrSky.

20160513_093703

I’ve used the X8R receivers in the past, but I bought an X4R-SB receiver specifically for this drone.

In it’s default configuration it will output PWM on the pins for Channels 1 through 3 and S.Bus on the 4th connector. It has a separate input connection on the side for S.Port telemetry devices and an analog data line.

By putting a jumper on the signal pins, as shown in the picture above, during the binding process, the output is changed so that CPPM channels 1-8 are on the first port, PWM Channel 9 and 10 on the second and third ports, and S.Bus on the fourth port.

I’ve got the S.Bus connection going to the main port on my CC3D flight controller, a cable to control the lights on the third port (channel 10) and the first two ports remain unused. I have the momentary switch SH on my Taranis configured to control Channel 10. Toggling it cycles the LEDs on my Eachine Racer through the  off and on colored states.

The telemetry cable is connected first to a FrSky SP-GPS – Smart Port GPS Sensor (GPS-V2) and daisy chain connected to a FrSky SP-FLVS – Smart Port Lipo Voltage Sensor.

20160513_10061520160513_10062720160513_10162020160513_101628

The GPS Sensor is a new item in my arsenal. I’ve used GPS chips connected to my flight controllers in the past, which allow the flight controller to direct its flight position using GPS. Since this is not connected into the flight controller, it’s purely a toy, though it may help finding the drone if I get confused and crash it away from myself. The last data received in my transmitter remains on the display, which I could then use to assist my search for the drone.

The voltage sensor connects to the balance plug on the battery, allowing me to monitor the state of the individual battery cells during flight, as well as having low battery alerts reported on the overall voltage.

For installation of this receiver in my drone, I’d come across a 3d model for a holder. I had never used a 3d printer before, though I’ve been fascinated with them for several years. I realized that Windows 10 has a 3d Modeling program installed by default, 3D Builder, and that it can print using an on-line service.  Because the service accepts the file and tells you how much it is going to cost delivered, this was an easy first try. I downloaded files for both a battery protector tray and the previously mentioned receiver tray, merged them into a single model file, and had them delivered for $27.15. Other than the time involved for the delivery I was happy with the result. I submitted the order on 4/26/2016. I received notice that it shipped on 5/10/2016. It finally arrived on 5/13/2016.

20160517_17394420160517_17375520160517_174014

The design here has the receiver extending in the model over the flight controller and under the video transmitter. I’m not positive that I’m going to use the battery tray. If I don’t use it, I just need to get screw extenders to install the receiver directly above the flight controller.

Eachine Racer 250

I decided to get into the FPV Racing Drone scene and found this unit available in an almost ready to fly version for $140.

20160513_101937

Almost ready to fly means that I need to add my own transmitter and receiver. Because it’s FPV, it also means that I need my own video receiver and display.

I have an FrSky Taranis transmitter, several compatible receivers, and a set of Fatshark Attitude V2 goggles,  so when the model arrived, I expected all I would need to do was charge the battery and it was ready to fly.

It turned out it was slightly more complicated, but the past three years of playing with drones meant that it wasn’t too involved.

First I found a useful wiki page dedicated to the racer, with plenty of information. It allowed me to understand that I’d got version 5 of the racer, with the difference from version 4 being that they’d removed a safety tray that kept the battery from crushing the flight controller.

I figured out that to use S.Bus to communicate between my receiver and the flight controller, I’d need to download software to my PC, connect the PC to the flight controller with a USB cable, and configure it’s inputs. The instructions I found all referred to OpenPilot. OpenPilot is an open source project that is now defunct, including the domain name itself not going anywhere. The replacement project that is functionally similar, is LibrePilot. I believe that http://opwiki.readthedocs.io/en/latest/user_manual/index.html is the old documentation from the OpenPilot project, while the LibrePilot home page has links to all of the new project.

After downloading LibrePilot and installing the software on my PC, I couldn’t change the settings on the firmware that had shipped on my drone without upgrading to new firmware and resetting all of the configuration.  After some hesitation, I allowed it to upgrade, then followed the wizard to configure the new firmware. I was impressed at the ease of setup, and later when I flew the device, it proved that the defaults worked nicely out of the box.

FirmwareMismatch2016-04-20 (1)2016-04-20 (3)2016-04-20 (4)

The original firmware had a rose icon, while the new firmware shows a warning icon. That could mean something important, but I never found any mention and the system seems to be working correctly. You can see that the drone shipped with a firmware dated 2015-03-12 and the new version is dated 2015-10-21.

Connecting the receiver to the flight controller using the S.Bus connection required using the MainPort connection on the flight controller. An appropriate cable shipped with the device that included 4 wires, colored black, red, green, and yellow. For S.Bus operation, the green wire is not used. I removed it from the flight controller side of the cable. When connecting to the receiver S.Bus, Black = (-) Red = (+) Yellow = (Signal).

20160517_173734

My goggles only support 8 channels. The drone video transmitter supports 32. Finding a match required a bit of reading, and then deciding on what was least likely to cause conflicts with other people near the field I regularly fly at. I chose D7 on my transmitter and CH6 on my goggles, which worked out to 5840MHz.  The newer version of my goggles supports 32 channels. (8 channels, on each of 4 bands.)

I found that http://www.rcgroups.com/forums/showthread.php?t=2266883 has a very nice explanation of the frequency bands used for FPV including details of how they overlap and recommendations for which frequencies to chose for the least interference between racers. It includes a google docs spreadsheet that’s been color coded to have the frequencies sorted in ascending order and make the bands more visible.

I got the video transmitter and receiver in sync by selecting the channel on my goggles, then cycling through all of the possible channels on the transmitter until I got the clearest video picture. My goggles auto select NTSC or PAL depending on the signal they receive. A friend using a full sized monitor wasn’t as lucky. That’s how I figured that the Eachine Racer 250 ships with a PAL camera.

The Eachine Racer 250 has a pair of bright white LEDs on the front, one on either side of the camera, and a LED lightbar on the back. The LED light bar on the back can be cycled through a series of colors by sliding a power switch located on the left side of the main board that turns all the LEDs off and on. There is also a two pin cable connector that can be plugged in to a receiver PWM output allowing remote light control.  To use remote light control, the local switch on the drone must be in the OFF position.  I’ll write more about this in a separate post.

After going through the LibrePilot setup wizard and putting the appropriate propellers on each motor, the racer flew completely as expected. It’s very responsive, and also very resilient to the basic crashes I’ve had so far.  The biggest learning experience for me has been to add throttle when I think I’m going to crash on the ground. With my larger drones, I’ve wanted to stop and recover the drone when it hits the ground. With this drone, it is much more likely to bounce and be able to recover itself if I can get it off the ground.  I recommend this drone as a good entry into the FPV racing drone scene. I’m sure that there are plenty of drones that are more resilient or responsive, but there’s also plenty more that can be spent than I did on this.