Using FFMPEG to Concatenate and Embed Subtitles

I recently upgraded my drone to a DJI Mavic Air 2. Among other things, it can create h.265 videos directly. It still uses the MP4 container format and the separate SRT format for storing video subtitles, the flight data. Following most camera standards, it creates video files that are individually smaller than 4 GB, which works out to be about five minutes in 4k video.

If I want to upload a longer raw video to social media, the video files need to be concatenated before uploading.

Concatenating the video with FFMPEG has been something I’ve known how to do for a long time using either of two methods. Today I learned how to properly embed the subtitles in either the MKV or MP4 container format.

The MP4 format is more widely supported than the MKV format, but is less flexible as to what it can contain.  The MKV (Matroska Multimedia Container) container format can hold almost any type of media, and so I’m able to copy the SRT format directly. The MP4 (MPEG-4 Part 14) container format only supports a limited selection of subtitle formats, so I’m required to have FFMPEG convert the SRT stream to a MP4 compatible stream.  If you are interested in video container formats, these tables are very helpful.

I’ll give several examples using the two video files and their associated subtitle files created by the drone named DJI00001.MP4, DJI00002.MP4, DJI00001.SRT, and DJI00002.SRT. The method I’m using should work for any number of files, up to the largest filesize you can store on your filesystem.

To simply concatenate the video files, create a text input file (I’m using mp4files.txt) with the contents as follows

file DJI00001.MP4
file DJI00001.MP4

then use the ffmpeg command to create a new concatenated file.

ffmpeg -f concat -safe 0 -i mp4files.txt -c copy ConcatenatedVideo.MP4

If you want to embed the subtitles, you need to create a second text file, do some stream mapping, and specify what format the subtitles should be. In this case I’m using srtfiles.txt

file DJI00001.SRT
file DJI00002.SRT

My FFMPEG command to create an MP4 file gets a lot more involved because now I’m specifying multiple inputs and have to specify the subtitle format.

ffmpeg -f concat -safe 0 -i mp4files.txt -f concat -safe 0 -i srtfiles.txt -map 0:v -map 1 -c:v copy -c:s mov_text ConcatenatedVideo.MP4

The FFMPEG command to create an MKV command is only a tiny bit different, and the resulting file is only a tiny bit smaller.

ffmpeg -f concat -safe 0 -i mp4files.txt -f concat -safe 0 -i srtfiles.txt -map 0:v -map 1 -c:v copy -c:s copy ConcatenatedVideo.MKV

When playing the ConcatenatedVideo files on my local machine, I can now enable or disable the closed caption track properly in the player for either format. Unfortunately in my initial testing with YouTube, neither format maintains the second stream of subtitles.

This is not all a waste of time and effort, because an advantage of embedding the subtitles into the container format is that the timing has been matched to the video, and can now be extracted in a concatenated form for use with YouTube.

ffmpeg -i ConcatenatedVideo.MKV -c copy ConcatenatedVideo.SRT

You can exclude the “-c copy” when extracting the subtitles and FFMPEG fill run it through its subrip codec and produce nearly identical results. It will only work with the MKV file because the subtitle format stored in the MP4 file is not easily converted to a SRT file.

Using the -f concat option invokes the concat demuxer in FFMPEG, which has the limitation that the format needs to be exactly the same for each file. If there are any changes between files you want to concatenate, you must use a more involved command invoking the concat filter. In a different project I ran into a command issue with the concat filter command when the command got to be much over 900 characters long.

 

FAA Drone Questionnaire

I got an email from the FAA asking me to participate in a questionnaire about recreational drone use and information the FAA can provide.

One of the questions was How long do you typically fly? My answer was 60 minutes.  I usually have three fully charged batteries, each of which will fly for approximately 28 minutes. I like to land and swap batteries with a few minutes to spare, giving me a total flight time per session of close to an hour.

The form doesn’t like any answer more than 30 minutes. One more example of how out of touch the FAA is with the rapidly advancing technology.

SD Card Holder

I’d been traveling with several microSD cards for both GoPro and Drone usage for the past year when I finally got around to buying this inexpensive plastic card to organize them.

Before buying it, I had read several reviews, and worried that cheap plastic might either make it brittle, or let the flash cards fall out of the enclosure. For $6.95 it was worth a try.

What I really found out was that I should have been using it for at least a year before I got it. The fact that I can carry 10 micro SD cards and they are all in order is a complete boon for organization.

Before using this, I’d often have two identical looking SD cards in my hands and forget which one was full and which one was empty. I’d also take a card out of my device and set it down, and nearly lose it.  Now I take a card out of my camera and immediately put it into this holder. I fill this holder from one end with empty cards and from the other with full cards and never have to remember which cards have something on them any more.

It’s essentially the same size as a credit card, but is quite a bit thicker. I wouldn’t carry it in my wallet, nor would I use the hole for a key ring, but having it in a pocket in my drone bag keeps all the memory cards in one place in a known order.

exiftool to manage DJI media files

DJI Drones don’t seem to remember the image count between formats of a media card. This creates a problem for me when I’m trying to backup and maintain my images and video.

Because the dates are all correct in the media files, retrieved from GPS data, organizing the files by naming them based on the date works for me.

Using ExifTool by Phil Harvey is a great solution for pulling the metadata from the files and renaming the files.

The command line that I was initially using is:

exiftool "-FileName<${CreateDate}.$filetype" -d %Y%m%d-%H%M%S%%-c -ext mp4 -ext dng -ext jpg dji*

It’s problem is that it orphans the SRT subtitle files from my videos that I’d like to keep matching the video files.

I’ve tried this variation to do it in one step but it doesn’t work, because the SRT files get renamed as MP4 files.

exiftool -verbose "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.%%le -ext mp4 -ext dng -ext jpg dji* -srcfile %f.srt

If anyone has a suggestion for how to rename all the media files in one directory I’d appreciate it. Even running two commands in sequence would be fine.

Update:

I’ve figured out that running these two commands in sequence will get me the results I am looking for:

exiftool "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.srt -ext mp4 -srcfile %f.srt dji*
exiftool "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.%%le -ext mp4 -ext dng -ext jpg dji*

I’m still looking for a way of doing it in a single command that may leave less room for error, but this is working for now.

Being a responsible drone pilot

I try to be a responsible drone pilot. I use the FAA B4UFly App on a regular basis to see what it has to say about locations that I’m interested in flying. Unfortunately, it lists every uncontrolled heliport as an airport and reports “Action Required” so often as to be nearly useless.

Yesterday I was sitting on the beach at Golden Gardens Park in Seattle, watching sailboats race offshore. I thought it would be good to check to see what the app would show. I was sitting at sea level, with a hill near me that would be at least 200 feet high. The rules for drones say that you are not allowed to fly over 400 feet above the surface without permission from the controlling authority. Aircraft are not allowed to fly below 700 feet without similar permission.

Golden Gardens is located about where the “E” in KENMORE sits on the map above. It’s under the Class B Airspace that starts at 3,000 feet, going up to 10,000 feet.

While understanding VFR charts  may be more than the average drone pilot should be expected to recognize, especially for a dense area like Seattle, the B4UFly App’s tendency to show warnings is similar to the boy who cried wolf. It’s impossible to recognize when an action is really appropriate.

AMA Membership and BestBuy

An interesting thing I found in the drone aisle in BestBuy was a card to join the Academy of Model Aeronautics.

I’ve been a member since I got involved with drones. I had to become a member to join the rc flying club I frequent. It’s good to see that it’s being easy to recognize in consumer stores, as opposed to the complete hobby level word of mouth that I’ve seen in the past.

Best Buy and DJI Accessories

I don’t shop too much locally for technology, because it always seems that what I want is not available locally. Yesterday I was wandering around killing time and went into the local Best Buy.  Instead of just looking at the normal things I might go for in a best buy, I looked around to see what was new. They now have an aisle dedicated to drones. There were complete drones from DJI, Yuneec, and several other manufacturers. More interesting to me was that they carried accessories for those drones as well.  I knew that they sold drones on-line, it just never occurred to me to look at them in person in the store.

I bought a Mavic Pro directly from DJI soon after they were released. I got the Fly-More package that entails several items including spare propellers. I’ve not needed to replace a propeller in my first 7 hours of flight time but it’s nice to know that propellers are available locally. DJI charges slightly less, $9.00, but they charge for shipping, and you have to wait for the item to arrive.

DJI Phantom and Wind

This week I took my drone to video Initial Point in Idaho. It’s a rocky outcrop about 20 miles south of Boise that was chosen as the initial survey point for the Idaho Territory in 1867. It has a concrete platform installed at the top with a survey marker embedded. There’s a rocky trail that can be driven to the top if your vehicle has a high ground clearance. I chose to walk to the top.

Initial Point Idaho

My car was parked at the base, I climbed up about 130 feet to the top. The wind was occasionally gusting at the base. It was constant at the top, with much higher gusts.

I carried the drone in my backpack to the top, and launched it from the platform. I manually flew it around the point, but was feeling extremely nervous doing so. I only flew about 36 feet above the platform during the entire flight.

The concrete platform has a metal railing surrounding it, and I didn’t trust the drone to return safely to land without hitting the railing so I manually had it land nearby. As I was hovering the drone before landing, it was holding a fairly constant 20° lean because of the wind.

After hiking back to my car I still had plenty of battery for another flight. From near my car I launched the drone and flew vertically to about 200 feet, centered the drone over the survey marker, and used the point of interest feature to create a video circling the point.

I don’t believe that the wind was any less on my second flight than it was on my first flight. The fact that I was in the wind on the first flight had me feeling significantly more nervous while flying than when I was out of the wind on the second flight. Watching the video from the first and second flight doesn’t appear significantly different. I would have liked the point of interest video slightly more if I’d been on top of the point the entire time, but I was too nervous to do it all while I was sitting in the wind myself.

DJI Battery Monitoring

DJI charges a significant amount of money for their batteries and calls them Smart Batteries. I’ve seen the statistics reported of how many times the battery has been power cycled, as well as details of how much flight time is available. This was a new and interesting feature to me.

I had left the drone in the back of my car overnight. The temperature had probably gotten into the high 30s, and was still in the mid 50s with the sun shining. The warning message “Battery Temperature Too Low. Warm battery to at least 15 degrees Celcius before flying” came up on my screen and would not let me initiate a takeoff.

I have used batteries in cold climates in the past. I know how temperature affects both current output from batteries and future usability. I’m impressed that DJI has built in this feature to their firmware.

DJI Phantom 3 Standard

DJI has been the standard drone that people visualize when they think of consumer drones for the past several years. It’s a white quadcopter that comes with a white remote control.

This drone is the camera platform I was hoping for when I got into drones three years ago. The flights are very automated, the camera controls are fully integrated, and I can start flying in a new location with very little setup time. The picture below was taken to demonstrate how close the drone came to returning to its launch location and automatically landing after flying over a thousand feet away from where I was standing. The drone had taken off from on top of the ring, and it landed less than 4 feet from the same location.

In early March DJI dropped the prices of their entire line, convincing me to buy the least expensive unit, now on sale for under $500. I ordered it on March 18th, received it on March 23rd, but didn’t successfully fly it until March 29th.

I live in downtown Seattle. I didn’t want my first flight to be anywhere that would draw excess attention. I’ve been flying drones for three years now, and have grown accustomed to things going wrong. I’d planned on flying at the RC field I regularly visit on the evening of  March 24th. When I got there, first I was not able to get the drone to power on, then when I figured out how to do that, I was forced to do a firmware update before I could fly. By the time I’d got all that out of the way I was running late for a meeting. I decided that the safe thing was to put everything away until my next opportunity.

The Phantom 3 Standard is delivered with two sets of propellers, the transmitter / remote control, a flight battery, an AC battery charger, and a few small extra parts related to the camera gymbal.  It is designed to work with an iPhone or Android phone to both control the unit and see what the camera is doing. I don’t have either of those phones, instead I’m using a Google Nexus 7 tablet. The device connects to the transmitter via WiFi. While my tablet is connected to the transmitter it is not able to connect to the internet.

My first flight with the drone was in a Seattle Park. The battery reports that it can fly up to 25 minutes. None of my previous drones would fly for more than about 13 minutes. The controller app on my tablet has plenty of feedback about the battery condition.

Normal flight mode for me has been that I tell the drone to take off via the app. It takes off and hovers about 4 feet in the air. Then I use the sticks to fly the drone to a location telling it to go up/down or rotate left/right with the left stick, and moving horizontally with the right stick. I usually enable the camera in movie mode before taking off, and only take it out of movie mode if I want to take still photos. When I’m ready to finish, I can either drive it back near myself, or toggle the left switch on the transmitter to cause it to go into return home mode. I’ve got it configured so that it will be at least 100 feet above the ground during return to home mode, which is good enough to clear most trees, but I make sure that’s true early in my flight just in case. If the drone loses contact with the transmitter for more than three seconds it will enter return to home mode.

After owning the drone for one month, I also purchased the DJI backpack specifically designed to carry the drone. It has been what really makes the drone fun for me because I can store the drone in the backpack, knowing it’s fully protected, and easily grab the entire thing and throw it in my car to go somewhere that might have interesting things to photograph.