Researching DVD Subtitle Format

I am attempting to stream webcam video from a BeagleBoneBlack to other computers over ethernet. I want to add an overlay with details about the video. I am capturing video from a Logitech C920 webcam, which is doing the hard work of creating the video on H.264 format, using FFMPEG to MUX the video into a network stream. The current video stream runs at 3Mb/s over ethernet, and seems to run at the same bitrate whether I’m sending video 30FPS at 1920×1080, 1280×720, or any other resolution I’ve tried. If I’m running the BBB at 1GHz FFMPEG uses only 3% load on the processor, while at 300MHz it uses 10% load. Either processor speed indicates that I should have plenty of CPU available for creating a subtitle frame a second.

If I transcode the H264 coming from the C920 to h.264 from FFMPEG the BBB CPU is 100% used and I’ve not been able to get over 5 FPS. This has led me to the idea of adding a second stream with much more compressible data and requiring the client computer to know how to enable subtitles.

My understanding of DVD Subtitles is that they are stored as image overlays. The images seem to be 3 color plus transparency, with the color indexed. They are RLE (Run Length Encoded) images but don’t seem to conform to any standard that would be created by an image library such as OpenCV.

The most useful links I’ve come across related to the DVD subtitles are these three:

Using FFMPEG to examine at a video that was ripped from a DVD into an MKV file with several subtitle layers shows the following:

Stream #0:0(eng): Video: mpeg2video (Main), yuv420p, 720x480 [SAR 32:27 DAR 16:9], SAR 186:157 DAR 279:157, 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc
Stream #0:1(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 448 kb/s (default)
Metadata:
  title           : 3/2+1
Stream #0:2(eng): Audio: ac3, 48000 Hz, 5.1(side), fltp, 384 kb/s
Metadata:
  title           : 3/2+1
Stream #0:3(spa): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s
Metadata:
  title           : 2/0
Stream #0:4(fre): Audio: ac3, 48000 Hz, stereo, fltp, 192 kb/s
Metadata:
  title           : 2/0
Stream #0:5(eng): Subtitle: dvd_subtitle (default)
Stream #0:6(spa): Subtitle: dvd_subtitle
Stream #0:7(eng): Subtitle: dvd_subtitle
Stream #0:8(spa): Subtitle: dvd_subtitle
Stream #0:9(fre): Subtitle: dvd_subtitle

All of the descriptions of creating subtitle tracks are directly related to creating textual subtitles using tools that are wonderful for mainstream movie content but not what I want to do. e.g.

I’ve not figured out how to create my own subtitle stream and am still looking for information on that. I’ve not figured out what parameters may need to be passed to FFMPEG to indicate that I’m passing in a subtitle track. I’ve not figured out if there’s a way in FFMPEG to indicate that the subtitles should be on by default, or forced subtitles, while still keeping them as a separate stream.

It doesn’t help that the DVD subtitle files seem to use the STL extension and that same extension is used for the input files for many 3D Printers.

WimTiVoServer reasons for existence.

I had multiple reasons for writing this program.

I wanted to work with the XMLLite library for XML processing which Microsoft has included as part of their operating system for several years. http://msdn.microsoft.com/en-us/library/windows/desktop/ms752838(v=vs.85).aspx I’ve used XML as my data storage and transfer medium for years but most of the time have manually processed the XML as opposed to using a prebuilt library. My reasoning behind my own parsing and generation was that I was building cross platform solutions that ran on embedded systems working with windows programs. It was easier to build the same code to run on both platforms and assure compatibility than to deal with the overhead and maintenance of third party libraries. I’ve still not formed a full opinion on using the library for reading XML, but it certainly makes creating XML easier by making sure tags are properly closed and nested, and the option of auto indentation in the formatting is nice.

I wanted experience using FFMPEG and its associated video processing libraries. With the release of FFMPEG v1 late last year it became much more capable in dealing with all of the container formats and encoding types that I was interested in, including the WTV containers that Windows Media Center uses and the MKV containers that are common on the internet for high definition files. In my functioning version of my server, I’m using the libraries directly linked into the program to parse the media files for metadata, but spawning a full copy of FFMPEG to do the required transcoding to send the proper format to the TiVo. I’m considering migrating entirely to the spawned FFMPEG process to simplify licensing if I want to make my program publicly available. It would also simplify future support for codecs and containers that FFMPEG may support if my service itself didn’t need to be relinked with updated libraries.

I’ve been frustrated with the state of the TiVo Desktop software provided by the TiVo company. It was designed so that it plugs into the windows video codec stack for video transcoding, as well as the apple protocol stack for Bonjour protocol support. Both of those lead to complications when upgrading programs. Apple regularly releases updates to iTunes. Whenever an update to iTunes needed to be installed, it caused problems because the TiVo service was running and keeping some of the apple services locked in use. It essentially required a full uninstall and reinstall of iTunes every time I needed to update it, with several machine reboots in that process.  Somehow I’d managed to get a codec installed on my machine that allowed the TiVo desktop to support the MKV container. Duplicating that installation on a new machine or server was not something I wanted to attempt. Since the modern FFMPEG supports MKV, I get that support without manipulating a video codec stack in windows.

The TiVo Desktop software only runs as a user process, when a user is logged in. Files are not served from a background service. This is an issue when I’d like to run the process on a headless server. There are ways around the issue, but my program solves it simply by being a pure service. The TiVo Desktop both announces itself via the UDP beacon process on port 2190 and also listens for other servers on the same port. Because my program is purely a server I saw no reason to listen for incoming beacons, and so I do not tie up the UDP port for receiving. This allows secondary programs to be started at the users discretion to listen for TiVo processes on the network.