Govee GVH5075 Thermometer Hygrometer

Last week I came across a deal on a small thermometer with display and Bluetooth access for under $10 so I had to give it a try. The fact that the data is available via bluetooth instead of via a web service was a major selling point for me. I am hoping to be able to to log the data via a Raspberry Pi4.

GVH5070 near my Raspberry Pi4

I installed the Govee Home app on my iPhone and it was able to find the device, communicate with it, and pull both current and accumulated data.

When I attempted to find it from my Pi4 it was much more difficult. I live in an apartment with units all around. I’m not just dealing with my own devices that may be visible, but my neighbors as well.

I managed to find the device using linux command line tools, but was not able to successfully connect. A friend suggested BLE Scanner 4.0 for my iPhone for discovering the details, and it was at least able to confirm what I should be looking for using the linux command line tools. I still had timeout issues with the iPhone app, but at least was able to confirm that I could connect to the device and retrieve GUID information.

This is my first time attempting to gather data from a Bluetooth device. I’m still in the research and test phase. I’m listing a bunch of the URLS I’ve found that have been helpful.

https://www.reddit.com/r/Govee/comments/e8ljbp/work_to_access_data_from_a_govee_h5075_indoor/
https://www.jaredwolff.com/get-started-with-bluetooth-low-energy/
https://github.com/neilsheps/GoveeTemperatureAndHumidity
https://www.raspberrypi.org/forums/viewtopic.php?f=37&t=241686
https://www.cnet.com/how-to/how-to-setup-bluetooth-on-a-raspberry-pi-3/
https://www.real-world-systems.com/docs/hcitool.1.html

From the command line on my Pi4 I already had the tools installed to try several Bluetooth commands. I believe they were installed as part of the bluez package. The first two commands below get details on the Raspberry Pi Bluetooth hardware, then the hcitool lescan command produced a lot of devices, and I found the line referencing the GVH5075 so I could use the address in further commands.

pi@WimPi4:~ $ sudo hcitool dev
Devices:
        hci0    DC:A6:32:1C:B5:74

pi@WimPi4:~ $ sudo hciconfig -a
hci0:   Type: Primary  Bus: UART
        BD Address: DC:A6:32:1C:B5:74  ACL MTU: 1021:8  SCO MTU: 64:1
        UP RUNNING 
        RX bytes:21284 acl:25 sco:0 events:791 errors:0
        TX bytes:4401 acl:26 sco:0 commands:172 errors:0
        Features: 0xbf 0xfe 0xcf 0xfe 0xdb 0xff 0x7b 0x87
        Packet type: DM1 DM3 DM5 DH1 DH3 DH5 HV1 HV2 HV3 
        Link policy: RSWITCH SNIFF 
        Link mode: SLAVE ACCEPT 
        Name: 'WimPi4'
        Class: 0x000000
        Service Classes: Unspecified
        Device Class: Miscellaneous, 
        HCI Version: 5.0 (0x9)  Revision: 0x13b
        LMP Version: 5.0 (0x9)  Subversion: 0x6119
        Manufacturer: Cypress Semiconductor Corporation (305)

pi@WimPi4:~ $ sudo hcitool lescan
LE Scan ...
7B:F9:68:96:C4:92 (unknown)
57:FA:0A:E7:61:A4 (unknown)
A4:C1:38:37:BC:AE GVH5075_BCAE
A4:C1:38:37:BC:AE (unknown)
15:FF:0C:3F:E7:35 (unknown)
57:FA:0A:E7:61:A4 (unknown)

pi@WimPi4:~ $ sudo hcitool leinfo A4:C1:38:37:BC:AE
Requesting information ...
        Handle: 64 (0x0040)
        LMP Version: 4.2 (0x8) LMP Subversion: 0x22bb
        Manufacturer: Telink Semiconductor Co. Ltd (529)
        Features: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

Another command that I attempted before I used the hcitool command was the bluetoothctl command. It scrolls a lot of data, but now that I have an idea what I’m looking at, I may be able to see announcement data from the thermometer periodically in the stream by filtering just to see the data coming from the MAC address.

pi@WimPi4:~ $ sudo bluetoothctl
Agent registered
[bluetooth]# scan on
Discovery started
[CHG] Controller DC:A6:32:1C:B5:74 Discovering: yes
[NEW] Device 57:75:EA:B6:EC:2B 57-75-EA-B6-EC-2B
[NEW] Device E7:E7:B4:AB:4A:1F 846B219FB80338A3E9
[NEW] Device 48:56:2E:FF:59:45 48-56-2E-FF-59-45
[NEW] Device 46:53:2F:D4:6F:A1 46-53-2F-D4-6F-A1
[NEW] Device 5C:C9:C5:C9:70:5F 5C-C9-C5-C9-70-5F
[NEW] Device 48:CF:F7:19:4A:3A 48-CF-F7-19-4A-3A
[NEW] Device 4E:30:D1:5D:0F:48 4E-30-D1-5D-0F-48
[NEW] Device 7D:4A:A3:81:32:22 7D-4A-A3-81-32-22
[NEW] Device 7E:0F:63:2B:DC:3E 7E-0F-63-2B-DC-3E
[NEW] Device 7F:5D:37:A2:4E:BA 7F-5D-37-A2-4E-BA
[NEW] Device 7F:6B:44:CD:3A:E5 7F-6B-44-CD-3A-E5
[NEW] Device 00:07:80:37:BD:35 00-07-80-37-BD-35
[NEW] Device 04:52:C7:BC:1C:E3 LE-Bose Revolve SoundLink
[NEW] Device 4F:84:D2:AC:59:FF 4F-84-D2-AC-59-FF
[NEW] Device 4E:F0:6A:DD:3D:7E 4E-F0-6A-DD-3D-7E
[NEW] Device 75:25:34:3F:B9:29 75-25-34-3F-B9-29
[NEW] Device 60:EC:A4:49:B6:67 60-EC-A4-49-B6-67
[NEW] Device 98:D6:BB:20:EB:3B 98-D6-BB-20-EB-3B
[NEW] Device 78:13:28:A8:0A:FF 78-13-28-A8-0A-FF
[NEW] Device 56:6F:B2:E0:40:E3 56-6F-B2-E0-40-E3
[NEW] Device 69:D9:38:44:5C:04 69-D9-38-44-5C-04
[NEW] Device 56:63:50:90:82:D6 56-63-50-90-82-D6
[CHG] Device A4:C1:38:37:BC:AE RSSI: -43
[CHG] Device A4:C1:38:37:BC:AE ManufacturerData Key: 0xec88
[CHG] Device A4:C1:38:37:BC:AE ManufacturerData Value:
00 03 32 62 64 00 ..2bd.
[CHG] Device A4:C1:38:37:BC:AE ManufacturerData Key: 0x004c
[CHG] Device A4:C1:38:37:BC:AE ManufacturerData Value:
02 15 49 4e 54 45 4c 4c 49 5f 52 4f 43 4b 53 5f ..INTELLI_ROCKS_
48 57 50 75 f2 ff c2 HWPu…
[CHG] Device 75:25:34:3F:B9:29 RSSI: -83
[NEW] Device 47:10:2F:15:99:2E 47-10-2F-15-99-2E
[NEW] Device B8:31:B5:8B:12:D2 ETOBAN386
[NEW] Device F0:6E:0B:D1:1B:BF ELRWLK345
[CHG] Device 75:25:34:3F:B9:29 RSSI: -72
[CHG] Device 7D:4A:A3:81:32:22 RSSI: -89
[CHG] Device 7D:4A:A3:81:32:22 RSSI: -81
[CHG] Device 98:D6:BB:20:EB:3B RSSI: -94
[NEW] Device A4:83:E7:20:06:5B A4-83-E7-20-06-5B
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Key: 0x05a7
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Value:
03 13 31 68 39 63 51 6f 4b 76 54 34 00 ..1h9cQoKvT4.
[NEW] Device 00:07:80:37:CA:7D 00-07-80-37-CA-7D
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Key: 0x05a7
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Value:
03 12 78 4d 32 49 31 6d 31 6a 6f 32 67 ..xM2I1m1jo2g
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Key: 0x05a7
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Value:
03 10 01 99 44 de ad be ef 00 0a 00 ca ….D……..
[CHG] Device 4E:30:D1:5D:0F:48 ManufacturerData Key: 0x004c
[CHG] Device 4E:30:D1:5D:0F:48 ManufacturerData Value:
10 06 10 1e b0 2a e1 be …..*..
[CHG] Device 98:D6:BB:20:EB:3B RSSI: -85
[NEW] Device 00:07:80:37:BE:C9 523
[CHG] Device 5C:C9:C5:C9:70:5F ManufacturerData Key: 0x004c
[CHG] Device 5C:C9:C5:C9:70:5F ManufacturerData Value:
10 06 5a 1e 56 a0 e1 eb ..Z.V…
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Key: 0x05a7
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Value:
03 13 31 68 39 63 51 6f 4b 76 54 34 00 ..1h9cQoKvT4.
[CHG] Device 75:25:34:3F:B9:29 RSSI: -81
[NEW] Device 6B:C2:D2:28:1E:A5 6B-C2-D2-28-1E-A5
[CHG] Device 5C:C9:C5:C9:70:5F ManufacturerData Key: 0x004c
[CHG] Device 5C:C9:C5:C9:70:5F ManufacturerData Value:
0c 0e 00 41 32 56 c8 79 5a 01 9d 63 d5 79 c7 80 …A2V.yZ..c.y..
10 06 56 1e 56 a0 e1 eb ..V.V…
[CHG] Device A4:C1:38:37:BC:AE RSSI: -35
[CHG] Device A4:C1:38:37:BC:AE ManufacturerData Key: 0xec88
[CHG] Device A4:C1:38:37:BC:AE ManufacturerData Value:
00 03 32 61 64 00 ..2ad.
[CHG] Device A4:C1:38:37:BC:AE ManufacturerData Key: 0x004c
[CHG] Device A4:C1:38:37:BC:AE ManufacturerData Value:
02 15 49 4e 54 45 4c 4c 49 5f 52 4f 43 4b 53 5f ..INTELLI_ROCKS_
48 57 50 75 f2 ff c2 HWPu…
[CHG] Device 48:CF:F7:19:4A:3A RSSI: -76
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Key: 0x05a7
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Value:
03 10 01 99 44 de ad be ef 00 0a 00 ca ….D……..
[NEW] Device 78:11:F9:E8:7A:DA 78-11-F9-E8-7A-DA
[CHG] Device 47:10:2F:15:99:2E RSSI: -84
[CHG] Device 69:D9:38:44:5C:04 RSSI: -83
[CHG] Device 47:10:2F:15:99:2E ManufacturerData Key: 0x004c
[CHG] Device 47:10:2F:15:99:2E ManufacturerData Value:
10 06 1c 1e 9a e0 28 9b ……(.
[CHG] Device 5C:C9:C5:C9:70:5F ManufacturerData Key: 0x004c
[CHG] Device 5C:C9:C5:C9:70:5F ManufacturerData Value:
0c 0e 00 42 32 7b fc b2 b6 a1 46 31 82 0f 67 02 …B2{….F1..g.
10 06 56 1e 56 a0 e1 eb ..V.V…
[CHG] Device 75:25:34:3F:B9:29 RSSI: -73
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Key: 0x05a7
[CHG] Device E7:E7:B4:AB:4A:1F ManufacturerData Value:
03 13 31 68 39 63 51 6f 4b 76 54 34 00 ..1h9cQoKvT4.
[CHG] Device 47:10:2F:15:99:2E ManufacturerData Key: 0x004c
[CHG] Device 47:10:2F:15:99:2E ManufacturerData Value:
10 06 14 1e 9a e0 28 9b ……(.
[NEW] Device 5C:53:86:8D:A4:61 5C-53-86-8D-A4-61
[NEW] Device 42:32:EC:5F:59:C5 42-32-EC-5F-59-C5
[bluetooth]# scan off
Discovery stopped
[CHG] Device E7:E7:B4:AB:4A:1F TxPower is nil
[CHG] Device E7:E7:B4:AB:4A:1F RSSI is nil
[DEL] Device E7:E7:B4:AB:4A:1F 846B219FB80338A3E9
[CHG] Controller DC:A6:32:1C:B5:74 Discovering: no
[CHG] Device 42:32:EC:5F:59:C5 TxPower is nil
[CHG] Device 42:32:EC:5F:59:C5 RSSI is nil
[CHG] Device 5C:53:86:8D:A4:61 RSSI is nil
[CHG] Device 78:11:F9:E8:7A:DA TxPower is nil
[CHG] Device 78:11:F9:E8:7A:DA RSSI is nil
[CHG] Device 6B:C2:D2:28:1E:A5 TxPower is nil
[CHG] Device 6B:C2:D2:28:1E:A5 RSSI is nil
[CHG] Device 00:07:80:37:BE:C9 RSSI is nil
[CHG] Device 00:07:80:37:CA:7D RSSI is nil
[CHG] Device A4:83:E7:20:06:5B RSSI is nil
[CHG] Device F0:6E:0B:D1:1B:BF TxPower is nil
[CHG] Device F0:6E:0B:D1:1B:BF RSSI is nil
[CHG] Device B8:31:B5:8B:12:D2 TxPower is nil
[CHG] Device B8:31:B5:8B:12:D2 RSSI is nil
[CHG] Device 47:10:2F:15:99:2E TxPower is nil
[CHG] Device 47:10:2F:15:99:2E RSSI is nil
[CHG] Device A4:C1:38:37:BC:AE RSSI is nil
[CHG] Device 56:63:50:90:82:D6 RSSI is nil
[CHG] Device 69:D9:38:44:5C:04 TxPower is nil
[CHG] Device 69:D9:38:44:5C:04 RSSI is nil
[CHG] Device 56:6F:B2:E0:40:E3 TxPower is nil
[CHG] Device 56:6F:B2:E0:40:E3 RSSI is nil
[CHG] Device 78:13:28:A8:0A:FF TxPower is nil
[CHG] Device 78:13:28:A8:0A:FF RSSI is nil
[CHG] Device 98:D6:BB:20:EB:3B RSSI is nil
[CHG] Device 60:EC:A4:49:B6:67 TxPower is nil
[CHG] Device 60:EC:A4:49:B6:67 RSSI is nil
[CHG] Device 75:25:34:3F:B9:29 TxPower is nil
[CHG] Device 75:25:34:3F:B9:29 RSSI is nil
[CHG] Device 4E:F0:6A:DD:3D:7E TxPower is nil
[CHG] Device 4E:F0:6A:DD:3D:7E RSSI is nil
[CHG] Device 4F:84:D2:AC:59:FF TxPower is nil
[CHG] Device 4F:84:D2:AC:59:FF RSSI is nil
[CHG] Device 04:52:C7:BC:1C:E3 TxPower is nil
[CHG] Device 04:52:C7:BC:1C:E3 RSSI is nil
[CHG] Device 00:07:80:37:BD:35 RSSI is nil
[CHG] Device 7F:6B:44:CD:3A:E5 TxPower is nil
[CHG] Device 7F:6B:44:CD:3A:E5 RSSI is nil
[CHG] Device 7F:5D:37:A2:4E:BA TxPower is nil
[CHG] Device 7F:5D:37:A2:4E:BA RSSI is nil
[CHG] Device 7E:0F:63:2B:DC:3E TxPower is nil
[CHG] Device 7E:0F:63:2B:DC:3E RSSI is nil
[CHG] Device 7D:4A:A3:81:32:22 TxPower is nil
[CHG] Device 7D:4A:A3:81:32:22 RSSI is nil
[CHG] Device 4E:30:D1:5D:0F:48 TxPower is nil
[CHG] Device 4E:30:D1:5D:0F:48 RSSI is nil
[CHG] Device 48:CF:F7:19:4A:3A TxPower is nil
[CHG] Device 48:CF:F7:19:4A:3A RSSI is nil
[CHG] Device 5C:C9:C5:C9:70:5F TxPower is nil
[CHG] Device 5C:C9:C5:C9:70:5F RSSI is nil
[CHG] Device 46:53:2F:D4:6F:A1 TxPower is nil
[CHG] Device 46:53:2F:D4:6F:A1 RSSI is nil
[CHG] Device 48:56:2E:FF:59:45 TxPower is nil
[CHG] Device 48:56:2E:FF:59:45 RSSI is nil
[CHG] Device 57:75:EA:B6:EC:2B TxPower is nil
[CHG] Device 57:75:EA:B6:EC:2B RSSI is nil
[DEL] Device 57:75:EA:B6:EC:2B 57-75-EA-B6-EC-2B
[DEL] Device 48:56:2E:FF:59:45 48-56-2E-FF-59-45
[DEL] Device 46:53:2F:D4:6F:A1 46-53-2F-D4-6F-A1
[DEL] Device 5C:C9:C5:C9:70:5F 5C-C9-C5-C9-70-5F
[DEL] Device 48:CF:F7:19:4A:3A 48-CF-F7-19-4A-3A
[DEL] Device 4E:30:D1:5D:0F:48 4E-30-D1-5D-0F-48
[DEL] Device 7D:4A:A3:81:32:22 7D-4A-A3-81-32-22
[DEL] Device 7E:0F:63:2B:DC:3E 7E-0F-63-2B-DC-3E
[DEL] Device 7F:5D:37:A2:4E:BA 7F-5D-37-A2-4E-BA
[DEL] Device 7F:6B:44:CD:3A:E5 7F-6B-44-CD-3A-E5
[DEL] Device 00:07:80:37:BD:35 00-07-80-37-BD-35
[DEL] Device 04:52:C7:BC:1C:E3 LE-Bose Revolve SoundLink
[DEL] Device 4F:84:D2:AC:59:FF 4F-84-D2-AC-59-FF
[DEL] Device 4E:F0:6A:DD:3D:7E 4E-F0-6A-DD-3D-7E
[DEL] Device 75:25:34:3F:B9:29 75-25-34-3F-B9-29
[DEL] Device 60:EC:A4:49:B6:67 60-EC-A4-49-B6-67
[DEL] Device 98:D6:BB:20:EB:3B 98-D6-BB-20-EB-3B
[DEL] Device 78:13:28:A8:0A:FF 78-13-28-A8-0A-FF
[DEL] Device 56:6F:B2:E0:40:E3 56-6F-B2-E0-40-E3
[DEL] Device 69:D9:38:44:5C:04 69-D9-38-44-5C-04
[DEL] Device 56:63:50:90:82:D6 56-63-50-90-82-D6
[DEL] Device 47:10:2F:15:99:2E 47-10-2F-15-99-2E
[DEL] Device B8:31:B5:8B:12:D2 ETOBAN386
[DEL] Device F0:6E:0B:D1:1B:BF ELRWLK345
[DEL] Device A4:83:E7:20:06:5B A4-83-E7-20-06-5B
[DEL] Device 00:07:80:37:CA:7D 00-07-80-37-CA-7D
[DEL] Device 00:07:80:37:BE:C9 523
[DEL] Device 6B:C2:D2:28:1E:A5 6B-C2-D2-28-1E-A5
[DEL] Device 78:11:F9:E8:7A:DA 78-11-F9-E8-7A-DA
[DEL] Device 5C:53:86:8D:A4:61 5C-53-86-8D-A4-61
[DEL] Device 42:32:EC:5F:59:C5 42-32-EC-5F-59-C5
[bluetooth]# exit

I’m posting all of this here and hopefully will be able to make progress on retrieving the data in the next few days.

iTunes, Microsoft Store, COM Interface Type Library

Several years ago I’d written a program to manipulate data in the iTunes library using the approved Apple COM API. Part of the way this works in a C program is to include a type library in the headers defining all of the function calls. When iTunes is installed in the traditional way, Apple embedded the type library in the executable, and the executable was installed in a traditional location.

#import "C:/Program Files (x86)/iTunes/iTunes.exe"
using namespace iTunesLib;

With the installation of iTunes from the Microsoft store, the iTunes executable no longer lives in that location. Today my application builds properly with the following import command, but it may change mysteriously with version changes and automatic updates via the store.

#import "C:/Program Files/WindowsApps/AppleInc.iTunes_12093.3.37141.0_x64__nzyj5cx40ttqa/iTunes.exe"
using namespace iTunesLib;

My program builds and runs more reliably than it used to, which I’m assuming is in part due to the fact that I appear to now be using a 64 bit version of iTunes, and all the extra work Apple put in to make iTunes more reliable on windows in general.

Finding the iTunes application itself was the hardest part of the transition. I’m happy the API still exists because Apple no longer hosts easy access to the documentation for the API, and http://www.joshkunz.com/iTunesControl/ seems to be the most complete and searchable information.

Part 3 of ROAV Dash Cam C1 Pro

After getting the video files from my dashcam strung together at high speed I realized I was just as interested in the GPS data that was stored in .info files alongside each .mp4 file.

Some digging around in forums led me to believe that the files were fairly simple comma separated files with the headers working out to be Datetime, Latitude, Longitude, FixType, SatCount, Altitude, SpeedKph, Heading, AccelerometerX, AccelerometerY, AccelerometerZ. The Datetime field is it’s own special format, but easy enough to interpret.

Because I’ve written a GPS data app in the past, I’d learned how to write Keyhole Markup Language (KML) files, as well as learning to use ZIP routines to package them into KMZ files. My original code had hard coded the KML tags because I didn’t want to rely on an external library requirement on a limited platform. In re-using the code I updated to use the XMLLite API. Microsoft may be discontinuing support for this API as well, but at least now it’s still included with current operating systems. The advantage of using the XML API to create the XML is knowing that all of the tags are properly and consistently formed and closed. A secondary feature was that it made it much simpler to explore different data formatting options for the folders in the KML structure.

ROAV-ConCatStructureI worked on the basic idea that the most interesting data from the source files was based on speed and altitude. Then I broke the data into segments by the day, so if I store multiple days worth of files the kml will automatically have reasonable breaks in it. I create a Placemark that includes a LineString with all the GPS coordinates for the day. I calculate the distance by multiplying the reported speed by the time between points.

The calculated distance isn’t as accurate as I’d like, but the code I had from years ago didn’t seem to get more accurate distances. My web references from years ago no longer work, which is the frustrating thing about pointing to documentation on the web. The code I’d used years before was used in a bicycle gps program, which meant that the speeds were slower and the distance traveled between points may have been smaller. The data the ROAV is writing to the log file may not have as many significant digits as the raw data available from the the GPS chip itself.

I create three more Placemarks for each day, each with a simple Point defined. Max Speed, Max Altitude, and Min Altitude. Each of those data points is selected from a simple scan of the input data.

ROAV-ConCatVideo

By creating a large KML file and then converting it to a KMZ, it becomes a manageable size. KMZ files load into google earth significantly faster then KML files. One of the interesting things that you can do in Google Earth is display the elevation profile of a path.

ROAV-ConCatVideoElevation

On this day you can see that I started at just under 4500 feet, drove over 6000 feet, then down to 1733 feet before ending the day near 2500 feet. When playing with the desktop app, you can drag a marker along the path and it will coordinate a marker along the elevation profile.

I’ve added this code to my ROAV-Concat program, as well as parameters that can tell the program to not output KML or MP4. this allows me to run the program and just generate the KMZ or MP4 file, though beyond during debugging I can’t think of a reason I’d not want both files generated.

I’m hoping that understanding all of this data will allow me to generate text to overlay the video with GPS data beyond what was embedded by the original dashcam.

Part 2 of FFMPEG and ROAV Dash Cam C1 Pro

While writing my software to concatenate and speed up the video files from my ROAV Dashcam I ran into an interesting issue with FFMPEG.

The -filter_complex option seems to stop parsing it’s parameters somewhere above 960 characters on the command line. I didn’t narrow down the exact point, or go digging in the FFMPEG source code to find the size. I expect this is an arbitrary buffer size in FFMPEG. I may contribute to the source code since it fails with no explanation, even when generating a report file. Learning the FFMPEG source structure in itself is a large task, meaning that I’ve not found time to do anything beyond find a workaround.

My workaround was to recognize when the command size will get long and fall back to using the -f concat option with a temporary file listing the input files instead of using the complex filtergraph.

The advantage of the complex filtergraph is twofold. It does not require a secondary input file or any cleanup. It can deal with input files that change resolution if necessary.

Here’s an example of the complex filtergraph:

ffmpeg.exe -report -i 2018_0705_101335_006.MP4 -i 2018_0705_101635_007.MP4 -i 2018_0705_101935_008.MP4 -i 2018_0705_102235_009.MP4 -i 2018_0705_102535_010.MP4 -i 2018_0705_102835_011.MP4 -i 2018_0705_103135_012.MP4 -i 2018_0705_103435_013.MP4 -i 2018_0705_103613_014A.MP4 -i 2018_0705_103615_015A.MP4 -i 2018_0705_104449_016A.MP4 -i 2018_0705_104750_017A.MP4 -i 2018_0705_105050_018A.MP4 -i 2018_0705_105350_019A.MP4 -i 2018_0705_105650_020A.MP4 -i 2018_0705_105950_021A.MP4 -i 2018_0705_110250_022A.MP4 -i 2018_0705_110550_023A.MP4 -i 2018_0705_110850_024A.MP4 -i 2018_0705_143415_025.MP4 -i 2018_0705_143716_026.MP4 -i 2018_0705_144016_027.MP4 -i 2018_0705_144316_028.MP4 -i 2018_0705_144616_029.MP4 -i 2018_0705_144916_030.MP4 -i 2018_0705_145216_031.MP4 -i 2018_0705_145516_032.MP4 -i 2018_0705_145816_033.MP4 -i 2018_0705_150116_034.MP4 -i 2018_0705_150416_035.MP4 -i 2018_0705_150716_036.MP4 -i 2018_0705_151016_037.MP4 -i 2018_0705_151316_038.MP4 -i 2018_0705_151616_039.MP4 -i 2018_0705_151916_040.MP4 -i 2018_0705_152216_041.MP4 -i 2018_0705_152516_042.MP4 -i 2018_0705_152816_043.MP4 -i 2018_0705_153116_044.MP4 -i 2018_0705_153416_045.MP4 -i 2018_0705_153716_046.MP4 -i 2018_0705_154016_047.MP4 -i 2018_0705_154316_048.MP4 -i 2018_0705_154616_049.MP4 -i 2018_0705_154916_050.MP4 -i 2018_0705_155216_051.MP4 -i 2018_0705_155516_052.MP4 -i 2018_0705_155816_053.MP4 -i 2018_0705_160116_054.MP4 -i 2018_0705_160416_055.MP4 -i 2018_0705_160716_056.MP4 -i 2018_0705_161016_057.MP4 -i 2018_0705_161316_058.MP4 -i 2018_0705_161616_059.MP4 -i 2018_0705_161916_060.MP4 -i 2018_0705_162216_061.MP4 -i 2018_0705_162516_062.MP4 -i 2018_0705_162816_063.MP4 -i 2018_0705_163116_064.MP4 -i 2018_0705_163416_065.MP4 -i 2018_0705_163716_066.MP4 -i 2018_0705_164016_067.MP4 -i 2018_0705_164316_068.MP4 -i 2018_0705_164616_069.MP4 -i 2018_0705_164916_070.MP4 -i 2018_0705_165215_071.MP4 -i 2018_0705_165516_072.MP4 -i 2018_0705_165815_073.MP4 -i 2018_0705_170115_074.MP4 -i 2018_0705_170415_075.MP4 -i 2018_0705_170715_076.MP4 -i 2018_0705_171015_077.MP4 -i 2018_0705_171315_078.MP4 -i 2018_0705_171615_079.MP4 -i 2018_0705_171915_080.MP4 -i 2018_0705_172216_081.MP4 -i 2018_0705_172515_082.MP4 -i 2018_0705_172815_083.MP4 -i 2018_0705_173115_084.MP4 -i 2018_0705_173415_085.MP4 -i 2018_0705_173715_086.MP4 -i 2018_0705_174015_087.MP4 -i 2018_0705_174315_088.MP4 -i 2018_0705_174615_089.MP4 -i 2018_0705_174915_090.MP4 -i 2018_0705_175215_091.MP4 -i 2018_0705_175515_092.MP4 -i 2018_0705_175815_093.MP4 -i 2018_0705_180115_094.MP4 -i 2018_0705_180415_095.MP4 -i 2018_0705_180715_096.MP4 -i 2018_0705_181015_097.MP4 -i 2018_0705_181315_098.MP4 -i 2018_0705_181615_099.MP4 -i 2018_0705_181915_100.MP4 -i 2018_0705_182215_101.MP4 -i 2018_0705_182515_102.MP4 -i 2018_0705_182815_103.MP4 -i 2018_0705_183115_104.MP4 -i 2018_0705_183415_105.MP4 -i 2018_0705_183715_106.MP4 -i 2018_0705_184015_107.MP4 -i 2018_0705_184315_108.MP4 -i 2018_0705_184615_109.MP4 -i 2018_0705_184915_110.MP4 -i 2018_0705_185215_111.MP4 -i 2018_0705_185515_112.MP4 -i 2018_0705_185815_113.MP4 -i 2018_0705_190115_114.MP4 -i 2018_0705_190415_115.MP4 -filter_complex [0:v][1:v][2:v][3:v][4:v][5:v][6:v][7:v][8:v][9:v][10:v][11:v][12:v][13:v][14:v][15:v][16:v][17:v][18:v][19:v][20:v][21:v][22:v][23:v][24:v][25:v][26:v][27:v][28:v][29:v][30:v][31:v][32:v][33:v][34:v][35:v][36:v][37:v][38:v][39:v][40:v][41:v][42:v][43:v][44:v][45:v][46:v][47:v][48:v][49:v][50:v][51:v][52:v][53:v][54:v][55:v][56:v][57:v][58:v][59:v][60:v][61:v][62:v][63:v][64:v][65:v][66:v][67:v][68:v][69:v][70:v][71:v][72:v][73:v][74:v][75:v][76:v][77:v][78:v][79:v][80:v][81:v][82:v][83:v][84:v][85:v][86:v][87:v][88:v][89:v][90:v][91:v][92:v][93:v][94:v][95:v][96:v][97:v][98:v][99:v][100:v][101:v][102:v][103:v][104:v][105:v][106:v][107:v][108:v][109:v]concat=n=110:v=1[v];[v]setpts=(1/60)*PTS,drawtext=fontfile=C\\:/WINDOWS/Fonts/consola.ttf:fontcolor=white:fontsize=80:y=main_h-text_h-50:x=50:text=WimsWorld[o] -map [o] -c:v libx265 -crf 23 -preset veryfast -movflags +faststart -bf 2 -g 15 -pix_fmt yuv420p -y “Output.mp4”

Here’s an example of the command where all the input files are defined in the temporary file:

ffmpeg.exe -report -f concat -safe 0 -i C:\Users\Wim\AppData\Local\Temp\Wim4BD3.tmp -vf setpts=(1/60)*PTS,drawtext=fontfile=C\\:/WINDOWS/Fonts/consola.ttf:fontcolor=white:fontsize=80:y=main_h-text_h-50:x=50:text=WimsWorld -an -c:v libx265 -crf 23 -preset veryfast -movflags +faststart -bf 2 -g 15 -pix_fmt yuv420p -y “Output.mp4”

 

FFMPEG and ROAV Dash Cam C1 Pro

I recently purchased a dedicated dashcam on sale to replace my GoPro setup for trip videos. This gives me a new need to understand a new file format.

2018-05-18

The Roav Dashcam stores sequential mp4 files. When configuring the camera it’s possible to set the loop time, which is the duration of each mp4. There’s also an option to watermark the files. I have it turned on, and the only thing I’ve noticed is the ROAV logo, timestamp, and speed in the bottom right. It does not appear to have a way of adjusting the size of the text.

My initial recordings were set to run at 1080p 60 fps. I wanted to concatenate multiple files, add some text of my own, and speed up the video. This was my first experience using the -filter_complex option of FFMPEG. Here’s what I came up with to put together three files, speed the output up by a factor of 60, and add some text. I’m dropping the audio completely. The ROAV can record audio inside the car, but I configured it not to, as I don’t want to hear what I was listening to on the radio or what I might be saying if I make a phone call..

ffmpeg.exe -hide_banner -i 2018_0512_130537_050A.MP4 -i 2018_0512_131537_051A.MP4 -i 2018_0512_132537_052A.MP4 -filter_complex "[0:v] [1:v] [2:v] concat=n=3:v=1 [v];[v]setpts=0.01666*PTS,drawtext=fontfile=C\\:/WINDOWS/Fonts/consola.ttf:fontcolor=white:fontsize=80:y=main_h-text_h-50:x=50:text=WimsWorld[o]" -map "[o]" -c:v libx265 -crf 23 -preset veryfast -movflags +faststart -bf 2 -g 15 -pix_fmt yuv420p  FirstMixSpeed60Concat.mp4

This first video was recorded at 1080p60. The camera can record at 1440p30 which I will be trying soon to see if things like license plates are more legible. The setpts factor that I’m currently using was 1/60, so that 1 minute of real time was compressed to 1 second of video, and just dropping the extra frames. I expect to need to change the setpts factor to 1/30 because of the decreased frame rate at the higher resolution.

FFMPEG and h.265

I’ve noticed that YouTube transcodes my videos after I upload them and wanted to know more. It turns out that they are internally using a form of h.265 video encoding, which reduces the data size significantly without reducing perceived quality over h.264 video compression.

I decided to run some tests using my GoPro time lapse program to see how much compression I’d get versus how much extra time for encoding.

First I had to read up on the settings for using h.265 in FFmpeg. According to https://trac.ffmpeg.org/wiki/Encode/H.265, If I add -c:v libx265 into my existing FFmpeg command line without changing anything, I’ll get an h.265 output with the defaults of -crf 23 and -preset medium.

The -preset value effects how much work is done in the compression, but shouldn’t affect the perceived quality. It will effect both time to create and output file size.

The -crf value effects the perceived quality. I’ve been using the defaults in my previous h.264 mp4 files, which should be approximately -crf 23 and supposedly -crf 28 in h.265 is equivalent to the lower h.264 value. A -crf 0 would be a completely lossless conversion. For my tests, I left crf at the default 23.

I ran all of these tests on a set of 7,129 photos I’d captured while sailboat racing on March 24th using my GoPro Hero 3+ Black. Each input image was 4000×3000 and I’m creating an output video with resolution 3840×2160 by cropping it at 3/4 of the height and scaling to fit.

The original h.264 conversion took 28:36 minutes to create and was 1,162,079,866 bytes long.

Here’s a trimmed down and sorted file listing. The first column is the time it took to create, second is filesize, and third is filename.

28:36 1,162,079,866 20180324-2160p30-cropped-h264.mp4
30:25 1,006,292,960 20180324-veryfast-2160p30-cropped-h265-crf20.mp4
21:35   328,700,100 20180324-ultrafast-2160p30-cropped-crf28.mp4
21:59   339,438,778 20180324-superfast-2160p30-cropped-crf28.mp4
25:12   350,085,434 20180324-veryfast-2160p30-cropped-crf28.mp4
25:17   349,720,030 20180324-faster-2160p30-cropped-crf28.mp4
27:24   348,582,310 20180324-fast-2160p30-cropped-crf28.mp4
52:07   365,050,252 20180324-medium-2160p30-cropped-crf28.mp4

I created a single test at -crf 20 because I was interested in seeing a higher quality video and how different it would be in size. It took slightly longer than the original h.264 compression and had a slight improvement in size.

Two things became obvious to me from this test. More time spent in compression doesn’t always mean better compression. For my application, running preset medium actually hurts the performance, both in file size and time taken.

I ran all of these tests only a single time on my desktop computer with an Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz, 3501 Mhz, 4 Core(s), 8 Logical Processor(s) running the latest version of Windows Version 10.0.16299 Build 16299. I was running FFmpeg version N-88668-g723b6baaf8 from https://ffmpeg.zeranoe.com/builds/. The variations in time could be related to background tasks running on the machine, and I should have run a more comprehensive battery of tests, averaging time and with more accurate timekeeping.

Odd Wildcard Matching in Windows 10

I recently ran into an odd behavior of more files matching a pattern than I expected. I’d used exiftool to modify the dates on files my GoPro produced. It creates backup files of the original images when it modifies the tags. Here’s the command I ran.

exiftool.exe -r "-AllDates+=4:7:6 17:40:00" -ext jpg f:\GoPro\20170807

Now I had about 4000 files with the .JPG extension and another 4000 files with a .JPG_original extension.

I ran my program that parses the directory structure and turns all those images into a time lapse movie, and it seemed to be including both the file extensions, making a very disjointed movie.

I loaded my source code in the debugger and it seemed to be doing a findfirst / findnext specifically looking for .JPG files, and not some other extension, but it was definitely retrieving files both with .JPG and .JPG_original extensions.

I then ran a couple of commands at the windows command prompt and was surprised to find the same results there.

dir F:\GoPro\20170807\372GOPRO\G*.JPG /p
dir F:\GoPro\20170807\372GOPRO\G???????.JPG /p

Each command returned both the JPG and JPG_original files.

dir F:\GoPro\20170807\372GOPRO\G*.JPG_original /p

returned just the JPG_original files.

dir F:\GoPro\20170807\372GOPRO\G??????.JPG /p

had one less question mark and correctly returned no files.

This is all unexpected behavior, though I’m glad to see that it was consistent with the operating system and not something specific to the C runtime. I’d love an explanation of what’s going on.

exiftool to manage DJI media files

DJI Drones don’t seem to remember the image count between formats of a media card. This creates a problem for me when I’m trying to backup and maintain my images and video.

Because the dates are all correct in the media files, retrieved from GPS data, organizing the files by naming them based on the date works for me.

Using ExifTool by Phil Harvey is a great solution for pulling the metadata from the files and renaming the files.

The command line that I was initially using is:

exiftool "-FileName<${CreateDate}.$filetype" -d %Y%m%d-%H%M%S%%-c -ext mp4 -ext dng -ext jpg dji*

It’s problem is that it orphans the SRT subtitle files from my videos that I’d like to keep matching the video files.

I’ve tried this variation to do it in one step but it doesn’t work, because the SRT files get renamed as MP4 files.

exiftool -verbose "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.%%le -ext mp4 -ext dng -ext jpg dji* -srcfile %f.srt

If anyone has a suggestion for how to rename all the media files in one directory I’d appreciate it. Even running two commands in sequence would be fine.

Update:

I’ve figured out that running these two commands in sequence will get me the results I am looking for:

exiftool "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.srt -ext mp4 -srcfile %f.srt dji*
exiftool "-FileName<${CreateDate}" -d %Y%m%d-%H%M%S%%-c.%%le -ext mp4 -ext dng -ext jpg dji*

I’m still looking for a way of doing it in a single command that may leave less room for error, but this is working for now.

FFMPEG and drawtext

Several years ago I wrote a program that consolidates time-lapse pictures into a directory and calls FFMPEG to create a video.

I had been wanting the time-code from when each picture was taken printed on the screen while the video was playing but had not figured out how to get it done until this weekend.

Video TimeCode

Frame from video showing the DateTimeOriginal timecode embedded.

I’d gone down multiple paths in an attempt to get this result before finally getting the drawtext feature to work. My program manually pulled the metadata from the images before feeding them to ffmpg. I’d tried creating both text files and image files for overlaying. none of those got the result that I was looking for.

When I finally got everything working, it seems simple, but the underlying problem has to do with the amount of string escaping required to get the command to work.

Here’s an example command I was issuing to ffmpeg that got the result I was looking for.

ffmpeg.exe -hide_banner -r 30 -i Wim%05d.JPG -vf crop=in_w:3/4*in_h,drawtext=fontfile=C\\:/WINDOWS/Fonts/OCRAEXT.ttf:fontcolor=white:fontsize=160:y=main_h-text_h-50:x=main_w-text_w-50:text=WimsWorld,drawtext=fontfile=C\\:/WINDOWS/Fonts/OCRAEXT.ttf:fontcolor=white:fontsize=160:y=main_h-text_h-50:x=50:text=%{metadata\\:DateTimeOriginal} -s 3840x2160 -pix_fmt yuv420p -n Test-2160p30-cropped.mp4

If you look at the -vf option parameter, I’m cropping my input pictures to 3/4 their original height, then using the drawtext feature twice. First I write the static text to the bottom right of the frame, then I extract metadata from the source image and write it to the bottom left of the frame.

Because I’m calling this from a program, I had extra escaping of the \ character in my code. All of the escaping required a lot of trial and error to get things working. I’m using OCRAEXT as my font, but I could be using any fixed spacing font. because of the fact that the time is changing every frame, it’s important that the font not be proportional to make it easy to read.

Flashing ESCs on Hobbylord BumbleBee

I bought a Bumblebee Quad from a local hobby shop a few months ago, and when I finally got around to trying to build it with a proper autopilot found that it’s ESCs used a protocol called UltraPWM that is a very uncommon protocol.

I came across this page, https://github.com/sim-/tgy/issues/13 , which leads me to believe that I should be able to flash the ESCs with a simonk tgy firmware and use the hardware I already have.

I came across a cable from hobbyking that is designed to make contact with the surface mounted atmel device and allow programming without any soldering or desoldering. http://hobbyking.com/hobbyking/store/__27195__Atmel_Atmega_Socket_Firmware_Flashing_Tool.html It was designed to be used with an atmel programming device that they also sell http://hobbyking.com/hobbyking/store/__27990__USBasp_AVR_Programming_Device_for_ATMEL_proccessors.html and so I thought I’d be good to go. The cable cost $20 while the programmer cost $4, but not needing to solder anything was a very positive solution for me.

USBasp AVR Programming Device for ATMEL proccessors

USBasp AVR Programming Device

Atmel Atmega Socket Firmware Flashing Tool

Atmel Atmega Socket Firmware Flashing Tool

What I didn’t recognize until it all arrived was that Hobbyking has updated the USB Programmer to use a 6 conductor connector, but not updated their programming cable from the 10 conductor cable. The message boards on hobbyking discuss the change, and have pinout descriptions, but it’s been very frustrating because getting the parts to do the correct wiring has not been as simple as plug and play.

Atmega contact points

Atmega contact points

Atmega contact points

Atmega contact points

Cable Pinout Description

Cable Pinout Description

This has been extremely frustrating to me as the parts I ordered were billed as no soldering required, but could not be simply plugged into each other.