Enable Raspberry Pi Camera Module 3 Wide after Arducam 64mp

My development pi has had an Arducam 64mp camera connected for my camera software development. I liked the quality of the camera but have at various times been frustrated with the software requirements to use the camera. It’s required both a custom kernel driver and a custom fork of the libcamera software packages. That’s meant that to use the 64mp camera I needed to reinstall the arducam suite after nearly every apt upgrade cycle, and definitely ones where the system kernel got updated.

I spent several days trying to get the remnants of the arducam64mp removed from my development system. I’d even built a fresh sd card image of Raspian Bullseye to make sure that the hardware was all connected and working properly.

In the end the fix was rather simple, if obscure.

First, remove or comment out the dtoverlay line from the /boot/config.txt file and make sure camera auto detect is enabled.

# dtoverlay=arducam-64mp
camera_auto_detect=1

Then run apt install with the –reinstall option for the libcamera packages and the raspberry kernel package.

sudo apt install --reinstall -y libcamera-apps libcamera-dev libcamera0 raspberrypi-kernel
sudo systemctl reboot

Then reboot. That should be then allow you to run libcamera-hello and verify that the new camera is working.

I’d found a reference How To Enable RP Cam V2 After Arducam 64MP that didn’t seem to work for getting the V3 wide camera working, perhaps because the focus hardware in the V3 camera made the 64mp think it was active.

I asked the question of how to remove the drivers on the Arducam Forum and then answered my own question.

On to playing with my new Camera Module 3! (I bought the Arducam cases from Amazon because I really liked the fit. The new wide camera protrudes from the front with enough clearance for the focus to function.)

C++ IPv6 Ping Code Example

My previous code example for IPv4 needed a bunch of modifications to work for an IPv6 address. The thing that took me the longest to figure out was that because IPv6 seems to send a lot more ICMP messages on the local network, I needed to filter the response messages to only the type I was listening for.

bool send_ping6(const std::string& ping_ip, const std::string& HostName4Output, const bool bOutput = false)
{
    bool rval = false;
    if (bOutput)
        std::cout << "[" << getTimeExcelLocal() << "] " << "send_ping6(" << ping_ip << ", " << HostName4Output << ");" << std::endl;
    struct timespec tfs;
    clock_gettime(CLOCK_MONOTONIC, &tfs);
    auto ping_sockfd = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6);
    if (ping_sockfd < 0)
    {
        if (bOutput)
            std::cout << "[" << getTimeExcelLocal() << "] " << "Socket file descriptor not received!!" << std::endl;
    }
    else
    {
        // set socket options at ip to TTL and value to 64,
        // change to what you want by setting ttl_val
        int ttl_val = 64;
        if (setsockopt(ping_sockfd, IPPROTO_IPV6, IPV6_UNICAST_HOPS, &ttl_val, sizeof(ttl_val)) != 0)
        {
            if (bOutput)
                std::cerr << "[" << getTimeExcelLocal() << "] " << "Setting socket options to TTL failed!" << std::endl;
        }
        else
        {
            struct icmp6_filter filt;
            ICMP6_FILTER_SETBLOCKALL(&filt);
            ICMP6_FILTER_SETPASS(ICMP6_ECHO_REPLY, &filt);
            setsockopt(ping_sockfd, IPPROTO_ICMPV6, ICMP6_FILTER, &filt, sizeof(filt));

            // setting timeout of recv setting
            struct timeval tv_out;
            tv_out.tv_sec = RECV_TIMEOUT;
            tv_out.tv_usec = 0;
            setsockopt(ping_sockfd, SOL_SOCKET, SO_RCVTIMEO, &tv_out, sizeof(tv_out));

            int msg_count = 0;
            int flag = 1;
            int msg_received_count = 0;
            // send icmp packet in a loop
            for (auto pingloop = 4; pingloop > 0; pingloop--)
            {
                // flag is whether packet was sent or not
                flag = 1;

                //filling packet
                struct ping_pkt pckt;
                bzero(&pckt, sizeof(pckt));
                for (auto i = 0; i < sizeof(pckt.msg) - 1; i++)
                    pckt.msg[i] = i + '0';
                pckt.msg[sizeof(pckt.msg) - 1] = 0;
                pckt.hdr.type = ICMP6_ECHO_REQUEST;
                pckt.hdr.un.echo.id = getpid();
                pckt.hdr.un.echo.sequence = msg_count++;
                pckt.hdr.checksum = checksum(&pckt, sizeof(pckt));

                usleep(PING_SLEEP_RATE);

                struct timespec time_start;
                clock_gettime(CLOCK_MONOTONIC, &time_start);

                struct sockaddr_in6 ping_addr;
                ping_addr.sin6_family = AF_INET6;
                ping_addr.sin6_port = htons(0);
                inet_pton(AF_INET6, ping_ip.c_str(), &ping_addr.sin6_addr);
                if (sendto(ping_sockfd, &pckt, sizeof(pckt), 0, (struct sockaddr*)&ping_addr, sizeof(ping_addr)) <= 0)
                {
                    if (bOutput)
                        std::cout << "[" << getTimeExcelLocal() << "] " << "Packet Sending Failed!" << std::endl;
                    flag = 0;
                }

                //receive packet
                struct sockaddr_in6 r_addr;
                auto addr_len = sizeof(r_addr);
                if (recvfrom(ping_sockfd, &pckt, sizeof(pckt), 0, (struct sockaddr*)&r_addr, (socklen_t*)&addr_len) <= 0 && msg_count > 1)
                {
                    if (bOutput)
                        std::cout << "[" << getTimeExcelLocal() << "] " << "Packet receive failed!" << std::endl;
                }
                else
                {
                    struct timespec time_end;
                    clock_gettime(CLOCK_MONOTONIC, &time_end);

                    double timeElapsed = ((double)(time_end.tv_nsec - time_start.tv_nsec)) / 1000000.0;
                    long double rtt_msec = (time_end.tv_sec - time_start.tv_sec) * 1000.0 + timeElapsed;

                    // if packet was not sent, don't receive
                    if (flag)
                    {
                        char szAddr[NI_MAXHOST] = { 0 };
                        inet_ntop(AF_INET6, &r_addr.sin6_addr, szAddr, sizeof(szAddr));
                        if (!(pckt.hdr.type == ICMP6_ECHO_REPLY && pckt.hdr.code == 0))
                        {
                            if (bOutput)
                                std::cout << "[" << getTimeExcelLocal() << "] " << "Error..Packet received from (" << szAddr << ") with ICMP type " << int(pckt.hdr.type) << " code " << int(pckt.hdr.code) << std::endl;
                        }
                        else
                        {
                            if (bOutput)
                                std::cout << "[" << getTimeExcelLocal() << "] " << PING_PKT_S << " bytes from (" << szAddr << ") (" << HostName4Output << ") msg_seq=" << msg_count << " ttl=" << "ttl_val" << " rtt= " << rtt_msec << " ms." << std::endl;
                            msg_received_count++;
                        }
                    }
                }
            }
            rval = msg_received_count > 0;
            struct timespec tfe;
            clock_gettime(CLOCK_MONOTONIC, &tfe);
            double timeElapsed = ((double)(tfe.tv_nsec - tfs.tv_nsec)) / 1000000.0;
            long double total_msec = (tfe.tv_sec - tfs.tv_sec) * 1000.0 + timeElapsed;
            if (bOutput)
                std::cout << "[" << getTimeExcelLocal() << "] " << "=== " << ping_ip << " ping statistics === " << msg_count << " packets sent, " << msg_received_count << " packets received, " << ((msg_count - msg_received_count) / msg_count) * 100.0 << " percent packet loss. Total time : " << total_msec << " ms." << std::endl;
        }
        close(ping_sockfd);
    }
    return(rval);
}

Because my calling routine is keeping the addresses for the hosts as strings, I’m calling each of these routines with those strings and converting them to proper addresses inside the function. I’m making a simple choice of whether it’s an IPv4 address or an IPv6 address by the fact that IPv4 addresses have “.” in them and IPv6 addresses have “:”.

bool send_ping(const std::string& ping_ip, const std::string& HostName4Output, const bool bOutput = false)
{
    bool rval = false;
    if (ping_ip.find('.') == std::string::npos)
        rval = send_ping6(ping_ip, HostName4Output, bOutput);
    else 
        rval = send_ping4(ping_ip, HostName4Output, bOutput);
    return(rval);
}

Here’s a bunch of links I found useful while creating this code:

C++ IPv4 Ping Code Example

I’ve written my own monitoring program to keep track of the availability of some of my machines. They register themselves in DNS using dynamic DNS protocols and occasionally change addresses. I realized that while recognizing when the address has changed is useful, I’d also like to know if the machine itself is reachable. Having code that would test the ICMP ping results directly in my code is useful, and this is what I ended up putting together after having found examples in variousl places on the web.

/ Define the Packet Constants
// ping packet size
#define PING_PKT_S 64
#define PING_SLEEP_RATE 1000000

// Gives the timeout delay for receiving packets in seconds
#define RECV_TIMEOUT 1

// ping packet structure
struct ping_pkt
{
    struct icmphdr hdr;
    char msg[PING_PKT_S - sizeof(struct icmphdr)];
};

// Calculating the Check Sum
unsigned short checksum(void* b, int len)
{
    unsigned short* buf = (unsigned short*) b;
    unsigned int sum = 0;

    for (sum = 0; len > 1; len -= 2)
        sum += *buf++;
    if (len == 1)
        sum += *(unsigned char*)buf;
    sum = (sum >> 16) + (sum & 0xFFFF);
    sum += (sum >> 16);
    unsigned short result = ~sum;
    return result;
}

bool send_ping4(const std::string& ping_ip, const std::string& HostName4Output, const bool bOutput = false)
{
    bool rval = false;
    if (bOutput)
        std::cout << "[" << getTimeExcelLocal() << "] " << "send_ping4(" << ping_ip << ", " << HostName4Output << ");" << std::endl;
    struct timespec tfs;
    clock_gettime(CLOCK_MONOTONIC, &tfs);
    auto ping_sockfd = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP);
    if (ping_sockfd < 0)
    {
        if (bOutput)
            std::cout << "[" << getTimeExcelLocal() << "] " << "Socket file descriptor not received!!" << std::endl;
    }
    else
    {
        // set socket options at ip to TTL and value to 64,
        // change to what you want by setting ttl_val
        int ttl_val = 64;
        if (setsockopt(ping_sockfd, SOL_IP, IP_TTL, &ttl_val, sizeof(ttl_val)) != 0)
        {
            if (bOutput)
                std::cout << "[" << getTimeExcelLocal() << "] " << "Setting socket options to TTL failed!" << std::endl;
        }
        else
        {
            // setting timeout of recv setting
            struct timeval tv_out;
            tv_out.tv_sec = RECV_TIMEOUT;
            tv_out.tv_usec = 0;
            setsockopt(ping_sockfd, SOL_SOCKET, SO_RCVTIMEO, &tv_out, sizeof(tv_out));

            int msg_count = 0;
            int flag = 1;
            int msg_received_count = 0;
            // send icmp packet in a loop
            for (auto pingloop = 4; pingloop > 0; pingloop--)
            {
                // flag is whether packet was sent or not
                flag = 1;

                //filling packet
                struct ping_pkt pckt;
                bzero(&pckt, sizeof(pckt));
                for (auto i = 0; i < sizeof(pckt.msg) - 1; i++)
                    pckt.msg[i] = i + '0';
                pckt.msg[sizeof(pckt.msg) - 1] = 0;
                pckt.hdr.type = ICMP_ECHO;
                pckt.hdr.un.echo.id = getpid();
                pckt.hdr.un.echo.sequence = msg_count++;
                pckt.hdr.checksum = checksum(&pckt, sizeof(pckt));

                usleep(PING_SLEEP_RATE);

                struct timespec time_start;
                clock_gettime(CLOCK_MONOTONIC, &time_start);

                struct sockaddr_in ping_addr;
                ping_addr.sin_family = AF_INET;
                ping_addr.sin_port = htons(0);
                inet_pton(AF_INET, ping_ip.c_str(), &ping_addr.sin_addr.s_addr);

                if (sendto(ping_sockfd, &pckt, sizeof(pckt), 0, (struct sockaddr*)&ping_addr, sizeof(ping_addr)) <= 0)
                {
                    if (bOutput)
                        std::cout << "[" << getTimeExcelLocal() << "] " << "Packet Sending Failed!" << std::endl;
                    flag = 0;
                }
                //receive packet
                struct sockaddr_in r_addr;
                auto addr_len = sizeof(r_addr);
                if (recvfrom(ping_sockfd, &pckt, sizeof(pckt), 0, (struct sockaddr*)&r_addr, (socklen_t*)&addr_len) <= 0 && msg_count > 1)
                {
                    if (bOutput)
                        std::cout << "[" << getTimeExcelLocal() << "] " << "Packet receive failed!" << std::endl;
                }
                else
                {
                    struct timespec time_end;
                    clock_gettime(CLOCK_MONOTONIC, &time_end);

                    double timeElapsed = ((double)(time_end.tv_nsec - time_start.tv_nsec)) / 1000000.0;
                    long double rtt_msec = (time_end.tv_sec - time_start.tv_sec) * 1000.0 + timeElapsed;

                    // if packet was not sent, don't receive
                    if (flag)
                    {
                        if (!(pckt.hdr.type == 69 && pckt.hdr.code == 0))
                        {
                            if (bOutput)
                                std::cerr << "[" << getTimeExcelLocal() << "] " << "Error..Packet received with ICMP type " << int(pckt.hdr.type) << " code " << int(pckt.hdr.code) << std::endl;
                        }
                        else
                        {
                            char szAddr[NI_MAXHOST] = { 0 };
                            inet_ntop(AF_INET, &r_addr.sin_addr, szAddr, sizeof(szAddr));
                            if (bOutput)
                                std::cout << "[" << getTimeExcelLocal() << "] " << PING_PKT_S << " bytes from (" << szAddr << ") (" << HostName4Output << ") msg_seq=" << msg_count << " ttl=" << ttl_val << " rtt= " << rtt_msec << " ms." << std::endl;
                            msg_received_count++;
                        }
                    }
                }
            }
            rval = msg_received_count > 0;
            struct timespec tfe;
            clock_gettime(CLOCK_MONOTONIC, &tfe);
            double timeElapsed = ((double)(tfe.tv_nsec - tfs.tv_nsec)) / 1000000.0;
            long double total_msec = (tfe.tv_sec - tfs.tv_sec) * 1000.0 + timeElapsed;
            if (bOutput)
                std::cout << "[" << getTimeExcelLocal() << "] " << "=== " << ping_ip << " ping statistics === " << msg_count << " packets sent, " << msg_received_count << " packets received, " << ((msg_count - msg_received_count) / msg_count) * 100.0 << " percent packet loss. Total time : " << total_msec << " ms." << std::endl;
        }
        close(ping_sockfd);
    }
    return(rval);
}

That works great if my address is an IPv4 address, but I required doing a lot of investigation to get a successful IPv6 address.

Moving Visual Studio Cross Platform Development to a new Machine

I upgraded my workstation for the holidays and needed to get things moved over as easily as possible.

In Visual Studio, under the tools menu, there’s an option to be able to Import and Export Settings… which got me an output file that made my new installation look mostly like my old installation.

What it didn’t transfer was the details of my connections to my Raspberry Pi devices. The following image is after I managed to get a working connection. It didn’t have anything displayed under Default Host Name etc.

I’d already run ssh-keygen on my new machine, creating a default security key on the local machine. I’d already connected via ssh to each of the hosts I regularly work on and imported the public key into the authorized_keys file and verified all appeared working.

After entering all of the details in the dialog box I got a rather unhelpful set of red boxes indicating that something had gone wrong. (192.168.0.66 and 192.168.0.67 are the same host, wired and Wi-Fi, on my local network, and I may use them interchangeably for images and text here)

Searching on the web I found that there is a console program that will configure the same thing, and I assumed correctly that I might get more descriptive error messages using the console than I was getting here. https://learn.microsoft.com/en-us/cpp/linux/connectionmanager-reference?view=msvc-170

**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.4.2
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************

C:\Program Files\Microsoft Visual Studio\2022\Enterprise>ConnectionManager.exe add visualstudio@192.168.0.66 --privatekey C:\Users\Wim\.ssh\id_rsa
Enter password (leave blank for no password):
Verifying connection with remote system.
Failed to add connection: Private key is invalid or is encrypted and no passphrase was given

I knew that the keys were good because I was using them to connect via SSH. I did some digging and realized that the physical file sizes on the new machine are larger than on the old machine. Looking at the length of the public keys themselves, I realized that my old keys were 2048 bits long, and the newly generated ones are 3072. I tried explicitly generating a new 2048 bit key with the command ssh-keygen -f id_rsa_2048 -b 2048 and using it, but I got similar results to what I was getting before. Finally, I copied my public and private key from the old machine and renamed them specifically for this usage on this machine.

C:\Program Files\Microsoft Visual Studio\2022\Enterprise>ConnectionManager.exe add visualstudio@192.168.0.66 --privatekey C:\Users\Wim\.ssh\id_rsa_visualstudio
Enter password (leave blank for no password):
Verifying connection with remote system.
The authenticity of host '192.168.0.66' can't be established.
ecdsa-sha2-nistp256 key fingerprint is SHA256:DGa1mVbMm3voQwVwtg06xHkANgs04zST9RP8CMfSoXY.
Are you sure you want to continue connecting (yes/no)? yes
Successfully added connection '1748855142;192.168.0.66 (username=visualstudio, port=22, authentication=PrivateKey)'.

That created the connection properly in Visual Studio. I went back into the Visual Studio interface, Tools->Options->Cross Platform->Connection Manager->Remote Headers Intellisense Manager and updated the headers and everything appears to be working now.

I don’t know why the keys I generated using the software on my new machine didn’t work, even when I specified the same number of bits.

Moving a subdirectory from one git repository to an entirely new repository

I use Microsoft Visual Studio as my primary development environment. I’ve got a long-term repository that I maintain on Azure Devops at https://wimsworld.visualstudio.com/. When I create a new test project, I often create it as a subdirectory inside the main repository. That gives me version control, without the hassle of creating a new repository every time.

The downside is that when a project grows into a real program, I want to make public, I have to figure out what to do about the code. I could take a snapshot, losing all of the history, or I could jump through hoops to get the project exported from the Azure git platform and imported into the GitHub platform. I like the history of how I got from the start to where I am, so I chose to go thourgh the hoops.

I should describe this with examples. My default code repository is WimsWorld and I often create projects as directories of that, so WimsWorld/WimsConstructionCam in this case. I created this project in July, and then a couple of months later I realized I should have created the program in WimsWorld/Linux/WimsConstructionCam, so I moved it one level deeper in the repository. Git handles all of that fine and I could trace back the history of the file all the way back to creation.

Now that it’s time for me to decide I may be interested in sharing this project publicly, I looked for step by step instructions and came across a nice video doing what I wanted.

This was easiest to do on a linux box at the command line, and since I have a dev system that has its SSH keys already registered as me in both Azure Devops and GitHub I could repeat the process multiple times to get it right.

git clone WimsWorld@vs-ssh.visualstudio.com:v3/WimsWorld/WimsWorld/WimsWorld
cd WimsWorld/
git remote rm origin
git filter-branch --subdirectory-filter Linux/WimsConstructionCam -- --all
cd ..
mv WimsWorld WimsConstructionCam
cd WimsConstructionCam/
git remote add origin git@github.com:wcbonner/WimsConstructionCam.git
git pull origin master --allow-unrelated-histories
git push origin master

The git –filter-branch –subdirectory command got rid of all of the history except for what was in the Linux/WimsConstructionCam/ directory, which seemed right, since all the files I wanted history on currently lived in that directory.

I had previously created a repository in GitHub and this got what appeared to have history in GitHub. On closer inspection, the history only went back a couple of months, and not back to when I’d created the project. I deleted my project in github, deleted the repository directory on my local machine and started again after some research.

First I tried to manually delete everything I didn’t want from my cloned repository, move all the files to where I wanted, commit them, then connect to the new repo and push. I got a problem message about a file that was too large. Sometime in my past history, I’d managed to push a Visual Studio debug symbol database into my repository and while I’d deleted the file, I’d not deleted the history of the file from the repository. That issue was actually a good thing because it confirmed to me that method of getting the files looking the way I wanted could be leaking credentials that I might have somewhere in my private repository.

I found that I needed the git tool git-filter-repo to do what I needed to do.

wget https://raw.githubusercontent.com/newren/git-filter-repo/main/git-filter-repo
git clone WimsWorld@vs-ssh.visualstudio.com:v3/WimsWorld/WimsWorld/WimsWorld
cd WimsWorld/
git remote rm origin
python3 ~/git-filter-repo --force --path WimsConstructionCam --path Linux/WimsConstructionCam
cd Linux/WimsConstructionCam/
git mv * ../..
cd ../..
git commit .
cd ..
mv WimsWorld WimsConstructionCam
cd WimsConstructionCam/
git remote add origin git@github.com:wcbonner/WimsConstructionCam.git
git pull origin main --allow-unrelated-histories
git push origin master

Using the git-filter-repo program I was able to remove the history of everything that was outside of the original folder and the current folder. Then I moved all of the files from the current subdirectory to the repository root and committed the changes. Then I connected the repository to the new remote repository, pulled the remote, and then pushed to the remote.

A funny issue I ran into during my second try into a brand-new repository in GitHub. I’d originally created an empty repository a few weeks ago and the default branch was called master, which matched the default branch on my old repository. They’ve changed to use the current politically correct term for the default branch, and it’s named main. After I’d got both branches in GitHub, I had to merge the two so I’m working from a single branch.

List of links:

Arducam Red Tint Issue on Raspberry Pi

no tuning file

After writing up my issue yesterday I posted the question on the amazon product details. I received an answer with a link to this page overnight. It references a page with several custom tuning files for their lenses. There did not seem to be a tuning file that matched the 175° FOV lens on the package I got, but there was both a 160° and a 200° version. I downloaded both files and ran a test capture with each. On initial viewing, either will work for my solution. I need to focus the lens to see if there are any noticeable differences.

–tuning-file imx219_160.json
–tuning-file imx219_200.json

The commands I used to download and test the tuning files are

wget https://www.arducam.com/wp-content/uploads/2022/05/imx219_160.json
wget https://www.arducam.com/wp-content/uploads/2022/05/imx219_200.json
libcamera-still -v 2 --nopreview --hflip --vflip --thumb none --tuning-file imx219_160.json --output `hostname`-160.jpg
libcamera-still -v 2 --nopreview --hflip --vflip --thumb none --tuning-file imx219_200.json --output `hostname`-200.jpg

I got a message about the tuning file being an older version, with pointers on how to convert it to the updated version.

WARN RPiController controller.cpp:43 This format of the tuning file will be deprecated soon! Please use the convert_tuning.py utility to update to version 2.0.

I still need to decide how I want to handle this in my automated photo processing, but having the information is hugely useful.

Links in one place:

Arducam 8MP IMX219 175 Degree Ultra Wide Angle Raspberry Pi Camera Module

I’ve been playing with several cameras attached to raspberry pi recently. Getting cameras is easy right now while getting Raspberry Pi is not. I purchased this model from Amazon and have it connected to a Raspberry Pi4. I had to upgrade the unit from running Raspian Buster to Raspian Bullseye to get support for this particular camera.

Amazon Listing

I was doing some searching for red tint problems and fixing them, and it seems to be something that should be able to fix in software. Most of the fixes refer to Jetson Nano, which is not the platform I’m working with, so the fixes don’t align with my platform.

Red Tint Problem on IMX219

I still need to manually focus the lens on this camera. It’s always a frustrating process on a small lens like this because my fingers obscure the view. I’m running with camera_auto_detect=1 while https://www.arducam.com/docs/cameras-for-raspberry-pi/native-raspberry-pi-cameras/8mp-imx219-standard-camera-modules/ recommends modifying /boot/config.txt and setting both camera_auto_detect=0 and dtoverlay=imx219. exiftool reports Camera Model Name : /base/soc/i2c0mux/i2c@1/imx219@10 with the current settings. I’ve verified that using the explicit camera overlay produces the same red tint as well as having exactly the same Model Name in the exif data.

I was able to find the file /usr/share/libcamera/ipa/raspberrypi/imx219.json which may be similar to the ISP files the Jetson platforms were using. I’m still working on the red tint.

Sunrise and Sunset in C++

I wanted to calculate sunrise and sunset at my location for a recent project I was working on where I wouldn’t necessarily have access to internet connectivity. I did some quick searching and found a Wikipedia page with equations and thought it would be straightforward to put those equations into code. Somehow, I was not able to get the routines working correctly.

I came across a NOAA page that used equations in a different form, and also had links to an excel spreadsheet I could download and play with. I was able to reverse engineer the spreadsheet into C++ code to get a working routine that calculates the sunrise and sunset to an accuracy that is good enough for my needs.

I’m sharing my code here. If you see anything I’ve done wrong, please let me know. If you find it useful, please let me know about that as well.

/////////////////////////////////////////////////////////////////////////////
// From NOAA Spreadsheet https://gml.noaa.gov/grad/solcalc/calcdetails.html
bool getSunriseSunset(time_t& Sunrise, time_t& Sunset, const time_t& TheTime, const double Latitude, double Longitude)
{
	bool rval = false;
	struct tm LocalTime;
	if (0 != localtime_r(&TheTime, &LocalTime))
	{
		// if we don't have a valid latitude or longitude, declare sunrise to be midnight, and sunset one second before midnight
		if ((Latitude == 0) || (Longitude == 0))
		{
			LocalTime.tm_hour = 0;
			LocalTime.tm_min = 0;
			LocalTime.tm_sec = 0;
			Sunrise = mktime(&LocalTime);
			Sunset = Sunrise + 24*60*60 - 1;
		}
		else
		{
			double JulianDay = Time2JulianDate(TheTime); // F
			double JulianCentury = (JulianDay - 2451545) / 36525;	// G
			double GeomMeanLongSun = fmod(280.46646 + JulianCentury * (36000.76983 + JulianCentury * 0.0003032), 360);	// I
			double GeomMeanAnomSun = 357.52911 + JulianCentury * (35999.05029 - 0.0001537 * JulianCentury);	// J
			double EccentEarthOrbit = 0.016708634 - JulianCentury * (0.000042037 + 0.0000001267 * JulianCentury);	// K
			double SunEqOfCtr = sin(radians(GeomMeanAnomSun)) * (1.914602 - JulianCentury * (0.004817 + 0.000014 * JulianCentury)) + sin(radians(2 * GeomMeanAnomSun)) * (0.019993 - 0.000101 * JulianCentury) + sin(radians(3 * GeomMeanAnomSun)) * 0.000289; // L
			double SunTrueLong = GeomMeanLongSun + SunEqOfCtr;	// M
			double SunAppLong = SunTrueLong - 0.00569 - 0.00478 * sin(radians(125.04 - 1934.136 * JulianCentury));	// P
			double MeanObliqEcliptic = 23 + (26 + ((21.448 - JulianCentury * (46.815 + JulianCentury * (0.00059 - JulianCentury * 0.001813)))) / 60) / 60;	// Q
			double ObliqCorr = MeanObliqEcliptic + 0.00256 * cos(radians(125.04 - 1934.136 * JulianCentury));	// R
			double SunDeclin = degrees(asin(sin(radians(ObliqCorr)) * sin(radians(SunAppLong))));	// T
			double var_y = tan(radians(ObliqCorr / 2)) * tan(radians(ObliqCorr / 2));	// U
			double EquationOfTime = 4 * degrees(var_y * sin(2 * radians(GeomMeanLongSun)) - 2 * EccentEarthOrbit * sin(radians(GeomMeanAnomSun)) + 4 * EccentEarthOrbit * var_y * sin(radians(GeomMeanAnomSun)) * sin(2 * radians(GeomMeanLongSun)) - 0.5 * var_y * var_y * sin(4 * radians(GeomMeanLongSun)) - 1.25 * EccentEarthOrbit * EccentEarthOrbit * sin(2 * radians(GeomMeanAnomSun))); // V
			double HASunriseDeg = degrees(acos(cos(radians(90.833)) / (cos(radians(Latitude)) * cos(radians(SunDeclin))) - tan(radians(Latitude)) * tan(radians(SunDeclin)))); // W
			double SolarNoon = (720 - 4 * Longitude - EquationOfTime + LocalTime.tm_gmtoff / 60) / 1440; // X
			double SunriseTime = SolarNoon - HASunriseDeg * 4 / 1440;	// Y
			double SunsetTime = SolarNoon + HASunriseDeg * 4 / 1440;	// Z
			LocalTime.tm_hour = 0;
			LocalTime.tm_min = 0;
			LocalTime.tm_sec = 0;
			time_t Midnight = mktime(&LocalTime);
			Sunrise = Midnight + SunriseTime * 86400;
			Sunset = Midnight + SunsetTime * 86400;
		}
		rval = true;
	}
	return(rval);
}

I’ve included the individual routines I created when trying to duplicate the Wikipedia equations. If anyone can point out why they didn’t work for me, I’d appreciate that as well.

/////////////////////////////////////////////////////////////////////////////
double radians(const double degrees)
{
	return((degrees * M_PI) / 180.0);
}
double degrees(const double radians)
{
	return((radians * 180.0) / M_PI);
}
double Time2JulianDate(const time_t& TheTime)
{
	double JulianDay = 0;
	struct tm UTC;
	if (0 != gmtime_r(&TheTime, &UTC))
	{
		// https://en.wikipedia.org/wiki/Julian_day
		// JDN = (1461 × (Y + 4800 + (M ? 14)/12))/4 +(367 × (M ? 2 ? 12 × ((M ? 14)/12)))/12 ? (3 × ((Y + 4900 + (M - 14)/12)/100))/4 + D ? 32075
		JulianDay = (1461 * ((UTC.tm_year + 1900) + 4800 + ((UTC.tm_mon + 1) - 14) / 12)) / 4
			+ (367 * ((UTC.tm_mon + 1) - 2 - 12 * (((UTC.tm_mon + 1) - 14) / 12))) / 12
			- (3 * (((UTC.tm_year + 1900) + 4900 + ((UTC.tm_mon + 1) - 14) / 12) / 100)) / 4
			+ (UTC.tm_mday)
			- 32075;
		// JD = JDN + (hour-12)/24 + minute/1440 + second/86400
		double partialday = (static_cast<double>((UTC.tm_hour - 12)) / 24) + (static_cast<double>(UTC.tm_min) / 1440.0) + (static_cast<double>(UTC.tm_sec) / 86400.0);
		JulianDay += partialday;
	}
	return(JulianDay);
}
time_t JulianDate2Time(const double JulianDate)
{
	time_t TheTime = (JulianDate - 2440587.5) * 86400;
	return(TheTime);
}
double JulianDate2JulianDay(const double JulianDate)
{
	double n = JulianDate - 2451545.0 + 0.0008;
	return(n);
}
/////////////////////////////////////////////////////////////////////////////
// These equations all come from https://en.wikipedia.org/wiki/Sunrise_equation
double getMeanSolarTime(const double JulianDay, const double longitude)
{
	// an approximation of mean solar time expressed as a Julian day with the day fraction.
	double MeanSolarTime = JulianDay - (longitude / 360);
	return (MeanSolarTime);
}
double getSolarMeanAnomaly(const double MeanSolarTime)
{
	double SolarMeanAnomaly = fmod(357.5291 + 0.98560028 * MeanSolarTime, 360);
	return(SolarMeanAnomaly);
}
double getEquationOfTheCenter(const double SolarMeanAnomaly)
{
	double EquationOfTheCenter = 1.9148 * sin(radians(SolarMeanAnomaly)) + 0.0200 * sin(radians(2 * SolarMeanAnomaly)) + 0.0003 * sin(radians(3 * SolarMeanAnomaly));
	return(EquationOfTheCenter);
}
double getEclipticLongitude(const double SolarMeanAnomaly, const double EquationOfTheCenter)
{
	double EclipticLongitude = fmod(SolarMeanAnomaly + EquationOfTheCenter + 180 + 102.9372, 360);
	return(EclipticLongitude);
}
double getSolarTransit(const double MeanSolarTime, const double SolarMeanAnomaly, const double EclipticLongitude)
{
	// the Julian date for the local true solar transit (or solar noon).
	double SolarTransit = 2451545.0 + MeanSolarTime + 0.0053 * sin(radians(SolarMeanAnomaly)) - 0.0069 * sin(radians(2 * EclipticLongitude));
	return(SolarTransit);
}
double getDeclinationOfTheSun(const double EclipticLongitude)
{
	double DeclinationOfTheSun = sin(radians(EclipticLongitude)) * sin(radians(23.44));
	return(DeclinationOfTheSun);
}
double getHourAngle(const double Latitude, const double DeclinationOfTheSun)
{
	double HourAngle = (sin(radians(-0.83)) - sin(radians(Latitude)) * sin(radians(DeclinationOfTheSun))) / (cos(radians(Latitude)) * cos(radians(DeclinationOfTheSun)));
	return(HourAngle);
}
double getSunrise(const double SolarTransit, const double HourAngle)
{
	double Sunrise = SolarTransit - (HourAngle / 360);
	return(Sunrise);
}
double getSunset(const double SolarTransit, const double HourAngle)
{
	double Sunset = SolarTransit + (HourAngle / 360);
	return(Sunset);
}

Here’s the code snippet I was using when trying to get the Wikipedia equations working.

double JulianDay = JulianDate2JulianDay(Time2JulianDate(LoopStartTime));
double MeanSolarTime = getMeanSolarTime(JulianDay, Longitude);
double SolarMeanAnomaly = getSolarMeanAnomaly(MeanSolarTime);
double EquationOfTheCenter = getEquationOfTheCenter(SolarMeanAnomaly);
double EclipticLongitude = getEclipticLongitude(SolarMeanAnomaly, EquationOfTheCenter);
double SolarTransit = getSolarTransit(MeanSolarTime, SolarMeanAnomaly, EclipticLongitude);
double DeclinationOfTheSun = getDeclinationOfTheSun(EclipticLongitude);
double HourAngle = getHourAngle(Latitude, DeclinationOfTheSun);
double Sunrise = getSunrise(SolarTransit, HourAngle);
double Sunset = getSunset(SolarTransit, HourAngle);
std::cout.precision(std::numeric_limits<double>::max_digits10);
std::cout << "         Julian Date: " << Time2JulianDate(LoopStartTime) << std::endl;
std::cout << "           Unix Time: " << LoopStartTime << std::endl;
std::cout << "         Julian Date: " << timeToExcelLocal(JulianDate2Time(Time2JulianDate(LoopStartTime))) << std::endl;
std::cout << "            Latitude: " << Latitude << std::endl;
std::cout << "           Longitude: " << Longitude << std::endl;
std::cout << "          Julian Day: " << JulianDay << std::endl;
std::cout << "       MeanSolarTime: " << MeanSolarTime << std::endl;
std::cout << "    SolarMeanAnomaly: " << SolarMeanAnomaly << std::endl;
std::cout << " EquationOfTheCenter: " << EquationOfTheCenter << std::endl;
std::cout << "   EclipticLongitude: " << EclipticLongitude << std::endl;
std::cout << "        SolarTransit: " << timeToExcelLocal(JulianDate2Time(SolarTransit)) << std::endl;
std::cout << " DeclinationOfTheSun: " << DeclinationOfTheSun << std::endl;
std::cout << "           HourAngle: " << HourAngle << std::endl;
std::cout << "             Sunrise: " << timeToExcelLocal(JulianDate2Time(Sunrise)) << std::endl;
std::cout << "              Sunset: " << timeToExcelLocal(JulianDate2Time(Sunset)) << std::endl;

Perils of Open Tabs

I have had several tabs open in my browser for an extended period of time, across multiple reboots of my computer, and connecting to several different networks.

Today I was traveling and happened to be using the WiFi in the Amex Centurion Lounge. Two of the background tabs in my browser reported privacy errors, and when I tried to go to them anyway, they were redirected to an ATT hotspot error page. What’s really frustrating is that now, I don’t know what one of the original URLS was.

Initial message when page didn’t load
Web Page Blocked

Both pages were hosted on opensource.com and I’m pretty sure were related to GIT, possibly with a group of handy commands to remember.

Govee H5182 Dual Probe Bluetooth Meat Thermometer

Govee H5182

Because I can’t seem to stop playing with the Govee line of Bluetooth thermometers, when this model came on sale for less than $20 I added it to my Amazon cart.

I knew that the Bluetooth protocol would be different purely based on multiple sensors, but I wanted to play with it anyway. With less than a month since I’d previously figured out a device, I was able to figure this one out in less than an hour, though I still don’t know the battery strength indicator in either this or the H5183 I decoded last month. I’ve added the code to my monitoring program https://github.com/wcbonner/GoveeBTTempLogger/  but have not yet published the changes. The existing code is designed around a single temperature, humidity, and battery reading per device. Even the H5183 broke the model slightly because it has two temperatures, current and alarm, and no humidity. I’m rethinking my data storage to be able to be more expandable, while still being backwards compatible as far as the log files, and will publish the new code when I’ve got it working.

This unit has the advantage of a display over the H5183. It can be used without connecting to a phone at all. You can turn it on, set the alarms, and monitor what’s going on. The one thing I did not figure out how to do using the buttons was to change the display from °F to °C, which I wanted to do to simplify debugging. Using the app I was able to update the settings and view the details.

Details from Amazon:

  • Remote Monitoring: With this wireless grill meat thermometer, you can spend less time waiting and more time multitasking within a 230ft Bluetooth range, Pull out your phone and check the meat temperatures of your grill, smoker, and oven at any time
  • Meat Temperature Settings: Govee Home App offers USDA-recommended food temperatures with various doneness levels for foods such as beef, lamb, chicken, pork, turkey, and more, Grill more efficiently, whether you’re a beginner or a professional
  • Smart Alerts: Avoid the risk of overcooking your food, Once your preset temperature is out of range, your meat thermometer will beep and send an instant notification to your phone via the Govee Home App
  • Food Grade Probes: Our food grade 304 stainless steel thermometer probes have an accuracy of ±1.8°F/1°C to reliably monitor the doneness of your food (under 302℉) ,*Please remember not to touch the probes immediately after use
  • Easy to Place: This Bluetooth meat thermometer has a strong magnetic backing and a folding stand to easily set up on the grill, smoker and oven when barbecuing, There is also a large backlight screen to make it easier to read at night

There is also a single probe version of this thermometer. I expect it works similar to this unit, but the price savings made it less interesting to me since I already have the H5183 in my kitchen.

Govee H5181