Thursday, April 20, 2017

UPDATED: A Silly Little Project

I have a Freetronics EtherTen Arduino Uno-compatible board that includes SD card and Ethernet support on-board.  It is a cool little board that I use a lot for prototyping a lot of projects.  I also have a Freetronics LCD and Keypad Shield that is usually attached to it.  This combo usually knocks around in my go bag, but in-between I wanted it to provide some useful function at my workspace when not otherwise occupied.

What I decided to do is build a two time zone clock that keeps synchronized to the NIST time servers.



So, the gist here was to have UTC and my local time zone displayed along with the local time zone date.  I didn't ever want to have to mess with setting it and wanted it to automatically handle daylight savings time.  The clock uses the millisecond timer to keep time, resetting to time.nist.gov every hour.  The update from the time server takes a couple seconds to accomplish, so we don't want to hammer on the network.  Once per hour appears to be completely adequate.

The code is a collection of code snips from the internet and some clever logic to avoid a bunch of if-then-else logic when calculating daylight savings time.  I can't really claim much ownership of anything other than this unique instance of these snippits.  My thanks goes out to the original authors for the inspiration and willingness to share code.

So, let's walk through the code from the top.  Here we have all necessary include files and initialization of the LCD and Ethernet bits.  We use UDP to communicate with the NIST server.

// UDP NTP Client - Implements an NTP set clock
//
// Displays UTC and Pacific time and date on 16x2 LCD display synchronized to internet time server time.nist.gov.
// Calculates daylights savings time, handles leap years
//
// Jeff Whitlatch ko7m - 23 Mar 2017

#include <Ethernet.h>
#include <EthernetUdp.h>
#include <LiquidCrystal.h>

// Freetronics 16x2 version
LiquidCrystal lcd(8,9,4,5,6,7);

// MAC address to use for Ethernet adapter
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

unsigned int localPort = 8888;       // local port to listen for UDP packets

char timeServer[] = "time.nist.gov"; // time.nist.gov NTP server

const int NTP_PACKET_SIZE = 48; // NTP time stamp is in the first 48 bytes of the message

byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing packets

// A UDP instance to let us send and receive packets over UDP
EthernetUDP Udp;

Here we set up a textual month array and process the setup() function which sets up the serial debug port, ethernet port and starts the UDP listener on the local port 8888.

const char *szMonth[] = { "Jan", "Feb", "Mar", "Apr", "May", "Jun",
                          "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" };

void setup()
{
  // Open serial communications and wait for port to open:
  Serial.begin(115200);
  Serial.println("Setting up ethernet");

  // Init the LCD display
  lcd.begin(16, 2);
  
  // start Ethernet and UDP
  if (Ethernet.begin(mac) == 0)
  {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for (;;);
  }
  Udp.begin(localPort);
}

One-time initialization of some time-related globals is next.

unsigned long secsSince1900 = 0;

int hour = 0;
int minute = 59;
int second = 58;
int day = 0;
int month = 0;
int year = 1900;
int dow = 0;
boolean fDST = false;

char buf[256];

Now, we get into the main loop which keeps the display updated predominantly.  Some optimization should be done here to only update the LCD when it changes, but in this simple code, I am updating it every time through the main loop.

Once per hour and on first boot, we send a NTP time request packet to the NIST server and parse the result that is returned.  We just extract the information of interest.  NTP is described in RFC 1305 and returns time as the number of seconds since 1 Jan 1900.

void loop()
{
  // Once every hour, update from the net, this takes a couple seconds so we don't do every time thru loop
  if (minute == 59 && second == 58)
  {
    sendNTPpacket(timeServer); // send an NTP packet to a time server
  
    delay(1000);
  
    if (Udp.parsePacket())
    {
      // We've received a packet, read the data from it
      Udp.read(packetBuffer, NTP_PACKET_SIZE); // read the packet into the buffer
  
      // the timestamp starts at byte 40 of the received packet and is four bytes,
      // or two words, long. First, extract the two words:
  
      unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
      unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]);
      // combine the four bytes (two words) into a long integer
      // this is NTP time (seconds since Jan 1 1900):
      secsSince1900 = highWord << 16 | lowWord;
      Serial.print("Seconds since Jan 1 1900 = ");
      Serial.println(secsSince1900);
    }
  }

Now Unix time is the number of seconds since 1 Jan 1970, so we need to convert.

  // Unix time starts on Jan 1 1970. In seconds, that's 2208988800:
  const unsigned long seventyYears = 2208988800UL;
  
  // subtract seventy years:
  unsigned long epoch = secsSince1900 - seventyYears;

Now, we can calculate the time of day.

  hour = (epoch / 60 / 60) % 24;
  minute = (epoch / 60) % 60;
  second = epoch % 60;

The algorithm implements a proleptic Gregorian calendar. That is, the rules which adopted the Julian calendar in 1582 in Rome are applied both backwards and forwards in time. This includes a year 0, and then negative years before that, all following the rules for the Gregorian calendar.  The accuracy of the algorithms under these rules is exact, until overflow occurs. Using 32 bit arithmetic, overflow occurs approximately at +/- 5.8 million years. Using 64 bit arithmetic overflow occurs far beyond +/- the age of the universe. The intent is to make range checking superfluous.


These algorithm internally assumes that March 1 is the first day of the year. This is convenient because it puts the leap day, Feb. 29 as the last day of the year, or actually the preceding year. That is, Feb. 15, 2000, is considered by this algorithm to be the 15th day of the last month of the year 1999. This detail is only important for understanding how the algorithm works.

Additionally the algorithm makes use of the concept of an era. This concept is very handy in creating an algorithm that is valid over extremely large ranges. An era is a 400 year period. As it turns out, the calendar exactly repeats itself every 400 years. And so we first compute the era of a year/month/day triple, or the era of a serial date, and then factor the era out of the computation. The rest of the computation centers on concepts such as:

  • What is the year of the era (yoe)? This is always in the range [0, 399].
  • What is the day of the era (doe)? This is always in the range [0, 146096].

Further details on the calculations can be obtained from the wonderful write-up here.

  // Algorithm: http://howardhinnant.github.io/date_algorithms.html#civil_from_days
  // an era is a 400 year period starting 1 Mar 0000.  
  long z = epoch / 86400 + 719468;
  long era = (z >= 0 ? z : z - 146096) / 146097;
  unsigned long doe = static_cast<unsigned long>(z - era * 146097);
  unsigned long yoe = (doe - doe/1460 + doe/36524 - doe/146096) / 365;
  unsigned long doy = doe - (365*yoe + yoe/4 - yoe/100);
  unsigned long mp = (5*doy + 2)/153;
  
  month = mp + (mp < 10 ? 3 : -9);
  day = doy - (153*mp+2)/5 + 1;
  year = static_cast<int>(yoe) + era * 400;
  year += (month <= 2);

  dow = dowFromDate(day, month, year);
  fDST = isDST(day, month, dow);

Now, we have all the bits necessary for printing, so we build a text buffer and send it to the LCD display.

  // Build the print buffer
  sprintf(buf, "%2d:%02d UTC %2d %s", hour, minute, day, szMonth[month-1]);
  lcd.setCursor(0, 0);
  lcd.print(buf);

  // Print the second time zone
  if (hour < 8) hour += 24;
  sprintf(buf, "%2d:%02d %s  %d", hour - 7, minute, fDST ? "PDT" : "PST", year);
  lcd.setCursor(0, 1);
  lcd.print(buf);
  
  // Keep the network alive every 60 seconds
  if (second % 60 == 0)
      Ethernet.maintain();
      
  // Accuracy depends on system clock accuracy, corrected once an hour.
  delay(1000);
  secsSince1900++;
}

I am not going to detail the NTP protocol here.  The following is sufficient for the meager needs of this simple application.  We build the UDP request packet and send if off to time.nist.gov.

// send an NTP request to the time server at the given address
void sendNTPpacket(char* address)
{
  // set all bytes in the buffer to 0
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  // Initialize values needed to form NTP request
  // (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12]  = 49;
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;

  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp:
  Udp.beginPacket(address, 123); //NTP requests are to port 123
  Udp.write(packetBuffer, NTP_PACKET_SIZE);
  Udp.endPacket();
}

The determination of daylight savings time is a bit of a moving target over time, but this function encapsulates the yes/no decision to allow for appropriate correction of the local time display along with the date.

// In most of the US, DST starts on the second Sunday of March and 
// ends on the first Sunday of November at 2 AM both times.
bool isDST(int day, int month, int dow)
{
  if (month < 3 || month > 11) return false;
  if (month > 3 && month < 11) return true;

  int prevSunday = day - dow;

  // In March, we are DST if our previous Sunday was on or after the 8th.
  if (month == 3) return prevSunday >= 8;

  // In November, we must be before the first Sunday to be DST which means the
  // previous Sunday must be before the 1st.
  return prevSunday <=0;
}
  This one is kind-of fun.  See if you can describe this one liner in words...  :) 
  // Day of week calculation
int dowFromDate(int d, int m, int y)
{
  return (d += m < 3 ? y-- : y - 2, 23*m/9 + d + 4 + y/4- y/100 + y/400)%7;

}

And, that is really all there is to it.  I am sure the reader can adapt the code for different time zone pairs, but if you run into difficulty I will be happy to assist if you drop me a note at ko7m at ARRL dot net.  For your convenience, here is the entire code file:

// UDP NTP Client - Implements an NTP set clock
//
// Displays UTC and Pacific time and date on 16x2 LCD display synchronized to internet time server time.nist.gov.
// Calculates daylights savings time, handles leap years
//
// Jeff Whitlatch - 23 Mar 2017

#include <Ethernet.h>
#include <EthernetUdp.h>
#include <LiquidCrystal.h>

// Freetronics 16x2 version
LiquidCrystal lcd(8,9,4,5,6,7);

// MAC address to use for Ethernet adapter
byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };

unsigned int localPort = 8888;       // local port to listen for UDP packets

char timeServer[] = "time.nist.gov"; // time.nist.gov NTP server

const int NTP_PACKET_SIZE = 48; // NTP time stamp is in the first 48 bytes of the message

byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing packets

// A UDP instance to let us send and receive packets over UDP
EthernetUDP Udp;

const char *szMonth[] = { "Jan", "Feb", "Mar", "Apr", "May", "Jun",
                          "Jul", "Aug", "Sep", "Oct", "Nov", "Dec" };

void setup()
{
  // Open serial communications and wait for port to open:
  Serial.begin(115200);
  Serial.println("Setting up ethernet");

  // Init the LCD display
  lcd.begin(16, 2);
  
  // start Ethernet and UDP
  if (Ethernet.begin(mac) == 0)
  {
    Serial.println("Failed to configure Ethernet using DHCP");
    // no point in carrying on, so do nothing forevermore:
    for (;;);
  }
  Udp.begin(localPort);
}

unsigned long secsSince1900 = 0;

int hour = 0;
int minute = 59;
int second = 58;
int day = 0;
int month = 0;
int year = 1900;
int dow = 0;
boolean fDST = false;

char buf[256];

void loop()
{
  // Once every hour, update from the net, this takes a couple seconds so we don't do every time thru loop
  if (minute == 59 && second == 58)
  {
    sendNTPpacket(timeServer); // send an NTP packet to a time server
  
    delay(1000);
  
    if (Udp.parsePacket())
    {
      // We've received a packet, read the data from it
      Udp.read(packetBuffer, NTP_PACKET_SIZE); // read the packet into the buffer
  
      // the timestamp starts at byte 40 of the received packet and is four bytes,
      // or two words, long. First, extract the two words:
  
      unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
      unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]);
      // combine the four bytes (two words) into a long integer
      // this is NTP time (seconds since Jan 1 1900):
      secsSince1900 = highWord << 16 | lowWord;
      Serial.print("Seconds since Jan 1 1900 = ");
      Serial.println(secsSince1900);
    }
  }

  // Unix time starts on Jan 1 1970. In seconds, that's 2208988800:
  const unsigned long seventyYears = 2208988800UL;
  
  // subtract seventy years:
  unsigned long epoch = secsSince1900 - seventyYears;

  hour = (epoch / 60 / 60) % 24;
  minute = (epoch / 60) % 60;
  second = epoch % 60;

  // Algorithm from http://howardhinnant.github.io/date_algorithms.html#civil_from_days
  // an era is a 400 year period starting 1 Mar 0000.  
  long z = epoch / 86400 + 719468;
  long era = (z >= 0 ? z : z - 146096) / 146097;
  unsigned long doe = static_cast<unsigned long>(z - era * 146097);
  unsigned long yoe = (doe - doe/1460 + doe/36524 - doe/146096) / 365;
  unsigned long doy = doe - (365*yoe + yoe/4 - yoe/100);
  unsigned long mp = (5*doy + 2)/153;
  
  month = mp + (mp < 10 ? 3 : -9);
  day = doy - (153*mp+2)/5 + 1;
  year = static_cast<int>(yoe) + era * 400;
  year += (month <= 2);

  dow = dowFromDate(day, month, year);
  fDST = isDST(day, month, dow);

  // Build the print buffer
  sprintf(buf, "%2d:%02d UTC %2d %s", hour, minute, day, szMonth[month-1]);
  lcd.setCursor(0, 0);
  lcd.print(buf);

  // Print the second time zone
  if (hour < 8) hour += 24;
  sprintf(buf, "%2d:%02d %s  %d", hour - 7, minute, fDST ? "PDT" : "PST", year);
  lcd.setCursor(0, 1);
  lcd.print(buf);
  
  // Keep the network alive every 60 seconds
  if (second % 60 == 0)
      Ethernet.maintain();
      
  // Accuracy depends on system clock accuracy, corrected once an hour.
  delay(1000);
  secsSince1900++;
}

// send an NTP request to the time server at the given address
void sendNTPpacket(char* address)
{
  // set all bytes in the buffer to 0
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  // Initialize values needed to form NTP request
  // (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12]  = 49;
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;

  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp:
  Udp.beginPacket(address, 123); //NTP requests are to port 123
  Udp.write(packetBuffer, NTP_PACKET_SIZE);
  Udp.endPacket();
}

// In most of the US, DST starts on the second Sunday of March and ends on the first 
// Sunday of November at 2 am both times.
bool isDST(int day, int month, int dow)
{
  if (month < 3 || month > 11) return false;
  if (month > 3 && month < 11) return true;

  int prevSunday = day - dow;

  // In March, we are DST if our previous sunday was on or after the 8th.
  if (month == 3) return prevSunday >= 8;

  // In November, we must be before the first Sunday to be DST which means the
  // previous sunday must be before the 1st.
  return prevSunday <=0;
}

// Day of week calculation
int dowFromDate(int d, int m, int y)
{
  return (d += m < 3 ? y-- : y - 2, 23*m/9 + d + 4 + y/4- y/100 + y/400)%7;

}

No comments:

Post a Comment