Summertime! Let's get out there and fox hunt! (Part 1)

With the recent 32 degree C (90 F) weather we have been having lately, I began thinking of summertime ham radio activities and I think it is time to build a hidden transmitter (fox) for our summer Salmoncon group activities.  My buddy Eldon (WA0UWH) and I have provided fox hunt activities for the group since about 2012.  This year I decided to make some changes.

Historically, we have provided 1 to 4 beacons on 30 metres and coached folks on the use of simple loop antennas in radio direction finding (RDF).  This year I have decided to add a VHF component to the hunt as most of the group owns VHF FM equipment.

Our summer camp site sits on about 50 acres up against the western slopes of the Cascade Mountains near North Bend, WA.  It is a truly idyllic location and a wonderful venue for our summer camp-out called Salmoncon.  If you are going to be in the area, we hope you will plan to drop in and visit.

Below and in a previous post I included a picture of a couple of our transmitter hunt participants from years past.  We have done the event every year and I have threatened to not do the event a couple of times but the gathering lynch mob convinced us otherwise.  So, it is a simple thing, but apparently it is quite popular.

What I want to do here is to provide the details of this year's hidden transmitter hardware and firmware so that others who might be interested in hosting such an event might be able to leverage the effort to best benefit.


Here is my little fox hunt transmitter that is able to transmit CW on any HF frequency you like while simultaneously sending an FM signal on 2 metres.

As you can see it fits nicely on the side of a standard 9V battery.  I have attached it with a little blob of putty that is used to adhere things to the wall in your home but is still removable without damaging the wall.  I find that it holds really well, but not as strongly as for example double sided foam tape.  Not shown are the wires that attach to HF and VHF antennas.  These connect to two of the pins along the side of the board.  In practice this gets wrapped in bubble wrap and stuffed into a short length of PVC pipe.  Pipe end caps provide water and dust protection.  Small eye bolts are used to connect the electronics to external antennas and to provide a convenient way to hang the transmitter up in a tree for example using the antenna wire as a support.

The board is a Parallax Propeller Mini board.  I have written previously on various projects that I have based on this processor.  This is a slick little board packing a 32 bit, eight core 160 MIPS 80 MHz processor which is complete overkill for this project, but a key feature of the Propeller chip makes it ideal for this application.

Each core has two counter modules.  Each counter module can control or monitor up to two I/O pins and perform conditional 32-bit accumulation of its FRQ register into its PHS register on every clock cycle.  Each counter module also has its own phase-locked loop (PLL) which can be used to synthesize frequencies up to 128 MHz.  In reality it can synthesize frequencies up to about 220 MHz, but Parallax recommends "for best stability" to limit frequency synthesis to 128 MHz maximum.  I am using it at 146 Mhz without issue.

Let me first discuss the CW HF beacon.  Typically we use four of these transmitters hidden around the venue.  Each is assigned a serial number 0-3 which is used to determine the content of the transmitted beacon.  We picked morse code characters that had the exact same transmission length for each of the beacons based on serial number.  V, F, L, and B.  The beacon will send this letter 5 times on a schedule discussed below.  By turning on all four transmitters simultaneously, the fox hunters will hear zero or more of them identifying at the same time.  The goal is to locate them all and we typically keep the game running for several days to give everyone a chance.  If there is interest and the foxes have been found, we move them.  (Don't forget to check the collar of any of the dogs on site for a transmitter...)

The Propeller chip is programmed in one of two languages.  SPIN or PASM.  Spin is a C-like (sort of) language while PASM is the Propeller ASseMbly language.  All of my code presented in this posting will be in the SPIN language.  I am not going to provide a tutorial on the language, for that you can hit the reference manual for the gory details.  If you get stuck, drop me a line at ko7m at arrl dot org and I will try to help you out.

The top part of the code declares the clock speed, PLL settings and crystal frequency.  This is followed by any constants used by the program.

' HF CW Fox Hunt Beacon
'
' ko7m - Jeff Whitlatch
'
con
  _CLKMODE = XTAL1 + PLL16X
  _XINFREQ = 5_000_000

The following constants are used in generating random numbers and to fine tune the timing accuracy of a delay function.

  Seed = 31414
  WMin = 381

Each transmitter gets a 6 second slot for transmitting, chosen at random and the transmitters re-sync random number generation every two minutes. 
  
  'Sync Each Ten Minutes So that Others can be be started Later, by watching the wall clock
  TimeSync  = 60 * 2            ' Seconds between time syncs (2 min)
  SlotTime  = 6                 ' Seconds
  TimeSlots = TimeSync / SlotTime ' Number of time slots (in this case 20)

We transmit at 15 words per minute on a frequency of 10.1395 MHz with a tone offset of 600 hz.  Here we also define which pin is used to generate the RF at this frequency.  Any available pin may be chosen for this task.

  wpmInit      =         15      ' Initial code speed in words per minute
  defaultFreq  = 10_139_500      ' Default transmit frequency
  defaultTone  =        600      ' Offset from transmit frequency
  defaultRFPin =         16      ' RF output default pin

This data section defines an array of bytes used to identify each transmitter.  These are the letters sent (repeated 5 times) during each time slot.  For example the transmitter with the serial number 2 will send VVVVV in morse code in randomly chosen time slots.

Here we also declare an object instance of the frequency synthesizer object used to generate RF energy.  This module is provided as part of the standard modules available with the development tools for the Propeller processor.  The source code is provided and nothing is done here that you cannot do yourself in your own code.
    
dat
  Ids      BYTE "BFVL"           'Equal Length Single Character IDs, one for each SN

obj
  Freq  : "Synth"

Local variables are now defined.

var
  long TimeSlot
  long Rand
  long WPM
  long Frequency                ' Current frequency
  long toneFreq                 ' Offset from frequency in Hz
  
  byte RFPin
  byte ditTime                  ' Time in milliseconds for a dit (dot)
  byte SN                       ' Serial number (set by switches)
  long Stack[128]
  BYTE Cog

Public functions now follow.  Main performs initialization and then runs the CW beacon routines.

pub Main
  Init
  doHFFox

The Start and Stop functions allow starting this code up on a different core and stopping it.  This allows me to implement both the HF and VHF transmitters in a way that they can function simultaneously.

PUB Start : fSuccess 
  fSuccess := (Cog := cognew(Main, @Stack) + 1) > 0

PUB Stop
  if Cog
    cogstop(Cog~ - 1)

This is the main function that does all the magic.  It schedules the random slot selections and at the appropriate time sends the identifier character in morse code based on the serial number.

During slots 1 and 5, the fox with serial number 0 transmits.  Slots 2 and 6, serial number 1 transmits and so forth.  During slot 9 all foxes transmit and during slot 0, nobody transmits.  The effect is that you may hear zero to four foxes all talking at once during each time slot.

pri doHFFox | Task, C
  C := cnt
      
  repeat
    waitcnt(C += (clkfreq * SlotTime) #> WMin)          ' Wait for our slot time                                       '
    C := cnt                                            ' Get the current clock value

    SN := 2                                             ' Serial number is hard coded - should be on switch input                        

    if TimeSlot == 0                                    ' Reset Sequence
      Rand := Seed

    Task := ||?Rand // 10                               ' Random task number (only use last digit)
    
    case (Task)
      1,5:
        if SN == 0
          sendId(SN)                    ' Send my ID at different times based on serial number and task                        
      2,6:
        if SN == 1
          sendId(SN)
      3,7:
        if SN == 2
          sendId(SN)
      4,8:
        if SN == 3
          sendId(SN)
      9:
        sendId(SN)                                      ' Everybody sends
      0:
        sendNothing                      ' Noboday sends (silence is heard for time required to identify)                       
    
    TimeSlot := (TimeSlot + 1) // TimeSlots

Now, we have the private functions.  Init handles initialization setting the frequency, morse code speed, RF pin to use, etc.

pri Init  
  TimeSlot  := 0
  Rand      := Seed
  WPM       := wpmInit
  ditTime   := 1200 / WPM                        ' Calculate dit time based on 50 dit duration standard
  Frequency := defaultFreq
  toneFreq  := defaultTone
  RFPin     := defaultRFPin                       

Now we have the various "send" functions that handle the actual morse code transmissions.  sendId send the single identifier character based on serial number 5 times.  sendNothing does exactly that.  sendCode works with sendSymbol to allow morse code transmission of strings of characters.

pri sendId(_SN)
  repeat 5
    sendSymbol(Ids[_SN])                                ' Send the associated ID for the serial number

pri sendNothing
  
pri sendCode(stringptr)
  repeat strsize(stringptr)
    sendSymbol(byte[stringptr++])                       ' Send each symbol of the stream

sendSymbol is the main function that provides the conversion of an ascii character into a series of morse code symbols that are used to key the RF energy on and off forming morse code characters.  The character itself is used as the index into a lookup table of morse code symbols for that character.  The function then forms properly constructed morse code and keys the transmitter.

pri sendSymbol(char) | cwBits, dotsToWait, iCodes
  if char == " "                                        ' Handle space character as 7 dot times                
    dotsToWait := 5
    delay(dotsToWait * ditTime)
  else
    if char => "a" and char =< "z"                      ' Convert lower case to upper case
      char := char - "a" + "A"
    iCodes := lookdown(char: "!", $22, "$&'()+,-./0123456789:;=?@ABCDEFGHIJKLMNOPQRSTUVWXYZ_")
    ' If unsupported character, ignore it
    if iCodes == 0
      return
    cwBits := cwCodes[iCodes - 1]                       ' Grab the CW bit codes for this symbol                        

    repeat while cwBits > 1
      if cwBits & 1
        dotsToWait := 3         ' dah
      else
        dotsToWait := 1         ' dit
        
      keyDown
      delay(dotsToWait * ditTime)   
      keyUp                      ' stop sending
      delay(ditTime)              ' one dot time between symbols
      cwBits >>= 1                ' get the next symbol (dit or dah)
                            
  delay(ditTime * 2)            ' two more dot times to give 3 dot character spacing                      

The following functions are use to control the frequency synthesizer to actually generate RF energy.  sendTone turns on an RF carrier at the requested frequency plus a tone offset value.  noTone turns off that RF carrier.  These functions are used by keyDown and keyUp to generate morse code keying transitions.  delay is a general purpose function to delay for the specified number of milliseconds.

pri sendTone(tone)
  Freq.Synth("A", RFPin, Frequency + tone)              ' 

pri noTone
  Freq.Synth("A", RFPin, 0)

pri keyDown
  sendTone(toneFreq)            ' Sidetone or transmit

pri keyUp
  noTone                        ' Stop sidetone or transmit
    
pri delay(Duration)
  waitcnt(((clkfreq / 1_000 * Duration - 3932) #> WMin) + cnt)

This data block is used  to define the morse code characters as described in the comments below.  Each character is stored in a byte where each one bit represents a dash (or dah) and a zero bit indicates a dot (or dit) in morse code. 
         
dat
  ' Morse code tables below are encoded as "1" being a dah and "0" being a dit.  The end of the character
  ' is signified by the most significant "1" bit.  Thus, while shifting the byte value right, we can easily
  ' test for the end of character condition by checking to see if the current value is > 1.
  '
  '            !         "         $         &         '         (         )         +         ,         -         .         /     
  cwCodes byte %1110101, %1010010, %11001000,%100010,  %1011110, %101101,  %1101101, %101010,  %1110011, %1100001, %1101010, %101001
  '            0         1         2         3         4         5         6         7         8         9
          byte %111111,  %111110,  %111100,  %111000,  %110000,  %100000,  %100001,  %100011,  %100111,  %101111
  '            :         ;         =         ?         @         
          byte %1000111, %1010101, %110001,  %1001100, %1010110 
  '            A         B         C         D         E         F         G         H        
          byte %110,     %10001,   %10101,   %1001,    %10,      %10100,   %1011,    %10000
  '            I         J         K         L         M         N         O         P        Q        R        
          byte %100,     %11110,   %1101,    %10010,   %111,     %101,     %1111,    %10110,  %11011,  %1010
  '            S         T         U         V         W         X         Y         Z        _ 
          byte %1000,    %11,      %1100,    %11000,   %1110,    %11001,   %11101,   %10011,  %1101100

          
In the next installment, I will describe the 2 metre FM fox.  Stay tuned!  For your reference I have pasted the entirety of the code above for convenience.  If you use this code, paste it into its own file and save it in a directory where the rest of the fox hunt transmitter files will be stored.  Name the file something like HFFox.spin.  The .spin suffix is used for SPIN program files.

' HF CW Fox Hunt Beacon
'
' ko7m - Jeff Whitlatch
'
con
  _CLKMODE = XTAL1 + PLL16X
  _XINFREQ = 5_000_000

  Seed = 31414
  WMin = 381
   
  'Sync Each Ten Minutes So that Others can be be started Later, by watching the wall clock
  TimeSync  = 60 * 2            ' Seconds between time syncs (2 min)
  SlotTime  = 6                 ' Seconds
  TimeSlots = TimeSync / SlotTime ' Number of time slots (in this case 20)

  wpmInit      =         15      ' Initial code speed in words per minute
  defaultFreq  = 10_139_500      ' Default transmit frequency
  defaultTone  =        600      ' Offset from transmit frequency
  defaultRFPin =         16      ' RF output default pin
    
dat
  Ids      BYTE "BFVL"                          'Equal Length Single Character IDs, one for each SN

obj
  Freq  : "Synth"

var
  long TimeSlot
  long Rand
  long WPM
  long Frequency                ' Current frequency
  long toneFreq                 ' Offset from frequency in Hz
  
  byte RFPin
  byte ditTime                  ' Time in milliseconds for a dit (dot)
  byte SN                       ' Serial number (set by switches)
  long Stack[128]
  BYTE Cog

pub Main
  Init
  doHFFox

PUB Start : fSuccess 
  fSuccess := (Cog := cognew(Main, @Stack) + 1) > 0

PUB Stop
  if Cog
    cogstop(Cog~ - 1)

pri doHFFox | Task, C
  C := cnt
      
  repeat
    waitcnt(C += (clkfreq * SlotTime) #> WMin)          ' Wait for our slot time                                       '
    C := cnt                                            ' Get the current clock value

    SN := 2                                             ' Serial number is hard coded - should be on switch input                        

    if TimeSlot == 0                                    ' Reset Sequence
      Rand := Seed

    Task := ||?Rand // 10                               ' Random task number (only use last digit)
    
    case (Task)
      1,5:
        if SN == 0
          sendId(SN)                                    ' Send my ID at different times based on serial number and task                        
      2,6:
        if SN == 1
          sendId(SN)
      3,7:
        if SN == 2
          sendId(SN)
      4,8:
        if SN == 3
          sendId(SN)
      9:
        sendId(SN)                                      ' Everybody sends
      0:
        sendNothing                                     ' Noboday sends (silence is heard for time required to identify)                       
    
    TimeSlot := (TimeSlot + 1) // TimeSlots

pri Init  
  TimeSlot  := 0
  Rand      := Seed
  WPM       := wpmInit
  ditTime   := 1200 / WPM                               ' Calculate dit time based on 50 dit duration standard
  Frequency := defaultFreq
  toneFreq  := defaultTone
  RFPin     := defaultRFPin                       

pri sendId(_SN)
  repeat 5
    sendSymbol(Ids[_SN])                                ' Send the associated ID for the serial number

pri sendNothing
  
pri sendCode(stringptr)
  repeat strsize(stringptr)
    sendSymbol(byte[stringptr++])                       ' Send each symbol of the stream

pri sendSymbol(char) | cwBits, dotsToWait, iCodes
  if char == " "                                        ' Handle space character as 7 dot times                
    dotsToWait := 5
    delay(dotsToWait * ditTime)
  else
    if char => "a" and char =< "z"                      ' Convert lower case to upper case
      char := char - "a" + "A"
    iCodes := lookdown(char: "!", $22, "$&'()+,-./0123456789:;=?@ABCDEFGHIJKLMNOPQRSTUVWXYZ_")
    ' If unsupported character, ignore it
    if iCodes == 0
      return
    cwBits := cwCodes[iCodes - 1]                       ' Grab the CW bit codes for this symbol                        

    repeat while cwBits > 1
      if cwBits & 1
        dotsToWait := 3         ' dah
      else
        dotsToWait := 1         ' dit
        
      keyDown
      delay(dotsToWait * ditTime)   
      keyUp                      ' stop sending
      delay(ditTime)              ' one dot time between symbols
      cwBits >>= 1                ' get the next symbol (dit or dah)
                            
  delay(ditTime * 2)            ' two more dot times to give 3 dot character spacing                      
                                
pri sendTone(tone)
  Freq.Synth("A", RFPin, Frequency + tone)              ' 

pri noTone
  Freq.Synth("A", RFPin, 0)

pri keyDown
  sendTone(toneFreq)            ' Sidetone or transmit

pri keyUp
  noTone                        ' Stop sidetone or transmit
    
pri delay(Duration)
  waitcnt(((clkfreq / 1_000 * Duration - 3932) #> WMin) + cnt)
         
dat
  ' Morse code tables below are encoded as "1" being a dah and "0" being a dit.  The end of the character
  ' is signified by the most significant "1" bit.  Thus, while shifting the byte value right, we can easily
  ' test for the end of character condition by checking to see if the current value is > 1.
  '
  '            !         "         $         &         '         (         )         +         ,         -         .         /     
  cwCodes byte %1110101, %1010010, %11001000,%100010,  %1011110, %101101,  %1101101, %101010,  %1110011, %1100001, %1101010, %101001
  '            0         1         2         3         4         5         6         7         8         9
          byte %111111,  %111110,  %111100,  %111000,  %110000,  %100000,  %100001,  %100011,  %100111,  %101111
  '            :         ;         =         ?         @         
          byte %1000111, %1010101, %110001,  %1001100, %1010110 
  '            A         B         C         D         E         F         G         H        
          byte %110,     %10001,   %10101,   %1001,    %10,      %10100,   %1011,    %10000
  '            I         J         K         L         M         N         O         P        Q        R        
          byte %100,     %11110,   %1101,    %10010,   %111,     %101,     %1111,    %10110,  %11011,  %1010
  '            S         T         U         V         W         X         Y         Z        _ 
          byte %1000,    %11,      %1100,    %11000,   %1110,    %11001,   %11101,   %10011,  %1101100

          

Satellite Tracking (Update)

In a previous post I discussed the possibility of tracking satellites through the use of 9 degrees of freedom (9DOF) sensors to indicate the position of an otherwise arbitrarily positioned antenna.

Interestingly enough a kind reader pointed out to me a recent publication of a nearly identical solution to what I had envisioned.  Not wanting to necessarily duplicate the work of someone else, I faced the dilemma of continuing my efforts without reference to this other body of work or not.  It would be nice if the other author had done a poor job and I could improve on his work by publishing an update with an improved implementation.  However, I would find it difficult to improve on the work that has already been published.

So, my solution is going to be to recommend that you go take a look at the fine work done by Elwood Downey over at http://www.clearskyinstitute.com/ham/AST/ and leverage the excellent write-up and complete code listings if you are interested in building such a device.

I will be taking my own advice and implementing Elwood's solution, no doubt customizing things along the way.  Once I have that functional, I will publish an article here on my implementation of his work.

Nice work Mr. Downey!

Mobile Lab

Hmmm...  This has got me thinking...



Read all about it here.

Windows 10

So after 26 years at Microsoft, you would think I would welcome Windows 10.  Not so much.  Especially given I came home last evening and opened up my laptop to find it in the middle of an auto-magic update to Windows 10.

That being said, thus far the experience has been fairly seamless.  Everything was still where I left it and with the exception of the touch pad having some quirkiness I have not found any issues with windows or my installed base of applications.

The other advice I have for you if you decide (or have it decided for you) to install Windows 10 is to actually read that bloody license agreement.  Once you recover from that experience, do not take the automatic settings offered to you and actually go in and turn off the options to kindly share all you browse history, all your contacts, all your usage patterns and "other data" with my old employer and unnamed third parties.  Turn off Cortana unless you actually wish to have a live microphone transmitting everything said in the room to another unnamed third party.

Ok, so enough ranting.  I am now in the Windows 10 camp, wiping the slime off from the process of how I got here and more than slightly disgusted with the business practices of my former employer.

And now for something completely different...

Spent the weekend out at the coast with a group of aviator friends at the only beach state airport in the lower 48, Copalis State Airport at Copalis Beach, WA.  The sand is rock hard at low tide and I think you could have landed a 747 out there.  The airport normally has about 220 aircraft operations per year.  We landed over 54 aircraft on Saturday from all over the Pacific Northwest, hung out for several hours and departed without leaving any trace of our having been there.  The weather was perfect with about 15 knots of wind straight out of the north making for a rather epic day at the beach.

Satellite tracking

The beginnings of a new project.  Software is coming together, but now it's time to pull together the hardware.  This will be an antenna azimuth/elevation  satellite tracker.  In previous posts I have explored the calculations necessary and the details of using hobby servos to create a simple proof of concept.

For now I am experimenting with the ATMega2560 board (on the left) as it has plenty of flash and RAM space for the task at hand.  I plan to control the device via a web page that it serves up, so for now I have an ethernet shield attached, though in the end the plan is to use WiFi.

Calculating satellite passes requires an accurate position for the observer in terms of latitude and longitude, but also elevation and time-of-day.  I am using a GPS module to provide this information (lower right).

While the mighty ATMega2560 is certainly capable of generating the necessary PWM signals to position servos, I have decided to offload that to separate controller based on the PCA9685 which provides I2C interfacing of 16 channels of 12 bit PWM data.  This particular board was originally intended to be used as an RGB LED controller, but my plan is to re-purpose it for this task.  It has its own internal 25 MHz oscillator that offloads the ATMega2560 from having to generate these signals.  That, plus the fact that I happened to have one on hand drove the decision to use this component to drive my servos.

After spending some time in the Avionics industry and working on electronic aviation attitude indicators, the math involved in Euler angle solutions to the orientation of a rigid body, plus the mapping of raw gyro, accelerometer and magnetometer sensor data into 3D space is a bit tedious.  Fortunately for me, Bosch has done an amazing job of taking MEMS data from on-die sensors and adding an ARM Cortex M0 processor to produce a single chip solution that abstracts away all the raw sensor fusion and spits out data directly in quaternions, Euler angles or vectors.

The plan is to use this sensor mounted on the antenna fixture to measure azimuth and elevation relative to the calculated position on earth with no careful alignment of antenna orientation required.  The Arduino will accept keplerian elements for the desired satellite and then automagically track the spacecraft as it passes overhead.

More updates as this comes together into a demonstrable solution.

Interesting problem

Today, I was writing some code that had some bitmap files embedded in them.  Not wanting  to waste space, I decided to use gzip to compress the bitmap.  After converting the compressed file to something that looks like a byte array in C, I get something like this:

unsigned char bitmap[] =
{
    0x1f, 0x8b, 0x08, 0x08, 0x2d, 0x87, 0xf9, 0x56,
    0x02, 0x0b, 0x69, 0x63, 0x6e, 0x5f, 0x73, 0x79,
    0x6e, 0x74, 0x68, 0x2e, 0x62, 0x6d, 0x70, 0x00,
    0xed, 0x99, 0x31, 0x0e, 0x82, 0x30, 0x14, 0x86,
    0x45, 0xa4, 0x5f, 0xf6, 0x0f, 0x21, 0x93, 0x3c,
    0x23, 0x70, 0xbf, 0xab, 0x06, 0xff, 0x9e, 0x00,

    .
    .
    .

    0x00, 0x00, 0x00, 0x00, 0x00, 0x42, 0xe1, 0x05,
    0x5e, 0xd7, 0xb8, 0x1c, 0x36, 0x40, 0x00, 0x00
};


So, nothing terribly interesting in that, but after compiling this into an executable, I get trapped by my virus scanner saying that my shiny new executable has a virus.  If I comment out the bitmap part of the code, no virus.

Apparently whatever byte pattern is created by compressing this particular bitmap gets flagged by my virus scanner, so in order to debug this code, I need to create an exception for the executable I am producing in the virus scanner.  Never had this happen before, but is certainly an artifact of the world we now live in.

I ran into another interesting trap while unit testing this code.  The basic problem at hand was to embed a series of bitmap files in executable code that could be reconstituted at will.  So, to unit test this code, I created the list of bitmap files and used gzip to compress them into a series of filename.bmp.gz compressed files.  Each of the bitmaps is 16k bytes and they compressed down to about 1k on average, so the savings is significant.

I created a utility that would take a list of file names and generate a C source code file similar to what is shown above which was then included in a unit test program.  The test program recreated the decompressed file using my decompression algorithm and wrote the resultant file back to the disk in a different folder.  What I expected was to see a directory full of identical files to the original set.  What I got instead was about 1/3 of the files had 2 or 4 bytes extra.  The rest were identical to the original.

This as you might guess had me more than a little concerned that my decompression algorithm, while not detecting any errors, including a CRC check of the decompressed memory image of the file, was still different on disk than the original by length and therefore content.

Loading the bitmaps up into an image viewer showed very minor corruption of the image.  Rats...

To check this out, I did a binary difference of all the affected files and I saw an interesting pattern emerge.  Every file that contained a 0x0A byte value in the original file, had a two byte sequence (0x0D, 0x0A) replacing it in the corrupt file.

For those that spend some time programming computers, this may be recognized as an automatic Unix/Linux end-of-line convention being replaced by a Windows end-of-line convention.  Unix/Linux uses (typically) line-feed characters only at the end of a line of text in a file whilst Windows (typically) uses a carriage-return/line-feed pair of characters.  (Thanks Microsoft...)

The cause of this result was failing to remember to open the new file in binary mode rather than non-binary mode which caused the file write operation to replace every line-feed character in the file with a carriage-return/line-feed pair.  A simple change to the file open to use binary mode and the resulting files were now identical to the originals.  Don't forget to unit test your code.

Audio Glitches

How do you find something that takes four hours to occur, and then when it does it is over in 10 ms?  I find that it is really hard to be there when it happens.  When it does happen and you can capture the event, how do you figure out the root cause?

This reminds me a bit of the Pitch Drop Experiment, probably one of the longest running scientific experiments I have ever encountered.  Fortunately my problem is reproduce-able in hours, not decades.

The audio signal path is, to say the least, complicated.  Audio originated from a USB source got transmitted to a dedicated micro-controller.  From there it traversed an I2S bus to a DSP where some sample rate conversion is done.  The I2S bus then continues on to an Audinate network audio device where the audio is deposited on a Dante channel and sent via an Ethernet network switch onto a private LAN network.  On the other side of the network, the audio goes through another network switch into another Audinate device and then across an I16S bus to an FPGA where signal switching can be accomplished.  From there across an I32S bus to another DSP where any effects and other audio mixing can be done.  The composite signal then returns through the same path in reverse to the USB device that originated the signal where it is recorded.  What could possibly go wrong?

Well, surprisingly little actually does go wrong, but when it does it can be a bit subtle, so how to track it down?  Well, there are eight signal path segments in each direction.  My approach is to try and isolate each segment and verify them individually.  I needed to keep the audio signal in the digital realm as I certainly didn't want to introduce the complexity of digital to analogue conversions, plus the issue reproduces in the digital realm.

Since one piece of equipment in the scheme was from a 3rd party, the first task was to verify it could source and record digital audio without error and without using any of the suspect equipment.  So, I purchased a digital audio USB interface that would allow me to loop back said audio while keeping it in the digital realm.  I chose a commercially available USB to S/PDIF adapter and just looped the S/PDIF output back to the input.  Now the USB device could playback and record its own signal and see if an exact copy was produced.  No problems found.

So, in the interest of divide and conquer, I started by sending audio all the way to the FPGA but then intercepting the outbound signal and routing it to a 32 channel hard disk audio recorder.  This allowed recording of the digital signal at the mid-point of the signal path while simultaneously recording it at the return end of the signal path.  Multiple four hour runs were free of errors at the signal mid-point, but still had glitches at the return end of the signal path.  This eliminated all but one signal path in the outbound direction.  To eliminate the remaining signal path, I looped back the audio signal at the DSP at the far end of the path back to the FPGA before sending it to the digital audio recorder.  Still clean.

So, now with half the signal path eliminated, I generated a 997 Hz sinusoidal tone in the same DSP (997 is a prime number) that was used to loop back the signal above in order to isolate the return path.  I have audio glitch testing software than is used to analyze the 48,000 audio samples per second over four hours for abnormalities  This ends up being several gigabytes of data.  Once a reproduce-able error is found, I find that generating a triangle waveform where each sample starts at zero and rises by a value of one to the maximum value and the decreases by a value of one to the minimum.  My analysis software can find glitches (basically looking for vertical edges over some threshold) with either sinusoidal or this kind of ramp-up/ramp-down waveform.  When it comes to looking for firmware issues of dropped samples or buffering issues, I usually use a sawtooth waveform.  At the sample rate of 48 kHz, a ramp from zero to maximum and a sudden drop to zero of a 16 bit sample is about 1.365 seconds.  It just sounds like a ~1 hertz click.  At 24 bits, it is a much slower changing waveform at nearly 350 seconds.  Using this kind of data provides predictability to the expected data, making it easier to recognize when things are not working and is of sufficient duration to ensure that no waveform can repeat itself inside any digital data buffering scheme.  Additionally, specialized code can be written to examine the binary data flowing across the system at any point and predict what should be happening at any given time.

Removing complexities in the Ethernet segment such as removing external network switches and using direct connections between pieces of equipment in order to simplify the setup failed to resolve the issue.

The return audio path ended up being the source of the audio glitch.  By having the ability to record the audio at each end of the complete audio path, the next task was to move one signal segment at a time down the return path and loop it back to the recorder.  This allowed isolating each signal segment and being able to rely on previously tested paths to perform the recording.  Again, using divide and conquer techniques, I did a binary search of the remaining signal segments.  If I had 8 segments to check, I went halfway down and looped back to the recorder.  If a problem was found, divide that signal path in half and retest until the offending segment was found.

In the end some restructuring of DMA priorities on one of the micro-controllers and some DSP work was involved in resolving the issue.