NTP Server Extreme accuracy for under $200 Using SI5351C

When you make your own NTP server, you probably spend a lot of time trying to get it as accurate and stable as possible. To get good results, you need to connect your server to a good time reference, like an atomic clock, GPS or a GPSDO to get the PPS (Pulse Per Second) to sync your server with the reference clock.

In both Linux and BSD, there is kernel support to slave the very inaccurate CPU clock crystal to the PPS signal, continually adjusting the frequency, because the crystals of computers continuously drift due to temperature and barometric pressure differences. So most of the work is to get a crappy crystal to behave. What if you just jumped over that step, so the CPU clock gets synchronized with the same lab reference that the PPS signal uses, using a SI5351C Clock Generator?

Read on to see a simple way of doing it and get an extremely accurate NTP server for under $200.

You may be wondering why not just either buy a complete NTP server or at least use a fast Linux computer directly, instead of trying to make it as small, energy efficient and low cost as possible. But there’s a reason. I’m planning to help install a bunch of NTP servers in different countries in Africa (yes Sarah Palin, Africa isn’t a country), so I need to keep the cost down, but still have it perform very accurately. I’ve managed to get a deal with a large internet provider, so we have room to install servers in each of their nodes around the continent. Because right now, the NTP server situation in Africa is quite abysmal.

For fun and (no) profit!

My first experiment has been with the Raspberry Pi 3. It’s a great little computer, and it’s possible to get good results on it. The Ethernet goes over the USB bus which makes it slow (besides the fact that it’s only 100Mbit/s.) So even when you have very accurate time locally on the server, when sent out to clients it degrades because of the USB to Ethernet conversion, and network latency.

So I looked around for another card with a proper 1Gbit Ethernet with a dedicated controller. I settled on the Odroid C2. It’s a quad-core ARM 64-bit computer, running at 1.56GHz. It has 1Gbit Ethernet, 2 GB of DDR3 RAM and runs some different flavors of Linux, so it’s perfect for my project. The below $50 price tag was a pleasant surprise as well. It also has provisions for using eMMC5.0 or SD-card, where the speed of the eMMC is much faster than the SD-card.

The NTP Server Setup

I use a Trimble GPSDO that I bought from eBay as my local time reference, giving me both a PPS signal and two 10MHz signals (which we’ll use later) coming out from it. The signals are synchronized with the atomic clocks onboard the GPS satellites, giving me an accuracy between 10 to 50 nanoseconds. The PPS pulse is fed into one of the GPIO pins on the computer as a time reference for syncing the kernel and the NTPD server application.

Unfortunately, the pulse width is too short for Linux to pick it up, so I’ve designed a pulse extender circuit which I’ve made Open Source Hardware, so the system can pick up the interrupt and make the adjustments. Usually, you use a feature of the Linux kernel, called PPS Consumer which adjusts the clock frequency of the entire system to get the very unstable local CPU crystal more in line with the correct time. I’ve been working for months trying to improve the latency and the stability of the kernel to make the oscillator as precise as possible.

But then it dawned on me. Why? Couldn’t I just replace the unstable clock with something better? My first idea was to install a more stable crystal, but you still have the problem of it drifting from the PPS signal. But I finally found a solution.

Slaved CPU Crystal

There is a clock generator IC called SI5351 from Silicon Labs. It’s an IC, where you have a local 25 or 27MHz crystal and then you can program it to output different clock signals at different frequencies between 100kHz to 200MHz, all derived from the local oscillator. Both Adafruit sells a breakout board, and there are others as well. But they use the SI5351A or SI5351B versions of the IC, where the differences are the number of outputs and that the B version can have a Voltage Controlled Oscillator. So the A version has three outputs, and the B versions have eight. But the most interesting one for me is the SI5351C, which can use an external 10-100MHz reference instead of the usual 25 or 27MHz crystal as its reference. And it also has eight outputs.

Oh, lightbulb over my head!

Apparently, the solution to my problem was all there. I have an extremely accurate atomic clock that is slaved by a GPSDO that outputs a 10MHz signal. So for testing, I built a breakout board PCB and did a test. You can read all about the SI5351C breakout board here.

The initial test was just to see how stable it was. The data sheet for the SI5351C said that the jitter from the clock generator would be around 70 nanoseconds. So I hooked up my contraption connected to an old Arduino (for programming the frequencies via I2C) to my old modified Philips PM6665 frequency counter and let it run for 24 hours. And it was spot on. I tested a lot of different frequencies, but the CPU on the Odroid C2 is 24MHz, so I let that frequency run for a long time. And it works, without a glitch. The only thing I needed to do was to add two resistors to form a voltage divider to get down to 1.5 volts to match the voltage of the old crystal.

Odroid C2 Modifications

Time to replace the CPU crystal on the Odroid C2 with my newfangled module. Looking at the schematic, both of the capacitors C159 & C160 needs to be removed. We don’t need to create the phase shift anymore. The same goes for R107. And naturally the crystal itself.

I use Capton© tape around all the components around the crystal to protect them from the heat from my SMD heat gun. A magnifying glass or microscope helps because the parts are tiny.

Here’s how the Odroid C2 PCB looks like without the heatsink. To remove the heatsink, just press down on the two tabs and squeeze the pins together on the other side.

Here’s a closeup of the area of the clock circuit.

After removing the components, it should look something like this. The crystal and C159, C160 and R107 are also removed.

Next up, let’s connect the clock generator board to the Odroid C2 board. Try to use thin wires which makes it easier to fit the heatsink back again without a problem.

And here’s the experimental setup as it runs until I find a suitable enclosure.

On my first prototype of the SI5351C Clock Generator card, I didn’t get the interrupt pin out from the IC. The board you can download here, has the line connected. So if the external reference should fail and the interrupt pin goes high, hopefully, I can switch over to the onboard oscillator quick enough, so the Odroid C2 board doesn’t hang. Still waiting for the new version of the PCB design from China to test that.

So, what’s the result then?

Well, I’ve been fiddling around and testing with different options. When leaving the Kernel Consumer turned on, I get very low values on the loopback statistics from NTP, but the NTPD daemon continually updates the board frequency, so it’s not as useful as just polling the PPS signal every 16 seconds in NTP. Here’s how the loopback statistics look.

And this is when I have four machines on 1Gbit Ethernet, having a sh*tload of stress tests topping out at 22000 requests per second on the Odroid C2! The processor was utilizing 22% of CPU load. So if you have clients using the regular connection time of once every 64 or 1024 seconds, it can easily handle a couple of hundreds of thousand clients or more.

Software

I’m using the excellent Etherkit SI5351 library. To get it up and running, this is all code you need.

#include "si5351.h" 
#include "Wire.h"

Si5351 si5351;

void setup()
{
  // Start serial
  Serial.begin(115200);

  // Initialize the Si5351 to use a 10 MHz clock input on CLKIN
  si5351.init(SI5351_CRYSTAL_LOAD_0PF, 10000000UL, 0);
  si5351.drive_strength(SI5351_CLK0, SI5351_DRIVE_8MA);
  // Set PLLA and PLLB to use the signal on CLKIN instead of the XTAL
  si5351.set_pll_input(SI5351_PLLA, SI5351_PLL_INPUT_CLKIN);
  si5351.set_pll_input(SI5351_PLLB, SI5351_PLL_INPUT_CLKIN);

  // Set CLK0 to output 24 MHz for CPU clock on Odroid C2
  si5351.set_freq(2400000000ULL, SI5351_CLK0);
  // Set CLK2 to output 25 MHz for Ethernet
  si5351.set_freq(2500000000ULL, SI5351_CLK2);
  // Set CLK4 to output 12 MHz for USB controller
  si5351.set_freq(1200000000ULL, SI5351_CLK4);
}

void loop()
{
  // Read the Status Register and print it every 10 seconds
  si5351.update_status();
  Serial.print("  SYS_INIT: ");
  Serial.print(si5351.dev_status.SYS_INIT);
  Serial.print("  LOL_A: ");
  Serial.print(si5351.dev_status.LOL_A);
  Serial.print("  LOL_B: ");
  Serial.print(si5351.dev_status.LOL_B);
  Serial.print("  LOS: ");
  Serial.print(si5351.dev_status.LOS);
  Serial.print("  REVID: ");
  Serial.println(si5351.dev_status.REVID);

delay(10000);
}

As you can see, I’m planning to replace the crystals for both Ethernet and USB as well. Why? Why not? It’s there!

There is a pin header for turning on or off the Odroid C2, so I’m planning to have the Odroid turned off until the SI5351C is fully configured, then turn on the computer. I’ll put up the schematic (one transistor!) when I finalize the enclosure and software.

The SI5351C Breakout Board

It uses a low noise 3.3-volt regulator; I have also added level shifting so that you can use both 5 Volt Arduino or if you prefer, any microcontroller that can do I2C at 3.3 Volt.

Naturally, this board can be used for a lot of other exciting stuff as well. How about a VFO for you HAM-guys out there or maybe time to replace those drifting crystals in your Ham-radio? With a cheap U-Blox GPS receiver and some modifications, you can get the reference 10MHz frequency you need. And you will be accurate down a couple of decimal places. Go nuts! You can read more about the board here.

So I decided to release the board as Open Source Hardware so that you can download it here. If you’re not up to doing SMD soldering (the SI5351C is 3x3mm, so you need an SMD oven), you can send me an email because I’m going to make some for sale. You can contact me at jacken@jacken.se