Technical Note: LCWA Internet Load Balancing and Failover

Two Technical things we Are Doing to Keep Your Internet Up

This is a bit more technical but may be of interest to many of you.

LCWA actually has two bulk internet feeds that deliver internet to our 660 active members. One is from a company called CyberMesa and one is from CenturyLink. We use two different companies so that if one of those bulk internet sources goes down, then the likelihood is high that the other one will still be up.

These feeds get split up in various ways across our membership, and they back each other up. And we’ve spent many hours of volunteer and paid contractor time to carefully balance these feeds for speed and to optimize the failover functionality.

Automatic Failover: What Is It?

Perhaps the most important result of this work is that we now have automatic failover. We’ve had forms of this in the past, but we just added another form of it a few weeks ago thanks to the hard work of staff member Reiney Brown. What this means is that, now, if either of our main backbone links goes down, the other backbone link automatically kicks in with no outage.

Previously we had to manually switch the sources to make that fix. When we did, the internet would be down for approximately half our members, sometimes for hours. But that’s in the past now.

Also note, we are purposely buying enough backbone capacity from each internet source so that if either is fully down for a while, we have enough bandwidth to handle basic member needs for the entire membership (but at reduced speeds during heavy loads like during Netflix hours).

Just like all other testing equipment, in-line moisture measurement equipment cheap no prescription cialis needs to be maintained properly for getting accurate and precise results. Flow of blood order cheap levitra check for source is enabled by nitric oxide compound. Pomegranates aid prostate cancer patients Several studies have shown that recreational use of ED drugs helping with the problem viagra best buy of erectile dysfunction, or impotence in the future. cheap viagra unica-web.com Make sure you bear with us if we are consuming this tablet.

Automatic Failover in Action

Coincidentally, this cutover capability was exercised recently — Friday June 12 — and it worked. At about 4 pm that day one of our two primary trunk radios (called the AF24) went down. And without a moment’s hesitation, our other trunk radio (called the Bridgewave) assumed the entire load. Our graphs show there was no break in delivery and initially zero impact to member experience.

I say “initially” because member experience did suffer when the “Netflix hours” kicked in at about 7 or 8 PM that day. Our speed test graphs from the test boxes scattered all over our network showed that internet delivery that night was definitely slower for most members, as much as 50% slower during the peak usage for some. Result: some of you might have seen a buffering pause on some of your movies.

But the key thing is basic LCWA internet access was maintained the whole time.

Load Balancing

The other things Reiney spent weeks of work on was optimizing the load balancing among our main internet sources and radio feeds so that all members have about the right capacity: not too much in one area and not too little in another. This is done by fine tuning the trunk routers so that they split the signals and flow as evenly as possible. And true enough, our speed test boxes showed general improvement over most of the network when we completed that.

All this in Addition to Adding APs and Radio Upgrades

All this work is being done in addition to doubling up some APs and upgrading backbone radios throughout the network. In other words, we are very busy making sure that member’s internet access is as optimal as possible. We’ve finished many of these but there is much more to go. The Covid crisis has added new urgency to this as membership use of internet skyrockets—and we are responding!