Restarting a Router Can Restore the Connection Without Explaining the Failure. QA Cafe’s CDRouter Looks for the Cause Before Deployment.

Anyone with a home router knows the routine. The connection slows to a crawl, so someone switches the box off and on again. Everything seems normal until the slowdown returns a few days later. The restart brought the connection back without explaining why it deteriorated.

For the broadband operator, that leaves a difficult question. The router passed its checks before deployment. So, what changed while it was running in the customer’s home?

Somewhere in those days, software may have been holding on to memory it should have released. The router kept working, just a little worse each day, until the slowdown became impossible to ignore.

QA Cafe built its CDRouter Stability and Memory Expansion to help engineers investigate those problems before deployment. We spoke with CTO Tim Winters about why finding a failure does not always explain what caused it.

A Fast Connection Still Needs to be Reliable

Broadband packages give customers an easily understood number: the speed they are buying. Reliability is harder to measure in a single number, but it affects how useful that connection is.

In its Comparing Customer Service report, Ofcom found that 45% of dissatisfied broadband customers cited an unreliable connection, compared with 30% who said their speed did not match what was advertised. 

Routers are only one link in that chain, but the gap shows that good speed alone does not guarantee a good experience.

Device manufacturers’ own records show the kinds of problems engineers can run into. ASUS’s February 2024 firmware notes for the RT-AX68U list fixes for a memory leak and an unexpected reboot tied to one VPN setup. They document how software behavior can affect a router’s operation.

The challenge is catching these problems while the router is still in the lab.

Knowing a Device Broke Does Not Explain the Breakdown

Engineers have been running their own long-term tests for years, keeping equipment busy under repeated activity so problems have time to emerge. Winters said those in-house tests only catch the failures someone thought to look for.

“You can script a device to run for 72 hours, but you usually end up knowing it broke without knowing when things started going wrong,” he said.

That means engineers have to create their own tools to generate traffic and track device resources. They also have to maintain those tools as devices change. Longer tests give faults more time to surface. However, recording what happens along the way gives engineers a better idea of why the failure occurred.

The Warning Signs May Appear Before the Failure

QA Cafe announced its CDRouter Stability and Memory Expansion on August 31. CDRouter Stability and Memory Expansion combines repeated testing with monitoring to catch signs of trouble over time.

Winters said CDRouter runs the tests and tracks the device’s memory and processor use at the same time. Those readings appear on the same timeline as the test results and logs. That gives engineers a clearer place to start looking.

Winters illustrated the difference with a hypothetical failure at hour 40. He says:

Because CDRouter runs the tests and manages the device, engineers can see what the device was doing when its resource use started to change.

The user guide describes tests that repeatedly make devices reconnect and request network addresses. It also checks whether data speeds stay close to where they were at the start of the test. Runs can last anywhere from one minute to one week, and memory tracking needs the device to report that data.

Thorough Testing Has to Fit the Qualification Schedule

Long-term stability testing is already part of established industry practice. The Broadband Forum’s TR-398 test plan includes a mandatory four-hour test that monitors connection availability under sustained activity. Four hours provides a useful baseline, but some faults can take days to surface.

The same principle appears elsewhere in infrastructure resilience: a recovery path has to be tested before an interruption, rather than assumed to work when it is finally needed.

The harder question is how operators fit extended testing into an already crowded schedule. Testing more than 150 devices gave Uniti a firsthand look at how much work device qualification can involve. The operator described manually testing more than 150 devices before adopting CDRouter. Its engineering team reported cutting a qualification process that could take up to two months by two weeks or more.

That result covers CDRouter as a whole rather than the new expansion. Still, it shows how much time device approval eats when every model needs enough testing before it can be deployed.

Winters said removing that setup work makes it easier to run stability checks more often. Instead of saving them for a major release, engineers can repeat them overnight or over a weekend.

Winters said repeating those checks across software builds could help engineers identify which change introduced a problem before final release testing.

That emphasis on testing before software is trusted with real-world consequences is showing up in other systems too. dxFeed, for instance, validates generated trading scripts before they run and is adding historical testing as those scripts move closer to automated trading. 

A ready-made test can also give operators and device makers a common way to discuss problems. According to Winters:

So, when a problem appears, the operator can hand the supplier the exact conditions that triggered it.

The Deployment Decision Comes Before the Customer’s Restart

Winters described QA Cafe’s role as turning problems reported by users into tests engineers can repeat. That keeps the tests in step with what devices actually face in customers’ homes.

For a customer, restarting a router may be enough to get back online. For the operator deploying it, a temporary recovery leaves the underlying reliability question unresolved. QA Cafe’s expansion brings that investigation into the lab, where engineers can examine the warning signs while there is still time to fix the software before deployment.