AI-Powered Resilience Testing Tools Now Recommend Fixes. Gremlin’s Foresight AI Retests Each One.

Code can pass review and still leave an application unprepared for a payment provider going offline. Nothing looks wrong until a dependency fails. And as AI generates more code, engineers have more changes to verify against real-world failure conditions.

Gremlin built its business around exposing these weaknesses through controlled failures before they cause actual outages. In October, it launched Foresight AI, which hands part of that work to supervised AI agents. The agents find weaknesses, recommend fixes, and rerun the tests to check whether those fixes work.

So, for engineering leaders evaluating AI-powered resilience testing tools, the launch raises a practical question. How much of the work between spotting a risk and resolving it can these systems actually handle?

Faster Coding Leaves More to Verify

Gremlin’s testing experience offers an interesting perspective on how AI-assisted development is changing software reliability.

As Kolton Andrus, CEO and founder of Gremlin, told SaaSTake: 

Research suggests faster code generation doesn’t necessarily reduce the work involved. It often shifts that effort from writing code to reviewing and verifying it. Google Cloud’s DORA team found in March 2026 that time saved writing code was often reassigned to auditing and verification. The same research linked wider AI use to faster software delivery, but also to less stable releases.

After all, that additional checking takes engineering time, so faster coding alone doesn’t guarantee productivity gains.

A June 2026 survey commissioned by New Relic puts numbers on the risk. Among 200 U.S. technology decision-makers at companies using AI in software engineering, 94% rated AI-generated code higher in quality during review. Yet 82% reported a production failure tied to AI-generated code in the past six months.

This is where reliability testing tools become important. They need to go beyond code review and examine how applications respond to failure conditions.

Dependency Failure Testing Shows What Code Review Misses

Applications rely on other services, such as databases and payment providers, to work. When one of these dependencies goes down, parts of the application that rely on it may stop working as expected. 

Dependency failures were the most common issue Gremlin encountered before AI-assisted coding became widespread. Andrus says they remain a frequent problem as teams ship more AI-generated code. He explained:

According to Andrus, AI-written code often leaves out the basic safeguards that tell an application what to do when a dependency fails. Timeouts stop an application from waiting forever, and retries try a failed request again. Circuit breakers temporarily stop requests to an unhealthy service when repeated failures reach a set threshold.

However, adding these safeguards doesn’t automatically mean they’ll work under every failure condition. Microsoft’s architecture guidance warns that aggressive retries can overwhelm a struggling service and delay recovery. It recommends testing how retries behave during failures and under heavy load.

Reliability testing has the same requirement elsewhere in infrastructure. QA Cafe’s CDRouter repeatedly tests routers under operating conditions so engineers can catch failures that shorter checks may miss. 

In other words, dependency failure testing should show what the customer actually experiences when a service disappears. Does checkout return a useful error, or does the customer sit waiting while the failure spreads?

For buyers comparing resilience testing tools, a live test shows whether an application can actually handle a dependency failure, rather than simply listing the safeguards a platform supports.

Foresight Recommends Fixes Based on Past Test Results

Chaos engineering tools introduce controlled failures so engineers can observe how systems respond. However, engineers still need to design these experiments, interpret the results, and decide how to address the problems they uncover. This can make application resilience testing difficult to maintain across large service environments.

Gremlin has already cut some of that effort with its native Dynatrace app, which brings testing and reliability scores into the dashboards engineers already use. Now, Foresight goes further and takes on part of that work itself, drawing on what Gremlin calls the Failure Atlas.

Andrus describes it as anonymized results from millions of tests and fixes, collected over more than a decade. Each record shows what a test did and how the system responded. It also captures what changed between a failed test and a passing one, giving the agents historical examples to draw on when recommending fixes.

Gremlin first used that data to decide which tests customers should run, and later to guide them on fixing what failed. Foresight’s agents now draw on the same record when they analyze systems or propose fixes. According to Andrus, that grounds their recommendations in failures and recoveries Gremlin has actually observed.

For buyers, this raises an important question during a demo: Can the vendor explain how each recommendation connects to the customer’s setup and the failure observed? It should also show what evidence would confirm that the fix worked.

For AI-powered remediation, a convincing explanation is only the starting point. Engineers still need a change they can review and a way to test whether it works.

What Happens When Teams Find Gaps in Test Coverage?

One early Foresight customer, a large global IT services company, already had more than 150 services connected to Gremlin.

According to Andrus, an engineer asked Foresight which services in a namespace weren’t covered by existing tests. It returned a list. Then, with one more prompt, it set those services up with health checks and ran reliability tests on them.

“Within a few minutes, the team understood their footprint and had a reliability baseline,” he said.

From there, the tests showed how each service responded to CPU or memory spikes and slower network conditions. For buyers evaluating automated resilience testing, the example raises a practical question: how much engineering effort does it take to bring an uncovered service into a regular testing cycle?

Human Approval Remains Part of the Product

Beta customers were interested in giving the agents more autonomy. However, Andrus says every one of them wanted human approval to remain in place for now. As he puts it: 

In practice, Foresight proposes tests and fixes, but a person must approve them before the agents can run the tests or apply the changes. Evinced uses a similar control in software remediation. Its agents can identify and fix accessibility problems, but developers review the proposed changes before they move forward. 

When comparing tools, buyers should check which actions require approval and whether engineers can review proposed changes before they are applied.

A Fix isn’t Proven Until the Test Passes Again

A recommended fix stays unproven until something tests it. Once an engineer approves a change, Foresight reruns the test that first exposed the weakness. If it passes, engineers have evidence that the change addressed the original weakness under the conditions tested.

The checks don’t stop after the initial fix, either. Foresight repeats tests as systems change, helping engineers catch weaknesses that may return after later changes.

That’s the difference buyers should look for between a suggested fix and one that has been validated through testing. A vendor should be able to show how its platform confirms a change fixed the original failure. Buyers can also ask which conditions the test covers, and whether later releases trigger another check.

The Strongest Demo Follows One Service from Gap to Proof

For buyers evaluating AI-powered resilience testing tools, a useful demo follows one service through the work the platform promises to handle. Engineers should be able to make sense of each finding and look over each change before it goes ahead. Ultimately, the result should be something they can check for themselves rather than take on faith.

Foresight brings these steps together through its historical failure data, supervised agents, and repeated testing. Its bet is that real failure data and a retest after each fix are what earn AI agents a place in production. 

After all, as more code comes from AI, the difference between these tools may depend less on how well they identify weaknesses and more on how effectively they verify the fixes.