How We Use AI Agents to Review and Improve Our Own Trading Agent
CapTradeAI trades around the clock, which means it makes more decisions in a day than any human team can manually review. Since April, we've been running a second layer of AI — one that reads every trade the agent makes, questions its own reasoning, and proposes fixes as ordinary code changes for a human to approve. Here's how it works.
A trading agent that runs 24/7 doesn't fail loudly. It fails quietly — a safety check that stops working under one specific condition, a threshold that made sense for last month's market and not this one, a piece of cleanup logic that accidentally erases state something else depends on. None of these show up as a crash. They show up, weeks later, as a slow bleed in the numbers. We built a second AI system whose only job is to catch that class of problem before it compounds.
The Problem: More Decisions Than Any Human Can Review
Every five minutes, the agent evaluates every tradeable asset for every user, weighs multiple independent strategies against each other, and either acts or holds. Across a month, that's tens of thousands of individual decisions, each with its own reasoning trail — which signals fired, which dissented, why the winning action won.
No human reviews that volume of decisions by hand, so the traditional answer is to watch the aggregates: win rate, profit factor, drawdown. The problem is that aggregates hide exactly the failures that matter most. A bug that quietly disables one specific safety check might cost real money for a month before it moves any headline metric enough to notice — buried inside a much larger number of trades that went fine for unrelated reasons.
What we needed wasn't a dashboard. It was something that reads individual decisions the way a human reviewer would — but at a volume no human reviewer has time for, every single day.
Two Agents, Two Distinct Jobs
We split the work into two roles that run on different schedules and have deliberately different permissions. Neither one trades, and neither one touches production on its own.
The Daily Reviewer
Runs once a day, read-only. Reads yesterday's individual trades — full reasoning attached, not just the outcome — cross-references a running log of previously known issues, and writes a findings report. It cannot edit a single file. Its entire output is a written report.
The Weekly Engineer
Runs once a week. First measures how its own last proposed change actually performed. Then, if the data supports it, proposes exactly one minimal, reversible improvement — as a normal code branch with a written justification. It never merges its own work.
The asymmetry is intentional. Reading and flagging is low-risk, so the reviewer can run constantly and say whatever it finds. Changing behavior is high-risk, so the engineer is throttled to one change at a time, and every change has to justify itself with a measurement before the next one is even considered.
What the Daily Reviewer Actually Looks For
It's not a generic "review these trades" prompt. It has a specific checklist built from everything we've learned matters:
The same failure shape showing up more than once — not a single bad trade, but a mechanism that keeps producing bad trades under the same conditions.
Every deployed change comes with something specific to watch for. The reviewer checks it against yesterday's evidence rather than assuming a fix stayed fixed.
A guardrail that never blocks anything is either perfectly calibrated or silently broken. A guardrail that blocks a high-conviction signal every single cycle is probably too strict. Both are worth a second look.
A strategy voting against a position it opened hours earlier with no new information. An exit priced below break-even for reasons that don't hold up under its own stated logic.
Every finding has to cite the specific evidence and the specific mechanism — not "this seems off," but this trade, this timestamp, this line of reasoning, here's what caused it. Vague findings don't get reported; "no issues today" is a valid and common outcome.
A Real Example of What This Catches
One of the more instructive finds involved a routine that runs when a position is fully closed, whose job is to clean up temporary tracking data so it doesn't accumulate forever. That cleanup was also deleting a timestamp that a completely unrelated safety check relied on — one designed to prevent the agent from immediately re-entering a position it had just exited, a basic guard against rapid back-and-forth trading.
The two systems shared a piece of state for unrelated reasons, and nothing enforced that they had to agree about what that state meant. The result: every time a position closed, the re-entry cooldown quietly reset to zero instead of counting down normally — silently disabled at precisely the moment it mattered most. It produced no errors and no crashes. It just let the agent do something it was explicitly designed not to do.
This is a genuinely common class of bug in systems that accumulate features over time: two mechanisms independently reuse the same piece of state for different purposes, and a change to one silently breaks the other. It's easy to miss in code review because each piece looks correct in isolation — the bug only exists in the interaction. Catching it took comparing a trade's actual timing against what the system's own rules said should have been possible, which is exactly the kind of check that's tedious for a human to do by hand across thousands of trades and mechanical for a daily reviewer to do every single time.
What the Weekly Engineer Does With a Finding
Findings from the daily reviewer accumulate into a shared, dated log — every fix that's shipped, every idea that was tried and disproven, every open question still being watched. The weekly engineer reads that entire history before doing anything, specifically so it doesn't re-propose something that was already tried and didn't work.
Its process is deliberately narrow:
If a previous change is still inside its measurement window, the engineer's entire run is spent evaluating that change against fresh data — no new proposal that week.
Every proposal has to cite specific numbers from the trading data — not "this strategy seems weak" but the actual attribution behind that claim.
Not a redesign. A single, reversible adjustment with an explicit description of exactly how to undo it if it doesn't work out.
The change is pushed as an ordinary branch with a written explanation. A human reads it, decides, and merges — or doesn't. The agent has no path to deploy itself.
The Guardrails
Giving an AI system read access to live trading data and write access to a code branch is not something to do casually, so the permissions are narrow on purpose:
- • No market access, ever. Neither agent can place an order, touch an exchange API, or move funds — that capability simply isn't available to them.
- • A hard file allowlist. The engineer can propose changes to strategy logic and configuration. It cannot touch authentication, payments, credentials, or the database layer, regardless of what it might otherwise conclude is a good idea.
- • Branch-only, human-gated deploys. Every proposal sits as a diff until a person reviews and merges it. Nothing reaches production without that step.
- • One open proposal at a time. The system enforces a cooldown so changes get measured individually instead of stacking on top of each other, which would make it impossible to tell what actually caused what.
Why We Built It This Way
None of this replaces good engineering practice — it's an application of it. Code review, incremental changes, measuring before and after, never deploying without a human sign-off: these are standard discipline for any serious software system. What's different here is the volume problem. A 24/7 automated trader generates more reviewable decisions per day than a team can realistically read, and the value of catching a subtle regression is highest in exactly the moment nobody's looking.
The result isn't a system that gets smarter on its own in some dramatic sense. It's steadier than that: a routine that reads what actually happened, writes down what it finds, and only ever proposes small changes that stand up to their own honest measurement.
The standard for any change is simple: better than what we already have, on real data, with a way to undo it if we're wrong. That's not a high bar for a headline. It's a very effective one for a system that runs unattended, every day, with real money behind it.
See the Agent in Action
Every mechanism described here exists to keep the live trading agent honest — around the clock, across multiple assets.