The Right Way to Use AI in Trading (This Week)

Use it or lose it

Yes, this is my first article without a podcast attached. The first of many, I hope. In time, I’d also like to produce more YouTube content that isn’t strictly interview-based.

I’ll admit it: I’ve been biting off more than I can chew. Courses, podcasts, trading, farming (yep!). As you know, I’m working on courses with Market Wizards, but I’m also building my own courses, which I plan to provide to members. Stay tuned for a roadmap article where I’ll lay out my vision and plans for the future of this Substack.

At the same time, I’m very busy with my own trading, and I’m taking notes as I go so I can share strategies and research very soon. Of course, I’m also neck-deep in using AI to help, and I’m building an AI course as well, especially for RT users. I’m spread so thin that I need all the help I can get.

Anyway, this article contains my reflections so far on using AI in trading, and I’ve been working with some experts in the space, so as usual, I’m leveraging that! Let me know what you think, and whether your views align with mine or differ. I’ll start with the basics (these are critical) and end with a more substantial and practical workflow and toolkit. With the likes of Hudson River Trading and Balyansy building out their own internal AI labs, hiring directly out of Google DeeepMind and OpenAI (according to Quant Enthusiasts), clearly we need to stay up-skilled in this space.

---

The alpha is not in the prompt. It’s in the process you build around it

“When we meet a fact which contradicts a prevailing theory, we must accept the fact and abandon the theory.”

— Claude Bernard, An Introduction to the Study of Experimental Medicine

That is a useful warning for anyone using AI in quant trading. Because AI is very good at giving you theories. It can generate strategy ideas, explain market behaviour, write code, interpret results, and wrap the whole thing in language that sounds unnervingly confident.

But confidence is not evidence.

In quant trading, the danger is not that AI will be useless. The danger is that it will be useful enough to make laziness feel like research. We are extremely good at backfilling explanations for results we want to believe. Show us a rising equity curve and an explanation from AI and we’ll have the story that justifies it before lunch.

If I see another “Claude Code built this strategy for me” YouTube video, I’ll throw a keyboard at the Substack wall.

In this article I want to lay out a more trustworthy way of using AI, and it’s this: you now have an intern with a PhD. If you can set it the right tasks, rightly expounded, it’s good to have around. But you’re the boss.

There is a singularly critical component to getting it right which lays at the heart of this article.

Things seem to be changing on a weekly basis in this space. The tips and tricks that were relevant 6 months ago are passé today. Hopefully the ideas expressed here are relevant for a bit longer than that, but no apologies if not. Still, it does seem like the skills have been iterative and cumulative. It was all about prompts and prompt engineering to begin with, but those capabilities have transferred into building better skills and workflow files (sitting in your project folder) that are essentially more detailed prompts in the background, guiding the agent accordingly. One thing that took some getting used to for me, was the idea of outsourcing research, let alone to a bot. I think scepticism is a healthy approach though, because you don’t want to lose touch with the data. You want to know it intimately.

Rightly set up, rightly sandboxed, prompted and instructed by an expert Portfolio Manager, the AI is a superpower. An intern you can have busily working away on one screen, while you work on the other. Ask random questions, let it deep dive analysis, summarise research, produce charts and spreadsheets, and build apps, in a fraction of the time it takes you. Perhaps ask it to do something creative, you might be genuinely surprised at its knowledge and resourcefulness. This is not “build me a strategy”, this is a research assistant that requires an expert to report back to, one that will validate everything.

But you need the stack and process laid out the right way. With that foundation, even a new trader can turbo-charge their development.

---

The Mistake Hiding Inside “Build Me a Strategy”

Look, we’re all constantly playing catch-up with AI models that advance on a weekly basis; we’re desperate to take as much advantage of them as possible, but the obvious and not-so-obvious traps are obviously lurking in any attempt to outsource our research efforts. Most significantly to the naïve new trader or coder. Lucky, a guru in the Collective, Keith, has an unhealthy addiction to both trading and AI and has been working with me to pull together a course on the whole topic, which we’ll provide to members. I’ll be focussed on using RealTest with AI, but the principles are universal.

Side note, once you have this kind of environment setup, you also have a coding mentor that can teach you how to use RealTest, and a structure to force you to be more disciplined about your robustness testing process.

Before we get to the good stuff, let’s cover the basics. If you go and do something silly like ask a model to “build you a profitable strategy”, you’ve handed it an objective it is extraordinarily good at hitting: produce something that looks right. And it will. The code runs. The equity curve climbs. Please do this, Keith and I would love to trade against you. ; ) But seriously, this doesn’t work because there are just too few boundaries and too many ways a toddler with a PhD in statistics can run down rabbit holes and come back with a sack of rabbits and a good story.

These failure modes aren’t speculative. A 2026 survey of generative AI in quantitative finance catalogues several that should give any systematic trader pause: temporal data leakage (where the model uses information it wouldn’t have had at the time), the hallucination of plausible but factually wrong figures, and a stochastic output process where identical inputs produce different outputs across runs. That last problem deserves its own line.

An assistant that answers differently each time you ask the same question.

Now picture trusting that assistant to tell you whether your strategy works.

There’s a subtler version that bites even when you stay engaged (if you aren’t setting your goals right). You start with a clean trend-following hypothesis. You ask the model to improve the results, so it adds a filter, then a smoothing layer, then a machine-learned overlay. Variance falls. The win rate climbs. The payoff compresses. And somewhere in that drift you’ve quietly built a mean-reversion system while still calling it trend following. You’re no longer testing the thing you set out to test. The verdict now answers a question you never asked.

---

What AI Coding Already Learned the Hard Way

Software engineers hit this wall before we did, and the way they climbed over it is worth borrowing wholesale.

In the course we’ll lay out the discipline that engineering teams have converged on for working with AI coding agents. The agents are clever but amnesiac: every session starts blank, so you give them a memory file, a committed document of project conventions and known traps that every new session reliably reads. One canonical file holds the truth. And the part that matters most: a human decides what goes in it.

The obvious temptation is to let the AI write that file for itself. Try to resist.

The evidence supports this. A 2026 ETH Zurich study tested coding agents three ways: with no context file, with an AI-generated one, and with a human-written one. The industry assumed more context would help. It didn’t. Context files tended to reduce task success rates compared with giving the agent no repository context at all, while inflating cost by more than 20%. And the AI-generated files were the worst performers of the lot; the human-written ones offered only a marginal gain.

The lesson crosses over cleanly. More AI, applied naively, made the work worse. What helped was a small, human-curated layer of hard-won truth, sitting on top of a deterministic system the model couldn’t talk its way around. For the coders, that system is the test suite and the build. It passes or it fails, and no amount of confident prose changes the result.

In trading, you already own that deterministic system. It’s your backtester.

---

The Closed Loop

Here is the model that actually works. It’s a “closed loop”.

You’re hiring an intern, but one without any ‘real-world experience’, so you keep an eye on them.

You form a hypothesis. The AI helps you sharpen it into a precise, testable specification and writes the code to express it. Ideally it is trained to push back, make you think, force you to express goals clearly. Then, your back-testing engine runs that code against reliable data and returns a result that has no interest in your feelings. You read the output, you judge it, and you feed the next question back in. There’s a secondary advantage here: you get the chance to try and formalise some of your methods. Tightening up your process, with clear explanations for each step, and their order, will help your trading enormously.

This is the entire article, so don’t miss it:

1. The key to using AI is to pair it with your own back-testing engine and reliable data in a “closed loop” testing environment.

2. Bundled in there are the training, skills and process that you provide as markdown files.

3. Orchestrating the entire process is an experienced trader with experience in AI, asking the right questions and probing in the right direction.

Closing the loop means keeping the work of the AI well contained in your ecosystem: your process, your back-testing engine, your data. Using your back-testing engine adds the much-needed deterministic tool into the mix. The AI can touch everything in the loop, except the verdict, and it’s on you to go back and validate everything.

That boundary is the key. The intern drafts, codes, explains, proposes, and tirelessly iterates. The engine decides what’s real. Keep those two jobs in separate hands and the loop has integrity. Collapse them into one and you’ve built an expensive way to lie to yourself.

---

Why the Engine and the Data Are Non-Negotiable

A backtester is boring in precisely the way you need it to be. Same script, same data, same answer, every single time. It’s reproducible. It can be audited. It has no incentive to please you.

An LLM has none of those properties. It’s probabilistic by construction, and it has been trained to be agreeable. Fine trait in an assistant. Catastrophic in a referee.

This matters because the graveyard is already overflowing. When researchers examined published trading strategies against live results, the institutional verdict was brutal: over 90% of academic strategies fail when implemented with real capital. Those were built by careful people, peer-reviewed, often elegant. The problem was never a shortage of ideas that backtest well. It was the chasm between back-testing well and actually working.

AI widens that gap, because it makes good-looking strategies free to produce. Marcos López de Prado has spent years making the point that if you try enough configurations, a spectacular backtest is essentially guaranteed by chance alone. Run a thousand variations and the best one will dazzle you whether or not it has any edge. An intern who can generate those thousand variations before lunch doesn’t solve that problem. It detonates it. Left unconstrained, the model simply becomes a very fast overfitting engine.

So the engine plus clean data is not a convenience bolted onto the workflow. It’s the only thing standing between the intern’s confident output and your account.

---

The Intern, Used Well

None of this makes the intern useless. A good intern changes everything about your workflow speed. You just have to give them intern-shaped work and check what comes back. Critically, you need to train it, very specifically. These skills (which I’ll provide to members) begin to codify your process, and give the AI a deep knowledge of how to work with your stack.

Where AI genuinely earns its keep:

• Triage and synthesis. Read the paper, summarise the method, surface the studies that contradict it. This is research-assistant labour, and the model is superb at it.

• Specification. Turn “momentum probably works better after high-volatility weeks” into a precise, unambiguous, testable definition with no hand-waving left in it.

• Code. Write and debug the strategy in your engine’s language, fix the parse error, explain why the trade list contains phantom fills. The tight loop of write, parse, test, fix one thing, re-run is exactly the kind of verifiable, fast-feedback task AI handles well.

• Adversarial variation. Ask it to break your hypothesis rather than bless it. Generate the variations that should destroy the edge if the edge is fragile. Most edges are fragile. Better you find out than the market.

• Interpretation. You hit a strange drawdown in a particular year. Ask it to propose explanations, then go and check them against the data yourself.

The thread running through all five: every output is verified by something other than the AI. You’re not asking for answers. You’re asking sharper questions, faster.

And this is the part that rests entirely on you. There’s a lot of art in the science. Having a process, a hypothesis, and some statistical structure already puts you ahead of most people, but the harder skill sits above that: deciding what to test, in what order, and for what purpose. Test the wrong thing, or ask the wrong question, and it no longer matters how sophisticated the engine is or how clean the data is. The output is worthless. Just delivered faster, and with more confidence. The intern’s quality is bounded by the quality of your prompts, your questions, and above all your judgement about what’s worth asking. Hand them a vague brief and you’ll get confident vagueness in return. Hand them a precise question inside a disciplined loop, and you get real results.

---

“But the AI Can Just Run the Backtest Itself”

The natural objection: agentic tools can now run the backtest too. So why keep a human in the loop at all?

Because of what that consolidation quietly does. Let one system generate the hypothesis, write the code, run the test, and report the result, and you’ve handed the entire chain to a single non-deterministic, leakage-prone, eager-to-please process. Every independent check has been removed. You may not even notice it happening.

Separation of powers here isn’t bureaucratic caution. It’s the reason a backtest means anything. The engine is trustworthy precisely because it didn’t write the strategy. The moment the author becomes the judge, the verdict is worthless. The quant is not the PM, they are two different roles.

Use the agent to run the test, by all means. Just never let it be the only thing that reads the result. The humanity of trading will remain in my opinion. The human agent needs to stay intimately connected to the data in a way that the machine never will – the human’s livelihood depends on it. I think that reality will remain an on-going divergence between human and AI capabilities. There’s still no “holy grail” right.

---

Key Takeaways

• The alpha is in the process: the loop, the engine, and your judgement. The prompt is one input among many, and not the important one.

  • The training ‘skills’ for the AI are where the power is. These codify your process & tell the intern exactly how to use your back-testing engine, what results to produce and how to analyse them.

  • The intern should also have ‘skills’ for pushing back, working according to your particular process, trained on your robustness testing techniques.

  • A deterministic backtester on reliable data is the only honest referee in the room. Reproducible, immune to flattery. That is exactly the point and why it must form part of the closed loop.

• Point a flawless engine at the wrong question and you get a worthless answer faster. Deciding what to test, in what order, and why is the skill that sits above all the tooling.

• Never ask an AI to “build you a strategy”. Ask it to help you express and test ideas as you research. Author and judge must always be different entities.

• AI drives the cost of plausible-looking ideas to nearly zero, which makes overfitting more dangerous, not less. Your entire edge has migrated to validation.

• Treat the model as a very smart intern: brilliant, fast, tireless, and fully capable of fabricating nonsense with a straight face. Give it intern-shaped work and verify all of it.

The traders who win the next decade won’t be the ones with the cleverest prompts. They’ll be the ones who never once let the intern sign off on its own work.

If you’re in the Collective, keep an eye out for a full course on getting started with RealTest, using AI, building out the tools and processes mentioned above. The bonus is that you build your own mentor, who can answer all your questions going forward. So, you may not need me anymore.

Have fun with it!

Simon