051 - Samir Varma - Prediction Fails While Classifying Risk States Succeeds!
The Real Edge Is Not Insight. It Is Experience.
Most traders think edge comes from insight.
A clever model. More data. A different indicator or filter. A parameter no one else has found.
Usually, it comes from repetition.
That is what stayed with me most from Samir Varma’s work. Yes, his framework is interesting. He is deliberately anti-alpha. He is not trying to forecast returns more precisely than the next person. He is focused on risk. More specifically, on classifying market states: when risk is low, be exposed; when risk is high, reduce or remove exposure.
That is a smart framework.
But the part that stuck in my head was simpler, and more useful:
Test until you throw up.
That, to me, is much closer to the truth of how real trading skill is built. Maybe it’s just my style because I’m not the smartest kid in the room.
Cleverness Is Overrated
In this business, hard work matters more than cleverness far more often than people want to admit.
Everyone likes the image of the brilliant quant uncovering some hidden law of markets. In reality, most durable progress comes from testing, refining, breaking, retesting, and learning what survives. You work through enough ideas, enough false starts, enough fragile back-tests, and eventually you develop something much more valuable than a single strategy.
You develop judgment.
That is the real asset.
Until you know a model inside out, you do not really own it. You may admire it. You may be excited by it. You may even think you trust it. But when live trading gets ugly, when the drawdown arrives, when the model starts behaving differently from the polished version in your head, confidence disappears very quickly unless it was earned properly.
That is why the best strategy is not the one with the prettiest optimisation report. It is the one you can actually trade, because you understand it deeply enough to know what kind of pain comes with the package.
Confidence like that is earned.
Not borrowed.
What “Test Until You Throw Up” Actually Means
To me, it means pushing an idea far enough that you run out of easy ways to fool yourself.
There are so many ways that you can validate and invalidate your hypothesis, and you’d like some automated pipeline that ‘did it all for you’, and maybe one day you’ll have that, but no one can miss this first step: just get to work.
Checking neighboring parameters instead of worshipping the best one. Add noise. Change markets. Study regimes. Run walk-forward tests. Looking at real out-of-sample data. Examine the ugly periods. Test the opposite of your hypothesis. Try to disprove the idea instead of protect it. Do all the things.
That process (however it looks for you right now) does two things at once.
It improves the model, if the model deserves to survive.
And it improves you, which matters even more.
Because what you are really building is not just a strategy. You are building a standard. A process for deciding when evidence is strong enough to trust and when it is just another flattering illusion.
That is where experience comes from. And if decades of experience don’t help, then trading must be the one and only field where that isn’t true. Of course it does.
Experience comes from doing the work long enough that your pattern recognition improves and your tolerance for nonsense drops. You begin to see commonalities in the things that work, you can draw more quickly from published ideas, you know broadly what works, and what works for you.
This is why we built Algo Vault. It’s a research library of hundreds of academic strategies and ideas, neatly summarised, code extracted, searchable, practical. We want to stay on top of the research, but we want to do it super efficiently, and in a way that gets straight to the point!
Experience Is the Edge
People often assume the advantage of an experienced quant is technical skill. More code, more maths, more tools.
That helps, obviously.
But the deeper edge is experience built through repeated exposure to what fails. After enough testing, you start spotting common patterns earlier. You get faster at separating signal from theatre. You develop a feel for what is fragile, what is robust, and what is probably just another curve-fit wearing a tie.
That is why building your own research process matters so much.
A borrowed strategy can make money.
A personal process builds judgment.
And judgment is a far more durable asset than any single model.
It also explains why seasoned traders can appear contrarian without being reckless. Experience gives them the confidence to distrust accepted norms when those norms are not useful in practice. It gives them the ability to prefer robustness over elegance, plateaus over peaks, and survival over intellectual vanity.
That is not anti-quant.
That is what serious quant thinking actually looks like.
Why Samir’s Framing Matters
This is why Samir’s approach to risk is so compelling.
His insight is not that markets can be forecast perfectly. It is that most people are solving the wrong problem. They are trying to predict exact future risk levels when a far more practical task is to classify whether the current regime is benign or hostile.
That is a profound shift.
Prediction asks for precision.
Classification asks for usefulness.
And usefulness is what trading pays for.
Returns are noisy. Risk is often stickier. Stress tends to cluster. Fragility tends to leave traces. So instead of asking, “Can I predict the next move?” the better question becomes, “Given the current state, how much risk should I carry?”
That is a serious trader’s question.
It also leads to better behaviour. Once you stop obsessing over precision, you start caring more about robustness. You stop hunting for the perfect parameter and start looking for a stable region. You stop falling in love with sharp peaks in a back-test and start preferring plateaus that can survive contact with reality.
That is the mentality experience gives you.
Less impressed by cleverness. More interested in resilience.
The Hardest Part of Research Is You
Sooner or later, every trader runs into the same obstacle.
Not the market. Not the model.
Themselves.
The real danger in research is the moment you start lying to yourself. The moment you ignore awkward evidence because the equity curve looks cleaner without it. The moment a tweak stops being honest investigation and starts becoming self-deception.
That is why process matters so much.
A good research process is not just a way to find strategies. It is a defense against your own nonsense.
And that tendency never fully disappears. Experience does not make you immune to overfitting, bias, or rationalisation. It just makes you better at recognising the smell of them before they become expensive.
That alone is a serious edge.
What This Means in Practice
So where should a trader start?
Start with something simple that already has some external validation. Trend following. Momentum. Mean reversion. Whatever fits your temperament. Then test it properly. Dump results into spreadsheets. Build crude tools. Improve them. Ask better questions. Study the weak points. Learn what changes matter and what does not. Learn to use AI with your back testing. This is the new frontier.
Keep the model simple enough that you can understand it and strict enough that you can interrogate it honestly.
Most importantly, build something you can actually trade.
Not admire.
Trade.
Because the best strategy is not the one that looks smartest in a presentation. It is the one you understand well enough to stick with when conditions get unpleasant and doubt gets loud.
That is the standard.
Final Thought
So yes, Samir’s model matters. The focus on risk classification over return prediction is useful and practical.
But the bigger lesson, at least for me, is about process.
Edge is built by testing hard enough and honestly enough that experience starts to form. By knowing your models inside out and back to front. By trying hard to break them before the market does it for you. By doing enough work that your confidence comes from evidence, not excitement.
That is what “test until you throw up” means.
Real work. Honest work. Repeated long enough that judgment emerges.
That is where experience comes from.
And for most traders, that is where the real edge is.
The market does not reward the trader who sounds smartest.
It rewards the one who has done enough work to stay alive.
Get in Touch with Samir
That’s a refreshing take. As a physicist, it makes me think of two important and related things: phase transitions and model risk from a second-line perspective. That’s basically what the second line of defense does (in part): tries to break models. It’s also a mindset that comports with the good-old scientific method. Scientific theories (and models) aren’t true–they’re more or less successful and have to be replaced from time to time. Steve Farmer's point is also well taken. I would complement that by saying conceptual soundness isn't just about predictive power, but also the integrity of the data.
This aligns perfectly with what I am seeing in prediction markets right now. Most retail traders on platforms like Kalshi try to build models that predict the exact temperature or inflation print. That is a losing game. I run a 62-member hybrid weather ensemble, but the predictive output is only the first step. The actual edge comes from classifying the execution risk. If the probability gap is massive but the order book imbalance or the bid-ask spread fails the risk threshold, my Python pipeline rejects the trade. I log every rejected signal into a PostgreSQL database. Reviewing why the system stayed out of a trade is vastly more valuable than reviewing why it won. Execution risk classification will always outperform pure prediction.