Hello! Welcome to the first lesson of our module on backtesting.
In our last lesson, you created a comprehensive pre-trade checklist, which serves as the complete logical blueprint for your ICT trading strategy. This checklist is the "what" and "when" of your trading decisions. Now, we move to a crucial question: "Would this strategy have actually worked in the past?"
This lesson addresses the learning outcome: Explain the purpose of backtesting and the danger of look-ahead bias. Answering this question is the first and most fundamental step in translating your manual strategy into a reliable automated trading bot. Without understanding how to properly test your ideas, even the best trading logic can lead to failure.
The Purpose of Backtesting: From Idea to Evidence
Before you launch a new marketing campaign, you conduct market research. You test your messaging on a focus group and analyze data to see if the idea has potential. Backtesting is the equivalent for a trading strategy. It is the process of simulating your trading rules on historical data to see how they would have performed.
To begin, let's establish a clear understanding of what backtesting is for.
Backtesting a Trading Strategy
The article 'Backtesting a Trading Strategy' from Onepagecode provides an excellent overview of what backtesting is and why it's so critical for any quantitative trader.
Please read the first two sections, 'What is Backtesting?' and 'Why is Backtesting Critical?'. Pay attention to the key objectives it helps you achieve.
As the article highlights, backtesting is not just about seeing if a strategy is profitable. It's a rigorous process that helps you:
- Validate Your Logic: Does your ICT checklist make sense when applied systematically over months or years of data?
- Assess Your Risk: What is the worst-case scenario? Backtesting reveals potential drawdowns (the peak-to-trough decline in account value) and the volatility of your returns.
- Refine and Improve: The initial results will almost certainly reveal weaknesses in your strategy, allowing you to make data-driven improvements.
- Build Confidence: Making the decision to risk real money is much easier when you have empirical evidence that your strategy has a statistical edge.
For your goal of building trading bots, this process is non-negotiable. An algorithm is just a set of rules, and backtesting is how you prove those rules are sound.
The Great Pitfall: "Too Good to be True"
While essential, backtesting is filled with traps that can make a losing strategy look like a holy grail. It's incredibly easy to produce a backtest with a perfect-looking equity curve that completely falls apart in live trading. Understanding why this happens is key to producing reliable tests.
Let's watch a short video that introduces this common problem and sets a healthy, skeptical mindset.
Out of 1,000s of backtests, here are the 6 main lessons for strategy success. (EP 38)
In 'Out of 1,000s of backtests, here are the 6 main lessons for strategy success,' the hosts at Unbiased Trading discuss why many backtests produce 'fantasy results'. This will frame our discussion on the most common biases.
Watch the first 6 minutes and 29 seconds. Focus on the core idea of why you should be skeptical of backtests that look 'too good to be true' and the general pitfalls mentioned.
The key takeaway is that markets are complex and efficient. Finding a genuine edge is hard. If your backtest suggests you've found an easy way to multiply your money, it's more likely that there's a flaw in your testing process than that you've discovered a secret money-printing machine. Of all the possible flaws, one is more subtle and dangerous than all the others: look-ahead bias.
Look-Ahead Bias: Seeing the Future
Look-ahead bias is the cardinal sin of backtesting. It occurs when your simulation uses information that would not have been available at the moment a trading decision was made. You are, in effect, giving your strategy a peek at the future.
Understanding Look-Ahead Bias and How to Avoid It in Trading Strategies
To formalize this, the article 'Understanding Look-Ahead Bias' from Marketcalls.in provides a crisp definition and a helpful analogy.
Read the section 'What is Look-Ahead Bias?' and 'How Models can look into the future?'. The weather forecasting analogy is a very clear way to think about this.
Any backtest infected with look-ahead bias is completely invalid. Let's consider how this could happen with the ICT concepts you've learned:
- Scenario 1: Trading Ranges. You want to backtest a strategy that sells in a premium and buys in a discount of the weekly range. You write a script that, for every day of the week, uses the high and low of that entire week (Monday-Friday) to define the range. This is look-ahead bias. On Tuesday, your algorithm cannot know what the high or low will be on Thursday or Friday. The test is using future data.
- Scenario 2: Swing Points. Your code identifies a swing high on a 1H chart. A swing high is only confirmed after the candle to its right closes lower. If your code makes a decision at the close of the swing high candle itself, it's using information (the confirming next candle) that wasn't available yet.
The core principle to avoid this is to constantly ask yourself: At this exact moment in the simulation, what information would I realistically have?
A Case Study: Spotting and Correcting Look-Ahead Bias
To make this concept concrete, let's watch a trader who intentionally creates a backtest with look-ahead bias, explains the flaw, and then shows the correct way to do it.
Volatility Strategy Backtesting: Look-Ahead Bias - Ep.82
In this video, 'Volatility Strategy Backtesting: Look-Ahead Bias', the presenter provides a practical, step-by-step demonstration of look-ahead bias and how to fix it using a method called walk-forward analysis.
Please watch from 08:40 to 11:37, and then from 15:41 to 17:23. First, focus on how he defines the bias in his test (using a median from the full dataset). Then, see how the 'walk-forward' method corrects this by only using past data. Finally, observe the difference in the final results.
The video demonstrates a critical concept:
- The Flawed Method: Used a single statistical value (the 50th percentile) calculated over the entire 20-year dataset and applied it back to the beginning of the test.
- The Correct Method (Walk-Forward): Periodically recalculated the statistical value using only the data accumulated up to that point in time. This simulates how a real trader would adapt as new data becomes available.
This walk-forward approach is a fundamental technique for building robust backtests, and we will explore it in more detail when we start coding.
Test your understanding!
You are building a bot to automate the ICT Silver Bullet strategy, which targets liquidity between 10:00 and 11:00 AM EST. To identify a high-probability setup, your bot's logic needs to know the high and low of the day so far to determine if price is in a premium or discount.
Your script is running at 10:05 AM. It calculates the day's range by using the high and low price from the completed daily candle data (i.e., the high and low from 9:30 AM to 4:00 PM EST). Is this a valid approach for your backtest? Why or why not?
Show answer
This is not a valid approach. It is a classic case of look-ahead bias. At 10:05 AM, the bot would only know the high and low of the session up to that point. It could not possibly know what the high or low for the rest of the day would be. Using the completed daily candle's data gives the bot information from the future, making the backtest results unrealistic and untrustworthy.
Conclusion
In this lesson, we've established the "why" and "what to watch out for" in backtesting. You've moved from having a trading plan to understanding the principles of how to test that plan against historical reality. This conceptual foundation is indispensable before writing a single line of code for your trading bot.
Key Takeaways:
- Purpose of Backtesting: To simulate a strategy on historical data to validate its logic, assess risk, and make data-driven decisions before trading live.
- Healthy Skepticism: Be wary of backtests that look too good to be true. They often contain errors or biases.
- Look-Ahead Bias: This is the critical error of using information that would not have been available at the time of a simulated trade. It invalidates a backtest.
- The Golden Rule: At every point in your backtest, you must only use data that would have been accessible at that precise moment in time.
Preview of the Next Lesson:
Now that you understand the purpose of backtesting and its most dangerous pitfall, we'll dive into the "how." In the next lesson, we will explore the two main architectural approaches to building a backtester: vectorized and event-driven backtesting. Understanding their differences is the next crucial step in designing the engine that will test your ICT strategies.