Topic: Let's Backtest That (Read 880 times)

BACK TO BACKTESTING

I posted here about once a week during the summer. I am interested in creating and implementing trading systems or bots. I generally use posts to discuss findings from backtesting various indicators, rules, or strategies.


I thought I jump back in by outlining the basic (not complete) process of building, backtesting, and trading a system.

1-. Come up with an idea.

This first step may sound simple, but it is different than "test 1,000 variations of an indicator and find the one that works." Now, research on indicators can be helpful, but use the indicators to try to implement your idea and not the other way around.

For example, if you are a trend trader, then using trending indicators like Donchian may be a good way to implement your idea. However, the idea must have some independent basis for why it should work.

2-. Write the code. Writing code can be tough. One reason why I recommend NinjaTrader is that they have a system building interface that allows you to build a system without any coding ability. Additionally, the language is based in C# which also makes it possible to do whatever you want given your ability to write code.

3-. Backtest on in and out of sample data. If you are really backtesting a system, then you will want to segregate some data, so that you can have a "test" set. This step helps, but cannot eliminate, data mining bias.

4-. Don't over optimize. Suppose you create a moving average crossover system and you realize that the 13/23 day values work the best, but that the 12/23 changes your profits significantly. It is highly likely that this combination is fluke. In fact the best use of optimizing is when you can see that a broad range of values have similar results. To return to the example, suppose that the 10-15/20-25 MA works about as well, then you can be more confident in your results.

5-. Once you've tweaked, tested and are happy with the results, then test on your out of sample data. If you find that it doesn't work well, then you've discovered a hard truth: Your system isn't that great. Now, the temptation is to re-optimize on the new data, but this effectively eliminates the out of sample data. Depending on the results you may need to throw out the system, revise it substantially or go back and re-tweak on in sample data.

6-. Test it live without money. Let it go on some live data without risking any money. After that, put a little down and see how it does. In other words, push on the gas slowly. Trust me this step can save you a lot of money.

7-. (Optional): Re-optimize? There is a debate among system traders about whether to merely let it run or to re-optimize every so often. I am probably in the Lee Leibfarth camp where you re-optimize every 2-4 weeks.

I am happy to discuss any of this in more detail.

You can backtest any idea for free using NinjaTrader. We provide a free daily updating link for historical data and the NinjaTrader license is free. We also provide You can find the information here:

http://signalstrengthfinance.com/bitconnector.php

If you are interested in any indicators/ideas that you want me to backtest, then let me know. I plan on going back to posting once a week.