Topic: Let's Backtest Some Indicators (Read 7776 times)

With this type of indicator you could reasonably predict when the next long term spike is about to occur. Using 10 to 1 leverage you could turn 10 BTC into 12 pretty easily. (minus the fees)

Hi, thanks very much for this amazing work.. Could you share the code for the MACD indicator is this is possible.. I have not knowledge in programming.. I would really thankful. I would like to test it with other parameters.

Whoosh! ... Thanks for the explanation.

Show me your time machine...

Please look at the data again, this is a backtest, not an estimation. Any backtest should be ran against the very simple and basic strategy of buy and hold which requires no TA, no bots, no effort.

So those who bought half a year ago should expect an 800% return before 2015? Yeah

Very nice buut... you gained 800% in 2 years? Buy and hold gives you that in less than a year. Maybe you can consider "tweaking" your backtest a bit more for more consistent results.

Thanks for the suggestions.  It would be easiest to give a final calculation that shows what the net profit would be in BTC at the end of the test period.   In that way, no additional position sizing would be necessary.

As far as using the SAR as a profit target on a bounce from recent lows...that is an interesting idea.  I would have to find a way to determining those entry points systematically and then use the SAR as a (stable or dynamic) profit target.  This gives me something to work on and then share with those interested.  Thanks for the suggestions!

About SAR rebound - we may have a chance to test it big-scale when/if the bubble come and burst.

About measuring in BTC - basically agree. What I mean is just a slight change to presentation of results,  so the reader won't have to calculate how his BTC holding have changed and won't have to translate "strategy made a profit, but performed worse than B&H" to "strategy made a loss".

re: SAR

I tried using the SAR myself, but find it alternatively too lagging and too "trigger happy" i.e. yielding too many, often unprofitable, signals. There's one positive observation I made however: since it is a momentum indicator with a "built in" price target, it can be useful for exiting a trade on a rebound, i.e. say I sell based on some indicator combination, buy back short term because I determine the "bottom is in", and then exit the trade = sell when price hits the SAR target. Usually relatively low time resolution, between 1h and 12h.



I thought the same at first, but then I came to the conclusion (please correct me if I'm wrong): a USD maximized holding and a BTC maximized holding are equivalent, as long as you just tack on one final 'buy' order to the end (i.e. calculate how many coins the final USD value buys you). Correct?

To see why: imagine you don't take a trade that is suggested by the tested system, and that turns out profitable. Your BTC value will be equal, but your USD to buy coins is higher at the next signal, so being USD optimal is equivalent to being BTC optimal. The only difference is that the final value of the holding is given in USD, so convert it to BTC according to current price.

@l_m_b_t Not entirely sure I understand your point about needing to include position sizing and curve fitting because of calculating BTC holdings. Then again, maybe I misunderstood wary's question.

Thanks for the suggestions!  Placing the data in terms of accumulation of bitcoins is an interesting goal.  I like it, but I wonder the best way to proceed.   If we proceed this way, then I would need to include a position sizing strategy in my backtests.  Position sizing strategies should differ from one strategy to another.  So, I would have to somehow choose one of them.    However, I don't want to be give any misleading results by unintentionally curve fitting my results. 

If other posters here have some suggestions.  I would be very happy to hear them.

Excellent point and worth repeating.  Everyone engaged in backtesting strategies needs to be aware of any and all pitfalls.   

Thanks, good work, very informative. Let God bless you with success in trading and with crowds of customers.  

(BTW, doesn't work here. You can use [ b ] [ /b ] or  [ i ]  [ /i ] or [ color=red ][ /color ] instead)

EDIT: A suggestion: since most of BTC traders trade to increase their BTC holdings, it would make sense to calculate profits and losses in BTC, rather than in dollars.

Past performance is not a predictor of future results.

Back testing can be useful if you can control your risk.

I got a suggestion to use the SAR on the weekly bars.  It does indicate the previous two bubbles pretty early.  The net profit doesn't change much, but i am assuming the person who used it didn't rely on the SAR as an exit signal. 

see below:



https://i.imgur.com/6K8GMmC.png

https://i.imgur.com/6K8GMmC.png

How About the Parabolic SAR

The Parabolic SAR (hereafter SAR)  is a trend indicator.   It was one of the indicators published by Welles Wilder in New Concepts in Technical Trading Systems.  It plots a dot above or below the price action to indicate the existence of a trend.  If the dot is plotted above the price, then a bearish trend is indicated.  A dot below the price action indicates a bullish trend.

The SAR indicates a trend reversal when the high/low move below the value of the dot plotted.  These trend changes are designed to detect reversals.    Hence the name, "Stop and Reverse."     Here is an example:

https://i.imgur.com/ewFiQwg.png

Before we go deeper into the data, I will just state that this indicator performs poorly at various points on the backtest.  Perhaps that is all you may want to know.  If it is, then it still shows the importance of backtesting an idea before trading with it.  Sometimes the best information you can get from a backtest is not to use it.

Additionally, as I've said before, an indicator is not a stand alone system.  So, if people have some interesting ideas on using the SAR in other ways, then I will do my best to integrate some new ideas.

Now that we have a bit of an introduction to the SAR let's see how it does on historical BTC prices.  We will start with a simple backtest that  assumes 1 BTC  per trade (no slippage and commissions) and merely enter when the SAR indicates a reversal long or short.  The backtest will run from 1/1/2012 until 7/1/2014.  The indicator gives 52 trades, a $472 net profit, a 59% peak-to-valley drawdown and a profit factor of 1.61.

https://i.imgur.com/HKPnpka.png

One thing to point out is that all the losses came from the short side of the signals.   The long part of the strategy had a net profit of $564.00  and a drawdown of 46%.

We can contrast can see that this underperforms a buy/hold strategy.  A buy/hold strategy with 1 BTC has a profit of $670 and a peak-to-valley drawdown of 73%.

https://i.imgur.com/zS8IUZ7.png

Much of this underperformance is due to whipsaws and early exits on long trends.  Each of these features is undesirable from the perspective of a trend following indicator.   Here is a replay of the results so you can get a feel for the SAR signals.

https://www.youtube.com/watch?v=MIaRYYoU6W4&feature=youtu.be

Here are some observations from the backtest:

1. The indicator has many whipsaws when prices are moving sideways.  This suggests that its usage in a system would benefit from a trend filter.  In fairness, Wilder suggested that it be used with the ADX to indicate trend following.

2. The indicator exits a trend early.  Generally, "letting your profits run" is the motto of trend following, so this is undesirable from that perspective.  (You can see from the backtest that the exit at $600 was far from the end of the trend).

3. Could it be used as a trailing stop (i.e., a stop that continually moves up to capture profit before the trend reverses)?  Perhaps, IMO there are better methods for finding a trailing stop for each particular strategy.

The backtests were performed using NinjaTrader which can be downloaded for free.  We provide historical BTC data that is updated daily and formatted for NinjaTrader  for free.

https://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

Also, the code is available for download and can be imported directly into NinjaTrader

https://www.signalstrengthfinance.com/how-does-the-parabolic-sar-work/

Trade Well and Prosper

The " Turtle Traders" are a legendary group of traders taught by commodity trader Richard Dennis.   He and his partner had a debate about whether or not trading could be taught.  They conducted an experiment by placing an ad in the Wall Street Journal and giving a group of traders some accounts and a system.  The turtles ended up making more than $150 million.    Richard Dennis came up with the idea after visiting a turtle farm in Singapore and thought that traders could be "grown" like turtles.

The Turtle Trading System's entry and exit parameters are quite simple.  They are based on the Donchian channel breakout indicator.  The Donchian channel merely measures the highest high and the lowest low in a series of bars.  For example, if you set the lookback to 10 bars, then the system will give you the highest high/lowest low in the last 10 bars.

The turtle trading system had two different entry systems that could be used. Each one based is based on the Donchian Channel breakout.  The one we'll consider goes long/short at the highest high/lowest low of the last 20 days.   It exits when when there is a new high/low 10 bars ago.   Here is an example:

https://i.imgur.com/SEIUCFq.png

If we backtest the indicator, then we will get some information about how it performed historically on BTC prices.  This information can be valuable when thinking about designing a system.   However, my primary purpose in this post is to distinguish between an indicator and a strategy.

Let's take the Turtle System which is given for fee by one of the former turtles:

http://bigpicture.typepad.com/comments/files/turtlerules.pdf

Notice that there are at least six things that go into the complete strategy:

1. Markets - What to buy or sell

2. Position Sizing - How much to buy or sell

3. Entries - When to buy or sell

4.  Stops - When to get out of a losing position

5. Exits - When to get out of a winning position

6. Tactics - How to buy or sell

Its important to notice that the entry/exit is only a small part of the entire strategy.  I will illustrate by running two  backtests and compare this with buy/hold.

The first will be a backtest of the Donchian indicator itself as determined by the Turtle trading strategy.  The second will be a backtest that includes the position sizing parameters of the Turtle system.  Each backtest will assume an account of $10,000 with profits reinvested and trade from 1/1/2012 thru 6/1/2014.

Here are the results for trading the Donchian channel indicator itself.  (Note: all profits are reinvested every trade because there is no position sizing strategy.)  The results include a 22 trades- 50% profitable- a profit factor of 1- a drawdown of 98%- and a net profit of $3,482.

https://i.imgur.com/5REjbmN.png

Here is a youtube playback of the system.  (You will notice the large drawdown occurs in April 2013).

http://youtu.be/aTlpsat74dY

As is obvious, buy/sell signals generated from the indicator severely underperformed buy/hold.    Buying $10,000 worth of Bitcoin would've netted you a nice profit of 1.38 mill.  As you can see below:

https://i.imgur.com/gqFHam9.png

Now, let's consider what happens when we include the position sizing algorithm that the Turtle's used.  We will follow their rules of only entering 4 units per trade.  Where a unit is defined by (0.01 x AccountSize)/ Dollar per point x the Average True Range).  This position sizing algorithm leads them to enter less money when markets are volatile and more when they are not.   Here are the results using the same time period.

There were 22 trades-50% profitable-a profit factor of 5.27- a drawdown of 98% and a net profit of 3.61 mill.  See below:

https://i.imgur.com/4efSiUa.png

I will also add the Youtube playback so you that you can see how the position sizing affected the trading.  Notice the awful trade that occurs in April 2013 doesn't  destroy all the equity because the trade size is risk adjusted for larger volatility.

http://youtu.be/cT5wu_ODap0

 

Here are some takeaways:

1. An indicator is not a system.  It should not be used as a complete system when trading.  It should not be evaluated as a complete system when backtesting.
2. An indicator that, by itself, underperforms buy/hold can outperform buy/hold when placed in a good strategy.

3.You can learn a lot from a Turtle.  Personally, the 34 pages providedis one of the most concise and helpful pieces on trading that I've read.  http://bigpicture.typepad.com/comments/files/turtlerules.pdf

As always, the code is given at my blog

http://www.signalstrengthfinance.com/distinguishing-a-strategy-from-an-indicator-a-lesson-from-the-turtles/

Historical Bitcoin Data can be downloaded for free in NinjaTrader format  at the link below:

http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

Trade Well and Prosper

Thanks for doing the backtest.  I guess it doesn't work as well as the standard MACD long only

The slow/fast signals are on shorter periods so its expected to have more trades.  I'm curious to why more drawdowns

Thanks...glad you like the thread.

I use NinjaTrader for my backtesting and order execution.  You can download NT for free here: http://ninjatrader.com/download-registration.php

We provide a continually updating link for BTC data that can be imported into NT for free as well as real time data in our Beta release of Bitconnector.   This will allow you to backtest and automate strategies fairly easily.   You can find both here: http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

I was interested in how the 3-10 oscillator would work. Here is the backtest for it on BTC (assuming that 3=fast, 10=slow and 16 = signal line)

 https://i.imgur.com/CiMkrda.png

Thats pretty cool.   Can I ask which software you use to backtest?   Did you automate yourself?

I use an indicator called 3-10 oscillator for trading stocks.   Its a MACD set to 3/10/16

Works pretty well but I'm manually doing the trades and sometimes I miss the signals.   I'd be interested to backtest this and also automate my trading system