Topic: Let's Backtest Some Indicators - page 2. (Read 7729 times)

I am fine with criticism.  I should note that I said it outperforms buy/hold on a risk adjusted basis.  In other words, you get a higher net profit and less of a drawdown.  I should also remind everyone that I did note that it underperformed for a significant amount of time.  I am not trying to promote the MACD as some holy grail.  Now to your question about trading?

Where and how do you perform these trades?  Well, I set it up for daily trading, so one could do this manually merely by trading at the opening of each bar UTC time.  You can also create automatic trading strategies.  I am using NinjaTrader for both backtesting and automated strategy execution. 

However, I want to emphasize AGAIN....that this is only backtested research and not a complete trading strategy. People can draw their own inferences as to its value.

I think it depends on what you mean by quiet time.  During times of really low volatility, the MACD can provide whipsaws.  Maybe you are asking this question, "Should you rely solely on the MACD when deciding to enter or exit the market during times of high volatility?"  IMO, no.

@BitchicksHusband

Okay, I'll jump to let_me_backtest_that's defense one more time... I'm starting to feel I'm whiteknighting here :D

Anyway. Your points are all absolutely valid questions.

However, let_me_backtest_that said clearly, that he doesn't provide a fully developed strategy. Call it "a hypothetical strategy based on a single signal", if you want.

This is not just a semantic difference. Understanding the difference between a single indicators's signal and a strategy is the difference between making profitable trades and incurring losses in the long run. I learned that lesson the last year.

The backtest is just the evaluation of a single indicator signal, and it potentially invites the interested trader to continue investigating it. Since the signal all by itself slightly outperforms b&h (in this very theoretical, non-practical setting), it is a somewhat interesting invitation.

First off, Oda, he said it beats buy and hold and I intend to make him prove it more than his simple statements.  He brought it up, he should have expected criticism.

My point: Buy and Hold is easier to perform perfectly than ADX.  So I have the following questions:

How do you perform all these trades?

How do you perform all these trades perfectly? 

Do you ever sleep?

Do you program a bot to do them for you? 

Do you leave your money on an exchange such as Mt. Gox?  If so, your carefully traded stash just went to zero.  If you say Bitstamp, how do you know Bitstamp isn't the next Mt. Gox?

All of these questions must be answered adequately in order to beat buy and hold in a cold storage wallet, which takes zero effort and gives 93% of the same result (by the numbers given).

Mistakes can easily lose you that 7%.

Your back-tests gave us very useful information and it is much appreciated. Especially appreciated is the videos, which gives much more clear picture for table-challenged persons like myself  
I realize that no single indicator can give perfect signals. I also realize that catching the exact tops&bottoms is Holy Grail that can be searched for but hardly can be found.  What I'm trying to figure out about 1d MACD is when it should be used and when avoided. It seems to be it is more useful for quiet time, i.e. not during the parabolic rise or immediately after. It is too lagging for it. The quiet time can be trending, just things should be moving slow enough for the indicator to catch up. Especially good it is for "buy" signals. For violent time and "sell" signals it should be used more cautiously, with more confirmations or not at all. Does it make any sense?

My purpose here is to give the backtested information so people can determine the usefulness of the indicator for themselves.  One reason I provide the videos is so that people can see the exact issues that you point out.  I will say this about the MACD, as with any lagging momentum indicator, it will not catch the exact top.  Also, any price that decreases precipitously in one or two bars will give back a lot of gains. 

What I am less comfortable with is the following inference: If the MACD lags (and thereby doesn't sell at the top of a rally), then it is only useful in non-trending periods.  Also, I want to point out that my purpose is not to give  full blown strategy.  I think a person could find various exit parameters alongside the MACD indicator that would be useful.  Just something to think about. 

Thanks for the tests!

I've looked at the MACD video. The indicator seems to be good for quiet periods, but not for the rally time.

At April's rally it
    1 Completely missed the rally's top
    2 Sold exactly at the following bottom
    3 Bought exactly at the following top

At October's rally it was planning to replicated April's disaster   , but was saved by an accident: it was going to sell at the bottom again, but since the rally had two overlapping peaks, it sold around the second peak.

Ergo: 1d MACD is good at quiet periods, but at rally time, because of the lag, it's signals becoming worse than useless (namely harmful).

(Sorry for harsh words. My anger is directed not to participants of this discussion, but to the indicator itself. I guess it wouldn't mind )

EDIT: grammar.

Thanks oda.krell-  You are right, this post is a an invitation to test people's own ideas with a some data and code to get started with.  I am responsible for some of the misunderstanding because the backtest length was a typo.  I am including the MACD backtest info up till 6-29-14.  People can see that it is similar to the other results.

http://i.imgur.com/bFcicsE.png

@BitchicksHusband Let it go, man. Just by eyeballing MACD, I can already tell that during the first 6 months of 2013, b&h would perform better. Then skip forward to 2013/2014, and MACD signal is better again.

What matters are overall profits, and this entire thread is basically an invitation to backtest your own ideas. Instead, you come in here, with your "only b&h makes sense" attitude, and take cheap shots. Why not _download_ the motherf'ing program and test it for yourself?

EDIT: that said, any reason why you don't run the test over the entire trading history, let_me_backtest_that?

EDIT 2: sounded more aggressive than intended. apologies.

Change it to first six months of 2013 and see if it still beats it.

Good point. I am not factoring in slippage, commissions or taxes which should be thoroughly taken into account before using this as a stand alone strategy.  I want to emphasize that this thread is just for testing indicators not fully developed strategies.

Yes, that's right.  I am buy on the open and selling on the close without slippage and commissions to simulate both the gains and losses of buy/hold.  Also, I don't see how the data is cherry picked, not intentionally anyway, I am using the same data set as the other indicator backtests.  The bitstamp data is pretty shoddy before 1/1/2012.   In fact, starting at the beginning of the most recent move to 1,200USD and decline would've given the best results because the MACD has had a very good record of shorting during the recent bear market.

AFAIK, he doesn't include slippage and fees in his calculations, so assuming no gaps between daily candles, I believe selling at candle close and buying at candle open is just his way to simulate a buy&hold strategy. It is (for the reasons mentioned above) equivalent to "real" b&h.

Outperformed "buy and hold DAILY".  That's not a valid measure.  Nobody buys and holds daily.

Also: "The backtest will cover 1/'1/2012-6/1/2012", the cherry-picked deadest period possible for buy and hold.

Interesting to read that trading MACD beats buy hold. Does that factor in the tax savings (long term capital gains vs. short term capital gains) inherent in the buy+hold strategy that MACD trading doesn't have?

I keep being surprised by how well daily MACD still seems to work for this market, even after it being mentioned and discussed over and over again. In fact, I suspect during the 2013/2014 bear market it would have performed even better than in your 2012 data backtest.

Yes, thanks for clarifying that.  I think its important to show how a going long and actively selling short with one's full account value can have some pretty horrendous consequences.

EDIT: I had the order backwards in the post.  It now makes more sense.

Very interesting, as before. Thanks!

"long/short" MACD strategy means on a buy signal, the system buys in with the entire account value, and on a sell signal, it sells and actively shorts? While "long" only means the system only buys and sells, without entering a margin position?

DOES MACD BEAT BUY AND HOLD?

The MACD is a fairly straightforward momentum indicator.  It consists of the difference between a 12 and 26 bar EMA.  The difference between these two moving averages is the MACD.  It is smoothed by a 9 Day EMA that is called the "signal line."

The MACD  is usually plotted with a histogram that represents the difference between the MACD and the signal line.  The histogram can give either positive or negative values.  On the traditional interpretation, a buy signal occurs when the signal line crosses the MACD and a sell signal occurs when the signal line crosses below the MACD.   See the example below:

http://i.imgur.com/Llda1qf.png

The MACD is a popular indicator on BTC forums.  How does it hold up historically? Does it beat  buy and hold?  How does a long and short strategy perform?  We will perform two backtests.  The first will be a MACD long and short strategy.  The second will be a long only strategy.  Finally, we will compare this to buy and hold.

We will make the following assumptions: (1) The backtest will cover 1/'1/2012-6/1/2012.  (2) We will reinvest profits beginning with a $1,000 account.  (3) We will keep the default parameters for the MACD.

EDIT: The backtest is performed over 1/1/2012-6/1/2014

How does the long/short strategy perform?

The strategy entered 41 total trades - 59% were profitable- net profit was $9,820- there was a 98% a drawdown and a profit factor of 6.42.  See below

http://i.imgur.com/UcJD6aH.png

I am also attaching a youtube replay of each trade in the system to get a better feel for how the system would've worked.

https://www.youtube.com/watch?v=3mwCZOr-6fc&feature=youtu.be

You will notice that the system trades a different number of BTC depending on the account size.  Once it hit a large drawdown at the end of April 2013, then the system did not have the equity to trade larger quantities of BTC.

http://i.imgur.com/vwQTIqI.png

Now let's backtest the long only system.

The strategy entered a total of 21 trades - 76.19% were profitable - net profit was $135,000,  there was a 19% drawdown and a profit factor of 60.41.

http://i.imgur.com/ZFZLGI5.png

Here is the youtube replay.

https://www.youtube.com/watch?v=jE_eDDAe10w&feature=youtu.be

Ultimately, MACD outperforms a buy/hold strategy as seen below.  The Buy/Hold had a net profit of $126,600 and a drawdown of 73%.  (For Buy/Hold I entered a buy at the open of everyday with the same amount of BTC 202 on each trade to make the performance equivalent. )

http://i.imgur.com/kz70wD9.png

Here are a few takeaways:

1. Ultimately, MACD long only does beat buy and hold on both a net profit and drawdown.   It caught every major uptrend even though it lagged the market.

2. However, MACD long only underperformed buy and hold during the large price increases.  At one point, someone would've seen their BTC holdings drop from 202 to 58 due to price increases, lag and some losing trades.

3.  The drawdown  in USD on the MACD strategy is superior to buy/hold.  Returns should always be related to risk.  Someone trading the MACD would've had a $1,982 drawdown in USD, while the buy/hold drawdown equaled $159,000.

You can download updated BTC historical data formatted for NinjaTrader here:

http://www.signalstrengthfinance.com/bitconnector-bitcoin-trading-on-ninjatrader/

You can download the code for your own analysis on my blog;

http://www.signalstrengthfinance.com/bitcoinchartsmacd/


Trade Well and Prosper

Nice!  You should charge for your services.