Pages:
Author

Topic: Goomboo's EMA10/21 Crossover Versus the rebuilder's Flipist Method (Read 5837 times)

hero member
Activity: 532
Merit: 500
I already have the daily scale up.  I personally think bitcoin is too volatile to really hold it long term in large quantities, so holding a position for large amounts for weeks or months would be unrealistic if you are using margin.

Right now and for some time to come, yes. You'd have to manage a balanced position by scaling it instead of going all in each time.

With Bitcoin, I use weekly data as an overall trending and reversal guide, then daily/hourly to spot scaling opportunities. It'd be more complex using multiple temporal dimensions, but even building on the simple EMA10/21 technique that way would be far more robust - even somewhat predictive.

The greater the time-frame, the harder it is to reverse the trend.

Next time I will add in scaling.  Anyone that wants to jump ahead check out here.  R has some other tools such as blotter that can keep track of positions.
hero member
Activity: 532
Merit: 500




It says EMA5/21 on the title of the chart?




Thanks for catching that.  It should be EMA10
sr. member
Activity: 1008
Merit: 250




It says EMA5/21 on the title of the chart?


legendary
Activity: 1615
Merit: 1000
Just popped in to say "nice work!"

I don't have the time to look through the whole thread right now, but I'll be sure to do it when I can.
legendary
Activity: 1316
Merit: 1005
I already have the daily scale up.  I personally think bitcoin is too volatile to really hold it long term in large quantities, so holding a position for large amounts for weeks or months would be unrealistic if you are using margin.

Right now and for some time to come, yes. You'd have to manage a balanced position by scaling it instead of going all in each time.

With Bitcoin, I use weekly data as an overall trending and reversal guide, then daily/hourly to spot scaling opportunities. It'd be more complex using multiple temporal dimensions, but even building on the simple EMA10/21 technique that way would be far more robust - even somewhat predictive.

The greater the time-frame, the harder it is to reverse the trend.
hero member
Activity: 532
Merit: 500
Very nice work.

Also try expanding the time frame to daily & weekly scales.

I already have the daily scale up.  I personally think bitcoin is too volatile to really hold it long term in large quantities, so holding a position for large amounts for weeks or months would be unrealistic if you are using margin.
hero member
Activity: 532
Merit: 500
Sure - although tbh I've only been using rgp for a couple of weeks, before that I was using eureqa but I didn't find it flexible enough (it's really just for symbolic regression).

rgp is well documented but if you don't have a lot of experience with GAs (i don't) then the tutorials at rsymbolic.org are a bit incomplete. Still, I have plenty of algorithms generated. Some overfit and just buy below a certain level and sell above it. It all comes down to your fitness function and i think mine leaves a bit to be desired.

Some of the useful trading algo's I've got are of the longlag-shortlag=0 variety, just like the EMA10 EMA21 variety.

I'll be happy to share code I've done so far - I'm sure you can improve on it.

Thanks, if you can put some here or pm to me I will take a look at it.  I requested some books on the subject.
donator
Activity: 2058
Merit: 1007
Poor impulse control.
Sure - although tbh I've only been using rgp for a couple of weeks, before that I was using eureqa but I didn't find it flexible enough (it's really just for symbolic regression).

rgp is well documented but if you don't have a lot of experience with GAs (i don't) then the tutorials at rsymbolic.org are a bit incomplete. Still, I have plenty of algorithms generated. Some overfit and just buy below a certain level and sell above it. It all comes down to your fitness function and i think mine leaves a bit to be desired.

Some of the useful trading algo's I've got are of the longlag-shortlag=0 variety, just like the EMA10 EMA21 variety.

I'll be happy to share code I've done so far - I'm sure you can improve on it.
legendary
Activity: 1316
Merit: 1005
Very nice work.

Also try expanding the time frame to daily & weekly scales.
hero member
Activity: 532
Merit: 500
Using modified code from here and here.  I ran some backtesting in R to compare the EMA 5 EMA 21 Crossover and a random buy/sell strategy based on a coin flip.

Why did I do this?  I was making my own trading strategies but got stuck on making one for bullish/bearish divergence and I thought some practice may help me find out what I am doing wrong.

Why not generate your own? The R package rgp lets you evolve GAs. I've got some that consistently give me better than 'buy and hold'.

Quote
In artificial intelligence, genetic programming (GP) is an evolutionary algorithm-based methodology inspired by biological evolution to find computer programs that perform a user-defined task. It is a specialization of genetic algorithms (GA) where each individual is a computer program. It is a machine learning technique used to optimize a population of computer programs according to a fitness landscape determined by a program's ability to perform a given computational task.

Wow, thanks for the recommendation.  I am an evolutionary biologist (or trying to be) and this makes a lot of sense.  How have you applied GP to trading strategies?
hero member
Activity: 532
Merit: 500
***EDIT*** Calculated Expectancy

Quote
Expectancy = (Probability of Win * Average Win) – (Probability of Loss * Average Loss)

Flipist Daily Flip

Code:
  (1) buy / (-1) sell
   Signal # Trades  % Win   Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L
1     -1       93   52.68817 7.132123 -6.507944   3.636423   -4.839239 1.095910  0.7514452
2      0        1    0.00000      NaN       NaN         NA          NA      NaN         NA
3      1       90   46.66667 4.551028 -4.431499   2.366660   -2.242254 1.026973  1.0554826


Flipist - Hourly Flip
Code:
 (1) buy / (-1) sell
    Signal # Trades  % Win   Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L
1     -1     2208   50.00000 1.123302 -1.034769  0.6283153  -0.6485819 1.085558  0.9687524
2      0        1    0.00000      NaN       NaN         NA          NA      NaN         NA
3      1     2206   49.50136 1.063690 -1.027253  0.6050835  -0.5874410 1.035470  1.0300328

Expectancy = -4.0892

EMA10/21 Crossover using Daily data.  There are 18 less trades when looking at the

Code:
  EMA10/21, buy (1) when EMA10 > EMA21, sell (-1) when EMA10 < EMA21
    Signal # Trades  % Win   Mean Win  Mean Loss  Median Win Median Loss Mean W/L Median W/L
1     -1      132    56.81818 6.341372 -5.713935 2.840616 -4.183966  1.109808 0.678929
2      0       21    0.00000      NaN       NaN       NA        NA       NaN       NA
3      1       52    57.69231  5.068756 -3.911978 3.199519 -1.747152  1.295702 1.831277

EMA10/21 Crossover using Hourly data.
Code:
 EMA10/21 Buy (1) when EMA10 crosses over EMA21, sell (-1) when EMA10 crosses below EMA21
    Signal # Trades    % Win   Mean Win Mean Loss Median Win Median Loss  Mean W/L Median W/L
1     -1       83    51.80723 0.8337976 -0.8799018  0.4516124  -0.5958119 0.9476030  0.7579782
2      0     4249     0.00000       NaN        NaN         NA          NA       NaN         NA
3      1       83    46.98795 0.6240971 -1.4239146  0.4776158  -0.5503347 0.4382968  0.8678642

Expectancy = -45.00281
hero member
Activity: 532
Merit: 500
You need emotional investor and EMA10/21 contrarian controls. I'm not sure how to implement the emotional investor one...

How would you do the controls then?
hero member
Activity: 532
Merit: 500
As expected with a random method with a greater sample size (number of trades) it is almost even between buy and sell signal.

Code:
  
-1 = short
1 = long
Signal # Trades    % Win Mean Win Mean Loss Median Win Median Loss Mean W/L Median W/L
1     -1     2208 50.00000 1.123302 -1.034769  0.6283153  -0.6485819 1.085558  0.9687524
2      0        1  0.00000      NaN       NaN         NA          NA      NaN         NA
3      1     2206 49.50136 1.063690 -1.027253  0.6050835  -0.5874410 1.035470  1.0300328








hero member
Activity: 532
Merit: 500
Now for the Flipist strategy.  This is the same strategy as before but now we can flip every hour if needed.

First the code.  I added in everything to this code, from making the signals, making the charts, and outputting the statistics.

Code:
############################################
#Backtesting Flipist Method
############################################
    # We will need the quantmod package for charting and pulling
    # data and the TTR package to calculate RSI(2).
    # You can install packages via: install.packages("packageName")
    # install.packages(c("quantmod","TTR"))
    library(quantmod)
    library(TTR)

    # Load data
    x = last(y,4415) #gets the last 184 daily positions from the data (starts at July 23, 2011)

    # Calculate the random indicator
    set.seed(43) #include this to reproduce my results
    x$flip <- rbinom(4415,1,0.5) #50% probability that 1 (heads/buy) will show and 50% probability that 0 (tails/sell) will show

    # Create the buy (1) and sell (-1) signals
    sigbuy <- ifelse(x$flip == 1, 1, 0)
    sigsell <- ifelse(x$flip == 0, -1, 0)

    # Lag signals to align with days in market,
    # not days signals were generated
    sigbuy <- lag(sigbuy,1) # Note k=1 implies a move *forward*
    sigsell <- lag(sigsell,1) # Note k=1 implies a move *forward*

    # Replace missing signals with no position
    # (generally just at beginning of series)
    sigbuy[is.na(sigbuy)] <- 0
    sigsell[is.na(sigsell)] <- 0

    # Combine both signals into one vector
    sig <- sigbuy + sigsell
    # Calculate Close-to-Close returns
    ret <- ROC(Cl(x))
    ret[1] <- 0

    # Calculate equity curves
    eq_up <- exp(cumsum(ret*sigbuy))
    eq_dn <- exp(cumsum(ret*sigsell*-1))
    eq_all <- exp(cumsum(ret*sig))

    # Equity Chart
    png("flipist.png",width=720,height=720)
    plot.zoo( cbind(eq_up, eq_dn),
    ylab=c("Long","Short"), col=c("green","red"),
    main="Flipist Strategy: 07-23-2011 to 01-22-2012" )
    dev.off()

    # Create a chart showing mtgoxUSD
    png("flipistchart.png",width=720,height=720)
    chartSeries(x, subset="last 4415 hours", type="line")

    # Add the total equity line
    addTA(eq_all)
    dev.off()

    # Evaluate the Strategy

    # install.packages("PerformanceAnalytics")
    require(PerformanceAnalytics)
   
    # chart equity curve, daily performance, and drawdowns
    png("performance.png",height=720,width=720)
    charts.PerformanceSummary(ret)
    dev.off()

    # This function gives us some standard summary
    # statistics for our trades.
    tradeStats <- function(signals, returns) {
    # Inputs:
    # signals : trading signals
    # returns : returns corresponding to signals

    # Combine data and convert to data.frame
    sysRet <- signals * returns * 100
    posRet <- sysRet > 0 # Positive rule returns
    negRet <- sysRet < 0 # Negative rule returns
    dat <- cbind(signals,posRet*100,sysRet[posRet],sysRet[negRet],1)
    dat <- as.data.frame(dat)

    # Aggreate data for summary statistics
    means <- aggregate(dat[,2:4], by=list(dat[,1]), mean, na.rm=TRUE)
    medians <- aggregate(dat[,3:4], by=list(dat[,1]), median, na.rm=TRUE)
    sums <- aggregate(dat[,5], by=list(dat[,1]), sum)

    colnames(means) <- c("Signal","% Win","Mean Win","Mean Loss")
    colnames(medians) <- c("Signal","Median Win","Median Loss")
    colnames(sums) <- c("Signal","# Trades")

    all <- merge(sums,means)
    all <- merge(all,medians)

    wl <- cbind( abs(all[,"Mean Win"]/all[,"Mean Loss"]),
    abs(all[,"Median Win"]/all[,"Median Loss"]) )
    colnames(wl) <- c("Mean W/L","Median W/L")

    all <- cbind(all,wl)
    return(all)
    }

    print(tradeStats(sig,ret))
hero member
Activity: 532
Merit: 500
Here is the EMA10/21 crossover using hourly data instead of daily.  This is still all buy or all sell.

I also found  how to get the number of trades.  

Code:
# This function gives us some standard summary
# statistics for our trades.
tradeStats <- function(signals, returns) {
# Inputs:
# signals : trading signals
# returns : returns corresponding to signals

# Combine data and convert to data.frame
sysRet <- signals * returns * 100
posRet <- sysRet > 0 # Positive rule returns
negRet <- sysRet < 0 # Negative rule returns
dat <- cbind(signals,posRet*100,sysRet[posRet],sysRet[negRet],1)
dat <- as.data.frame(dat)

# Aggreate data for summary statistics
means <- aggregate(dat[,2:4], by=list(dat[,1]), mean, na.rm=TRUE)
medians <- aggregate(dat[,3:4], by=list(dat[,1]), median, na.rm=TRUE)
sums <- aggregate(dat[,5], by=list(dat[,1]), sum)

colnames(means) <- c("Signal","% Win","Mean Win","Mean Loss")
colnames(medians) <- c("Signal","Median Win","Median Loss")
colnames(sums) <- c("Signal","# Trades")

all <- merge(sums,means)
all <- merge(all,medians)

wl <- cbind( abs(all[,"Mean Win"]/all[,"Mean Loss"]),
abs(all[,"Median Win"]/all[,"Median Loss"]) )
colnames(wl) <- c("Mean W/L","Median W/L")

all <- cbind(all,wl)
return(all)
}

print(tradeStats(sig,ret))

Code:
 Signal # Trades    % Win  Mean Win  Mean Loss Median Win Median Loss  Mean W/L Median W/L
1     -1       83 51.80723 0.8337976 -0.8799018  0.4516124  -0.5958119 0.9476030  0.7579782
2      0     4249  0.00000       NaN        NaN         NA          NA       NaN         NA
3      1       83 46.98795 0.6240971 -1.4239146  0.4776158  -0.5503347 0.4382968  0.8678642

Here are the graphs.

The equity curve graph with long (green) and short (red).  ***EDIT*** This graph's title should say EMA10/21, not EMA5/21



I tried to plot the EMA10 and 21 on here but the time span is so large it is hard to see.



Here is the cumulative return along with hourly return and draw-down.  Surprisingly or not it is the same as the other daily data set.

hero member
Activity: 532
Merit: 500

... when I run the data there is only 1 trade at the beginning of December.  So what to do?  I can change the data to hourly.


do you suppose if you ran the flippest method many times and averaged the result at each point in time, you'd end up with a 'returns' chart that was pretty-much the same as the market price?

I would expect in a trending market the ratio would be about breaking even, but I never looked before so I can only guess.
hero member
Activity: 532
Merit: 500
To convert the data to hourly is pretty easy.  There is some function in one of the quantitative financial packages in R.  But I forget which one.  But that one function I had before can do the same thing with only a slight modification.

Code:
       
ohlc <- function(ttime,tprice,tvolume,fmt) {
             ttime.int <- format(ttime,fmt)
             data.frame(time = ttime[tapply(1:length(ttime),ttime.int,function(x) {head(x,1)})],
             mtgoxUSD.Open = tapply(tprice,ttime.int,function(x) {head(x,1)}),
             mtgoxUSD.High = tapply(tprice,ttime.int,max),
             mtgoxUSD.Low = tapply(tprice,ttime.int,min),
             mtgoxUSD.Close = tapply(tprice,ttime.int,function(x) {tail(x,1)}),
             mtgoxUSD.Volume = tapply(tvolume,ttime.int,function(x) {sum(x)}),
             mtgoxUSD.Adjusted = tapply(tprice,ttime.int,function(x) {tail(x,1)}))
             }

data <- ohlc(data$time,data$price, data$volume,"%Y%m%d%H") #converts data in CSV to OHLC,[b] notice the %H[/b]

The original data is taken from bitcoincharts and then put through this function.  The output is hourly data.

I also modified the code so that only crossovers indicate buy and sell signals.  Here is the new code for the EMA10/EMA21 crossover.

***EDIT*** Named the graph wrong.

Code:
   library(quantmod)
    library(TTR)
    # Load presorted data
    x = last(y,4436) #gets the last 4436 hourly positions from the data

    # Calculate the EMA indicators
    x$ema10 <- EMA(Cl(x),10)
    x$ema21 <- EMA(Cl(x),21)
    x$ema10.old = as.double(lag(x$ema10))
    x$ema21.old = as.double(lag(x$ema21))

    #omit NA's
    x = na.omit(x)

    # Create the long and short signals
    sigbuy = ifelse ((x$ema10 > x$ema21) & (x$ema10.old < x$ema21.old), 1, 0)  #buy signal.  yesterday's EMA10 was below yesterday's EMA21 and today it crossed over
    sigsell = ifelse ( (x$ema10 <= x$ema21) & (x$ema10.old > x$ema21.old), -1, 0) #sell signal.  yesterday's EMA10 was above yesterday's EMA21 and today it crossed under

    # Lag signals to align with days in market,
    # not days signals were generated
    sigbuy <- lag(sigbuy,1) # Note k=1 implies a move *forward*
    sigsell <- lag(sigsell,1) # Note k=1 implies a move *forward*

    # Replace missing signals with no position
    # (generally just at beginning of series)
    sigbuy[is.na(sigbuy)] <- 0
    sigsell[is.na(sigsell)] <- 0    

    # Combine both signals into one vector
    sig <- sigbuy + sigsell

    # Calculate Close-to-Close returns
    ret <- ROC(Cl(x))
    ret[1] <- 0

    # Calculate equity curves
    eq_up <- exp(cumsum(ret*sigbuy))
    eq_dn <- exp(cumsum(ret*sigsell*-1))
    eq_all <- exp(cumsum(ret*sig))

    # Equity Chart
    png(filename="emacrossnew.png",width=720,height=720)
    plot.zoo( cbind(eq_up, eq_dn), plot.type="single", ylab=c("Long","Short"), col=c("green","red"), main="EMA10/21 Crossover Strategy:\n 2011-07-23 to 2012-01-22" )
    dev.off()

    # Create a chart showing mtgoxUSD
    png("EMAcrosschart-new.png",width=720,height=720)
    chartSeries(x, subset="last 4415 hours", type="line")
    
    # Add the total equity line and EMA lines
    addEMA(n=c(10,21),col=c("orange","blue") )
    addTA(eq_all)
    dev.off()

    # Evaluate the Strategy
    # install.packages("PerformanceAnalytics")
    require(PerformanceAnalytics)
    
    # chart equity curve, daily performance, and drawdowns
    png("performance-EMAcrossnew.png",height=720,width=720)
    charts.PerformanceSummary(ret)
    dev.off()
mav
full member
Activity: 169
Merit: 107

... when I run the data there is only 1 trade at the beginning of December.  So what to do?  I can change the data to hourly.


interesting 'predicament' but I guess there's no arguing with the notion of 'don't buy' during that big fall, and 'do buy' when confidence came back... I guess we're all action junkies at heart huh?

I find it very interesting that no matter the strategy, essentially you end up following the general market trend no matter what. Of course there are some differences between strategies, so the end result is 'one strategy is relatively better than the other', although when both loose it really reflects the market more than the strategy I reckon.

Also it's interesting that this only works for 'trivial' amounts, as anything more than that would have affected the market and the historical data becomes moot.

Great analysis, it's nice to see a bit of coding to pass the time.

edit:

do you suppose if you ran the flippest method many times and averaged the result at each point in time, you'd end up with a 'returns' chart that was pretty-much the same as the market price?
hero member
Activity: 532
Merit: 500
I was going to modify the buy sell signals of the EMA10/EMA21 crossover strategy to:

Code:
   sigbuy = ifelse ((x$ema10 > x$ema21) & (x$ema10.old < x$ema21.old), 1, 0)  #buy signal.  yesterday's EMA10 was below yesterday's EMA21 and today it crossed over
    sigsell = ifelse ( (x$ema10 <= x$ema21) & (x$ema10.old > x$ema21.old), -1, 0) #sell signal.  yesterday's EMA10 was above yesterday's EMA21 and today it crossed under

Which means, if yesterday's EMA10 is below yesterday's EMA21 AND EMA10 is greater than or equal to EMA21 today then buy

Also, if yesterday's EMA10 is above yesterday's EMA21 AND EMA10 is less than or equal to EMA21 today then sell

But when I run the data there is only 1 trade at the beginning of December.  So what to do?  I can change the data to hourly.

hero member
Activity: 532
Merit: 500
The EMA 10 / 21 Crossover Strategy is:
1.  Buy when EMA 10 crosses over EMA 21.  So EMA 10 > EMA 21, BUY
2.  Sell when EMA 10 crosses below EMA 21.  So EMA 10 < EMA 21, SELL

I think most systems based on SMA and EMA crossovers (eg GMMA) give 'buy' or 'sell' signal at the actual point of crossover - ie when (in this case) EMA10==EMA21. The rest of the time they indicate a hold. You shouldn't be changing positions that often.

Edit: Nice chartage though. What package are you using for the bottom one?



The bottom chart comes from PerformanceAnalytics, the middle is from quantmod, and the first is from from the zoo package.

I will try this for the signals.

Code:
    sigbuy = ifelse ((ema10 >= ema 21) & (ema10.old < ema21.old), 1, 0)  #buy signal.  yesterday's EMA10 was below yesterday's EMA21 and today it crossed over
    sigsell = ifelse ( (ema10 =< ema 21 & (ema10.0ld > ema21.old), -1, 0) #sell signal.  yesterday's EMA10 was above yesterday's EMA21 and today it crossed under
Pages:
Jump to: