Topic: The Bot that was and will not ever be.... (Read 2827 times)

Also... bots don't sleep, they don't get bored and bots have much quicker reaction times.

That almost sounds satirical to me.  All trading is gambling, whether done by people or by bots.  Some people are better than others at it.  The difference with bots is that they have no emotions.

> A trading bot is limited by the stochastic process.

An important note is that if the stochastic process is not stationary, you cannot use past trades to predict future trades, and any testing of the bot using historical data is meaningless.

Is technical analysis scientific at all? Are there any peer-reviewed journals on this topic? Are there any scientifical proofs (I mean publications) or disproofs of stationarity of real markets?

My analysis:

1. Why are you wasting your effort on the BTC market? Go try it on Forex.

2. Testing over the course of 10 days, or whatever, means nothing. NOTHING. Test it on years of data.

3. Using a genetic algorithm would really require you to continue breeding on a monthly basis. Here's how it works:

• Pick a one year period three years in the past. Breed generations on that period until you have a fit set of bots. Let them then trade the succeeding six months. If they don't survive, then go back to the drawing board.
• Now advance the clock six months and repeat. Go back to the drawing board if they fail.
• Keep advancing the clock until the current time. Let the winners trade.
• Keep breeding going forward.

4. Picking local minima is the wrong way to trade. Especially when you move into markets that allow leverage. There is no reason to pick bottoms. What you want to do is pick freight trains that are moving and jump on board. It doesn't matter that the train began moving in Philadelphia and you're in Kansas. You just want the train to keep going west when you get on board.

I can only guess that what you've done is pick a set of past data to test on, and instituted an inherent bias into your testing.  The graph you've got shows July 6 to 11.  Given this period, we can easily choose a line which the price will cross over and under several times.  This knowledge is inherently biased- from today forward, we have no indication that an arbitrary line will be crossed.

What you're suggesting is that hidden signals in the data can choose this, and it simply isn't that easy.

This is important lest someone tries to do this and loses their shirt.  HFT makes money in a diverse set of well understood principles of existing exchanges- not the process you've described. On the other hand, stock bots like this do not make money.  What you're describing is closer to the many stock bot scams which promise suckers a return and never perform better than chance.


Forget about volume, if you can consistently turn around an average 2.5% every day, on a 5btc pot per day, I would pay $80k for the algo.  Not gonna happen though.

In rereading what I wrote, I thought I should clear this up.

In the case of an algorithmic trading bot, that is trained based on past market behavior, there is a level of prediction involved.  The prediction is not in WHAT the market will do , but WHICH SIGNALS and at WHAT LEVELS those signals need to be at to decide an action.

It isn't "Buy now because the price will go up" , it is "buy now because it appears the market may be moving in such a direction to indicate that the short term minima is approaching"

The difference is subtle, but there.

Also, these signals change... hence I had the_brain and the_brawn... sticking with a single set of trigger values and trading behavior would result in poor long term behavior.  

If i were to continue working on it, one of the first things to do is to change the way the_brain decides it's actions... right now, it picks the best algorithm and signal values based on the previous 12 hours of the market... it would be better to base the decision on hours -13 to -1 and then test the top 100 results agains hour -1 to 0.  That might give a tighter response.

After a certain point, the bot development was only intellectual exercise due to the fact that it's obvious that the market can not take the kind of volume needed to make the kind of money that would make it interesting.
  

Initially, I was not inclined to defend myself against your first response, because it was naive and a knee jerk response to the post.

Your second post on the other hand is slightly more interesting due to the introduction of the fact that the market is intrinsically unpredictable.

You are correct... the market is by it's very nature unpredictable and there are many external and internal pressures that affect the price and movement of the btc->usd market.

Unlike an analyst who tries to PREDICT what the market will do by looking at past performance, external factors, and various indicators and comparing that to other, similiar products, a trading bot is REACTIVE.

In the simplified examples I gave of how a trading bot works, do you see at any point where it says "buys shares based on prediction that it will raise above X value"?  The answer is no.  What I said it that based on signals of HOW THE MARKET IS BEHAVING, it reacts and performs an action being one of 3 base reactions: buy, sell or hold.  There is a little complexity in how it performs the buy and sell actions ( being unit buy, unit multibuy, max buy, unit sell, unit multisell, or hard stop )

In testing, and in the couple of days I performed live trades, it averaged 2.5% increase from the daily starting value.  Some days it dropped , some days it did better.

The real issue is not in weither it can sustain a 2.5% growth, the real issue is can the MARKET scale to allow it to do so.



apetersson and rizzn have both hit the nail on the head.  A trading bot , with a little luck , could hold 2.5% gain on 200$ ( as tested ) every day without any issue... the market can absorb the < 5btc trades that the bot executes ( and in some cases < .2btc ) without being affected.

My bot "hard bought" and "hard sold" ( being buy's set at 100$usd and sells at 0.01usd to ensue that they executed completely since MTGox fills orders by consuming outstanding orders in value order... ie my 100$buy would first buy every btc it could get its paws on at 14.0001, then 14.0005, etc. , and sales working in the inverse fashion )

If you wanted to use a trading bot with an account funded with 10000$ , RIGHT NOW if I sold 260 btc ( not even 3500$ worth ) , according to the depth charts it would move the price down $0.05usd... thats the problem... you can not move any sort of volume needed to scale the bot to make the return on that 2.5% growth worth it, or sustain it.

Am I saying my bot would continue to make 2.5% a day?  No... I said that it MADE 2.5% a day average based of a tiny trading account and a very short period of time... if algorithmic trading did not make money, HFT would not exist... do they make 2.5% a day?  of course not... do they have weeks where they DO make 2.5% a day? Yes.
 
If I thought it could hold 2.5% a day, and the market could BEAR the growth involved there, I wouldn't be posting here... I would be laughing as I watch the zeros get added to my account.

Noted. I edited the post to have a big "warning" up top

Thanks for pointing this out.

It isn't about volume, a 2.5% per day increase simply isn't possible.  You can't do it with pennies either.

Its like saying you built an electric car.  Cool, I'm following you.  But then when you say it has 1000km range everyone stops listening (or you should).  An electric car is has a hard limit by the joules per gram in your battery.  A trading bot is limited by the stochastic process.

for those of us who skim over stuff without taking the time to comprehend, you might want to clarify that it is this market trading bot you are killing and not your mining monitor project.

I've considered that - I've been scaling my personal investment in BTC, and using standard trading practices it's impossible to make money without engaging in market making to a certain extent.

There's plenty of information on marketmaking, though, and using techniques usually reserved for multi-billion or multi-trillion dollar hedgefunds on something like isn't boring to me. I'll probably continue to work on my own bot - but I love that you've done a lot of research for me here. Thanks for baring the kimono.

i think the point is
he cannot scale the volume up since it would distort the market and thus the bot would stop working..

I came to the same conclusion as you, bots work but volume is not important enough to make them really profitable...

so you are killing your 2.5% a day super bot ... oh yeah
what a pity 

Proof?


How is 2.5% per _day_ not significant?  This would make you $80,000 on a $10 investment in one year.

I call bullshit.  Anyone can make a trading bot.  Making a bot that profits 2.5% per day will not happen.

I'm too busy writing other Bitcoin related projects to write a 6,000 line trading bot in Perl (or any other language).

Ha!

It's not even 6k lines counting my hand rolled framework ( it's all in Perl, so there is > 100k lines of cpan modules, but that's not something I had to write ) ... anyone could write it themselves

Heck, you should write it yourself... it's a good learning experience.

Pete

I forgot...

These trading algorithms were based more around a HFT methodology... HFT is more "fun" from a simulation and live standpoint since there are lots of smaller trades going on...

I had a version called "the_tortoise" that was also profitable, but it was a longer term , multi day holds and stops ... just boring to look at and simulate

Peter Azuolas

Source code?

For clarification, this post has to do with me killing my trading bot project, and nothing to do with MiningMonitor itself.  MiningMonitor is doing very well for us, and if nothing else, we use it ourselves for our miners

After fiddling around, I think it's time to show off, then shelve my pet project.

For those who don't read the Mining boards, I am the lead developer behind MiningMonitor.com ( Board Post ).

---------------

Starting as a possible feature of MiningMonitor, I developed an advanced interface to MTGox:

http://f.cl.ly/items/1O2c3H1N3V0Y1y1X0Q3B/Screen%20shot%202011-07-13%20at%205.08.44%20PM.png
( Yes, High Stop/Low stop aren't real terms... one is a plain sell order "dark" since it's not an outstanding order on MTGox, and and the other is a simple stop order )

Technically the screenshot above shows you a very simple bot.  If the price is above X or below Y, sell.

Going from there, it almost seemed logical to write a bot. Simple trading bots are a breeze to write... they are really just based around 4 rules

1) Check spot if time since last check is > window
2) If price is below x, buy one unit of btc, note actual price paid ( for easy math, you can buy 1.003btc so 1btc get put in your account )
3) If price is above Y , sell one unit of btc given that spot_price = pricepurchased + (pricepurchased * desired gain )
4) Set a stop to prevent unbound loss ( either overall, or on a per unit purchased basis )

With a little manipulation of X and Y, it's easy to turn a small profit...

A more complex bot makes buying and selling discussions based on various indicators, EMA, WMA, HMA, Ease of movement, the MACD, my personal favorite for the current market RSI, etc.  Using an indicator based bot allows for maximizing the margin since mins and maxs are identified by the signals

1) Assuming a certain timewindow ( could be a per trade window if you want ) , get low, high, total volume, open, close
2) Calculate indicators

Assuming for this example we are using RSI , though in reality, multiple combined indicators give you your buy and sell singnals

3) if RSI is < 30, buy a unit of btc  , note effective price paid

4) if RSI is > 70, sell unit of btc given that spot_price > pricepurchased + desired minimum gain per transaction

5) Set stops

---

Both of these examples are simplified for brevity.


---

So, once I had a simple bot, I started thinking of a more complex bot. But , given the number of possible signals, and the number of ways to tune them, what is a programmer to do?

Use a genetic algorithm of course

http://f.cl.ly/items/290Q2o3l3k1g3S0M0r3t/Screen%20shot%202011-07-15%20at%205.21.34%20PM.png

And if you're doing that, you need a way to visualize the results

http://f.cl.ly/items/1W2O0Y3X1Q0C462B3N1w/Screen%20shot%202011-07-12%20at%206.02.19%20PM.png

----

Writing a higher level bot/genetic algorithm is again, not that difficult to do:

  

sub fitness {
   # load the data
   foreach my $trade (@$ref) {
      # determine the current values of your indicators
      # pass data to algorithm which decides what to do
      # perform action ( buy, sell, hold )
      # repeat
   }
   my $final_networth = $btc * $spot_price + $cash;
   my $performance = ($final_networth-$start_networth)/$start_networth*100;
   print "Genome: $genome Fitness: $final_networth Performance: $performace\%\n";
   return $final_networth;
}


With your algorithm, or trade bot, deciding what it should do based on the data passed in

I developed a number of different trading algorithms:

hurrr_i_am_a_bot
force_1btc
let_the_btc_hit_the_floor
tiny_bubbles
fuck_it_all_RSI_alone
kitchen_sink
( there were a couple of more, but I took them out back and shot them in the head ... they had fundamental flaws in their designs )

Each had it's strength and weaknesses.... for example fuck_it_all_RSI_alone loves a market like the one we are in... very small movements , lots of bouncing around in a constricted range.

kitchen_sink is the most adaptive, and can live in any market since it uses all the indicators, but it needs to be tweaked alot to stay profitable.

----

So the actual trading bot has two parts:

the_brain
the_brawn

Every hour or so , the_brain  takes the last 12 hours of trading data, uses genetic algorithms to decide which trading algorithm and settings for said algorithm works best, and tells the_brawn to use them.  This allows for automated switching based on current market conditions to a trading algorithm.

The_brawn just reads the live trading data, applies the various metrics to the data, and feeds the data, and settings to the algorithm the_brain decided on, takes back a market order ( buy, sell, twiddle thumbs ), pushes it off to mtgox if needed and repeats.

-----

What is the end result?  It's profitable... ~2.5% a day fully automated... but I'm killing off the project

-----

But Pete, that sounds great! Why kill the project?

Couple of reasons....

1) the_brain & the_brawn are a little fragile, it would take a decent amount of work to make it usable/shareable

2) Security, if I wanted to let people have access to the bot, they need to give me the MTGOX username/password ... so the original plan was to simply charge $XX per month and give each person their own VPS loaded with DB and software... that way even if one DB got penetrated, everyone's user/pass wasn't exposed

3) Bored now... I've taken it to the point where beyond this would require a much higher understanding of finance and signals

4) Value: Dudes, and dudettes, it's not worth it... the market is too small

---

Number 1 is laziness, 2 is paranoia, 3 is just a fact, but number 4 is the real reason.

The market for BTC has such low volume, and is of such a small size, that a bot that could make any real money would actually affect the market in doing so... since there are no Fill or Kills on MTGox , trying to buy/sell units of BTCs > ~5 at any certain times can move the price of BTC by pennies ( fractions of a percent ).

My personal bot running has been averaging around 2.5% a day, but that's peanuts... 2.5% of 300$ is $7.50 a day... an hour of freelance work makes me 6->14 TIMES that ...  

And since I've satisfied my intellectual curiosity... I thought I would share with you folks a post mortem, and give some direction to people screwing around with the idea of a bot some pointers.


Hope you enjoyed reading,

Peter Azuolas

tl;dr: Developed interface to MTGox, built simple trade bot, built genetic algorithm test bed for trading algoritms, turned it into a live bot, make 2.5% a day, killing project because intellectually satisfied and not worth it anymore