Topic: ncurses based MtGox live monitor and trading-bot-framework - page 8. (Read 33853 times)
The channel #cryptocoin-bots is quiet at the moment, but was used for such discussion a while ago....
Maybe we could make a separate forum thread where people post their bots and discuss them.
That's a good idea.
(half the distance leads to 1/4 the profit but 4 times more trades [and more mtgox fees, so it would actually be less profitable]).
Correct me if I am wrong but Mtgox charges a 0.6% fee per trade so it makes little difference whether you make large* or small ones. But, as you say, you do need to ensure that you make approximately >(0.6*2)% profit on trades or you will simply be slowly giving away your account to Mtgox in the form of fees.* Large being relative here.
If it does work automatically, how does it decide at which point to "balance out"
It will place orders in the distance of 7% price change. So for example if price is at $100/BTC and you have 1000USD and 10BTC in your account is exactly balanced, it would place a sell order 0.327 at 107 and a buy order 0.376 at 93. Then it will wait until price moves and one of these orders is filled and automatically cancel the other and place 2 fresh orders in +- 7% distance around the new price. Thats all. It should happen automatically. BUT it will only work if you don't confuse the logic by having other open orders in your account! It will try to tell bot orders from manually entered orders by looking at the last digit of the order price but thats only an ugly hack! Make sure you dont have any other open orders whose price ends with "7" (or whatever you configured the value of MARKER)!
Standard disclamer for algorithmic trading: If you don't understand the logic of the bot by reading the source code and maybe inserting some more debug output then please don't use it. Please don't ask me, you cannot run a bot if you are not able to debug it yourself! That is simply not possible!
If the distance (default = 7%) is chosen too narrow then it might need to place orders faster than mtgox lag allows and it might come out of sync.
If it is messed up completely (happens maybe once a week) press c to cancel all orders and if it won't cancel them all or is behaving strange then cancel them manually, maybe restart goxtool or press u to get a manually updated list of open orders, use F6 to check your open orders to make sure they are all canceled. then press i to see how far you are out of balance at current price and if needed press b to rebalance manually, press p to place 2 fresh orders.
And as i have mentioned before already: make sure you ALWAYS have an up to date version of goxtool. git pull (and restart if there were updates) at least once a day, I'm still fixing things in goxtool itself occasionally that make it more reliable.
prof7bit, I didn't try out the balance bot, but from the description it seems doesn't work automtically but you have to tell it to place orders manually?
If it does work automatically, how does it decide at which point to "balance out" (i.e. how much divergence from 50/50 does it tolerate)? (I've written a similar bot in my traidor "framework". Never got it to fully work, even after 2nd try, I kept scrwing it up, then came the api change at gox (this was quite a while ago) but the goal was the same. I thought about putting the asks/bids just below/above walls. Also it didn't place just 2 bids, but a whole array all the way up/down so it wouldn't run into problems with lag)
If it does work automatically, how does it decide at which point to "balance out" (i.e. how much divergence from 50/50 does it tolerate)? (I've written a similar bot in my traidor "framework". Never got it to fully work, even after 2nd try, I kept scrwing it up, then came the api change at gox (this was quite a while ago) but the goal was the same. I thought about putting the asks/bids just below/above walls. Also it didn't place just 2 bids, but a whole array all the way up/down so it wouldn't run into problems with lag)
I just tested the balancer.py now, and i must say it had a huge spread.
Had about 7BTC, divided in two, and got 433 USD in cash. Current price is 120.
Started the bot with p, and it put in a sell at 129.03137, and a buy at 112.14877
Had about 7BTC, divided in two, and got 433 USD in cash. Current price is 120.
Started the bot with p, and it put in a sell at 129.03137, and a buy at 112.14877
Yes, its a compromise between how often should it trade and how big should be the effect of the the rebalancing (the profit after a consecutve buy and sell). I cannot mathematically prove it but my gut feeling tells me in a normally trading market the overall profit should be the same (half the distance leads to 1/4 the profit but 4 times more trades [and more mtgox fees, so it would actually be less profitable]).
Faster trades also means more problems when mtgox becomes slow (order lag) or disconnects often during fast price moves. You can experiment with smaller numbers by changing the value of DISTANCE, don't make it smaller than 1.2, at 1.2 you will pay 100% of your profits as mtgox fees, maybe 2 would be a good minimum DISTANCE to experiment with when the market is slow but I guarantee that as soon as there are sudden large price moves mtgox will start lagging at exactly the wrong times and it won't be able to cancel and place new orders fast enough.
This bot is not intended to make you rich quickly, you need to run it for months to see an effect. Write down the BTC price and your account balance once a day and plot the curves on top of each other after a few months. You could also program a simulation if you find historical price data to experiment with different distances, I haven't done this yet.
I just tested the balancer.py now, and i must say it had a huge spread.
Had about 7BTC, divided in two, and got 433 USD in cash. Current price is 120.
Started the bot with p, and it put in a sell at 129.03137, and a buy at 112.14877
Had about 7BTC, divided in two, and got 433 USD in cash. Current price is 120.
Started the bot with p, and it put in a sell at 129.03137, and a buy at 112.14877
Do you have the bot in a separate git repo that can be tracked?
No, not yet. Would it be possible to add a slot to the strategy so that on a gox reconnection the strategy can then re-asses if anything needs to be done (a cancel and re-place orders in the case of the balancer).
Yes, something like that is on my todo list because i have exactly the same problems with the balancer.bot. As a workaround I have slightly modified the _balancer.py code to also trade when it suddenly detects only one order while the trade signal has been missed:
Thanks for this, esp the 'b' command I just deposited this morning and would have been very helpful.
Do you have the bot in a separate git repo that can be tracked?
For those playing along at home, here is a command summary & launch instructions for prof7bit's balancer bot:
Code:
# "p": place 2 orders above and below price.
# "c": cancel orders and effectively stop the bot
# "u": update own order list, depth, history, wallet and everything
# "i": display how much it is out of balance at current price (negative: must sell, positive: must buy)
# "b":balance immediately (cancel orders and then buy/sell at market to rebalance)
./goxtool.py --protocol=socketio --use-http --strategy=_balancer.py
# "c": cancel orders and effectively stop the bot
# "u": update own order list, depth, history, wallet and everything
# "i": display how much it is out of balance at current price (negative: must sell, positive: must buy)
# "b":balance immediately (cancel orders and then buy/sell at market to rebalance)
./goxtool.py --protocol=socketio --use-http --strategy=_balancer.py
Would it be possible to add a slot to the strategy so that on a gox reconnection the strategy can then re-asses if anything needs to be done (a cancel and re-place orders in the case of the balancer).
Yes, something like that is on my todo list because i have exactly the same problems with the balancer.bot. As a workaround I have slightly modified the _balancer.py code to also trade when it suddenly detects only one order while the trade signal has been missed:
Code:
"""
The portfolio rebalancing bot will buy and sell to maintain a
constant asset allocation ratio of exactly 50/50 = fiat/BTC
"""
import strategy
import goxapi
DISTANCE = 7 # percent price distance of next rebalancing orders
MARKER = 7 # lowest digit of price to identify bot's own orders
COIN = 1E8 # number of satoshi per coin, this is a constant.
def add_marker(price, marker):
"""encode a marker in the price value to find bot's own orders"""
return price / 10 * 10 + marker
def has_marker(price, marker):
"""return true if the price value has the marker"""
return (price % 10) == marker
def mark_own(price):
"""return the price with our own marker embedded"""
return add_marker(price, MARKER)
def is_own(price):
"""return true if this price has our own marker"""
return has_marker(price, MARKER)
class Strategy(strategy.Strategy):
"""a protfolio rebalancing bot"""
def __init__(self, gox):
strategy.Strategy.__init__(self, gox)
self.last_trade = 0
self.temp_halt = False
def slot_before_unload(self, _sender, _data):
pass
def slot_keypress(self, gox, (key)):
"""a key has been pressed"""
if key == ord("c"):
# remove existing rebalancing orders
self.debug("canceling all rebalancing orders")
self.temp_halt = True
self.cancel_orders()
if key == ord("p"):
# create the initial two rebalancing orders. Before you
# do this the portfolio should already be balanced.
self.debug("adding new initial rebalancing orders")
book = self.gox.orderbook
self.temp_halt = False
self.place_orders((book.bid + book.ask) / 2)
if key == ord("u"):
gox.client.channel_subscribe()
if key == ord("i"):
price = (gox.orderbook.bid + gox.orderbook.ask) / 2
vol_buy = self.get_buy_at_price(price)
self.debug("BTC difference:",
goxapi.int2float(vol_buy, "BTC"))
if key == ord("b"):
price = (gox.orderbook.bid + gox.orderbook.ask) / 2
vol_buy = self.get_buy_at_price(price)
if abs(vol_buy) > 0.01 * COIN:
self.temp_halt = True
self.cancel_orders()
if vol_buy > 0:
self.debug("buy %f at market" %
goxapi.int2float(vol_buy, "BTC"))
gox.buy(0, vol_buy)
else:
self.debug("sell %f at market" %
goxapi.int2float(-vol_buy, "BTC"))
gox.sell(0, -vol_buy)
def cancel_orders(self):
"""cancel all rebalancing orders, we identify
them through the marker in the price value"""
must_cancel = []
for order in self.gox.orderbook.owns:
if is_own(order.price):
must_cancel.append(order)
for order in must_cancel:
self.gox.cancel(order.oid)
def get_buy_at_price(self, price_int):
"""calculate amount of BTC needed to buy at price to achieve
rebalancing. price and return value are in mtgox integer format"""
currency = self.gox.currency
fiat_have = goxapi.int2float(self.gox.wallet[currency], currency)
btc_have = goxapi.int2float(self.gox.wallet["BTC"], "BTC")
price_then = goxapi.int2float(price_int, currency)
btc_value_then = btc_have * price_then
diff = fiat_have - btc_value_then
diff_btc = diff / price_then
must_buy = diff_btc / 2
return goxapi.float2int(must_buy, "BTC")
def place_orders(self, center):
"""place two new rebalancing orders above and below center price"""
currency = self.gox.currency
step = int(center * DISTANCE / 100.0)
next_sell = mark_own(center + step)
next_buy = mark_own(center - step)
sell_amount = -self.get_buy_at_price(next_sell)
buy_amount = self.get_buy_at_price(next_buy)
if sell_amount < 0.01 * COIN:
sell_amount = int(0.01 * COIN)
self.debug("WARNING! minimal sell amount adjusted to 0.01")
if buy_amount < 0.01 * COIN:
buy_amount = int(0.01 * COIN)
self.debug("WARNING! minimal buy amount adjusted to 0.01")
self.debug("new buy order %f at %f" % (
goxapi.int2float(buy_amount, "BTC"),
goxapi.int2float(next_buy, currency)
))
self.gox.buy(next_buy, buy_amount)
self.debug("new sell order %f at %f" % (
goxapi.int2float(sell_amount, "BTC"),
goxapi.int2float(next_sell, currency)
))
self.gox.sell(next_sell, sell_amount)
def slot_trade(self, gox, (date, price, volume, typ, own)):
"""a trade message has been receivd"""
# not interested in other people's trades
if not own:
return
# not interested in manually entered (not bot) trades
if not is_own(price):
return
text = {"bid": "sold", "ask": "bought"}[typ]
self.debug("*** %s %f at %f" % (
text,
goxapi.int2float(volume, "BTC"),
goxapi.int2float(price, gox.currency)
))
self.last_trade = price
self.check_trades()
def slot_owns_changed(self, orderbook, _dummy):
"""status or amount of own open orders has changed"""
self.check_trades()
def check_trades(self):
"""find out if we need to place new orders and do it if neccesary"""
# bot temporarily disabled
if self.temp_halt:
return
# still waiting for submitted orders,
# can wait for next signal
if self.gox.count_submitted:
return
# we count the open and pending orders
count = 0
count_pending = 0
book = self.gox.orderbook
for order in book.owns:
if is_own(order.price):
if order.status == "open":
count += 1
else:
count_pending += 1
# as long as there are ANY pending orders around we
# just do nothing and wait for the next signal
if count_pending:
return
# if count is exacty 1 then one of the orders must have been filled,
# now we cancel the other one and place two fresh orders in the
# distance of DISTANCE around current price.
if count == 1:
price = self.last_trade
self.last_trade = 0
if not price:
price = (book.ask + book.bid) / 2
self.debug(
"*** missed trade message, using current price %f" %
goxapi.int2float(price, self.gox.currency)
)
self.cancel_orders()
self.place_orders(price)
The portfolio rebalancing bot will buy and sell to maintain a
constant asset allocation ratio of exactly 50/50 = fiat/BTC
"""
import strategy
import goxapi
DISTANCE = 7 # percent price distance of next rebalancing orders
MARKER = 7 # lowest digit of price to identify bot's own orders
COIN = 1E8 # number of satoshi per coin, this is a constant.
def add_marker(price, marker):
"""encode a marker in the price value to find bot's own orders"""
return price / 10 * 10 + marker
def has_marker(price, marker):
"""return true if the price value has the marker"""
return (price % 10) == marker
def mark_own(price):
"""return the price with our own marker embedded"""
return add_marker(price, MARKER)
def is_own(price):
"""return true if this price has our own marker"""
return has_marker(price, MARKER)
class Strategy(strategy.Strategy):
"""a protfolio rebalancing bot"""
def __init__(self, gox):
strategy.Strategy.__init__(self, gox)
self.last_trade = 0
self.temp_halt = False
def slot_before_unload(self, _sender, _data):
pass
def slot_keypress(self, gox, (key)):
"""a key has been pressed"""
if key == ord("c"):
# remove existing rebalancing orders
self.debug("canceling all rebalancing orders")
self.temp_halt = True
self.cancel_orders()
if key == ord("p"):
# create the initial two rebalancing orders. Before you
# do this the portfolio should already be balanced.
self.debug("adding new initial rebalancing orders")
book = self.gox.orderbook
self.temp_halt = False
self.place_orders((book.bid + book.ask) / 2)
if key == ord("u"):
gox.client.channel_subscribe()
if key == ord("i"):
price = (gox.orderbook.bid + gox.orderbook.ask) / 2
vol_buy = self.get_buy_at_price(price)
self.debug("BTC difference:",
goxapi.int2float(vol_buy, "BTC"))
if key == ord("b"):
price = (gox.orderbook.bid + gox.orderbook.ask) / 2
vol_buy = self.get_buy_at_price(price)
if abs(vol_buy) > 0.01 * COIN:
self.temp_halt = True
self.cancel_orders()
if vol_buy > 0:
self.debug("buy %f at market" %
goxapi.int2float(vol_buy, "BTC"))
gox.buy(0, vol_buy)
else:
self.debug("sell %f at market" %
goxapi.int2float(-vol_buy, "BTC"))
gox.sell(0, -vol_buy)
def cancel_orders(self):
"""cancel all rebalancing orders, we identify
them through the marker in the price value"""
must_cancel = []
for order in self.gox.orderbook.owns:
if is_own(order.price):
must_cancel.append(order)
for order in must_cancel:
self.gox.cancel(order.oid)
def get_buy_at_price(self, price_int):
"""calculate amount of BTC needed to buy at price to achieve
rebalancing. price and return value are in mtgox integer format"""
currency = self.gox.currency
fiat_have = goxapi.int2float(self.gox.wallet[currency], currency)
btc_have = goxapi.int2float(self.gox.wallet["BTC"], "BTC")
price_then = goxapi.int2float(price_int, currency)
btc_value_then = btc_have * price_then
diff = fiat_have - btc_value_then
diff_btc = diff / price_then
must_buy = diff_btc / 2
return goxapi.float2int(must_buy, "BTC")
def place_orders(self, center):
"""place two new rebalancing orders above and below center price"""
currency = self.gox.currency
step = int(center * DISTANCE / 100.0)
next_sell = mark_own(center + step)
next_buy = mark_own(center - step)
sell_amount = -self.get_buy_at_price(next_sell)
buy_amount = self.get_buy_at_price(next_buy)
if sell_amount < 0.01 * COIN:
sell_amount = int(0.01 * COIN)
self.debug("WARNING! minimal sell amount adjusted to 0.01")
if buy_amount < 0.01 * COIN:
buy_amount = int(0.01 * COIN)
self.debug("WARNING! minimal buy amount adjusted to 0.01")
self.debug("new buy order %f at %f" % (
goxapi.int2float(buy_amount, "BTC"),
goxapi.int2float(next_buy, currency)
))
self.gox.buy(next_buy, buy_amount)
self.debug("new sell order %f at %f" % (
goxapi.int2float(sell_amount, "BTC"),
goxapi.int2float(next_sell, currency)
))
self.gox.sell(next_sell, sell_amount)
def slot_trade(self, gox, (date, price, volume, typ, own)):
"""a trade message has been receivd"""
# not interested in other people's trades
if not own:
return
# not interested in manually entered (not bot) trades
if not is_own(price):
return
text = {"bid": "sold", "ask": "bought"}[typ]
self.debug("*** %s %f at %f" % (
text,
goxapi.int2float(volume, "BTC"),
goxapi.int2float(price, gox.currency)
))
self.last_trade = price
self.check_trades()
def slot_owns_changed(self, orderbook, _dummy):
"""status or amount of own open orders has changed"""
self.check_trades()
def check_trades(self):
"""find out if we need to place new orders and do it if neccesary"""
# bot temporarily disabled
if self.temp_halt:
return
# still waiting for submitted orders,
# can wait for next signal
if self.gox.count_submitted:
return
# we count the open and pending orders
count = 0
count_pending = 0
book = self.gox.orderbook
for order in book.owns:
if is_own(order.price):
if order.status == "open":
count += 1
else:
count_pending += 1
# as long as there are ANY pending orders around we
# just do nothing and wait for the next signal
if count_pending:
return
# if count is exacty 1 then one of the orders must have been filled,
# now we cancel the other one and place two fresh orders in the
# distance of DISTANCE around current price.
if count == 1:
price = self.last_trade
self.last_trade = 0
if not price:
price = (book.ask + book.bid) / 2
self.debug(
"*** missed trade message, using current price %f" %
goxapi.int2float(price, self.gox.currency)
)
self.cancel_orders()
self.place_orders(price)
there are also some new keyboard commands:
"u": update own order list, depth, history, wallet and everything
"i": display how much it is out of balance at current price (negative: must sell, positive: must buy)
"b": balance immediately (cancel orders and then buy/sell at market to rebalance)
Bug Report: please check if the terminal is a terminal emulation and not a real tty otherwise the CUI looks ugly because of this code
EDIT: im aware of the following code, still its not nice. altough its good that you can disable it with ease.
Code:
# update the xterm title (this is not handled by curses)
if self.gox.config.get_bool("goxtool", "set_xterm_title"):
last_candle = self.gox.history.last_candle()
if last_candle:
title = goxapi.int2str(last_candle.cls, self.gox.currency).strip()
title += " - goxtool -"
title += " bid:" + goxapi.int2str(book.bid, self.gox.currency).strip()
title += " ask:" + goxapi.int2str(book.ask, self.gox.currency).strip()
curses.putp("\033]0;%s\007" % title)
if you want i can provide a screenshot if self.gox.config.get_bool("goxtool", "set_xterm_title"):
last_candle = self.gox.history.last_candle()
if last_candle:
title = goxapi.int2str(last_candle.cls, self.gox.currency).strip()
title += " - goxtool -"
title += " bid:" + goxapi.int2str(book.bid, self.gox.currency).strip()
title += " ask:" + goxapi.int2str(book.ask, self.gox.currency).strip()
curses.putp("\033]0;%s\007" % title)
EDIT: im aware of the following code, still its not nice. altough its good that you can disable it with ease.
Code:
if self.gox.config.get_bool("goxtool", "set_xterm_title"):
@prof7bit
I noticed that sometimes with the balancer + gox connection issues sometimes the balancer gets out of whack. Last night with the spike at $130 was a prime example.
Would it be possible to add a slot to the strategy so that on a gox reconnection the strategy can then re-asses if anything needs to be done (a cancel and re-place orders in the case of the balancer).
I noticed that sometimes with the balancer + gox connection issues sometimes the balancer gets out of whack. Last night with the spike at $130 was a prime example.
Would it be possible to add a slot to the strategy so that on a gox reconnection the strategy can then re-asses if anything needs to be done (a cancel and re-place orders in the case of the balancer).
A very very basic overview on rebalancing would be that it protects you from volitilty of one asset.
and it is producing a small profit in the long run and outperforms a buy and hold strategy. This is because you are always selling higher than you have bought. Its only a small amount because only a tiny fraction of your BTC is bought or sold on every step but if BTC price is very volatile and does two digit percent changes every single day and huge ups and downs this can become significant quite fast.
Its a very conservative strategy and probably one of the few strategies that can cannot accidentally burn all your money in one night if the bot is crashing/hanging while you are sleeping.
Maybe we could make a separate forum thread where people post their bots and discuss them.
I run it just like this.
Code:
./goxtool.py --protocol=socketio --use-http
Thanks, that does seem to work a lot better.
Been running it for a week, it is surprisingly robust considering the problems with mtgox, and me suspending my laptop a couple of times per day.
What API configuration are you using? Do you trade or just use it to watch the market?
I installed it last weekend with git.
Code:
git clone https://github.com/prof7bit/goxtool.git
I run it just like this.
Code:
./goxtool.py --protocol=socketio --use-http
I tried other configurations, but I can't say anything good or bad considering the conditions. I wouldn't call it a controlled experiment in other words.
I traded manually with it this week a bit, and it worked fine. I changed the colors up a bit since I have 256 color xterm. Until today I was just entering trades manually but then I checked the thread and saw the _balancer so I did a git stash and a git pull and I have been playing with it this evening. Fun stuff, thanks to OP for posting the example. It's better (more complete) than what I was throwing together. A good starting point.
PS If anyone is interested, I found a book on Amazon http://www.amazon.com/Trading-Systems-Methods-Website-Wiley/dp/1118043561
Been running it for a week, it is surprisingly robust considering the problems with mtgox, and me suspending my laptop a couple of times per day.
What API configuration are you using? Do you trade or just use it to watch the market?
A very very basic overview on rebalancing would be that it protects you from volitilty of one asset.
Is this actually usable? socketio seems to drop constantly and lag horribly, especially on orders. websocket just sits there and doesn't update.
Been running it for a week, it is surprisingly robust considering the problems with mtgox, and me suspending my laptop a couple of times per day.
Quote
Please read about portfolio rebalancing to understand the implications and why you would want (or not want) to do this.
So, about that, got any good links or books to recommend? Googling for "portfolio rebalancing" gets me a bunch of noise about how I should own a diversity of stocks, bonds, and other garbage. Googling for "Forex rebalancing" is only a bit better.
Yea. The problem lies with mtgox though, not with the code.
Is this actually usable? socketio seems to drop constantly and lag horribly, especially on orders. websocket just sits there and doesn't update.
prof7bit, the people demand bitfloor support 
Belay that order, disregard.
Jump to: