Pages:
Author

Topic: Gekko - a javascript trading bot and backtesting platform - page 18. (Read 147899 times)

legendary
Activity: 1428
Merit: 1000
i've made a small web interface for it.

screenshot: https://www.dropbox.com/s/od0ac7djvvdyrps/gekkoweb.png
source: https://www.dropbox.com/s/54y2crazfnyghnd/gekko.zip

i am not familiar with npm or node. so you have to install a few dependencies by hand (eg npm install express).

there are currently some drawbacks:
 - portfolio is only updated after a successfull buy or sell
 - web interface does work as soon as a the first advice is made.
 - only ema does work as the web if does show some internal data from it

i use it in "advanced mode" using data from mtgox and trading on mtgox and btce (but only with small amounts)

i did not test it with other configurations, so use it at your own risk (i am not responsiple for any losses etc).

as this is a pure personal project i dont think i'll make other extensions, but i am always open for suggestions

regards
flower

Woow this looks great! I am pretty sure we can use a lot of your code and put it into Gekko (if you don't mind).

So we are working on a new version for Gekko that has a plugin system which makes it easy to build stuff on top of Gekko. The demo server I built uses this system already.

It also uses websockets in favor of a rest style API. Though I am pretty sure we can leverage your code quite a bit (especially the frontend stuff) Smiley

Awesome stuff!

just take what you need..

one suggestion for graphs and multiple indicators: let the indicator export the jqplot options and data (i am planning to do this myself to play with it).

i would really like to see a bitfinex exchange plugin. and ways to decide a risk strategy for margin trading (but the exchange part of bitfinex would be a nice first step)

btw great work. i really enjoyed working with it!
sr. member
Activity: 287
Merit: 250
i've made a small web interface for it.

screenshot: https://www.dropbox.com/s/od0ac7djvvdyrps/gekkoweb.png
source: https://www.dropbox.com/s/54y2crazfnyghnd/gekko.zip

i am not familiar with npm or node. so you have to install a few dependencies by hand (eg npm install express).

there are currently some drawbacks:
 - portfolio is only updated after a successfull buy or sell
 - web interface does work as soon as a the first advice is made.
 - only ema does work as the web if does show some internal data from it

i use it in "advanced mode" using data from mtgox and trading on mtgox and btce (but only with small amounts)

i did not test it with other configurations, so use it at your own risk (i am not responsiple for any losses etc).

as this is a pure personal project i dont think i'll make other extensions, but i am always open for suggestions

regards
flower

Woow this looks great! I am pretty sure we can use a lot of your code and put it into Gekko (if you don't mind).

So we are working on a new version for Gekko that has a plugin system which makes it easy to build stuff on top of Gekko. The demo server I built uses this system already.

It also uses websockets in favor of a rest style API. Though I am pretty sure we can leverage your code quite a bit (especially the frontend stuff) Smiley

Awesome stuff!
legendary
Activity: 1428
Merit: 1000
i've made a small web interface for it.

screenshot: https://www.dropbox.com/s/od0ac7djvvdyrps/gekkoweb.png
source: https://www.dropbox.com/s/54y2crazfnyghnd/gekko.zip

i am not familiar with npm or node. so you have to install a few dependencies by hand (eg npm install express).

there are currently some drawbacks:
 - portfolio is only updated after a successfull buy or sell
 - web interface does work as soon as a the first advice is made.
 - only ema does work as the web if does show some internal data from it

i use it in "advanced mode" using data from mtgox and trading on mtgox and btce (but only with small amounts)

i did not test it with other configurations, so use it at your own risk (i am not responsiple for any losses etc).

as this is a pure personal project i dont think i'll make other extensions, but i am always open for suggestions

regards
flower
sr. member
Activity: 287
Merit: 250
Yeah sounds about perfect, I'd want to to watch LTC/BTC, USD/BTC and USD/LTC.

I know it's a long shot but would a GUI be possible at any point? I can use it ok but there are a couple of things I'm unsure of how to do such as the historical testing.

Well those would be perfectly possible I think.

So this is the roadmap for Gekko:

0.1.0:

Quote
TODO before we can merge the localDB branch into Master (Gekko 0.1.0)

*(We only merge stable stuff into master).*

- Fix remaining bugs in the datastore (#106).
- Move filestore from neDB to our own.
- Finalize [actor API](https://github.com/askmike/gekko/blob/localDB/docs/internals/Actors.md)
- Add test cases for all vital parts of core
  - ~~Trade fetcher~~
  - ~~candle store~~
  - candle calculator
- Add test cases for all trading methods and indicators:
  - ~~EMA~~
  - EMAC
  - MACD
- Verify that all tests are even valid
- Add a trader actor (hook up to portfolioManager)
- make the Trade fetcher able to handle exchanges that do provide historical data (either `since` or `tid`).
- Add basic backtesting capabilities.

More info here and here.

After that all works we are planning some exciting stuff like a webbased GUI, sneak peak of my early early prototype (more of a proof of concept really):



The GUI is going to be the main interface in that you can backtest through here, manage histories, manage traders, manage profits, etc. Though this is going to take a while if I keep on developing this whole project on my own.
sr. member
Activity: 308
Merit: 250
I run on an amazon AWS free instance - single core windows VPS with 615MB RAM. Only have 13GB diskspace so under 2GB per currency pair would be fine.
Barely any RAM free though so it would have to be efficient swapping to disk or running off of 100MB.

Ok awesome that sounds reasonable. Do you have any preferences of how many markets you want to watch (in the case Gekko can trade every market on BTC-e at the same time in one algo for example)? Because a single market does not drain that much recourses.

So this is what Gekko is probably going to look like right now:

- persists about 100k / market data to disk (less than 40MB per year) - in 1 minute candles
- around ~30 MB of memory usage per market.

Yeah sounds about perfect, I'd want to to watch LTC/BTC, USD/BTC and USD/LTC.

I know it's a long shot but would a GUI be possible at any point? I can use it ok but there are a couple of things I'm unsure of how to do such as the historical testing.
sr. member
Activity: 287
Merit: 250
I run on an amazon AWS free instance - single core windows VPS with 615MB RAM. Only have 13GB diskspace so under 2GB per currency pair would be fine.
Barely any RAM free though so it would have to be efficient swapping to disk or running off of 100MB.

Ok awesome that sounds reasonable. Do you have any preferences of how many markets you want to watch (in the case Gekko can trade every market on BTC-e at the same time in one algo for example)? Because a single market does not drain that much recourses.

So this is what Gekko is probably going to look like right now:

- persists about 100k / market data to disk (less than 40MB per year) - in 1 minute candles
- around ~30 MB of memory usage per market.
sr. member
Activity: 308
Merit: 250
If anyone is interested in running Gekko, could you weigh in on the discussion with:

- On what machine do you want to run Gekko (desktop, laptop, raspberry PI or VPS/cloud instance)
- How big can the history be that Gekko stores on disk (in MB).
- How much memory can Gekko consume per market you want to watch (1-10MB, 10-100MB or more?)


I run on an amazon AWS free instance - single core windows VPS with 615MB RAM. Only have 13GB diskspace so under 2GB per currency pair would be fine.
Barely any RAM free though so it would have to be efficient swapping to disk or running off of 100MB.
newbie
Activity: 9
Merit: 0
Sounds great, would be awesome if you could offer it back as an alternative trading method. Just a couple of things to keep in mind with your suggestion:

1. If you never start with a sell and Gekko stops & restarts (crash, user turned off) after a buy if can't do anything until you manually sell.
2. While this sounds great in theory it can be dangerous: if the bot bought at ~1200 during the last peak it won't do anything ever anymore. Even though there is money to be made even though the price is < 1200.

> Right now I'm trying to configure the thresholds to make the bot work well with 5 minute candles since one hour is way too much.

If you set the interval in the config to 5, Gekko will calculate 5 minute candles instead of 60 minute. It will also recheck every 5 minutes and make a trade decision.

That's my point: No bots should be left unattended.  The "aggressive" option off is meant so you can decide for yourself, but the bot should still alert you (via email) that the downtrend has come so you can take a decision.

I insist: In my back tracking simulations the bot have made more profit if I let it lose money, but there are some simulated trades that I frankly would have panicked.  This makes me think about a thousand of alternatives, including letting the bot decide how much you can lose like configuring a percentage or something.  Good traders always lose money in some bad trades, but they always make good money in good trades, so it's part of the business.

Bitcoin trading using 5 minute candles based EMA is VERY TRICKY (so the ocasional trader reading this: please stick to the 60 minutes intervals), and looking for crossovers between 10 and 21 exponential moving averages in 5 minutes intervals just don't work; but there are some numbers that do Tongue

I love this bot! though I use it as an advisor that alerts me, I'm not using it like an automatic trader bot.
sr. member
Activity: 287
Merit: 250
Hi, this might be a bug:

Code:
2014-01-02 19:46:57 (INFO):     ADVICE is to BUY @ 768.000 (0.406)

/home/pi/arne/gekko/portfolioManager.js:153
  if(this.minimalOrder.unit === 'currency')
                      ^
TypeError: Cannot read property 'unit' of undefined
    at [object Object]. (/home/pi/arne/gekko/portfolioManager.js:153:23)
    at [object Object].bound [as getMinimum] (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at [object Object]. (/home/pi/arne/gekko/portfolioManager.js:169:22)
    at [object Object].bound [as buy] (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at [object Object]. (/home/pi/arne/gekko/portfolioManager.js:126:12)
    at bound (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:229:13
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:139:25
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:226:17
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:553:34

Trying to buy @ BTC-e.
Where should the minimalOrder be set?
Thanks for the good work (y)

Ah :/ will look at this Sad

This is definitely a great bot but it's kinda too agressive.  The "agressiveness" should be an option. 

What I mean is, this bot will always advice to sell even if it bought for a higher price before.  Even though in my testing is better to let the bot lose money, I think some of us prefer to take that decision manually.  So I modified it a little:

1.- Never start with a SELL.  Always start with a BUY.

2.- Never sell for a price lower than the last buy operation.  Hold until year 2140 (when the last bitcoin will be mined) if it's necessary.

Right now I'm trying to configure the thresholds to make the bot work well with 5 minute candles since one hour is way too much:  Many things happen in one hour in the bitcoin world.

Anyway, this is great work though.  Congrats.

Sounds great, would be awesome if you could offer it back as an alternative trading method. Just a couple of things to keep in mind with your suggestion:

1. If you never start with a sell and Gekko stops & restarts (crash, user turned off) after a buy if can't do anything until you manually sell.
2. While this sounds great in theory it can be dangerous: if the bot bought at ~1200 during the last peak it won't do anything ever anymore. Even though there is money to be made even though the price is < 1200.

> Right now I'm trying to configure the thresholds to make the bot work well with 5 minute candles since one hour is way too much.

If you set the interval in the config to 5, Gekko will calculate 5 minute candles instead of 60 minute. It wil also recheck every 5 minutes and make a trade decision.
newbie
Activity: 9
Merit: 0
This is definitely a great bot but it's kinda too agressive.  The "agressiveness" should be an option.  

What I mean is, this bot will always advice to sell even if it bought for a higher price before.  Even though in my testing is better to let the bot lose money, I think some of us prefer to take that decision manually.  So I modified it a little:

1.- Never start with a SELL.  Always start with a BUY.

2.- Never sell for a price lower than the last buy operation.  Hold until year 2140 (when the last bitcoin will be mined) if it's necessary.

Right now I'm trying to configure the thresholds to make the bot work well with 5 minute candles since one hour is way too much:  Many things happen in one hour in the bitcoin world.

Anyway, this is great work though.  Congrats.
newbie
Activity: 47
Merit: 0
Hi, this might be a bug:

Code:
2014-01-02 19:46:57 (INFO):     ADVICE is to BUY @ 768.000 (0.406)

/home/pi/arne/gekko/portfolioManager.js:153
  if(this.minimalOrder.unit === 'currency')
                      ^
TypeError: Cannot read property 'unit' of undefined
    at [object Object]. (/home/pi/arne/gekko/portfolioManager.js:153:23)
    at [object Object].bound [as getMinimum] (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at [object Object]. (/home/pi/arne/gekko/portfolioManager.js:169:22)
    at [object Object].bound [as buy] (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at [object Object]. (/home/pi/arne/gekko/portfolioManager.js:126:12)
    at bound (/home/pi/arne/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:229:13
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:139:25
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:226:17
    at /home/pi/arne/gekko/node_modules/async/lib/async.js:553:34

Trying to buy @ BTC-e.
Where should the minimalOrder be set?
Thanks for the good work (y)
sr. member
Activity: 287
Merit: 250
@ whydifficult

Forgive me please, if you will, for the tone of my last... Had a rough day (one of many), and this issue was just the bitter icing on a dirt cake.  Ya' know what they say about excuses and back-sides...lol

I do understand that Gekko is a WIP, and may possibly crash.  I also understand that default settings may not be the best for a given set of circumstances.

I guess what I do not yet understand is why, in 2 separate situations on 2 different exchanges (and I believe on different systems), it consistently lost money rather than making any gains whatsoever, and crashed:

1.  CEX.io in sim-mode with settings changed according to recommendations made here in this thread

2.  BTC-e in trading mode with defaults, but with no indication that I've yet seen that the defaults are not appropriate for it.

Might I ask the following, as it sounds like you're re-writing much of it already:

How difficult might it be to set it up such that depending upon which exchange / currency pair you set it to run with, it sets safe or at least lower risk defaults accordingly?  As well, it might be valuable to set up the startup scripts so that if the service fails, it can be stopped and restarted.  I haven't had any time (nor do I have much supporting knowledge) to check into how this might be done, but it was an idea that I had...

If I weren't so behind on modern software development (last I programmed was Basic on C-64 and crApple 2c's...hehe), I'd dig in and help.  For that matter, I'm still hoping to, to some minimal extent.  If I weren't so dead broke, I'd donate now.  I will donate once Gekko's helping me dig outta' this pit I'm in!!

Again, my apologies for previous attitude, and appreciation for your work on this!

(I'm sorry if I sound a bit grumpy, this is not to you personally but I'm going to link to this general answer every time this happens.)

Gekko is not a money making machine. Gekko is a tool for traders to automate buying and selling according to simple mathmetical methods.

Gekko works as long as it is able to run the implemented algorithmes correctly over either realtime market data or historical market data. Configuring the algorithmes in a way they are profitable is not up to Gekko and also not up to me (I wrote the default config).

The default config was based on this thread by a trader. It appears during the last few months the proposed settings are not as profitable as they once were. So if you are losing money there might be two cases:

- You configured an algorithm in a way that it won't generate profit. -> blame yourself, read that thread and first backtest potential new configurations before letting Gekko play with your money again - also use the paper trading (Gekko will simulate profits without your money at risk).
- There is a bug in Gekko so it implements the algos wrongly or interpets the market data wrong, etc. -> blame me & tell me about what is wrong.

--

I'm getting tired of people leeching Gekko when the default settings are profitable and not taking part in any discussion and start complaining when it isn't anymore. Everybody is free to come up with more profitable settings and I'll put them in the default config.

--

edit:  Shoulda' gone to bed, but...  This got my attention in the config's:
  sellTreshold: -0.25,
  buyTreshold: 0.25

Forgive my financial ignorance (I'm quickly learning), but might these be reversed?  In my mind, you'd want to sell at EMA+N.nn rather than -N.nn, and buy at -N.nn, but I can't be sure that I'm interpreting things correctly, or even in the same ball-park...

So those thresholds are part of the EMA method, a trendwatching algo. It is trying to detect trends: if it thinks the price is going up it will advice to buy (so you can sell later when the price is higher and have profit) and if the price is going down it will advice to sell.

sellThreshold determines how fast the trend has to be going down for it to sell (hence negative).
buyThreshold determines how fast the trend has to be going up for it to buy (hence positve).

Note you might not find these parameters in Goomboo's thread, that's mainly because in a flat market Gekko will detect any minor price movement as a new trend, advice to buy/sell and the exchange will eat all your money away in fees.



how to set a maximum transaction volume in BTC/USD??
Example: maxUSD: 50

So program will not use all acc money.

Not possible in this version, coming up in the next. The only thing you can do for now is get all your other funds out of the exchange.
newbie
Activity: 18
Merit: 0
how to set a maximum transaction volume in BTC/USD??
Example: maxUSD: 50

So program will not use all acc money.

newbie
Activity: 19
Merit: 0
@ whydifficult

Forgive me please, if you will, for the tone of my last... Had a rough day (one of many), and this issue was just the bitter icing on a dirt cake.  Ya' know what they say about excuses and back-sides...lol

I do understand that Gekko is a WIP, and may possibly crash.  I also understand that default settings may not be the best for a given set of circumstances.

I guess what I do not yet understand is why, in 2 separate situations on 2 different exchanges (and I believe on different systems), it consistently lost money rather than making any gains whatsoever, and crashed:

1.  CEX.io in sim-mode with settings changed according to recommendations made here in this thread

2.  BTC-e in trading mode with defaults, but with no indication that I've yet seen that the defaults are not appropriate for it.

Might I ask the following, as it sounds like you're re-writing much of it already:

How difficult might it be to set it up such that depending upon which exchange / currency pair you set it to run with, it sets safe or at least lower risk defaults accordingly?  As well, it might be valuable to set up the startup scripts so that if the service fails, it can be stopped and restarted.  I haven't had any time (nor do I have much supporting knowledge) to check into how this might be done, but it was an idea that I had...

If I weren't so behind on modern software development (last I programmed was Basic on C-64 and crApple 2c's...hehe), I'd dig in and help.  For that matter, I'm still hoping to, to some minimal extent.  If I weren't so dead broke, I'd donate now.  I will donate once Gekko's helping me dig outta' this pit I'm in!!

Again, my apologies for previous attitude, and appreciation for your work on this!



edit:  Shoulda' gone to bed, but...  This got my attention in the config's:
  sellTreshold: -0.25,
  buyTreshold: 0.25

Forgive my financial ignorance (I'm quickly learning), but might these be reversed?  In my mind, you'd want to sell at EMA+N.nn rather than -N.nn, and buy at -N.nn, but I can't be sure that I'm interpreting things correctly, or even in the same ball-park...
sr. member
Activity: 287
Merit: 250
I've put out my first bounty, whoho!

If anybody wants to make some money by contributing to Gekko:

Quote
I don't really understand why we are looking at trends for CEX.io, as the value of 1 GHS depends not only on the value for which you can sell it, but also on how much BTC it will mine for you. I propose a new method that doesn't look at trends but uses an online mining profit calculator to determine how much BTC 1 GHS is going to bring you, if the price is < profit, buy all the GHS and if the profit is < price sell all the GHS.

I will write this when my todo list is a bit smaller, so I'm putting a bounty out if someone can do it before me: If someone can implement it (in the localDB branch) I will pay that person 0.01 BTC (Remember I am poor because I opensourced Gekko)!

Expectations:

- Same coding style as the EMAC / core.
- Uses an online mining calculator to figure out how much profit 1 GHS will bring you. The settings need to be configurable in `config.js` (settings are stuff like `block halving time` / `diff increase time` / `diff increase %`).
- If it will profit more than the current price, emit `long` advice. If it will profit less than current price, emit `short` advice.
- Needs to be plugged into the tradeAdvisor just like the EMAC is.

'm obv. only paying out the bounty once, and to the first pull request that satisfies all expectations.

Here is the context.
sr. member
Activity: 287
Merit: 250
I'm developing a new version of Gekko that is able to deal with limited APIs such as: Bitstamp, CEX.io & BTC-e (so all altcoin markets as well). There is a discussion taking place about how Gekko should store historical data. Here is the context and here is the specific issue.

If anyone is interested in running Gekko, could you weigh in on the discussion with:

- On what machine do you want to run Gekko (desktop, laptop, raspberry PI or VPS/cloud instance)
- How big can the history be that Gekko stores on disk (in MB).
- How much memory can Gekko consume per market you want to watch (1-10MB, 10-100MB or more?)

If you want to input on the implementation side that's also fine. Either do it here, on github or on IRC: #gekkobot (freenode).
hero member
Activity: 714
Merit: 500
Any hints with this?
Is mtgox-apiv2 not up to date with latest node?
Very new to this "node" technology.

First:
Code:
sudo npm install
npm ERR! Error: failed to fetch from registry: mtgox-apiv2

So i went to the url, https://registry.npmjs.org/mtgox-apiv2 yes its there, nice.

Then I did,
Code:
sudo npm install mtgox-apiv2 --registry http://registry.npmjs.org/
npm http GET http://registry.npmjs.org/mtgox-apiv2
npm http 304 http://registry.npmjs.org/mtgox-apiv2
npm ERR! Error: No compatible version found: mtgox-apiv2@'>=1.0.0- <1.1.0-'
npm ERR! No valid targets found.

node -v v0.6.19
npm -v 1.1.4


Does this not support my node version? I just grabbed node from git.
thanks for any help.

EDIT:
Sorry turns out maybe i must of grabbed the node source from the wrong github, getting new source for node now https://github.com/joyent/node

To anyone else having this issue make sure you "apt-get remove nodejs" turns out ubuntu comes prepacked with a shitty old version.
sr. member
Activity: 287
Merit: 250
Code:
  short: 10,
  long: 21,
  // amount of candles to remember and base initial EMAs on
  candles: 100,
How come the default number of candles are 100 when the long EMA is only using 21? Is this only usable for the backtesting function? Is is safe to set candles: 21 if I'm not using backtesting?

Nope, EMA works by calculating an average out of the current price + the last average price. So just like the blockchain it calculates new EMAs on top of all the old ones. The older the candle the (exponentially) less value it has in the current EMA. So:

- 100 candles means that it needs to calculate at least 100 EMAs on top of each other.
- 21 long means that the the old EMAs are more important than the new one. The result is a line that reacts slower to trends and lags behind a little (because new candles have little impact).
- 10 short means that the old EMAs are less important than the new one. The result is a line that reacts faster to trends and lags less behind than the long one.

If you lower the 100 it will result in the first EMAs calculated not really being EMA values.
member
Activity: 113
Merit: 10
Code:
  short: 10,
  long: 21,
  // amount of candles to remember and base initial EMAs on
  candles: 100,
How come the default number of candles are 100 when the long EMA is only using 21? Is this only usable for the backtesting function? Is is safe to set candles: 21 if I'm not using backtesting?
sr. member
Activity: 287
Merit: 250
Great explanation, thank you.

I take this means that
Code:
{"$$indexCreated":{"fieldName":"s","unique":true,"sparse":false}}
does not always need to be the very first entry of the database file and the lines don’t need to be ordered by incrementally the "s"-key value?

Other than that the localDB branch works fine for me. Looking forward to it because I just ran into this issue again:

https://github.com/askmike/gekko/issues/51

Oddly enough it loads the first entries just fine, but gekko fails on the last (most recent) candles, retries a few times and then shuts down.I assume there are no connectivity or DNS issues, otherwise it wouldn’t be able to download the first 99 candles.

I also really don't like their API, hopefully I'll get the new method working quickly.

- Store all trades
- Store candles at the size that is in the current config
- Store 1 minute candles.

The first option creates the biggest databases.
...
So I went for the first option: I know exactly how much data per day is required and I am able to calculate every candle (> 1 minute) on the fly based on those.
Do you mean that you went for the third option?

oops, yes that's what I meant.
Pages:
Jump to: