Gekko - a javascript trading bot and backtesting platform - page 12.

cryptcollector

full member

Activity: 227

Merit: 100

Quote from: kuzetsa on June 23, 2014, 02:32:55 AM

It's not going to be easy to get cexio backtesting working in a realistic / sane way Sad

I hear you, i've kind of given up trying to play the GHS market at the moment, its just not viable, unless you actually sit there and watch it for the sudden drops and spikes that have a gap of about 0.0002 between top and bottom.

I've decided to point my gekko (so to speak Wink

) at btc-e and LTC/BTC market.

I'm going to see about using CEX GHS to reinvest in itself, but at a slightly lower price than market value (take advantage of the sudden price drops in any given candle to a set amount).

I'm also going to point 10% of my mining power from hardware to CEX to fuel the reinvest untill i can get a reasonable sustainment from the GHS itself.

Point of this? Just a suggestion, if you are going to concentrate on a solely CEX.IO for your own fork of gekko, it might be a nice function to have where you never sell GHS but take advantage of market dumps to get maximum ghs for your generated BTC.

Just my tuppence worth :-)

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: cryptcollector on June 22, 2014, 04:32:14 PM

I noticed that there used to be a backtest for gekko, is this still there, I take it the only problem there is, is that the data in CSV format is not that easily obtainable.

Would be a good feature to still have, even if perhaps the accumulated DB data was to be used for this purpose natively.

But even if not, getting the info out of the DB data would be a good alternative.

((...snip...))

I never touched the old backtesting code, mainly because I wasn't personally using it before the DB rewrite, and come to think of it you're right -- looks like it never got ported to the new gekko framework... And from what I can tell, the current state of gekko is entirely event driven & asynchronous so I'm not sure how porting the backtest feature into the current version of gekko could even be done.

Moving forward, my own fork of gekko is probably ONLY going to support cexio, but a big thing to consider with GHS is that a port of the old backtesting code would probably be even more invalid than it was in the first place since over time, the constant increase in your BTC balance is strongly linked to how much GHS you have in your portfolio. There is a problem with the idea of selling your GHS in hopes of a short-term gain from a strategy where you buy back the GHS at a lower price, since you will often LOSE more money because your portfolio would not have the same mining income gains you would have earned from simply holding onto your GHS.

Working backtesting support (which would need to take into account timestamps, and then simulate hypothetical mining based on the historical conditions for the bitcoin network difficulty) for cexio is not anything I'm planning on implementing. I'm far more concerned with taking care of cleaning up bugs which were introduced during the "LocalDB" branch...

... and then MAYBE adding a new trading method which accounts for the real value of GHS (mining income based on realtime network conditions and difficulty, as well as deducting the maintenance fee... which itself will require tracking the USD bitcoin exchange rate, as the current cexio maintenance fees are based on USD exchange rate for bitcoin)

It's not going to be easy to get cexio backtesting working in a realistic / sane way Sad

cryptcollector

full member

Activity: 227

Merit: 100

I noticed that there used to be a backtest for gekko, is this still there, I take it the only problem there is, is that the data in CSV format is not that easily obtainable.

Would be a good feature to still have, even if perhaps the accumulated DB data was to be used for this purpose natively.

But even if not, getting the info out of the DB data would be a good alternative.

I'm still playing about with settings, but ghs is really flat at the moment, so not getting much response. Not so sure what I should set as a good threshold for it. I've expanded to 30 min candles, so I'm waiting once more for the DB to catch up 😠...

I thank you for your time and taking an interest in my troubles.

When I get this thing going proper and start to see results I will gladly contribute in one way or other.

Thanks 😃

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: cryptcollector on June 21, 2014, 05:58:12 PM

kuzetsa

((...snip...))

I created a new API Key as suggested as I was getting the same error as on Windows once i got gekko running right, and this solved the problem on linux. Will test the new key on the windows installation and update.

Thanks your help, Now i just have to wait for 13 hrs for the database to get up to speed again. Is there a quicker way of getting the data that gekko needs? or do we just need to wait it out?

Glad to hear a new API key was all you needed.

As for your other issue of building a database, there are two options:

"the data that gekko needs" will vary depending on your chosen settings for the technical indicators you're running gekko with (MACD, PPO, etc. etc. etc.) but the MACD / PPO / other trading method defaults can be easily changed, as can the candle sizes. I'm personally using rather exotic settings on my own, and require less than an hour of data for gekko to work as I want it to.
If you really need to use the current settings (which requires more data) then you could simply copy your databases from the old gekko installation. So long as there are no gaps in the data, this should work. Being able to copy databases between gekko bot installations was a design goal when the gekko database backend was changed several months ago. It was a major rewrite, but was worth it

Edited to add:

Oh, I just realized you mentioned the "13 hours to rebuild database" more than 13 hours ago. Nevermind then I guess, but for future reference, you can copy databases to a fresh install to speed things up.

cryptcollector

full member

Activity: 227

Merit: 100

kuzetsa

Computer clock was set okay and syncing time from the internet.

I decided to setup gekko on Ubunutu 12.04, but came across another issue, but found the solution in an earlier post and finally got it running right, I've also implemented your changes.

I created a new API Key as suggested as I was getting the same error as on Windows once i got gekko running right, and this solved the problem on linux. Will test the new key on the windows installation and update.

Thanks your help, Now i just have to wait for 13 hrs for the database to get up to speed again. Is there a quicker way of getting the data that gekko needs? or do we just need to wait it out?

kuzetsa

sr. member

Activity: 369

Merit: 250

The particular error you're having makes me wonder if maybe your computer's clock isn't accurately set.

This could affect the trade API, as the exact time is sent by gekko using the cexio API when a trade happens.

Only advice I have at this time:

Check to make certain your system clock is accurate. Gekko's cexio trade API uses a timestamp, and even if this doesn't fix the issue it's good to have an accurate clock ...because reasons^TM

Try reinstalling gekko, complete with the "npm install" step
Try a fresh config file (it's a bit of work, but there could theoretically be an issue with your settings if it's an old config)
Try a fresh API key, since at some point in the past 10 months it looks cex.io added a new IP address whitelist feature, and possibly they also changed the format for API keys?
(the new API key I generated today looks a little bit longer than the old one)

Code:

2014-06-21 14:48:46 (DEBUG):    inserting candle 1127 (2014-06-21 18:47:00 UTC) volume: 1.1627554099999997
2014-06-21 14:48:46 (DEBUG):    calculated MACD properties for candle:
2014-06-21 14:48:46 (DEBUG):             short: 0.00745433
2014-06-21 14:48:46 (DEBUG):             long: 0.00745423
2014-06-21 14:48:46 (DEBUG):             macd: 0.00000010
2014-06-21 14:48:46 (DEBUG):             signal: -0.00000007
2014-06-21 14:48:46 (DEBUG):             macdiff: 0.00000017
2014-06-21 14:48:46 (DEBUG):    In uptrend since 9 candle(s)
2014-06-21 14:48:46 (DEBUG):    Processed trades, sleeping for a minute...
2014-06-21 14:48:47 (INFO):     Attempting too BUY 0.04942629630374857 GHS at cex.io
2014-06-21 14:48:47 (DEBUG):    BUY 0.04942629 GHS @ 0.00745485 BTC
2014-06-21 14:48:47 (DEBUG):    BUY order placed.  Order ID 1562717868
2014-06-21 14:49:18 (INFO):     BUY was successfull

For now, all I can seem to figure is that it just works^TM and there's nothing wrong with gekko's cexio support other than not having the fee.
(cexio now has a fee on all trades, so I had to modify the API slightly)

At some point I can clean up my code a bit (instead of a quick and dirty hack) to push a new update to the github repo for my
"correctly supports cexio" fork of gekko with the updated code to account for cexio now having a trading fee.

Updated with more info:

The hardcoded code with "zero fee" is outdated, as is the code comment:

Code:

FROM FILE: ~/gekko/echanges/cex.io.js


Trader.prototype.getFee = function(callback) {
  // cexio does currently don't take a fee on trades
  // TODO: isn't there an API call for this?
  callback(false, 0.0);
}

However, when I changed the value from 0.0 to a non-zero value, I had some other issues with the fee STILL not being used:

Code:

FROM FILE: ~/gekko/core/portfolioManager.js

Manager.prototype.trade = function(what) {
  if(what !== 'BUY' && what !== 'SELL')
    return;

  this.action = what;

  var act = function() {
    var amount, price;

    if(what === 'BUY') {

      // do we need to specify the amount we want to buy?
      if(this.infinityOrderExchange)
        amount = 10000;
      else
        amount = this.getBalance(this.currency) / this.ticker.ask;

I couldn't even find evidence of the fee being used, so I changed the line to:

Code:

        amount = ((1 - this.fee) * this.getBalance(this.currency)) / this.ticker.ask;

also note, it appears as though the API doesn't use an exact 0.2% fee, so I actually used a higher value in ~/gekko/echanges/cex.io.js

Code:

  callback(false, 0.0075);

that value corresponds to is 0.75% instead of 0.2%, and even then it sometimes still complains about insufficient funds (though not always) so I think maybe there is a rounding error somewhere in gekko I haven't managed to track down... Otherwise, like I said, gekko works fine.

Hope this info helps

Note: Feel free to hack around in your own gekko code and do better, as my current solution to the fee not being used (was a hardcoded 0.0) really is a quick and dirty hack rather than having gekko read the value from the config... which I don't even think works correctly for ANY exchange, so that's not just an issue with gekko's cexio support

cryptcollector

full member

Activity: 227

Merit: 100

Quote from: kuzetsa on June 21, 2014, 10:47:38 AM

I haven't worked on gekko in several months, and neither has askmike. Fortunately, cexio was the one I was personally trying to support so I'll update you further if I'm able to get it working, and will hopefully be able to provide assistane if the current state of gekko's cexio API support isn't completely broken (again, broken AGAIN, sometimes the APIs get changed by the various exchanges and there's sometimes nothing we can do to fix our code)

I'll update this thread with status of gekko's cexio API support this week / over the weekend / before the end of the month... maybe today but no promises

Update:

Took me less than an hour to install a clean install of gekko and configure for cexio's API... So far, at or around noon, local NY time I have gekko working fine with cexio in "watcher" mode (not live trading... yet) but that doesn't mean much since you said the error was in trading mode (I have a little GHS and BTC on hand on my cexio account for testing, so we'll see what happens next time there's a trend)

Hi kuzetsa,

Cheers for this, if there's anything I can do to help eg, testing or anything feel free to ask.
I was beginning to think is was me Cheesy

Look forward to the fix.

---added
Sorry should have also said originally that the alternate API key/Secret i used were for different accounts on CEX.io and different installation from different systems as well. All were running under Windows.

Crypt

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: cryptcollector on June 20, 2014, 07:18:07 PM

I've eventually got the history I needed for CEXIO and when I came to activate the actual trading Gekko just started to give me this error.

2014-06-21 01:07:26 (DEBUG): cex.io returned an error, retrying.. Nonce must be incremented waiting for 10000 ms

No matter what I try to do it just keeps on going, if I revert to just simulation is goes away. The API key and Secret are in and correct, I've even checked it with a different API key and got the same effect.

Anybody had this problem, and more importantly, has anybody solved it?

I'm going nuts here, I cant find this problem anywhere with regerad to gekko, not even by googling

Thanks

I haven't worked on gekko in several months, and neither has askmike. Fortunately, cexio was the one I was personally trying to support so I'll update you further if I'm able to get it working, and will hopefully be able to provide assistane if the current state of gekko's cexio API support isn't completely broken (again, broken AGAIN, sometimes the APIs get changed by the various exchanges and there's sometimes nothing we can do to fix our code)

I'll update this thread with status of gekko's cexio API support this week / over the weekend / before the end of the month... maybe today but no promises

Update:

Took me less than an hour to install a clean install of gekko and configure for cexio's API... So far, at or around noon, local NY time I have gekko working fine with cexio in "watcher" mode (not live trading... yet) but that doesn't mean much since you said the error was in trading mode (I have a little GHS and BTC on hand on my cexio account for testing, so we'll see what happens next time there's a trend)

cryptcollector

full member

Activity: 227

Merit: 100

I've eventually got the history I needed for CEXIO and when I came to activate the actual trading Gekko just started to give me this error.

2014-06-21 01:07:26 (DEBUG): cex.io returned an error, retrying.. Nonce must be incremented waiting for 10000 ms

No matter what I try to do it just keeps on going, if I revert to just simulation is goes away. The API key and Secret are in and correct, I've even checked it with a different API key and got the same effect.

Anybody had this problem, and more importantly, has anybody solved it?

I'm going nuts here, I cant find this problem anywhere with regerad to gekko, not even by googling

Thanks

Starving_Marvin

sr. member

Activity: 511

Merit: 250

Open and Transparent Science Powered By Blockchain

The candle size is 60 and history size is 100. I like these settings. Not perfect but better than my previous ones. I just don't understand why does all the historical data suddenly disappear if I close Gekko for a minute? Because it was trading a few hours ago. The same re-loading time doesn't seem to occur if I don't login into btc-e but just press ctrl+c and exit.

Can I please have some help on getting the log to work? So I could see my balance without logging into btc-e. The email system gives weird errors for me...

ernie-

sr. member

Activity: 252

Merit: 250

Quote from: Starving_Marvin on May 07, 2014, 06:02:43 AM

Quote

The bot was working great for four days. Then I logged in to btc-e to see how my balance was doing. After that Gekko didn't do any trades anymore. But the messages etc. were normal in the terminal. Today when I restarted Gekko it said that it has to load another three days worth of info. So I guess I need to always close Gekko before I log into btc-e?

This time I closed Gekko before I logged into btc-e to see my balance. Just for a minute. Then I restart Gekko and it says it needs four days to gather new data to make trades. WHY? It worked fine a minute ago before restarting.

Also how can I get logs? The log folder is empty. Trade advisor etc. are enabled but where does the advice go? Annoying situation because I can't log into btc-e because everything seems to go crazy and I can't even see logs or anything.

Don't set it so high, it's your choice that it takes 4 days to gather data, I have mine set for 2hours.

Starving_Marvin

sr. member

Activity: 511

Merit: 250

Open and Transparent Science Powered By Blockchain

Quote

The bot was working great for four days. Then I logged in to btc-e to see how my balance was doing. After that Gekko didn't do any trades anymore. But the messages etc. were normal in the terminal. Today when I restarted Gekko it said that it has to load another three days worth of info. So I guess I need to always close Gekko before I log into btc-e?

This time I closed Gekko before I logged into btc-e to see my balance. Just for a minute. Then I restart Gekko and it says it needs four days to gather new data to make trades. WHY? It worked fine a minute ago before restarting.

Also how can I get logs? The log folder is empty. Trade advisor etc. are enabled but where does the advice go? Annoying situation because I can't log into btc-e because everything seems to go crazy and I can't even see logs or anything.

Starving_Marvin

sr. member

Activity: 511

Merit: 250

Open and Transparent Science Powered By Blockchain

Quote from: Starving_Marvin on March 21, 2014, 03:46:48 AM

I got this kind of error. I'm using the newest version of gekko from github and newest node.js

TypeError: Object false has no method 'find'
at Manager.getCandles (/home/xxx/gekko/core/candleManager.js:560:35)
at bound [as getCandles] (/home/xxx/gekko/node_modules/lodash/dist/lodash.js:729:21)
at iterator (/home/xxx/gekko/core/candleManager.js:516:10)
at bound (/home/xxx/gekko/node_modules/lodash/dist/lodash.js:729:21)
at /home/xxx/gekko/node_modules/async/lib/async.js:227:13
at /home/xxx/gekko/node_modules/async/lib/async.js:111:13
at Array.forEach (native)
at _each (/home/xxx/gekko/node_modules/async/lib/async.js:32:24)
at async.each (/home/xxx/gekko/node_modules/async/lib/async.js:110:9)
at _asyncMap (/home/xxx/gekko/node_modules/async/lib/async.js:226:9)

Any help please?

Turns out, this was because of the email advisor. Disabled it and never saw the error again.

The bot was working great for four days. Then I logged in to btc-e to see how my balance was doing. After that Gekko didn't do any trades anymore. But the messages etc. were normal in the terminal. Today when I restarted Gekko it said that it has to load another three days worth of info. So I guess I need to always close Gekko before I log into btc-e?

Thom

member

Activity: 112

Merit: 10

Be kind man, don't be mankind

Gekko won't play nice with cex.io now that they've introduced fees.

Code:

2014-05-01 10:33:31 (INFO):     Attempting too BUY 0.0032098427735274496 GHS at cex.io
2014-05-01 10:33:31 (DEBUG):    BUY 0.00320984 GHS @ 0.00716546 BTC
2014-05-01 10:33:31 (ERROR):    unable to buy: Error: Place order error: Insufficient funds.

~~Haven't tested it for larger transactions yet, but~~ trades this size and larger are doable manually on the cexio website.

EDIT: sometimes manages trades, sometimes not. :/

Code:

2014-05-07 06:35:31 (ERROR):    unable to buy: Error: Place order error: Insufficient funds.
2014-05-07 06:40:26 (INFO):     Attempting too BUY 0.11609350532371158 GHS at cex.io
2014-05-07 06:40:26 (ERROR):    unable to buy: Error: Place order error: Insufficient funds.
2014-05-07 06:45:26 (INFO):     Attempting too BUY 0.11609367731459945 GHS at cex.io
2014-05-07 06:45:26 (ERROR):    unable to buy: Error: Place order error: Insufficient funds.
2014-05-07 06:50:26 (INFO):     Attempting too BUY 0.11609333333333334 GHS at cex.io
2014-05-07 06:50:26 (ERROR):    unable to buy: Error: Place order error: Insufficient funds.
2014-05-07 06:55:27 (INFO):     Attempting too BUY 0.11609333333333334 GHS at cex.io
2014-05-07 06:55:27 (ERROR):    unable to buy: Error: Place order error: Insufficient funds.

ernie-

sr. member

Activity: 252

Merit: 250

Quote from: Starving_Marvin on April 29, 2014, 09:36:22 AM

Is it possible to have several Gekkos running on the same computer trading different currencies on different exchanges?

seems to work.

uzer23

newbie

Activity: 3

Merit: 250

this bot is soo cool. But I really need a solution for fetching the initial data. I must wait 2 days until I get results using 1 hour candles and appropriate history ... Sad

And it would be so great so simulate strategies for given timeframes for historic data. I saw this in a TODO list of yours, hope you find some time someday

Thanks for the bot!!

Starving_Marvin

sr. member

Activity: 511

Merit: 250

Open and Transparent Science Powered By Blockchain

Is it possible to have several Gekkos running on the same computer trading different currencies on different exchanges?

cryptocxeq

newbie

Activity: 23

Merit: 0

what should I put my candleSize and historySize at for DEMA if I want to emulate the 2hr EMAs, but don't want to wait forever to get data (esp. since I'm crashing all the time...)

I keep getting this error on btce

TypeError: Cannot read property 'c' of undefined
   at Base.tick (C:\Users\Zack\Desktop\gekko-candle-rewrite\core\baseTradingMethod.js:74:21)
   at bound [as tick] (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)
   at Actor.processCandle (C:\Users\Zack\Desktop\gekko-candle-rewrite\plugins\tradingAdvisor.js:39:15)
   at bound (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)
   at EventEmitter.emit (events.js:95:17)
   at bound [as emit] (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)
   at Manager.relayCandle (C:\Users\Zack\Desktop\gekko-candle-rewrite\core\marketManager.js:60: Cool

   at bound (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)
   at EventEmitter.emit (events.js:117:20)
   at bound [as emit] (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)

and this one on bitstamp

C:\Users\Zack\Desktop\gekko-candle-rewrite\methods\DEMA.js:44
  var message = '@ ' + price.toFixed( Cool

+ ' (' + diff.toFixed(5) + ')';
   ^
TypeError: Cannot call method 'toFixed' of undefined
   at method.check (C:\Users\Zack\Desktop\gekko-candle-rewrite\methods\DEMA.js:44:30)
   at bound (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)
   at Base.tick (C:\Users\Zack\Desktop\gekko-candle-rewrite\core\baseTradingMethod.js:86:10)
   at bound [as tick] (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)
   at Actor.processCandle (C:\Users\Zack\Desktop\gekko-candle-rewrite\plugins\tradingAdvisor.js:39:15)
   at bound (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)
   at EventEmitter.emit (events.js:95:17)
   at bound [as emit] (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)
   at Manager.relayCandle (C:\Users\Zack\Desktop\gekko-candle-rewrite\core\marketManager.js:60: Cool

   at bound (C:\Users\Zack\Desktop\gekko-candle-rewrite\node_modules\lodash\dist\lodash.js:729:21)

my settings are basically unchanged:

  enabled: true,
  exchange: 'BTCe', // 'MtGox', 'BTCe', 'Bitstamp', 'cexio' or 'kraken'
  currency: 'USD',
  asset: 'BTC'
  method: 'DEMA',
  candleSize: 60,
  historySize: 50

Mailer on.

ernie-

sr. member

Activity: 252

Merit: 250

Quote from: ceslick on April 09, 2014, 06:50:58 PM

Hi All,

My problem seems to be a bit different..

Gekko runs but always tells me that it does not have enough information needs two days of information to start giving advice.

Any ideas?

Also anyone able to give me samples of there config for cex.io?

Cheers

Chris

You have set your candleSize and historySize high.

If you have 5min candleSize with 60 historySize that's 5min x 60 or 5hrs worth of information to gather before gekko will start giving advice.

ceslick

full member

Activity: 161

Merit: 100

digging in the bits... now ant powered!

Hi All,

My problem seems to be a bit different..

Gekko runs but always tells me that it does not have enough information needs two days of information to start giving advice.

Any ideas?

Also anyone able to give me samples of there config for cex.io?

Cheers

Chris

Topic: Gekko - a javascript trading bot and backtesting platform - page 12. (Read 147876 times)