Topic: Gekko - a javascript trading bot and backtesting platform - page 19. (Read 147876 times)

Do you mean that you went for the third option?

Great explanation, thank you.

I take this means that does not always need to be the very first entry of the database file and the lines don’t need to be ordered by incrementally the "s"-key value?

Other than that the localDB branch works fine for me. Looking forward to it because I just ran into this issue again:

https://github.com/askmike/gekko/issues/51

Oddly enough it loads the first entries just fine, but gekko fails on the last (most recent) candles, retries a few times and then shuts down.I assume there are no connectivity or DNS issues, otherwise it wouldn’t be able to download the first 99 candles.

What data is it storing?

I had three options to store historical trade data in a way that would let me access the data again:

- Store all trades
- Store candles at the size that is in the current config
- Store 1 minute candles.

The first option creates the biggest databases. I need to store the data in a way that works cross platform and I want to limit the memory footprint. On markets which lots of volumes this can mean a couple of MB in a busy hour. This is not really an acceptable size for me.
The second option means that if you want to change from 60 min to 30 min candles you cannot use previous history.

So I went for the first option: I know exactly how much data per day is required and I am able to calculate every candle (> 1 minute) on the fly based on those.

How is it stored?

I didn't write all the code for storing the local database myself, instead I used neDB. Databases are hard to do correctly so I'm happy I don't have to reinvent the wheel here.

The first line tells the database that there is an index on the `s` field, which means we can query specific `s` recods faster. The other lines are part of the neDB filesystem (stored in JSON). Each record is a 1 minute candle with the following properties:

- s: start minute since UTC midnight (we store each day in a different database, check it in the filename)
- o: open price of this minute
- l: lowest price of this minute
- h: highest price of this minute
- c: close price of this minute
- v: total trade volume of this minute

Can I re use other data?

I am going to write tools that can convert this database into a popular CSV format that I see a lot in the forums, also the other way around (but if the CSV candles are > 1m this means you loose accuracy in the data).

I’m currently running the localDB branch on the side and it seems to record the data just fine from what I can tell.

The db file produced seems to be similar to a CSV, but I noticed that it isn’t ordered by the first fieldname ("s").

Right now my file has entries with "s" values from 388 to 482 in a random order, then a line that says


and then entries 483 to 522 neatly ordered by their incremental "s" value.

Could you maybe give me a very short explanation of what the field names stand for?
I just wanted to make sure that this is intended behaviour.

1. Gekko is an open source project: it might contain bugs that will crash it. That said, it is able to recover after connection interrupts.
2 I am working on a new version that does store history, I am working on this on my own in my spare time and I need people to help me test it, read more here.
3. You are so right, I am now going to change Gekko so that it will never sell when the price is low again and never buy when the price is high again...

If you download Gekko (open source software), turn it on at defaults settings, load real money on the account and expect to pull money out of the air I can't really help you at all.

Gekko is a (free and open source) trading bot that watches trends according to how you configure it yourself. If you feel the default trendwatcher is flawed:

1. Don't use it for real money.
2. Complain about it to the guy who popularized it.
3. Come up with better settings that will lead to profit (using Gekko's backtester & paper traders or external tools).
4. Configure Gekko to use your better settings.

-----

I have some awesome news guys, but I do need your help!

I am now working on a big rewrite that will store the trade history persistently on disk, this means that it will store all data and doesn't rely on API's to always be able to fetch newest stuff anymore. This means:

- Able to watch CEX.io & Bitstamp again
- Able to watch BTC-e (including all altcoin markets)
- Ability to add basically all other exchanges without requiring them to handover older historical data.

It isn't done yet and at this moment it will only save historical trades (so no advice / mail / trading yet) but I need people who want to help me test to get this solid. More info here!

Ouch...

OK, so again, I have the issue that it seems to be trading backwards.

This, now, is on BTC-e, all defaults, trading for real. Before, it was happening on CEX.io in sim-mode.

Here's the log of my trades (all times UTC):

Date     Amount        Type       Price         Total               Manual or Gekko

03:57   B0.0170397   bought    $630.00     $10.73501101   Manual
04:18   B0.0170057   sold        $635.50     $10.80712235   Manual
04:32   B0.0170656   bought    $632.00     $10.78545921   Manual
(So far so good, right?)
13:55   B0.0170315   sold        $617.10     $10.51013866   Gekko (Are you freakin' kidding me?)
Then Gekko crashed.  It was not running when I got home, so I re-ran it.
12:43   B0.0159465   bought    $657.77     $10.48912931   Gekko (Umm... Really?)

I'm not sure what caused Gekko to crash.  If I had to take a guess just now, I'd say it was likely a flakey moment in my InterWebs connection (quite frequent, unfortunately).  What I CAN say is that:

1.  Gekko darn well should be able to stand up to a loss of Internet, and recover from it when returned
2.  Gekko should re-read logs from last run to determine if the Buy order it's creating is going to lose us money
and
3.  When the price is low, it's time to BUY, NOT SELL...

The error was a bug that has since been fixed.

The looping of 'BTCe returned ad error, retrying' has probably something do to with bitcoincharts (make sure the interval is above 15min). I am currently working on a version that doesn't use bitcoincharts anymore but the limited BTC-e [,CEX.io & Bitstamp] API. We have a lot of problems with the bitcoincharts API and Gekko is trusting an unknown third party for data it uses to give you trading advice.

Darn, I am sorry for that. Fixed now.

Tried to go into real trading @ BTCE and had a message 'BTCe returned ad error, retrying' looping
Also tried to go into real trading @MtGox and had this @ launch :
2013-12-22 16:51:20 (INFO):     Using normal settings to monitor the live market
2013-12-22 16:51:20 (INFO):     Profit reporter active on simulated balance
2013-12-22 16:51:20 (DEBUG):    Setup email adviser.
2013-12-22 16:51:20 (DEBUG):    getting balance & fee from Mt. Gox

TypeError: Cannot read property 'message' of null
    at Trader.checkUnauthorized (/home/ubuntu/gekko/exchanges/mtgox.js:88:9)
    at bound [as checkUnauthorized] (/home/ubuntu/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at calculate (/home/ubuntu/gekko/exchanges/mtgox.js:102:10)
    at bound (/home/ubuntu/gekko/node_modules/lodash/dist/lodash.js:729:21)
    at Request._callback (/home/ubuntu/gekko/node_modules/mtgox-apiv2/mtgox.js:48:11)
    at Request.self.callback (/home/ubuntu/gekko/node_modules/mtgox-apiv2/node_modules/request/index.js:148:22)
    at Request.EventEmitter.emit (events.js:98:17)
    at Request. (/home/ubuntu/gekko/node_modules/mtgox-apiv2/node_modules/request/index.js:876:14)
    at Request.EventEmitter.emit (events.js:117:20)
    at IncomingMessage. (/home/ubuntu/gekko/node_modules/mtgox-apiv2/node_modules/request/index.js:827:12)

Putting the 'secret' key seems to trigger this issue ... any idea ?
(API keys have the correct rights setup, I can use them from within my own software)

Nice catch, should be fixed now.

I might be wrong but mailer seems broken (just pulled the repo, was fine before)
TypeError: Cannot read property 'what' of undefined
    at module.exports.send (C:\Users\lixnet\Downloads\gekko-master\mailer.js:65:
19)
    at EventEmitter.emit (events.js:117:20)
    at bound [as emit] (C:\Users\lixnet\Downloads\gekko-master\node_modules\loda
sh\dist\lodash.js:729:21)
    at TradingMethod.advice (C:\Users\lixnet\Downloads\gekko-master\methods\expo
nential-moving-averages.js:170:12)
    at bound [as advice] (C:\Users\lixnet\Downloads\gekko-master\node_modules\lo
dash\dist\lodash.js:729:21)
    at TradingMethod.calculateEMAs (C:\Users\lixnet\Downloads\gekko-master\metho
ds\exponential-moving-averages.js:98:10)
    at bound [as calculateEMAs] (C:\Users\lixnet\Downloads\gekko-master\node_mod
ules\lodash\dist\lodash.js:729:21)
    at EventEmitter.emit (events.js:92:17)
    at bound [as emit] (C:\Users\lixnet\Downloads\gekko-master\node_modules\loda
sh\dist\lodash.js:729:21)
    at CandleCalculator.calculateCandle (C:\Users\lixnet\Downloads\gekko-master\
methods\realtime-candle-fetcher.js:177:

You need to find that out for yourself! You can use the the backtester for that, more info in the docs.


I am currently in the phase of adding a bunch of exchanges, if anyone feels generous pull requests welcome! (Here is an explanation of how to add a new exchange to Gekko)


---

Lots of updates and bugfixes!

Note that cex.io is currently broken due there changed API.


I hope we'll be able to implement a fix soon.

Hi, I play with gekko since a bunch of days, any plans to add more currencies like LTC, NMC or adding others platforms such as cryptsy ? I found a module for its api in nodejs : https://github.com/nothingisdead/npm-cryptsy-api

hi,

i will test it on btc-e the next few days.
which EMA setting is the best

interval = 60  (i think its to long)
short = 10?
long = 21?
candles: 100
sellTreshold: -0.25
buyTreshold: 0.25

thx a lot

Keep up the good work!

Thank you for creating Gekko. Getting ready to try it out very soon.

Time to give this a try !

Hmmm...

Mine's been running in Simulated Mode for 3-4 days, with a CEX.io API key-set.

Currency is BTC, asset is GHS, and thresholds were each changed to -0.025 and 0.025 respectively, as I read that this is what most folks are using.

The result:  Each and every Profit Report (simulated) shows a NEGATIVE number.  The first was -9 and change, and it's progressively decreased to currently -15.xxx.

I've gotta' wonder, if I actually enabled trading with the defaults (and just the changes noted above), would I actually be losing money?  If so, how might I edit things to prevent this?

I can understand that there will be losses / chances of loss in the trading arena, but to lose at EVERY SINGLE TRANSACTION?  That'd be silly...

an indicator, like Heiken Ashi, or ichimoku cloud would be nice...

works...missspelled a currency asset pairs...