Gekko - a javascript trading bot and backtesting platform - page 9.

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: itod on August 23, 2014, 06:41:37 PM

Quote from: kuzetsa on August 23, 2014, 02:42:46 PM

I personally implemented that node.js version of this reduced lag MACD code, but it is directly 100% based on Patrick Mulloy's:
"reduced lag" EMA method to improve MACD signals, known as DEMA from over 20 years ago.
...
For description of the algorithm, the original author (Mulloy) published detailed information which can be found in the
February 1994 issue of Technical Analysis of Stocks & Commodities Magazine, article titled: "Smoothing Data With Less Lag"

If anyone local wants to borrow a hardcopy I'd be fine with that, otherwise you might have trouble finding one online which isn't behind a paywall Sad

Just ask and you shall be given Wink

:
ftp://80.240.216.180/Transmission/Фaйлы/S&C%20on%20DVD%2011.26/VOLUMES/V12/C02/SMOOTHI.PDF

BTW, on that FTP server you have a complete 13 years of that magazine, a treasure.

Edit: forum software removes the ":" sign after the "ftp" and before "//", just add it yourself, or copy/paste the complete string of the link to other tab.

Neat, yeah. Thanks itod - that russian server apparently has an electronic PDF version, it's the same article as the one I used for reference.

Mostly my point was that it's not hard to add new trading methods to gekko, and that particular specification was "low hanging fruit" since I had a copy of mulloy's well-written article at my disposal... I just wanted to try something cool which wasn't already in gekko

Edited to add:

Holy crap, I was curious and trying to translate that russian URL into english, and once I converted the %20 into spaces, noticed that
it says: [ S&C on DVD 11.26 ]

Thanks so much, I'm gonna get a copy of that DVD -- I hate hard copies they take up way too much space and are just awful to keep in good condition long-term... Turns out, apparently, that Фaйлы is russian for "files"

itod

legendary

Activity: 1974

Merit: 1076

^ Will code for Bitcoins

Quote from: kuzetsa on August 23, 2014, 02:42:46 PM

I personally implemented that node.js version of this reduced lag MACD code, but it is directly 100% based on Patrick Mulloy's:
"reduced lag" EMA method to improve MACD signals, known as DEMA from over 20 years ago.
...
For description of the algorithm, the original author (Mulloy) published detailed information which can be found in the
February 1994 issue of Technical Analysis of Stocks & Commodities Magazine, article titled: "Smoothing Data With Less Lag"

If anyone local wants to borrow a hardcopy I'd be fine with that, otherwise you might have trouble finding one online which isn't behind a paywall Sad

Just ask and you shall be given Wink

:
ftp://80.240.216.180/Transmission/Фaйлы/S&C%20on%20DVD%2011.26/VOLUMES/V12/C02/SMOOTHI.PDF

BTW, on that FTP server you have a complete 13 years of that magazine, a treasure.

Edit: forum software removes the ":" sign after the "ftp" and before "//", just add it yourself, or copy/paste the complete string of the link to other tab.

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: NAME_REMOVED

Hey I saw that you seem to have one of the most up to date branches for the Gekko project. I was wondering if I could ask you a few quick questions about the project. First, how hard was it to create a new bot strategy for Gekko? It seemed to me as if the project was designed to utilize the one strategy as opposed to users being able to easily write new bots with different strategies.

((...snip...))

^ I got a private message on the forums from someone asking how easy it is to add a new trading method.

No. gekko can use ANY strategy.

Adding support is pretty easy, just have a look at these two consecutive commits:

Step 1: commit "de84075..." <-- add the code to implement your new trading method method
Step 2: commit "7e5dfdb..." <-- make some changes so the config.js will be able to use the new method
Step 3: there is no step 3, just use gekko as normal, and configure using config.js

I personally implemented that node.js version of this reduced lag MACD code, but it is directly 100% based on Patrick Mulloy's:
"reduced lag" EMA method to improve MACD signals, known as DEMA from over 20 years ago.
...
For description of the algorithm, the original author (Mulloy) published detailed information which can be found in the
February 1994 issue of Technical Analysis of Stocks & Commodities Magazine, article titled: "Smoothing Data With Less Lag"

If anyone local wants to borrow a hardcopy I'd be fine with that, otherwise you might have trouble finding one online which isn't behind a paywall Sad

jeezy

legendary

Activity: 1237

Merit: 1010

Quote from: kuzetsa on July 19, 2014, 05:13:17 PM

Now with smarter "sanity check" logic (based on a moving stats window) to do better TA science
...makes sure the price is actually good before reinvesting, and it worked better than I expected:

Code:

Start: 2014-07-16 01:17:20 PM
Stop: 2014-07-19 04:48:38 PM

Purchased @ 0.00487063 (volume weighted average price of actual / live / real GHS reinvest orders)
moving median, market price (min) 0.00485721 -0.28% (occasionally a better price was missed)
moving median, market price (avg) 0.00495821 1.77% (on average, purchase price was better)
moving median, market price (max) 0.00536019 9.13% (occasionally the bot did really good)

^ The latest version I've been testing does a good job reinvesting GHS...

Relevant settings from config.js:

Code:

// ZERO settings:
config.ZERO = {
  // EMA weight (α)
  // the higher the weight, the more smooth (and delayed) the line
  short: 12.576709,
  long: 17.786152,
  signal: 10.992447,
  // how optimistic is the MACD extrapolation going to be?
  crystalball: 0.00000907,
  // how large is the stats window for sanity checking?
  window: 1361,
  // the difference between the EMAs (to act as triggers)
  thresholds: {
   down: -9999,
   up: 0.00000001,
  }
};
((...snip...))

As you can see, EMA weights don't have to be integers because of the way EMA works
(it's really an infinite impulse response filter, and technically not a moving average).

Anyway, here's the new version:

/releases/tag/zerolag "zerolag" EMAs [2014 July 16th]

It's been well-tested, and so it is the "latest" recommended build,
currently @ https://github.com/kuzetsa/gekko/releases/latest

Thanks for sharing this Sarah. I will make sure to have a look at the code and give it a test spin.

kuzetsa

sr. member

Activity: 369

Merit: 250

Now with smarter "sanity check" logic (based on a moving stats window) to do better TA science
...makes sure the price is actually good before reinvesting, and it worked better than I expected:

Code:

Start: 2014-07-16 01:17:20 PM
Stop: 2014-07-19 04:48:38 PM

Purchased @ 0.00487063 (volume weighted average price of actual / live / real GHS reinvest orders)
moving median, market price (min) 0.00485721 -0.28% (occasionally a better price was missed)
moving median, market price (avg) 0.00495821 1.77% (on average, purchase price was better)
moving median, market price (max) 0.00536019 9.13% (occasionally the bot did really good)

^ The latest version I've been testing does a good job reinvesting GHS...

Relevant settings from config.js:

Code:

// ZERO settings:
config.ZERO = {
  // EMA weight (α)
  // the higher the weight, the more smooth (and delayed) the line
  short: 12.576709,
  long: 17.786152,
  signal: 10.992447,
  // how optimistic is the MACD extrapolation going to be?
  crystalball: 0.00000907,
  // how large is the stats window for sanity checking?
  window: 1361,
  // the difference between the EMAs (to act as triggers)
  thresholds: {
   down: -9999,
   up: 0.00000001,
  }
};
((...snip...))

As you can see, EMA weights don't have to be integers because of the way EMA works
(it's really an infinite impulse response filter, and technically not a moving average).

Anyway, here's the new version:

/releases/tag/zerolag "zerolag" EMAs [2014 July 16th]

It's been well-tested, and so it is the "latest" recommended build,
currently @ https://github.com/kuzetsa/gekko/releases/latest

jeezy

legendary

Activity: 1237

Merit: 1010

Quote from: ?? on ??

Just wanted to let you guys know I open sourced my own bot project (also written in Node.JS):

https://bitcointalksearch.org/topic/deleteme-683755

My bot is currently a basic platform that works but it still has a huge road to go.

Currently I'm targetting the technical audience, so if you are skilled in Javascript/Node you might want to have a look :-).

Interesting, will have a look at the source code later on.

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: Hermann1337 on July 09, 2014, 04:00:13 AM

i configured it to be running on BTCe like told in the documentation and use RSI as method

it wont output a correct simulation with this configuration?

Sorry, I don't know. If you're using askmike's original code it might work. (I really do not know) I'm not supporting BTCe exchange, RSI method, or sell orders, and the "simulation" code isn't something I've ever tried to debug or even use.

Hermann1337

legendary

Activity: 1003

Merit: 1000

Quote from: kuzetsa on July 08, 2014, 08:46:42 PM

Quote from: Hermann1337 on July 08, 2014, 10:18:30 AM

Hi guys!

i installed latest gekko from https://github.com/kuzetsa/gekko on my archlinux on raspberry pi

Working fine so far.

I configured it to catch short term data and show me some advice.

But only output i get is:

((...snip...))

Okay now i have some results

Quote

2014-07-08 16:48:56 (INFO): (PROFIT REPORT) original simulated balance:
1.16275 BTC
2014-07-08 16:48:56 (INFO): (PROFIT REPORT) current simulated balance:
1.16254 BTC
2014-07-08 16:48:56 (INFO): (PROFIT REPORT) simulated profit:
-0.00021 BTC (-0.01840%)
2014-07-08 16:49:45 (INFO): processing 6 trade(s)
2014-07-08 16:51:21 (INFO): processing 1 trade(s)

Note that my version explicitly is NOT made to:

[buy low] / [sell high] _{(repeatedly, over and over again.)}

In fact, my version doesn't currently sell.
...ever.
...never ever
It's only set up to reinvest and buy more GHS on cex.io

As a result, the profit simulator won't tell you anything meaningful:
It doesn't simulate GHS mining, so it will never show any simulated profit.

i configured it to be running on BTCe like told in the documentation and use RSI as method

it wont output a correct simulation with this configuration?

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: Hermann1337 on July 08, 2014, 10:18:30 AM

Hi guys!

i installed latest gekko from https://github.com/kuzetsa/gekko on my archlinux on raspberry pi

Working fine so far.

I configured it to catch short term data and show me some advice.

But only output i get is:

((...snip...))

Okay now i have some results

Quote

2014-07-08 16:48:56 (INFO): (PROFIT REPORT) original simulated balance:
1.16275 BTC
2014-07-08 16:48:56 (INFO): (PROFIT REPORT) current simulated balance:
1.16254 BTC
2014-07-08 16:48:56 (INFO): (PROFIT REPORT) simulated profit:
-0.00021 BTC (-0.01840%)
2014-07-08 16:49:45 (INFO): processing 6 trade(s)
2014-07-08 16:51:21 (INFO): processing 1 trade(s)

Note that my version explicitly is NOT made to:

[buy low] / [sell high] _{(repeatedly, over and over again.)}

In fact, my version doesn't currently sell.
...ever.
...never ever
It's only set up to reinvest and buy more GHS on cex.io

As a result, the profit simulator won't tell you anything meaningful:
It doesn't simulate GHS mining, so it will never show any simulated profit.

wizzardTim

legendary

Activity: 1708

Merit: 1000

Reality is stranger than fiction

Quote from: Hermann1337 on July 08, 2014, 10:18:30 AM

Hi guys!
.
.
.

shouldnt there be something like short or long advices?

Yes, that will come after some time, when gekko has gathered many trades in order to calculate advice (may take from 2 hours till 2 days to do that)

Hermann1337

legendary

Activity: 1003

Merit: 1000

Hi guys!

i installed latest gekko from https://github.com/kuzetsa/gekko on my archlinux on raspberry pi

Working fine so far.

I configured it to catch short term data and show me some advice.

But only output i get is:

Quote

2014-07-08 15:31:31 (INFO): We have all required history, going to calculate
advice from this point.
2014-07-08 15:32:19 (INFO): processing 14 trade(s)
2014-07-08 15:33:08 (INFO): processing 9 trade(s)
2014-07-08 15:33:56 (INFO): processing 1 trade(s)
2014-07-08 15:34:45 (INFO): processing 4 trade(s)
2014-07-08 15:35:33 (INFO): processing 4 trade(s)
2014-07-08 15:36:21 (INFO): processing 1 trade(s)
2014-07-08 15:37:10 (INFO): processing 4 trade(s)
2014-07-08 15:37:58 (INFO): processing 2 trade(s)
2014-07-08 15:38:46 (INFO): processing 4 trade(s)
2014-07-08 15:39:35 (INFO): processing 1 trade(s)
2014-07-08 15:42:00 (INFO): processing 1 trade(s)
2014-07-08 15:42:48 (INFO): processing 10 trade(s)
2014-07-08 15:43:37 (INFO): processing 4 trade(s)
2014-07-08 15:44:25 (INFO): processing 2 trade(s)
2014-07-08 15:45:13 (INFO): processing 2 trade(s)
2014-07-08 15:46:02 (INFO): processing 6 trade(s)
2014-07-08 15:46:50 (INFO): processing 1 trade(s)
2014-07-08 15:48:27 (INFO): processing 2 trade(s)
2014-07-08 15:49:15 (INFO): processing 6 trade(s)
2014-07-08 15:50:04 (INFO): processing 5 trade(s)
2014-07-08 15:50:52 (INFO): processing 1 trade(s)
2014-07-08 15:54:05 (INFO): processing 3 trade(s)
2014-07-08 15:54:54 (INFO): processing 1 trade(s)
2014-07-08 15:55:42 (INFO): processing 3 trade(s)
2014-07-08 15:56:31 (INFO): processing 22 trade(s)
2014-07-08 15:57:19 (INFO): processing 3 trade(s)
2014-07-08 15:58:07 (INFO): processing 2 trade(s)
2014-07-08 15:58:56 (INFO): processing 6 trade(s)
2014-07-08 16:00:33 (INFO): processing 5 trade(s)
2014-07-08 16:01:21 (INFO): processing 5 trade(s)
2014-07-08 16:02:10 (INFO): processing 3 trade(s)
2014-07-08 16:02:58 (INFO): processing 3 trade(s)
2014-07-08 16:03:46 (INFO): processing 1 trade(s)
2014-07-08 16:04:35 (INFO): processing 5 trade(s)
2014-07-08 16:05:23 (INFO): processing 5 trade(s)
2014-07-08 16:06:12 (INFO): processing 7 trade(s)
2014-07-08 16:07:00 (INFO): processing 6 trade(s)
2014-07-08 16:07:48 (INFO): processing 12 trade(s)
2014-07-08 16:08:37 (INFO): processing 9 trade(s)
2014-07-08 16:09:25 (INFO): processing 2 trade(s)
2014-07-08 16:10:13 (INFO): processing 4 trade(s)
2014-07-08 16:11:50 (INFO): processing 3 trade(s)
2014-07-08 16:12:38 (INFO): processing 1 trade(s)
2014-07-08 16:13:27 (INFO): processing 2 trade(s)
2014-07-08 16:15:03 (INFO): processing 5 trade(s)
2014-07-08 16:16:40 (INFO): processing 3 trade(s)
2014-07-08 16:17:28 (INFO): processing 6 trade(s)
2014-07-08 16:18:17 (INFO): processing 1 trade(s)

shouldnt there be something like short or long advices?

EDIT:

Okay now i have some results

Quote

2014-07-08 16:48:56 (INFO): (PROFIT REPORT) original simulated balance:
1.16275 BTC
2014-07-08 16:48:56 (INFO): (PROFIT REPORT) current simulated balance:
1.16254 BTC
2014-07-08 16:48:56 (INFO): (PROFIT REPORT) simulated profit:
-0.00021 BTC (-0.01840%)
2014-07-08 16:49:45 (INFO): processing 6 trade(s)
2014-07-08 16:51:21 (INFO): processing 1 trade(s)

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: tofm2 on July 07, 2014, 04:42:35 AM

Hi,

I have Gekko running on my linux box. Unfortunately, it is only giving console text output.

I guess I have missed something for enabling its true puwer.

Do I have to coulpe it with a web server or something ??

thanks for your info.

It's NOT a TA tool made for manual, human-powered trading where you look at charts and whatnot...

If you really wanted to you could convert the console output into graphs, or even enable the builtin web module
(it has a web server built-in, but I've never tested it and it's not very well documented.)

I'd recommend using something like ninjatrader if you need a graphical interface with charts.

It was never made for that, and here's a quote from the TL;DR "pretty long"
very first (original) post from page 1 of this thread:

Quote from: whydifficult on May 18, 2013, 08:14:13 AM

((...snip...))

So Gekko is not

A fully automated trading bot that you turn on and will generate profit withouth you having to do anything.
A trading platform for human day traders with a GUI and charts.
A High frequency trading bot designed to operate on < minute resolution.
An exchange.

tofm2

newbie

Activity: 40

Merit: 0

Hi,

I have Gekko running on my linux box. Unfortunately, it is only giving console text output.

I guess I have missed something for enabling its true puwer.

Do I have to coulpe it with a web server or something ??

thanks for your info.

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: kuzetsa on July 04, 2014, 10:51:20 PM

I really don't test or debug the historical datastore "local DB" system since there's other bugs I'm working on fixing and the database rewrite itself originally took several months to complete, so there's a lot of code involved in figuring out what's wrong

CRITICAL BUGFIX!!!

I took the time to audit the code today since the zerolag method benefits from being initialized with more than one day worth of historical data...

/commit/4ef18a95f3d508f8e230f7a70520413c5de23369 (the code changes for the actual fix)
(for those who read code)
.
/kuzetsa/gekko/releases/critical.bugfix (download link for the critical bugfix)
/kuzetsa/gekko/releases/latest ("latest" currently is the same build
...and all future releases will still have this bug fixed)

Bug has been fixed since a little before 2014 July 5th, 15:00 (3 PM) Local NY time, but I've been taking time to update documentation and release notes and whatnot.

Edited to add:

In case it wasn't clear why I posted this:

History database has been fixed, and older data now works reliably & no longer gets thrown out because of the bug.
(technically it wasn't the database that was the issue, but the code which loads it at startup)

Easy to fix / easy to spot "off by one" bug which nobody was looking at or trying to fix for the past several months up until I started doing audits recently

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: jeezy on July 05, 2014, 07:33:27 AM

Have it running in testmode for the last couple of days, your implemantation of buy&hold on the best spots seems to work fine.

This morning I've pushed a new prerelease version of the zerolag method which has fewer signals
(tries to only buy once per day, and has better signal filtering to better reject short-term noise)

Now has a "crystalball" parameter because I didn't have a better idea on what else to call it.

Download link: The new prerelease "zerolag" method ~ ^{today's update [2014 July 5th]}

exact changes to the code
(the "crystal ball" feature itself just lets the signal occur before the EMA crossover itself if momentum is positive)

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: Najska on July 05, 2014, 05:08:18 AM

kuzetsa,

Thank you very much! I don't use softwares with *lack* of developmental support. I didn't know the original author stopped the support. So that means the end of Gekko

((... snip ...))

In all fairness, when is the last time satoshi nakamoto did work on bitcoin

Just because the original author left doesn't mean there's a "[total] lack of support"

As I work on it, I try to keep detailed, well-documented commits live on the "master" branch. I rarely make a change which is poorly documented (I include code comments) and the commit messages and comments on each commit has plenty of info as well.

jeezy

legendary

Activity: 1237

Merit: 1010

Have it running in testmode for the last couple of days, your implemantation of buy&hold on the best spots seems to work fine.

Najska

full member

Activity: 152

Merit: 100

kuzetsa,

Thank you very much! I don't use softwares with lack of developmental support. I didn't know the original author stopped the support. So that means the end of Gekko for me, until maybe one day I want to involve in supporting it. I already have a code running, collecting data and executing orders, but I thought immigrating to Gekko might serve better for future interests.

I use 2 hourly intervals, and RSI(14) needs somewhat 250 candles, which makes about 20 days for a reliable signal.

kuzetsa

sr. member

Activity: 369

Merit: 250

Quote from: Najska on July 04, 2014, 05:01:20 PM

Hi!

I don't want to wait for 20 days to have a reliable signal from RSI indicator, so how can I add the historical data on top of the data collected by the bot in its runtime?

Bitcoincharts.com has the historical data, and it's easy to put them in the form of {"s":1239,"o":628.372,"h":628.372,"l":628.372,"c":628.372,"v":0.033,"p":628.372,"_id":"plNRczdJWMI7ds5V"}, but what's "s", can it be negative? And what's _id? And I guess p is the adjusted price. Is it vwap?

Thank you very much in advance

Code:

{
  start: [moment object],
  o: [open of candle],
  h: [high of candle],
  l: [low of candle],
  c: [close of candle],
  p: [average weighted price of candle],
  v: [volume]
}

That's the exact spec... as for the "_id" it's related to the format used for the database...

The "s" is the "start" time for when that candle opened... It's in "moment" format:

http://momentjs.com/docs/

... and the _id is a fancy key identifier specific to nedb... but I have no clue what format that _id is in.

not sure if that helps you to know gekko stores its data in a "nedb" database:

https://github.com/louischatriot/nedb

(it's an open-source database for node.js, the language gekko is written in)

Changing gekko over to using the nedb database for storing historical data was one of the last things to happen before the original author stopped supporting gekko completely Sad

But I'm just gonna warn you in advance, I think there's a bug which prevents gekko from reliably loading data more than 24 hours old... it doesn't happen every time, but if you ever have to restart gekko there's a chance the history could be invalidated due to corruption and you'll lose all but the most recent 24 hours or so worth od data.

As I understand, the way RSI is traditionally works requires long-term (generally more than 2 days) of data, so you I really hope you're using a fork of gekko which doesn't have issues with database correction...

I really don't test or debug the historical datastore "local DB" system since there's other bugs I'm working on fixing and the database rewrite itself originally took several months to complete, so there's a lot of code involved in figuring out what's wrong Sad

Najska

full member

Activity: 152

Merit: 100

Hi!

I don't want to wait for 20 days to have a reliable signal from RSI indicator, so how can I add the historical data on top of the data collected by the bot in its runtime?

Bitcoincharts.com has the historical data, and it's easy to put them in the form of {"s":1239,"o":628.372,"h":628.372,"l":628.372,"c":628.372,"v":0.033,"p":628.372,"_id":"plNRczdJWMI7ds5V"}, but what's "s", can it be negative? And what's _id? And I guess p is the adjusted price. Is it vwap?

Thank you very much in advance

Topic: Gekko - a javascript trading bot and backtesting platform - page 9. (Read 147818 times)