Always interesting to see new methods (especially ML based) here. Thanks for the stimulus
A few remarks:
* While I'm guilty of 'black box' posting myself, it would of course be a lot more interesting to look a bit behind the veil of your algorithm. I gather it's an ANN-based predictor. Anything more you want to say about it?
What NN architecture specifically? And, maybe most importantly, a few words about the structure (and features) of your input, and what exactly you have been training here?
(EDIT: As in: Vector or scalar input? Price only, or other factors? Uniform time scale, or several? Any idea how to describe, qualitatively, what your input represents, and what the generalization is your network (ideally) learned?)
* The way it looks now, the network predicts a moderate rise over the next 5 days, to around $560/565. As it stands now, those predictions are a bit "isolated", so ideally, you would add the following:
(a) An additional output of your predictor that represent the certainty of your network wrt its prediction. Probably could be done as another error function you need to train (training aims to reduce the combined error then, of 'price deviation' and 'confidence deviation').
This suggestions is probably similar to rebuilder's request above, but possibly, to do what he asks, you'd first need to implement something like (a).
(b) Tracking how your method performed on the site. Even if (a) above is too much effort, it should be not too much effort to graph the history of prediction vs. actual price. Possibly, you were already planning to do so. There's probably more than one way to implement that one in detail, since I expect the output of your NN to get closer to the actual price as the prediction distance gets smaller.
Perhaps, even though it might be a bit of a hack, you could discretely color-code the prediction output vs. price for different distances, e.g. at time N, you plot the actual price of N, plus the price predicted by the NN at N-5 days (in red), plus the prediction of N-4 days (in orange), etc., up to maybe N-1 day.
(Apologies to the data scientists and statisticians
... as you can see, I have no damn clue how to properly plot error for a rolling window method)
Thanks for the detailed feedback, I really appreciate you taking the time to write this stuff out!
So yeah I guess it is a bit of a black box post... sorry about that. I usually don't go into details in an initial post about something like this because the majority of people wouldn't really know what I'm talking about since you'd have to have a background in machine learning.
Anyway the predictions are created by feed forward neural networks with varying numbers of layers and sizes. Generally it's 4 or 5 layers, and sizes are mostly in the 100-200 node range, but they are all different. The inputs for bitcoin are change in both price and volume leading up to the point at which predictions are being made, and for currencies it is just price. Also I'm happy to go into more detail about stuff if you'd like to contact me directly at
[email protected].
a. A confidence interval would be wonderful, and others have suggested that too. Unfortunately, the way the model works, that data isn't really available. I may experiment with more probabilistic models in the future because I do agree that a confidence interval would be extremely valuable and interesting to see.
b. This feature actually exists already. It hasn't shown up yet because there aren't 24 hours worth of predictions yet (at this point, the site went up about 21 hours ago. A little later tonight, you should start seeing those charts. I'm excited to see them actually because the euro/usd and aud/usd predictions have not been tested yet at all.
Thanks again for the feedback! Like I said, feel free to shoot me an email if you'd like to get into more detail about the actual neural network.
).
