Topic: Ideal buy/sell strategy for maximizing profit (Read 3050 times)

Yes, of course the specific data you train the neural network on is important to determining whether or not it's going to overfit. However, when predicting bitcoin prices there is really only one available set of training data, so that is kind of irrelevant in these circumstances. We're always going to be training on historic bitcoin price data, so the factors that affect whether or not it overfits this data are generally going to be the structure. More complex structures allow it to represent more complex functions.

I think you understood correctly. A simple neural network cannot overfit a very large and complicated set of training data. People here seem to think that neural networks of any size and structure can represent any function, but this is not true. For example, you could have a neural network with 800 inputs and 800 outputs, but no matter how much you trained it on whatever data you give it, it will never be able to map a simple XOR function.

If this model were overfitting, it would perform a lot better than it's performing on historic data, and its average error would be much lower. I've seen these neural networks overfit, and when they do, I restructure them and change parameters so that they don't. I've tested these historic data, data from other exchanges, and future data. The model is not overfitting. And no, it doesn't look at dates. It is very obvious that doing this would lead to overfitting lol... I don't know why anyone would ever input dates into a neural network for predicting prices. This model only looks at past prices right now (though hopefully it will look at sentiment data too soon) and the past prices are represented in various different ways in different neural networks.

This sounds strange to me. Most important must be what data to train it on, don't you agree? While the most important goal of the neural net is that it actually does what it is supposed to do, in this case predict the future? The actual technical design, the amount of nodes, feedback propagation etc., can only work to fit data faster and better.

Surely even a very simple network can overfit the training data set, as opposed to the working data set? (There may be a language issue here so perhaps I misunderstand.)

This is a very good example of what I was trying to say above. (And a more complex network could very well have made this fit even better. Yay.)

A big warning sign for me is that the author is proud that historical data "fits". I am just as interested modelling bitcoin price data as anyone around here, but I defer judgement until I've seen it work its magic on future data.

The data is presented to the neural network in different ways for different neural networks that I'm testing. However, this isn't really what's important to determining a model's likelihood of overfitting. What is most important is the number of input nodes, output nodes, layers of hidden nodes, and nodes in each hidden layer.


It actually is possible. Maybe I'm misunderstanding what you are trying to say but it sounds like you may have a bit of a conceptual misunderstanding of neural networks. The weights aren't assigned to nodes, but rather to edges between nodes. Furthermore, each weight holds one piece of data - a weight is represented by a double. It's just one number. Also a trained neural network will fit to its input data to a certain degree (assuming it was well structured and trained), but this doesn't mean that it is overfitting.

So basically, when I say that the neural network is not complex enough to overfit this data, I mean that the structure is not complex enough that it could learn the data closely enough to overfit. My data has millions transactions, which end up being compressed into something like 300,000 prices spaced evenly. The patterns in this data are very complex - complex enough that in order to overfit, my neural network requires something ridiculous like 130+ input nodes and 700+ hidden nodes. Smaller neural networks simply would not be able to represent a function that is that complicated.

I'd be interested in hearing how he is presenting the BTC data to the neural network.  I once saw someone present dates along with price data to a neural network he designed.  It ended up fitting itself to the dates, buying and selling on specific dates, without really looking at the prices at all.  Suffice it to say that doesn't work in forward-testing with out-of-sample data.

I didn't really want to say it's likely, but you can in principle set the no. of parameters low enough, given the points of data, s.t. the risk overfitting is arbitrarily small*. Keep in mind that this was my responsne to K128kevin2's claim that "overfitting is impossible with this model", which I found (and still find) hard to believe.


* If you challenge me on this, I'm afraid I'll have to go through my trusty old Bishop to find the answer

Yes, that's what I mean - although I would say that it is unlikely enough that I feel confident saying that it's impossible. I mean, I've trained neural networks on this data that do overfit, and this isn't one of them.


Yes I definitely see the advantages to partitioning data into training/test data. The difference, however, in doing that with this data versus doing that with more common applications of neural networks (such as character recognition or audio recognition) is that we can create an infinite amount of training data for those things. We can draw thousands of hand written characters for training and thousands more for testing. However, there is a finite amount of bitcoin data available. Not training on all of the available data will result in less accurate predictions. Even watching the neural network that is up on the main page of my site now, you can see that it has been getting more and more accurate as more data becomes available for it to train on. As long as I know it's not overfitting, I feel that it is best to train it on all available data.

That's not possible for a neural net.  The weights assigned to each node of a neural net can hold an enormous amount of data.  A trained neural net by definition fits to the input data.  You don't see it if you examine the node data because that data is spread throughout the entire network.

You mean you have few enough model parameters to make overfitting the given data unlikely. Yes?

This still allows for subtle effects how well your network generalizes to new input, especially for something as error sensitive as trading. There's a reason why in ML the data is usually partitioned into a training set and a test set (or three, including a validation set), and the two are kept separate.

I've tested it on the same data and on different data, and the results are very similar. The issue with training and testing on the same data is generally overfitting, but overfitting is impossible with this model. The testing I've done with neural networks on future data has yielded pretty much the same results as tests on historic data.

Just out of curiosity: the data used to determine the profitability of the network's outputs wasn't used in the training phase of the network, correct?

Agreed.

I'm working on a simplistic trading bot. It would be great to integrate this sort of prediction into the bot, rather than me trying to hack together some sort of trending logic

I can see what you are saying but I think maybe you misunderstand what I'm doing when I simulate trade. The neural network that looks at the historic data has no idea where the price is going to go when it simulates trade. It doesn't know pre-determined ideal points of where to sell or buy - it basically uses a formula (or more accurately, a function) to determine when it is good to buy or sell, and it makes these decisions on the fly with no knowledge of what will happen next (as would be the case with future data). It's purely based on recent prices. Also I can assure you that creating a trading strategy that can do this and be profitable on historic data is very far from trivial lol... it takes a ton of time, work, and effort.

Thx, now I got what was meant. The term I know for that is prognostic validity. Backtesting the algorithm is a necessary first step, though. If it fails generating profit on past data the probability of doing it in the future is rather slim.

The only thing that matters is future profits.

Decide on your algorithm now, and come back in a week (.. month, or year) and tell us how much it actually could have made.

It is trivial to find and algorithm have would have traded for immense profits historically. I know if could do much "better" than what the author published above, only by following known fixpoints. But that strategy would not own me even a millibitcoin in the future.

And netiher would the author's. Or...? The proof is in the pudding, as they say.

Huh? What do you mean with real time graph? Pofits with buy and hold obviously equal price development. +100x price development = +100x profit. What should be simulated about that?

I spoke to some people today at Coinalytics (bitcoin startup out in California). They may be developing an API for this data actually!

For now the code is not going to be open source yet, but I am thinking about developing some open source trading algorithms. Maybe I'll develop like a framework for simulating trade based on any strategy and make it easy to implement and test different strategies. It could be interesting to see what kinds of strategies people come up with.

Also on an unrelated note, here is a picture of the profitability of trading based on the data on this new section of the site versus just buying and holding:


EDIT: The values on the left are in USD, so if it is currently holding bitcoin instead of USD it just multiplies number of bitcoins held times the value of bitcoin.

For API I would love to simply have the updated buy and sell prices. But thinking of it I can probably grab those and parse them into a program from your site anyways. Maybe if you made a JSON page that would be just as good.

By Whitepaper I mean your ideas and thinking behind your algorithms and specifics on how they actually work. SO yes open source along with the conceptual or philosophical reasoning as well.

Thanks again for your work and contribution to the community!

Ugh I remember that conversation... I do not want to get back into that again.

For the new app, the neural network looks at prices in like 5 min intervals for the past couple hours, hour intervals for the past like 1.5 ish days I think, and like every other day I think for the past about 30 days. I don't remember exactly because I trained it a while ago, although I have the exact parameters written down somewhere.

There are 65 inputs, 30 hidden nodes, and 2 output nodes.

Sorry. I was interested and amazed by the low average error. So I wanted to test this. I took 6 or 7 random points in time and wrote your 24h prediction down. After 24hours I checked the real value. I measured an error of 8%, not 1.2% like your average error. There were two possible explainations for that:
1. Your average error is the error of your learning data and useless for real predictions.
2. I was very unlucky. Very Very unlucky. Rolling a dice 6 times in a row and get an 1 all the time is more possible.

Anyhow. Is it a secret how far you go back in time for the learning? How many neurons do you use?

Hey so I'm not sure I quite understand what you are saying here. Can you remind me what we were talking about in our last conversation? Sorry lol...


Sorry, it has trouble with Hotmail and occasionally with some other less common emails. Send an email to [email protected] from the email that you signed up with and I can get your account activated.


I'm glad you like it! What would you be looking for in an API? And by whitepaper do you mean to make the algorithm open source?