Topic: New more accurate network hashrate forecasts - page 2. (Read 5559 times)

Im looking at the 2 month ahead prediction: 3548TH -71% +251%
-71% means -2519 so 1028 TH correct?
+251% means +8905 = 12453 TH

Sorry if I misunderstood that.


Not that I dont appreciate it, but I guess Im curious why you would spend so much time calculating it, or  for what  purposes such a wide range would be accurate enough?

No. The next forecast was for 1152 Thps with a 95% confidence interval of -17% to +21%. That's about 956 to 1393 Thps. How on earth did you calculate your values?

And yes, the forecast is lower than the current seven day average by about 50Ghps. So?

I think it's very likely it will be within the 95% confidence interval. and possible it will be within the 50% confidence interval





You still don't really understand how forecasting works.

Temperature is a stationary time series. The network hashrate is not. If the temperature was exponentially increasing per day, and I could forecast the next day's temperature within +/- 20% with 95% confidence, I'd be quite happy with that.





Or confidence intervals.

Firstly, in this context they indicate the probability of a given occurrence. This means they show the uncertainty associated with a given forecast. How many other estimates of future hashrates provide a confidence interval for their estimate that not just made up?

So if you know a given forecast only provides accuracy that is too poor for your purposes, why would you use it? Or even comment on it?

Yeah, chances seem good considering your prediction ranges between 1PH (less than today, how likely is that) and >12PH
I dont even have to look at a weather map to be able to predict tomorrow temperatures will be between -20C and +40C with 95% confidence.

How do you get that? Difficulty lags hashrate, and times between retargets change change with hashrate. Since I'm forecasting a weekly average hashrate confidence interval, I don't think you can derive an exact figure from that. I do have a nice difficulty forecast which I'll post if there's any call for it.

Even if you could, as mentioned in the blog and several times on this page, the confidence intervals for the forecast are more the important than the forecast.

To avoid confusion in future, I might just provide several confidence intervals for the forecast, and not the forecast itself. Does that sound like a good idea?

Looks like you're saying that the difficulty factor will DOUBLE every month from now on.

Which current numbers? I'm forecasting the average weekly network hashrate - that's the total hashes submitted in a week divided by 604800. The week I'm forecasting ends on the 29th.

The last seven days' average hashrate is 1150.225 Thps. So the forecast will probably be under by 20% or so.

Ok.

How much is your forecast off by the current numbers?

I'm not sure what you mean. Can you rephrase your question?

Am I looking at this the wrong way or is your
eight weeks forecast off by 20% 3 days before your first week prediction?

I agree. The forecast will give a better estimate of the expected hashrate increase than x% per month, but I don't know how accurate shipping forecasts are. Again, the confidence intervals are very important since they provide a degree of confidence in the forecast. They should be used as discussed in the blog post, not as a defnite prediction of the network hashrate.


I can't really do that. Regressing to data that is error prone amplifies errors, and I have no idea how accurate that sort of model would be. Think back to before the network hashrate all but decoupled from the exchange rate - would you have been able to estimate when it occurred and the effect it could have?

The new model for the data is derived each week in a way that has minimised historical errors. I expected the forecasts to under predict in the upcoming weeks, but since the confidence intervals are bootstrapped, this is taken into account. For example, you can be reasonably certain that 50% of forecasts will be within the 50% confidence interval of the network hashrate to which that forecast relates.

The best option would be to estimate shipping times and amounts, which I'd done previously - but that ended up being much more error prone than this forecast method. It all depends on how accurate the data is, and so far most manufacturers have been notoriously unreliable and vague.

I think perhaps you do need to figure out a way to quantify the real world for inclusion in your model.

Your forecast seems to be like trying to describe the weather of Florida by taking wind and precipitation readings during a hurricane. There is value in doing this, because it does describe the hurricane, but when the hurricane ends and you start forecasting that "wind is below expectations", it doesn't mean your forecast is incorrect, it just means you are not accounting for if we are in or out of hurricane season.

We know that Moore's law predicts transistor count in a CPU and up until this year, bitcoin was not profitable enough for semiconductor manufacturers to make ASICs. Now it is profitable, so we're playing catch up, but eventually we will converge on Moore's law. If there was some way to approximate chip counts and feed it into your forecast, I think that would be very valuable. Then, you could forecast what the impact of making x chips available on the market is.

The model does well for historical increases in hashrate, even when the network hashrate increased much more rapidly in the past than it can possibly increase now, but rather poorly for abrupt decreases in hashrate (such as June 2011). Modelling the acceleration of the log of the hashrate ie diff(diff(log(hashrate))) make following upward trends more possible for the model. Of course, if someone dumps 2Phps on the network in two weeks' time then the forecast will be inaccurate. However if it take more than a week for al the machine to come online then the model should be flexible enough to respond. The percentage error will increase, though.

However I think you're missing the point of the forecasts. I'm not attempting to predict hashrate, rather to provide an indicator a bit more accurate (and well founded) than "difficulty (or hashrate) will increase by an average of 30% per month".


All good points, and as you say it's impossible to include whether a manufacturer will deliver on time and how much they will deliver. As I make clear in the post, I'm not attempting to figure that out  - that's a mugs game.

The confidence intervals are also important to pay attention to. They include historical fits to the network hashrate, which as I mentioned has already seen hashrate increases far outstripping whats happening now.

I appreciate what you're saying, but I'm pretty confident that confident that 95% of the weekly average hashrates will be inside the 95% confidence interval. However you might be right. Time will tell.

How well does this model work when you retroactively apply it to the periods where we transitioned from CPU to GPU or FPGAs ?

I appreciate you put a lot of work in to this, but building mathematical models to make predictions that dont factor in facts like that $10M's worth of next generation hardware is due to be delivered over the next few months, makes it usability rather questionable.  Its not like charts can predict if KnC will ship all that its promised in 2 weeks or wont ship anything at all, and that alone would roughly double difficulty.

I finally got around to updating the my hashrate forecasts.

These new forecasts are ARIMA models of the weekly average hashrate, forecast to eight weeks in the future, and with bootstrapped and surprisingly accurate confidence intervals. the model make no reliance on the previous BTC exchange rate relationship with the network hashrate, and also do not take into account manufacturer shipping dates and amounts.

More details here: http://organofcorti.blogspot.com/2013/09/105-forecasting-network-hashrate.html

The forecasts:




The errors:



Enjoy! Post any questions you might have, but do read the blog post first - it will probably answer most of your questions.