Topic: New more accurate network hashrate forecasts (Read 5559 times)

I will at some point - just need things to settle down a bit first.

omg please continue to report on that information, I think that is VERY useful data.

I've got the user hashrate distribution data which I update each week. However the average is not a very good indicator, and neither is the median. The hashrates per user account are generally pareto distributed, often with and undefined mean.

Secondly, I only have data for about half the network. When I had about 75% of the network I could extrapolate to the rest of the network. Now, it's just too inaccurate.

The last post I did on the subject was: http://organofcorti.blogspot.com/2013/07/network-electricity-consumption-16th.html


Thanks for the offer - PM sent.

organofcorti,

A thought - do you have enough information from the pools to be able to specify average GH/s per user account and get a total # of user accounts? I think if you compared that to hardware available at the time (in terms of either GH/s/$ or GH/s/W), you could perhaps determine a relationship between available GH/s/[$,W] and GH/s/user. Making the assumption that most people have a fixed amount of power/income they can provide to this effort, it might show how future changes to efficiency/pricing can impact total network hashrate. I think that would yield more accurate results than basing it on historical information.

Also, re: your other comments, I do know PHP fairly well and have a Linode that I'd be happy to host with. PM me if you're interested and we can put something together.

20% fee is only if you bet at the tail end of the betting window. If you place your bet in the beginning, its 0.
I love the idea of that site, but I just wish the subject of the bet wouldnt be someone else's ambiguous estimate of network hashrate, but something unambiguous like difficulty, number of blocks found, etc.

Again, that never occurred to me. It does sound like the best way to go about it.

Can I make a suggestion? Start a thread called "cool bitcoin shit I know" which I for one would find very helpful.

Futureblock is interesting. I think I'd use my skills for evil and bet, although the 20% house fee is a bit off-putting.

Id offer my help, but my PHP skills are really not very good. Perhaps you can team up with one of the sites that currently publish similar information, like thegenesisblock, blockchain.info or even futureblock.

I didn't think of getting a blockchain account and using the api - that's a great idea, and reading the api it looks like it would be pretty easy to implement. Now I just have to work on the first of the two problems

You dont have to run a local bitcoind, you could just use blockchain.info's api:
http://blockchain.info/api/json_rpc_api

Thanks for all your input, Its About Sharing, Puppet and Dabs. You've all given me some idea of interesting problems to work on. Sorry it's taken so long for me to get back to you.


The I can forecast four retargets (~ 6 weeks at the moment) ahead with reasonable accuracy, better than the 6 week average hashrate forecast anyway. I'll post here once I get a post up. I really do want to get the forecasts up weekly but finding the time is tricky.

Puppet's comment addresses a way around the posting problem:


Live updating would be fantastic. I have a domain, and I could easily write something automated to regenerate charts at a two hour lag (to allow for orphaned blocks) but I have two problems:

 * I have absolutely no experience in running or coding a site or a server
 * Not many hosts are ok with their server running bitcoind - I remember one guy who was kicked off his host for exactly that (I think they thought he was mining or something)

So I could do (and already do for myself) exactly what you're suggesting but I have no idea how to do it or even how to go about learning to do it, or if I have enough time to learn to do it. I'd welcome any hints or suggestions.


I actually do a separate model for Difficulty - it's more accurate than estimating based on an estimated hashrate, and also forecasts the date of the difficulty change too.

The model can't do a worst case scenario though, and forecasting that far ahead would be too inaccurate by any measure of use. Death and Taxes' thread is what you're after. A visualisation of the data in that thread is probably a good idea and I can do that - but it won't be a modelled forecast with confidence intervals and so forth.

I promise I'll get around to posting forecasts weekly (weekly average hashrate, daily average hashrate and difficulty) but I'm really focussed on forecasting BTC prices. I didn't think it could be done with any degree of usefulness, but as I learn and experiment more I'm finding out I was wrong. It's really exciting (if you're in to that sort of thing).

I didn't see that before and wondered who donated. Thanks for that!

ooc, can you translate your next forecast into bitcoin mining difficulty numbers? Like, 200 million, 300 million, 500 million, 1 billion, ... and when we can expect this?

Worst case scenario when all the new hardware comes online and hashing (this company said 2000 TH by January 2014, that company said 1000 TH, etc.)

Organ, may I suggest an analysis that I personally would find more interesting?
Not sure how to put it, but giving us an idea of how accurate live updating difficulty predictions and hashrate estimates are.

For instance, right now, when I visit
http://dot-bit.org/tools/nextDifficulty.php

I see a projection of 235M, but its based on ~600 blocks. What is the spread on that ?

Or put another way, I would like to see something similar as dot bit, but guessing *current* network hashrate and next difficulty adjustment with 50 and 95% confidence intervals.

This would have to be live updating to be really interesting tough.

I'm just looking for a difficult predictor to compliment what is already out there. It seems you are taking the hashing projections quite deep and so you probably will have one of the more accurate difficulty predictors. As far as timeline, it would be nice to have a 4 week projection. But what you are already doing with the range can allow you to take it out further (but perhaps that doesn't serve a real useful purpose so far out). That range (leeway with your blue shadows looking forward) is great.

Thanks for the quick reply btw,
IAS

I've already done this, just haven't gotten around to posting. Explaining what I'm doing takes way longer than actually doing it, and if I don't do a good job in the explanation then I get responses such as those at the top.

As for difficulty predictors, it depends on what you want. If you want something more than 4 retargets ahead, then you're better off checking out Death and Taxes' thread.

Thanks for sharing this. Quite interesting.

Are you planning on also publishing the equivalent in the Difficulty rate?
If not, would it be difficult to do that or ?

I ask because I'm looking for some new difficulty predictors. Any suggestions?

Thx,
IAS

I've done that, along with an exponential smooth. The confidence intervals make it a very rough guide. I'll get around to posting something when I have time. Or after I've made my millions first, heh. From memory I think a random walk + drift model based on ~ the last three months was as good as the ETS model.

Since I've lately been working on working out "Gambler's Ruin" in brownian motion, this got me to thinking that a brownian motion model might be fun to try. Rather than forecasts I'd be able to plot times of first passage (eg time to 150 usd/btc, 160 usd/btc etc) with confidence intervals - sort of a reverse forecast since the result would be times rather than prices. If I do get some time, I'll post the results. Even if it's not very accurate, it would be very cool.

Excellent work in my opinion, I've been looking at length for such kind of research. Satoshi sent to support your site.
Would you consider performing ARIMA analysis on BTC price?

So the point of the exercise is demonstrating just how pointless the exercise is :p
/tongue-in-cheek.

No, the eight week forecast interval is extremely wide. Part of the reason for this is that the network hashrate is increasing exponentially, so the confidence intervals are the exponential of the modelled confidence intervals. I wasn't sure I should include anything after the first four weeks, but I thought readers might find it interesting to see how it becomes harder to accurately estimate the network hashrate into the future. A tiny change the percentage rate of change now can make peta-hashes of difference a couple of months from now.


Getting an accurate and reliable model is the hard part. Once you have that, the forecasts follow. I could provide forecasts of the hashrate a billion years from now, if you want. It would have a fairly wide confidence interval though.


Although they only directly apply to the model, I think these confidence intervals should also give you an idea of how much confidence to place in anyones estimate of the hashrate months from now.  If someone estimates that the hashrate will increase by thirty percent per month for the next four months then they're probably going to be subject to a much larger confidence interval for the hashrate. Exponential changes in the hashrate mean exponential errors, and I think it's something that people don't generally take into account.