Pages:
Author

Topic: Weekly pool and network statistics - page 13. (Read 91461 times)

legendary
Activity: 1260
Merit: 1000
November 18, 2013, 02:23:55 AM
It's possible, yes...

I think I found one other place that is not showing the shares properly and it might affect the share count for a round... I need to investigate it further, but it's kinda late here.  I will get to it ASAP.  The share count should definitely be more accurate but I'm not sure it's 100% accurate without further investigation.  The way I have PPS, Anonymous PPS and DGM shares in the tables makes collating all those shares properly somewhat difficult, as they all reside in different tables in the database, which makes paying the shares much easier.  If it had it to do over, I'd have designed it very differently, but with all projects that kind of grow as time goes along, it's kind of convoluted now.

The pool originally started as Prop, then switched to GM, then to DGM, then added PPS, then added Anonymous PPS... ugh.

donator
Activity: 2058
Merit: 1007
Poor impulse control.
November 18, 2013, 01:45:02 AM
@Organofcorti:  The crazy luck I think have may been due to anonymous PPS shares not being included in the round totals for share count.  I've corrected that issue and share counts should be proper going forward.


Thanks Inaba - I've updated the blog post to reflect that. Do you think this could have also created the hashrates spikes I've been seeing?
legendary
Activity: 1260
Merit: 1000
November 17, 2013, 02:28:32 PM
@Organofcorti:  The crazy luck I think have may been due to anonymous PPS shares not being included in the round totals for share count.  I've corrected that issue and share counts should be proper going forward.
sr. member
Activity: 658
Merit: 250
November 17, 2013, 01:27:17 PM
What does "CDF" stand for?
Cumulative distribution function.

Aha, well I wish I could say that was helpful.

What is the theory of operation behind that term?
Did you read wikipedia?

In short, it reflects luck. The lower it is, the better the luck is.
CDF can tell you the probability of a block being found with a given number of shares.
50% value corresponds to expected value, i.e. neutral luck.

50% is indeed the "median", rather than "mean" (expected value).

If my understand is correct, block finding is like a poisson process and number of blocks found should follow poisson distribution.
Of course, the rate parameter changes continuously as miners go on and off, and every 2016 blocks at difficulty adjustment.


This is correct - the CDF measures the probability that a random variable will be larger or (in this case) smaller than a given random variable.

Here, the random variable is the average (d1 shares per round)/network difficulty for a given number of rounds, which is described by the Erlang distribution.

You'll notice for example that Itzod, with an average (d1 shares per round)/network difficulty of 1.19  had a CDF of 0.73, with 7 blocks solved. This means that 73% of the time a pool solves 7 blocks, the average  (d1 shares per round)/network difficulty  will be lower - and luck will be better.

OTOH   GHash.IO  had an average of  1.11 for 261 blocks soved, with a CDF of 0.96, so 96% of the time 261 blocks are solved, the average will be less than 1.11.

The more blocks solved, the "narrower" the CDF.

I've tried to explain it simply so I've left some things out. If it doesn't make sense, ask again and I'll try to make it a bit more clear.



Very good explanation, thanks! I had always wondered how the CDF could vary so much on pools with similar (d1 shares per round)/network difficulty values, but that bolded part made it embarrassingly simple to understand.
donator
Activity: 2058
Merit: 1007
Poor impulse control.
November 17, 2013, 05:40:32 AM
17th November 2013 weekly pool and network statistics


Other weekly pool and network statistics posts

Welcome, miners.


Changelog:

    Added BTCDig - thanks for the API, dbitcoin.

Usual pools missing from results:

    Deepbit - no blocks solved this week.
    Ozcoin -no user hashrate data this week. Cloudflare prevents my attempts to screen scrape the relevant tables, so I won't be able to include Ozcoin in Figure 8 until that changes.

Errors:

    Nil

Pools with coinbase signature:

    ASICMiner: "Mined By ASICMiner"
    Alydian5335: "Alydian5335"
    Bitparking: "bitparking"
    BitMinter:  "BitMinter"
    BTCGuild :  "Mined by BTC Guild", "BTC Guild DE", "BTC Guild 2", "BTC Guild US2", plus a new signature, "BTC Guild GW"
    CoinLab: "CoinLab"
    Discus Fish: "七彩神仙鱼" and "Made in China"
    EclipseMC: "EMC"
    Eligius: "Eligius"
    50BTC.com: "Hi from 50BTC.com" and  "50BTC.com"
    GHash.IO: "ghash.io" (not current)
    GIVE-ME-COINS.com: "Mined at GIVE-ME-COINS.com"
    HHTT: "HHTT"
    Megabigpower "megabigpower.com"
    175btc.com": "Mined By 175btc.com"
    Ozcoin: "ozcoin"
    Pierce and Paul: "For Pierce and Paul"
    Triplemining: "Triplemining.com"
    Slush's pool: "slush"

Recent coinbase messages:

    258692 "To my honey, by bitfish."
    259575 259622 259625  "EMC: Organofcorti lives!"
    263952 "btcpoolman"

Pool hopping:

    Nil.

1. EclipseMC has crazy good luck

An average of 0.49 shares per round / mining difficulty for 41 solved blocks has a CDF of 0.000028. This means that luck as good as or better than that should occur only once every 35, 661 repeats of 41 blocks solved. This is very unlikely - there has only been about 6, 500 groups of 41 blocks solved ever. So either Eclipse has had extremely good luck, or there's a problem with the data. Given the strange changes in hashrate per round, it may be the latter so I'll find some time to audit the script I use to pull Eclipse's data.

2. Most pools lose some of their network; GHash.IO challenges BTCGuild.

The network hashrate is back up to 126% of expected, and the only pools to show an increase in their proportion of the network are GHash.IO, Discus Fish and ASICMiner. Since the latter two are small pools/entities, is seems that GHash.IO has increased it proportion of the network at everyone else's expense. In other words, much of the extra 262 extra blocks solved can be attributable to GHash.IO. At this rate it won't be long before they overtake BTCGuild.

Now would be a good time to throw some hashes as some of the smaller pool. Spreading your hashes amongst several pools will reduce your overall income variance and also help keep the network less centralised.

3. 50BTC now at 2Thps.

Seems like miners are returning. Check the latest gossip in the 50BTC thread on the bitcointalk.org forum.



As usual, please post comments if there's anything you don't understand, with which you disagree, or just think is wrong.
You can view this weeks charts at http://organofcorti.blogspot.com.au/2013/11/november-17th-2013-weekly-pool-and.html

You can view all previous charts at http://organofcorti.blogspot.com.au/search?q=weeklypoolstatistics and other posts and fun things at http://organofcorti.blogspot.com. Follow me on Twitter @oocBlog for notification of new posts as soon as I publish.






legendary
Activity: 3586
Merit: 1098
Think for yourself
November 12, 2013, 09:18:06 AM
I've tried to explain it simply so I've left some things out. If it doesn't make sense, ask again and I'll try to make it a bit more clear.

Digesting.
Thanks all,
Sam

So, looking at Eclipse, .68 for (d1 shares per round)/network difficulty, looks pretty good.  But the chance of getting that luck is only 4%?


Not quite, but almost.

Rather than the probability of an average of 0.68 (over 26 rounds) being equal to 0.04 (or 4%), it's actually the probability of an average of 0.68 or less.



There we go, I got it now.
Thanks,
Sam
donator
Activity: 2058
Merit: 1007
Poor impulse control.
November 12, 2013, 07:58:35 AM
I've tried to explain it simply so I've left some things out. If it doesn't make sense, ask again and I'll try to make it a bit more clear.

Digesting.
Thanks all,
Sam

So, looking at Eclipse, .68 for (d1 shares per round)/network difficulty, looks pretty good.  But the chance of getting that luck is only 4%?


Not quite, but almost.

Rather than the probability of an average of 0.68 (over 26 rounds) being equal to 0.04 (or 4%), it's actually the probability of an average of 0.68 or less.(if you're trying to calculate this at home, keep in mind these are round values and won't give exactly the same results).

legendary
Activity: 3586
Merit: 1098
Think for yourself
November 12, 2013, 07:25:13 AM
I've tried to explain it simply so I've left some things out. If it doesn't make sense, ask again and I'll try to make it a bit more clear.

Digesting.
Thanks all,
Sam

So, looking at Eclipse, .68 for (d1 shares per round)/network difficulty, looks pretty good.  But the chance of getting that luck is only 4%?
legendary
Activity: 3586
Merit: 1098
Think for yourself
November 12, 2013, 07:18:09 AM
I've tried to explain it simply so I've left some things out. If it doesn't make sense, ask again and I'll try to make it a bit more clear.

Digesting.
Thanks all,
Sam
donator
Activity: 2058
Merit: 1007
Poor impulse control.
November 12, 2013, 06:05:09 AM
What does "CDF" stand for?
Cumulative distribution function.

Aha, well I wish I could say that was helpful.

What is the theory of operation behind that term?
Did you read wikipedia?

In short, it reflects luck. The lower it is, the better the luck is.
CDF can tell you the probability of a block being found with a given number of shares.
50% value corresponds to expected value, i.e. neutral luck.

50% is indeed the "median", rather than "mean" (expected value).

If my understand is correct, block finding is like a poisson process and number of blocks found should follow poisson distribution.
Of course, the rate parameter changes continuously as miners go on and off, and every 2016 blocks at difficulty adjustment.


This is correct - the CDF measures the probability that a random variable will be larger or (in this case) smaller than a given random variable.

Here, the random variable is the average (d1 shares per round)/network difficulty for a given number of rounds, which is described by the Erlang distribution.

You'll notice for example that Itzod, with an average (d1 shares per round)/network difficulty of 1.19  had a CDF of 0.73, with 7 blocks solved. This means that 73% of the time a pool solves 7 blocks, the average  (d1 shares per round)/network difficulty  will be lower - and luck will be better.

OTOH   GHash.IO  had an average of  1.11 for 261 blocks soved, with a CDF of 0.96, so 96% of the time 261 blocks are solved, the average will be less than 1.11.

The more blocks solved, the "narrower" the CDF.

I've tried to explain it simply so I've left some things out. If it doesn't make sense, ask again and I'll try to make it a bit more clear.

hero member
Activity: 868
Merit: 1000
November 10, 2013, 04:37:43 PM
What does "CDF" stand for?
Cumulative distribution function.

Aha, well I wish I could say that was helpful.

What is the theory of operation behind that term?
Did you read wikipedia?

In short, it reflects luck. The lower it is, the better the luck is.
CDF can tell you the probability of a block being found with a given number of shares.
50% value corresponds to expected value, i.e. neutral luck.

50% is indeed the "median", rather than "mean" (expected value).

If my understand is correct, block finding is like a poisson process and number of blocks found should follow poisson distribution.
Of course, the rate parameter changes continuously as miners go on and off, and every 2016 blocks at difficulty adjustment.
legendary
Activity: 3586
Merit: 1098
Think for yourself
November 10, 2013, 03:39:27 PM
What does "CDF" stand for?
Cumulative distribution function.

Aha, well I wish I could say that was helpful.

What is the theory of operation behind that term?
Did you read wikipedia?

In short, it reflects luck. The lower it is, the better the luck is.
CDF can tell you the probability of a block being found with a given number of shares.
50% value corresponds to expected value, i.e. neutral luck.

Nope I hadn't read wikipedia.  I thought that was a Bitcoin specific term.  But reading, well looking at, the wikipedia article doesn't look too helpful since I'm not a statistician.

Your explanation does help though.
Thanks,
Sam
legendary
Activity: 1386
Merit: 1009
November 10, 2013, 02:25:31 PM
What does "CDF" stand for?
Cumulative distribution function.

Aha, well I wish I could say that was helpful.

What is the theory of operation behind that term?
Did you read wikipedia?

In short, it reflects luck. The lower it is, the better the luck is.
CDF can tell you the probability of a block being found with a given number of shares.
50% value corresponds to expected value, i.e. neutral luck.
legendary
Activity: 3586
Merit: 1098
Think for yourself
November 10, 2013, 01:58:52 PM
What does "CDF" stand for?
Cumulative distribution function.

Aha, well I wish I could say that was helpful.

What is the theory of operation behind that term?
legendary
Activity: 1386
Merit: 1009
November 10, 2013, 01:54:27 PM
What does "CDF" stand for?
Cumulative distribution function.
legendary
Activity: 3586
Merit: 1098
Think for yourself
November 10, 2013, 01:46:01 PM
What does "CDF" stand for?
legendary
Activity: 1750
Merit: 1007
November 10, 2013, 12:06:24 PM
Thanks again for your work.  I'm actually relieved that the BTC Guild CDF wasn't *that* bad.  That 24 hour period of bad luck was brutal, and felt like it was going to drag the whole week down to a 0.99-1.00 CDF.
donator
Activity: 2058
Merit: 1007
Poor impulse control.
November 10, 2013, 08:16:58 AM

10th November 2013 weekly pool and network statistics


Other weekly pool and network statistics posts

Welcome, miners.


Changelog:

    Added GIVE-ME-COINS.com.
    A reader reported that the confidence intervals for the network hashrate in figure 2 were very hard to see. I have removed the gray background and reduced the line width to make the confidence interval a bit clearer.
    The gray background has also been removed from the pie chart - it also has some transparency so I think a white background will allow better contrast.

Usual pools missing from results:

    50BTC and Deepbit - no blocks solved this week.

Errors:

    Nil

Pools with coinbase signature:

    ASICMiner: "Mined By ASICMiner"
    Alydian5335: "Alydian5335"
    Bitparking: "bitparking"
    BitMinter:  "BitMinter"
    BTCGuild :  "Mined by BTC Guild", "BTC Guild DE", "BTC Guild 2", "BTC Guild US2", plus a new signature, "BTC Guild GW"
    CoinLab: "CoinLab"
    Discus Fish: "七彩神仙鱼" and "Made in China"
    EclipseMC: "EMC"
    Eligius: "Eligius"
    50BTC.com: "Hi from 50BTC.com" and  "50BTC.com"
    GHash.IO: "ghash.io" (not current)
    GIVE-ME-COINS.com: "Mined at GIVE-ME-COINS.com"
    HHTT: "HHTT"
    Megabigpower "megabigpower.com"
    175btc.com": "Mined By 175btc.com"
    Ozcoin: "ozcoin"
    Pierce and Paul: "For Pierce and Paul"
    Triplemining: "Triplemining.com"
    Slush's pool: "slush"

Recent coinbase messages:

    258692 "To my honey, by bitfish."
    259575 259622 259625  "EMC: Organofcorti lives!"
    263952 "btcpoolman"

Pool hopping:

    Nil.


 

1. Network drops back even further - 1150 blocks solved this week.

Perhaps it's the effect of KNC finishing shipping and the changes in difficulty that caused, but the number of blocks solved this week was 1150 - only 14% more than expected. Hopefully the next change in difficulty wont be as extreme as the last few.

2. Most big pools have bad luck this week.

In order of crappiness of luck (worst luck to best luck):

Code:
                Pool Mean shares/round/difficulty  CDF
1              Itzod           1.19                0.73
2        Giga's pool           1.12                0.75
3           GHash.IO           1.11                0.96
4              Slush           1.08                0.76
5           BTCGuild           1.05                0.82
6          Bitminter           1.04                0.65
7            Eligius           0.94                0.22
8         Bitparking           0.79                0.36
9             p2Pool           0.71                0.11
10           Polmine           0.69                0.24
11           Eclipse           0.68                0.04
This means is was probably a bad luck week for the network. However, it should be noted that the CDF for all pools were within a 95% confidence interval and so when averaged over the week none had unusually good or bad luck.

4. 50BTC still at 4Ghps.

Still no news, almost no-one mining, but lots of anger in the 50BTC thread on the bitcointalk.org forum.


5. Polmine: more bouncy hashrates.

(see figure 4) A commenter on last weeks' post named 'lenny' had this to say:

    Polmine's spikes in hashrate happening, because main server is running out of processing power. Right now, according to main pool operator - megavega, background thread is processing backlog of accepted shares during last couple of days, process has reached 60% so far. That's why hashrate right now seems to be higher than it's really is, and was reported lower than it really was in last couple of days.

This may be correct, however the "hashrate spikes" have been occurring since the start of the dataset - so while this may explain one of last weeks' spikes, it doesn't explain them all. Any other ideas?


Polmine's hashrate per round:

Code:
        Date   Thps
1 2013-11-04  14.36
2 2013-11-04   6.17
3 2013-11-05 178.77
4 2013-11-07  11.78
5 2013-11-09   5.23
6. Proofreading

Proofreading payments total so far: 0.105 btc.

Thanks to everyone who has submitted error reports so far! I appreciate all the help I can get. Note that with the recent increase in the exchange rate, the current payment per error has automatically dropped to 0.001 btc.
 
As usual, please post comments if there's anything you don't understand, with which you disagree, or just think is wrong.
You can view this weeks charts at http://organofcorti.blogspot.com.au/2013/11/november-10th-2013-weekly-pool-and.html

You can view all previous charts at http://organofcorti.blogspot.com.au/search?q=weeklypoolstatistics and other posts and fun things at http://organofcorti.blogspot.com. Follow me on Twitter @oocBlog for notification of new posts as soon as I publish.






donator
Activity: 2058
Merit: 1007
Poor impulse control.
November 03, 2013, 07:55:15 AM
3rd November 2013 weekly pool and network statistics


Other weekly pool and network statistics posts

Welcome, miners.

Changelog:
Updated the table and all charts but the user hashrate density chart to Thps. User hashrates are still best measured in Ghps.
"Unknown" has been in the pie chart for a while; now I've also added it to the table, along with confidence intervals for the hashrate.
Pie chart has been changed to be prettier (and hopefully easier to read).
Pool hopping chart for Deepbit has been changed to include the last ~128 days. I'll reduce this if Deepbit's share of the network rises again.

Usual pools missing from results:
50BTC - no blocks solved this week.

Errors:
Nil

Pools with coinbase signature:
ASICMiner: "Mined By ASICMiner"
Alydian5335: "Alydian5335"
Bitparking: "bitparking"
BitMinter:  "BitMinter"
BTCGuild :  "Mined by BTC Guild", "BTC Guild DE", "BTC Guild 2", "BTC Guild US2", plus a new signature, "BTC Guild GW"
CoinLab: "CoinLab"
Discus Fish: "七彩神仙鱼" and "Made in China"
EclipseMC: "EMC"
Eligius: "Eligius"
50BTC.com: "Hi from 50BTC.com" and  "50BTC.com"
GHash.IO: "ghash.io" (not current)
HHTT: "HHTT"
Megabigpower "megabigpower.com"
175btc.com": "Mined By 175btc.com"
Ozcoin: "ozcoin"
Pierce and Paul: "For Pierce and Paul"
Triplemining: "Triplemining.com"
Slush's pool: "slush"
Recent coinbase messages:
258692 "To my honey, by bitfish."
259575 259622 259625  "EMC: Organofcorti lives!"
263952 "btcpoolman"

Pool hopping:
Nil.

1. Network drops back to solving 1330 blocks during the past week.
Since February this year, (the start of the ASIC induced network hashrate increase ) ~ 1200 blocks have been solved per week, (8.4 minutes per block). For the last couple of weeks as KNC miners were delivered, the network was solving ~ 1500 blocks per week (6.72 minutes per block). Now, the network has returned to its previous background acceleration - until the next batch of ASICs ship.

2. 50BTC at 4Ghps.
Almost no one is mining at 50BTC - after the last announcement, I guess they're waiting to see what happens next.

3. Eclipse and Polmine: Ups and downs, again.
(see figure 4) I've noticed Eclipse's bouncing hashrates for a while now. It's not beyond reason that the 40Thps spikes this week could be due to intermittently added hashrates. However, the hashrate changes per block that Polmine has been experiencing must be due to data error:
        
Code:
Date        Thps
1 2013-10-29   2.9978219
2 2013-10-30   3.0793512
3 2013-10-31 147.9324923
4 2013-11-02   0.2168786
5 2013-11-02 101.8922103

I haven't been able to figure out what the problem might be just yet.

5. Proofread for me.
Most of my posts are completed around 2am, and when it's late my proofreading abilities are somewhat poor. Plus my spacebar seems to be sticking and I don't always notice it. So if you find a typo or spelling error in my blog posts, email me the error or errors (highlighted in red) and if you're the first to email them, I'll pay you per error:
 
As usual, please post comments if there's anything you don't understand, with which you disagree, or just think is wrong.
You can view this weeks charts at http://organofcorti.blogspot.com.au/2013/11/november-3rd-2013-weekly-pool-and.html

You can view all previous charts at http://organofcorti.blogspot.com.au/search?q=weeklypoolstatistics and other posts and fun things at http://organofcorti.blogspot.com. Follow me on Twitter @oocBlog for notification of new posts as soon as I publish.






donator
Activity: 2058
Merit: 1007
Poor impulse control.
October 27, 2013, 05:57:34 AM
27th October 2013 weekly pool and network statistics


Welcome, miners.

Changelog:
Polmine added to the weekly statistics. A big thank you to Polmine for creating a special API for yours truly, and another big thank you to bitcointalk forum member pajak666 for making the request, and then making sure I didn't forget to add it to the weekly stats.
Updated some of the charts to Thps. Yet to update the table, poolhopping chart and  user hashrate density chart.
GHash.IO is back in the charts. Thanks for getting the glitch sorted for me, Alex. The stats just weren't the same without GHash.IO.
Usual pools missing from results:
None
Errors:
Nil
Pools with coinbase signature:
ASICMiner: "Mined By ASICMiner"
Alydian5335: "Alydian5335"
Bitparking: "bitparking"
BitMinter:  "BitMinter"
BTCGuild :  "Mined by BTC Guild", "BTC Guild DE", "BTC Guild 2", "BTC Guild US2", plus a new signature, "BTC Guild GW"
CoinLab: "CoinLab"
Discus Fish: "七彩神仙鱼" and "Made in China"
EclipseMC: "EMC"
Eligius: "Eligius"
50BTC.com: "Hi from 50BTC.com" and  "50BTC.com"
GHash.IO: "ghash.io" (not current)
HHTT: "HHTT"
Megabigpower "megabigpower.com"
175btc.com": "Mined By 175btc.com"
Ozcoin: "ozcoin"
Pierce and Paul: "For Pierce and Paul"
Triplemining: "Triplemining.com"
Slush's pool: "slush"
Recent coinbase messages:
258692 "To my honey, by bitfish."
259575 259622 259625  "EMC: Organofcorti lives!"
263952 "btcpoolman"

Pool hopping:
Nil.

1. Network solves 1508 blocks this week.
Another extreme block solve rate, leading to another ~ 50% increase in difficulty. I think most miners are going to have trouble making enough to pay for electricity soon. On the upside, there will be very few CPU and GPU miners left, and once most pools adopt a minimum share submission difficulty, there will be fewer still.

2. The large get larger.
It seems that miners want the reduction in variance that a large pool brings - the top five pools all saw an increase in their share of the network, where as the lower five saw a reduction in their share of the network. I hazard a guess many of these are 50BTC refugees (see section 5, below) used to the low variance a large pool brings, and they probably feel safer at a large well established pool.

I would like to point out here that spreading your hashrate among various pools not only helps keep the network less centralised, but also reduces your variance significantly over using just one pool. Send a few shares to a smaller pool and help the network.

3. Eligius takes 2nd.
Eligius continues it's meteoric rise in hashrate. Anyone know why? It's a recent change and extreme. Good luck to wizkid!

4. Eclipse and Polmine have similar ups and downs.
(see figure 4) Eclipses hashrate continues to bounce, and Polmine's is quite similar. I've double checked my scripts, so I have no idea why this would be. Big hashrates being added and removed every second block? Errors in the data? I have no idea. But it is very unusual.

5. 50BTC is really down.
The news that 50BTC.com was hacked and many coins stolen hit the community hard. This was quite a large pool (although no longer) and many miners have been affected. Although they seem to be attempting to work their way through the mess, 50BTC.com spokespeople have been slow to reassure their English speaking miners. Many have been upset that they haven't been able to withdraw BTC, and many have changed pools. FWIW, I think 50.BTC have planned to be here for the long haul -  but after this last hack they may not be able to recover.

Good luck 50.BTC - we hope you can recover and prosper.


As usual, please post comments if there's anything you don't understand, with which you disagree, or just think is wrong.
You can view this weeks charts at http://organofcorti.blogspot.com/2013/10/october-20th-2013-weekly-pool-and.html

You can view all previous charts at http://organofcorti.blogspot.com.au/search?q=weeklypoolstatistics and other posts and fun things at http://organofcorti.blogspot.com. Follow me on Twitter @oocBlog for notification of new posts as soon as I publish.






Pages:
Jump to: