Author

Topic: KanoPool kano.is lowest 0.9% fee 🐈 since 2014 - Worldwide - 2432 blocks - page 1978. (Read 5352097 times)

legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
You lost me at "we use diff as a double everywhere"...... Wink

Jokes aside, you can even submit every share to bitcoind and test that way if you like. Doesn't matter to me. I just don't think skewing stats in the block list based on these non-blocks is reasonable.
Block stats are nothing to do with me. I was just explaining why I submit them to bitcoind.

Oh, yeah I already understood that. I personally think the diff as a double thing is a coder speedup more than a performance speedup, but it is what it is and you have to cover edge cases like that for sure. But when bitcoind says "nope!" then it should just be ignored and not thrown in the block list as if it actually meant something. And if it must be for posterity then the next block's stats should at least display correctly.

Just weird seeing the stats for real blocks being displayed so poorly due to basically a random share that is worthless.
Well I don't have much money so yeah I'm poor.

Edit: oh ok I'll answer properly even though your comment is silly.
Yeah on my block page I show all blocks found, even if they are orphans, and also show the 'Unworthy' blocks - yep extra information and nothing hidden ... unlike ... ...

If we submit a share to bitcoind it will show there, no matter what, nothing hidden.

But then of course if someone is actually trying to look at the pool luck, not at the block by block level where it really doesn't show you anything useful unless you got no idea about statistics Smiley but instead at the top there's a great table of block statistics with all sorts of numbers you wont find on the rich Eligius web page Smiley

Edit2: though that may be coz, if you did put the CDF[Erl] on your web site and showed a year of it, you'd get people screaming "Da Fuk!" why is it so bad?!?
legendary
Activity: 1223
Merit: 1006
You lost me at "we use diff as a double everywhere"...... Wink

Jokes aside, you can even submit every share to bitcoind and test that way if you like. Doesn't matter to me. I just don't think skewing stats in the block list based on these non-blocks is reasonable.
Block stats are nothing to do with me. I was just explaining why I submit them to bitcoind.

Oh, yeah I already understood that. I personally think the diff as a double thing is a coder speedup more than a performance speedup, but it is what it is and you have to cover edge cases like that for sure. But when bitcoind says "nope!" then it should just be ignored and not thrown in the block list as if it actually meant something. And if it must be for posterity then the next block's stats should at least display correctly.

Just weird seeing the stats for real blocks being displayed so poorly due to basically a random share that is worthless.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
You lost me at "we use diff as a double everywhere"...... Wink

Jokes aside, you can even submit every share to bitcoind and test that way if you like. Doesn't matter to me. I just don't think skewing stats in the block list based on these non-blocks is reasonable.
Block stats are nothing to do with me. I was just explaining why I submit them to bitcoind.
legendary
Activity: 1223
Merit: 1006
...
As for the unworthy stuff... while you've posted two blocks that didn't make the cut... I'd be more curious as to if you had any blocks that the pool thought wouldn't make the cut that bitcoind (and the bitcoin network) actually accepted.  Probably not.  Why?  Because math is math.  Just because bitcoind is in C++ and ckpool is in C and eloipool is in python doesn't change the comparison of two numbers.  It's either a block or it isn't.  This whole "unworthy" thing is just nonsense.
...
This is effectively directed at me so while I was trying not to get dragged into this...

We use diff as a double everywhere in ckpool which makes it super fast without nonsense converting to hex and doing comparisons and so on. Doubles are only accurate to integer sizes at 2^53.  Given diff is expressed differently (heh) in bitcoind compared to a double, and numbers of 2^53 may one day not be unbelievable in bitcoin, then add to the fact that even the fractional accuracy gets less the closer to 2^53 a double gets, and that while there are standards for how rounding should occur with doubles, but diff isn't expressed directly as a double in the blockchain... testing the extremely rare very close diff share is not going to hurt.

You lost me at "we use diff as a double everywhere"...... Wink

Jokes aside, you can even submit every share to bitcoind and test that way if you like. Doesn't matter to me. I just don't think skewing stats in the block list based on these non-blocks is reasonable.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
Payout 386010 sent (finally)
c8943e4a9a18625e18c333940134efe90481e8d68c86e2f8cc375f88a596c295
and confirmed
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
...
As for the unworthy stuff... while you've posted two blocks that didn't make the cut... I'd be more curious as to if you had any blocks that the pool thought wouldn't make the cut that bitcoind (and the bitcoin network) actually accepted.  Probably not.  Why?  Because math is math.  Just because bitcoind is in C++ and ckpool is in C and eloipool is in python doesn't change the comparison of two numbers.  It's either a block or it isn't.  This whole "unworthy" thing is just nonsense.
...
This is effectively directed at me so while I was trying not to get dragged into this...

We use diff as a double everywhere in ckpool which makes it super fast without nonsense converting to hex and doing comparisons and so on. Doubles are only accurate to integer sizes at 2^53.  Given diff is expressed differently (heh) in bitcoind compared to a double, and numbers of 2^53 may one day not be unbelievable in bitcoin, then add to the fact that even the fractional accuracy gets less the closer to 2^53 a double gets, and that while there are standards for how rounding should occur with doubles, but diff isn't expressed directly as a double in the blockchain... testing the extremely rare very close diff share is not going to hurt.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
...
More to come on that soon as I gather up the data I have and get it over to him for independent analysis.
Make sure you learn about 'confidence level' and apply it to your reply, coz your comment about my pool size and the number of orphans clearly suggests you have no idea about it.

Edit:
...
As for the unworthy stuff... while you've posted two blocks that didn't make the cut... I'd be more curious as to if you had any blocks that the pool thought wouldn't make the cut that bitcoind (and the bitcoin network) actually accepted.  Probably not.  Why?  Because math is math.  Just because bitcoind is in C++ and ckpool is in C and eloipool is in python doesn't change the comparison of two numbers.  It's either a block or it isn't.  This whole "unworthy" thing is just nonsense.
...
Haven't been a programmer for very long or not much experience with different languages hey? Smiley
legendary
Activity: 1223
Merit: 1006
...
I read the * note and the ? note.  Doesn't make block 378548 a lucky block, but it's shown as one.  Seems weird to try and display it based on only the shares since the "unworthy" block...

Just like 368390... it wasn't lucky at all, but the stats have it shown in green...

Yep, as I already said

...
The single line stats are on the single line.
...

On the right they show the same number so yes for those 4 blocks you'd have to add the 2 numbers together
... like the code link does for the top.

I understand your explanation.  And for orphan blocks it would make sense to display things like this.  Throwing these completely fake "unworthy" blocks in the mix just screws things up and makes real blocks look luckier than they really are.  That's all.
Looking at a single block to gauge the luck of the pool is pretty pointless.

The table at the top shows the pool luck ... pretty well with no confusion Smiley
... there's even a pretty good graph of the luck and all sorts of other information you can't see without an account Smiley

... and yep my usual answer to the question about ... "did you do it correctly" ... is ... "of course".
https://bitbucket.org/ckolivas/ckpool/src/0635c75560ee1823650d81008da2385f69843624/pool/page_luck.php?at=master&fileviewer=file-view-default#page_luck.php-91
I'd suggest a quite reasonable assumption with my code is that I did do it correctly.
If you find a mistake somewhere, I'll fix it, but asking if I did it correctly is pretty pointless unless you've found a bug somewhere.
All code has bugs ... but yeah unless you actually find one ... there's not much point asking if I did something correctly Tongue

However, with the blocks page itself, if I start doubling up data on the page, like you are suggesting, then I can see that causing problems for anyone trying to do anything with the data ... and I'm certainly not going to hide anything.

Yes I gather that you assume it's impossible for your code to get a different difficulty calculation than bitcoind ... even though your code is python and bitcoind is C++ - however we don't assume that to be the case and have a small margin of error to ensure we can't throw a block away.
The result is: yes so far twice, 2 "Unworthy" shares on the blocks page.
That information in itself is interesting - so much so that I also have my ckdb console report any shares that are within 5% of being a block (or are a block or better) and also stale/invalid shares that are within 5% of being a block ... or even stale but 'block worthy'
Yep a stale block would suck, and may or may not make it onto the blocks page depending on if it was stale vs bitcoind - in both cases it will show there as an orphan

Now lastly related to that ... regarding this:
...
Orphan rates have been on par and normal despite kano's trolling.
...
You checked that yet? Smiley

But this one ...
Kano's own pool has been running about a year, roughly 25% of the time that the current version of Eligius has been online and has mined 368 blocks as of this writing, a whopping 3% of the blocks Eligius has mined and has data on.  I don't know about you, but I'm reasonably certain Eligius has much more valid long term statistical data.  So while kano's young pool may have about 0.5% stale rate over 368 blocks, let's check back in about 30 years when that total block count catches up to what Eligius's today and see where things actually stand.  Kano saying Eligius's orphan rate is high is like a guy at the roulette table who is up a few % after a few hours going around telling people that this roulette table must be better. lol.  Pretty sure I'll stick with legitimate long term stats on the topic when discussing it.
lulz worthy ...

If you want to actually play with numbers, you need to check what your numbers actually mean.
The statistical term that matters is 'confidence interval' and it's not linear Smiley
https://en.wikipedia.org/wiki/Confidence_interval

Lots to reply to here...

First, yes, looking at a single block to gauge luck is pointless.  But looking at a single block to see how lucky that particular round was is actually important.  You incorrectly report that.

As for the unworthy stuff... while you've posted two blocks that didn't make the cut... I'd be more curious as to if you had any blocks that the pool thought wouldn't make the cut that bitcoind (and the bitcoin network) actually accepted.  Probably not.  Why?  Because math is math.  Just because bitcoind is in C++ and ckpool is in C and eloipool is in python doesn't change the comparison of two numbers.  It's either a block or it isn't.  This whole "unworthy" thing is just nonsense.

As for the rest of your post once again bring up orphan related things out of the blue again... I've been chatting with organofcorti about it.  He said that his data is 1.37% orphans based on data self-reported by pools representing only a fraction of the total network and that it's entirely possible he doesn't have enough data to calculate the actual network wide orphan rate.  More to come on that soon as I gather up the data I have and get it over to him for independent analysis.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
...
I read the * note and the ? note.  Doesn't make block 378548 a lucky block, but it's shown as one.  Seems weird to try and display it based on only the shares since the "unworthy" block...

Just like 368390... it wasn't lucky at all, but the stats have it shown in green...

Yep, as I already said

...
The single line stats are on the single line.
...

On the right they show the same number so yes for those 4 blocks you'd have to add the 2 numbers together
... like the code link does for the top.

I understand your explanation.  And for orphan blocks it would make sense to display things like this.  Throwing these completely fake "unworthy" blocks in the mix just screws things up and makes real blocks look luckier than they really are.  That's all.
Looking at a single block to gauge the luck of the pool is pretty pointless.

The table at the top shows the pool luck ... pretty well with no confusion Smiley
... there's even a pretty good graph of the luck and all sorts of other information you can't see without an account Smiley

... and yep my usual answer to the question about ... "did you do it correctly" ... is ... "of course".
https://bitbucket.org/ckolivas/ckpool/src/0635c75560ee1823650d81008da2385f69843624/pool/page_luck.php?at=master&fileviewer=file-view-default#page_luck.php-91
I'd suggest a quite reasonable assumption with my code is that I did do it correctly.
If you find a mistake somewhere, I'll fix it, but asking if I did it correctly is pretty pointless unless you've found a bug somewhere.
All code has bugs ... but yeah unless you actually find one ... there's not much point asking if I did something correctly Tongue

However, with the blocks page itself, if I start doubling up data on the page, like you are suggesting, then I can see that causing problems for anyone trying to do anything with the data ... and I'm certainly not going to hide anything.

Yes I gather that you assume it's impossible for your code to get a different difficulty calculation than bitcoind ... even though your code is python and bitcoind is C++ - however we don't assume that to be the case and have a small margin of error to ensure we can't throw a block away.
The result is: yes so far twice, 2 "Unworthy" shares on the blocks page.
That information in itself is interesting - so much so that I also have my ckdb console report any shares that are within 5% of being a block (or are a block or better) and also stale/invalid shares that are within 5% of being a block ... or even stale but 'block worthy'
Yep a stale block would suck, and may or may not make it onto the blocks page depending on if it was stale vs bitcoind - in both cases it will show there as an orphan

Now lastly related to that ... regarding this:
...
Orphan rates have been on par and normal despite kano's trolling.
...
You checked that yet? Smiley

But this one ...
Kano's own pool has been running about a year, roughly 25% of the time that the current version of Eligius has been online and has mined 368 blocks as of this writing, a whopping 3% of the blocks Eligius has mined and has data on.  I don't know about you, but I'm reasonably certain Eligius has much more valid long term statistical data.  So while kano's young pool may have about 0.5% stale rate over 368 blocks, let's check back in about 30 years when that total block count catches up to what Eligius's today and see where things actually stand.  Kano saying Eligius's orphan rate is high is like a guy at the roulette table who is up a few % after a few hours going around telling people that this roulette table must be better. lol.  Pretty sure I'll stick with legitimate long term stats on the topic when discussing it.
lulz worthy ...

If you want to actually play with numbers, you need to check what your numbers actually mean.
The statistical term that matters is 'confidence interval' and it's not linear Smiley
https://en.wikipedia.org/wiki/Confidence_interval
legendary
Activity: 1223
Merit: 1006
@ck - On a serious note, very nice.  Guess I'll have to work on one-upping you later on to get Eligius back to the top of that list while still doing full validation Smiley
Don't worry, when I remove the 0txn blockchange templates shortly you can move ahead in marketing once more - but if you're still behind then, especially considering my entire pool and bitcoind run on a 4 vcpu 4Gb ram vps...

Nah, it'll be fine.  Presumably you'd be back at work change speeds at least about where kano is now, which is behind me on average.  I still have a hundred ms or so of speedup possible with my proxy, too, that I haven't had time to code.

As long as my changes are on par with the top full-validating pools, I don't really care much.  I'm not into marketing. Tongue
Solo is back to creating full sized blocks on blockchanges (up to 1MB). Kano doesn't have half the code I used on solo but he has MUCH more horsepower at his disposal.

*best Tim the Tool Man impression* Har har har.  More horsepower!  Cheesy Cheesy
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
@ck - On a serious note, very nice.  Guess I'll have to work on one-upping you later on to get Eligius back to the top of that list while still doing full validation Smiley
Don't worry, when I remove the 0txn blockchange templates shortly you can move ahead in marketing once more - but if you're still behind then, especially considering my entire pool and bitcoind run on a 4 vcpu 4Gb ram vps...

Nah, it'll be fine.  Presumably you'd be back at work change speeds at least about where kano is now, which is behind me on average.  I still have a hundred ms or so of speedup possible with my proxy, too, that I haven't had time to code.

As long as my changes are on par with the top full-validating pools, I don't really care much.  I'm not into marketing. Tongue
Solo is back to creating full sized blocks on blockchanges (up to 1MB). Kano doesn't have half the code I used on solo but he has MUCH more horsepower at his disposal.
legendary
Activity: 1223
Merit: 1006
...
I read the * note and the ? note.  Doesn't make block 378548 a lucky block, but it's shown as one.  Seems weird to try and display it based on only the shares since the "unworthy" block...

Just like 368390... it wasn't lucky at all, but the stats have it shown in green...

Yep, as I already said

...
The single line stats are on the single line.
...

On the right they show the same number so yes for those 4 blocks you'd have to add the 2 numbers together
... like the code link does for the top.

I understand your explanation.  And for orphan blocks it would make sense to display things like this.  Throwing these completely fake "unworthy" blocks in the mix just screws things up and makes real blocks look luckier than they really are.  That's all.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
...
I read the * note and the ? note.  Doesn't make block 378548 a lucky block, but it's shown as one.  Seems weird to try and display it based on only the shares since the "unworthy" block...

Just like 368390... it wasn't lucky at all, but the stats have it shown in green...

Yep, as I already said

...
The single line stats are on the single line.
...

On the right they show the same number so yes for those 4 blocks you'd have to add the 2 numbers together
... like the code link does for the top.
legendary
Activity: 1223
Merit: 1006
@ck - On a serious note, very nice.  Guess I'll have to work on one-upping you later on to get Eligius back to the top of that list while still doing full validation Smiley
Don't worry, when I remove the 0txn blockchange templates shortly you can move ahead in marketing once more - but if you're still behind then, especially considering my entire pool and bitcoind run on a 4 vcpu 4Gb ram vps...

Nah, it'll be fine.  Presumably you'd be back at work change speeds at least about where kano is now, which is behind me on average.  I still have a hundred ms or so of speedup possible with my proxy, too, that I haven't had time to code.

As long as my changes are on par with the top full-validating pools, I don't really care much.  I'm not into marketing. Tongue
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
@ck - On a serious note, very nice.  Guess I'll have to work on one-upping you later on to get Eligius back to the top of that list while still doing full validation Smiley
Don't worry, when I remove the 0txn blockchange templates shortly you can move ahead in marketing once more - but if you're still behind then, especially considering my entire pool and bitcoind run on a 4 vcpu 4Gb ram vps...
legendary
Activity: 1223
Merit: 1006
I just noticed something weird on your pool stats page.  Your "unworthy" blocks make the luck/CDF stats for the actual block found afterwards completely wrong, don't they?

For example, you have an "unworthy" 378389 block with 129,706,365,253 shares since 377920 was found.  But the next block actually found, 378548, shows only 37,753,537,093 shares and an incorrectly low CDF based on shares since 377920, where in fact it took 167,459,902,346 shares to find 378548, which would be a pretty high CDF (bad luck).  Makes 378548 look lucky, when in fact it wasn't.  Looks the same for all blocks after an "unworthy" block.  Not sure on orphans, and I'll assume your aggregated stats at the top of the page don't have this issue, but I could be wrong.
Did you read the bottom line of the page? ...
There's a * on each Unworthy and Orphan
The single line stats are on the single line.
(and yeah Unworthy isn't a block, it's just a "very close to being a block" share - but not good enough to be a block.
The awesome stats at the top, take the numbers into consideration correctly as per the comment at the bottom.
https://bitbucket.org/ckolivas/ckpool/src/0635c75560ee1823650d81008da2385f69843624/src/ckdb_data.c?at=master&fileviewer=file-view-default#ckdb_data.c-3035

Edit: I would add re my last comment, I consider blank blocks the cause of SPV, and one really no different to the other in their bad effect on BTC, except that SPV is a faster way to do it.
SPV can be done with enough validation to avoid anything like the last forks.
The only real down side of proper SPV vs blank blocks is some time in the far distant future someone may make one block that causes problems with SPV - they'd need to be very lucky or have lots of hash rate Tongue

I read the * note and the ? note.  Doesn't make block 378548 a lucky block, but it's shown as one.  Seems weird to try and display it based on only the shares since the "unworthy" block...

Just like 368390... it wasn't lucky at all, but the stats have it shown in green...
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
I just noticed something weird on your pool stats page.  Your "unworthy" blocks make the luck/CDF stats for the actual block found afterwards completely wrong, don't they?

For example, you have an "unworthy" 378389 block with 129,706,365,253 shares since 377920 was found.  But the next block actually found, 378548, shows only 37,753,537,093 shares and an incorrectly low CDF based on shares since 377920, where in fact it took 167,459,902,346 shares to find 378548, which would be a pretty high CDF (bad luck).  Makes 378548 look lucky, when in fact it wasn't.  Looks the same for all blocks after an "unworthy" block.  Not sure on orphans, and I'll assume your aggregated stats at the top of the page don't have this issue, but I could be wrong.
Did you read the bottom line of the page? ...
There's a * on each Unworthy and Orphan
The single line stats are on the single line.
- and yeah Unworthy isn't a block, it's just a "very close to being a block" share - but not good enough to be a block.
The awesome stats at the top, take the numbers into consideration correctly as per the comment at the bottom.
https://bitbucket.org/ckolivas/ckpool/src/0635c75560ee1823650d81008da2385f69843624/src/ckdb_data.c?at=master&fileviewer=file-view-default#ckdb_data.c-3035

Edit: I would add re my last comment, I consider blank blocks the cause of SPV, and one really no different to the other in their bad effect on BTC, except that SPV is a faster way to do it.
SPV can be done with enough validation to avoid anything like the last forks.
The only real down side of proper SPV vs blank blocks is some time in the far distant future someone may make one block that causes problems with SPV - they'd need to be very lucky or have lots of hash rate Tongue
legendary
Activity: 1223
Merit: 1006
I just noticed something weird on your pool stats page.  Your "unworthy" blocks make the luck/CDF stats for the actual block found afterwards completely wrong, don't they?

For example, you have an "unworthy" 378389 block with 129,706,365,253 shares since 377920 was found.  But the next block actually found, 378548, shows only 37,753,537,093 shares and an incorrectly low CDF based on shares since 377920, where in fact it took 167,459,902,346 shares to find 378548, which would be a pretty high CDF (bad luck).  Makes 378548 look lucky, when in fact it wasn't.  Looks the same for all blocks after an "unworthy" block.  Not sure on orphans, and I'll assume your aggregated stats at the top of the page don't have this issue, but I could be wrong.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
I've no idea exactly what all his changes are he's been trialling ... I only use specific ones I've seen and then in a reduced version Tongue
But in my case there are never any blank blocks.
legendary
Activity: 1223
Merit: 1006
You and ck have apparently made changes to bitcoind to mitigate this issue to the point where your pool is almost as fast as Eligius at block changes and ck's solo pool is somehow faster.  The latter is pretty impressive without the blank block speed up because it means that not only is GBT time next to nothing, the time needed to actually validate the block is next to nothing.  So, not trivial changes.
I do indeed still include full validation. All of the changes are in bitcoind on the validation side but they are all non-portable changes, and things that I have no interest in getting past the committee that is bitcoin development, and ultimately I hate to say it but we're running businesses here. I still indeed do full validation but I noticed the current emphasis on benchmarketing block changes so I figured I'd do everything possible - this includes my own variation of blank blocks on blockchange from bitcoind (no, not the patch you are using). I am almost certainly not going to leave it this way but I had set out to try and beat every fully validating node out there Smiley

Aha!  Blank blocks! *points* *points*  I knew it! Tongue  Kano!  Kano!  CK is mining blank blocks!  *tattles*

lol

@ck - On a serious note, very nice.  Guess I'll have to work on one-upping you later on to get Eligius back to the top of that list while still doing full validation Smiley
Jump to: