Topic: Does a high pool difficulty lower anyone's profits? - page 3. (Read 4420 times)

In other pools, VARDIFF select 64 difficulty for my hashrate,

How VARDIFF decide on that? (maybe this could help us to find the answer)

Looking at a graph of a poisson distribution, it starts low, peaks, then falls off. That's not the right distribution to use here. We need to start high, and fall off. The peak should be the very first sample.

If you are using a difficulty such that you have a 50% chance of getting a share each hash, the distribution will look like so:
1st hash: 50%
2nd hash: 25%
3rd hash: 12.5%
4th hash: 6.25%
and so on, dividing by two each time.

So whatever kind of distribution you call that.

Alright. In the main forum, I posted my simulation script showing the skew towards fast miners. If you want the background, read that.

The main criticism was a fair one - I was "cheating" and using 0.5x and 1.5x of the average as the range for my random values for the share time, and block times. It was a shortcut that, admittedly skewed the results, but in such a negligible way that I felt it would still expose the effect. It was essentially cutting off the tiny sliver on the far right end the pierces out into infinity (plus some of the end on both sides). Its a minuscule amount of the distribution, but it was certainly a fair criticism.

So I did some research. The appropriate distribution is the poisson distribution. This is the same distribution that is used by bitcoin itself to calculate the expected difficulty for the next difficulty change. It is used in the prediction of the network, to maintain the statistical probability of one block every 10 minutes. Bitcoin would collapse without this prediction.

So it turns out theres a wonderful module in python called numpy. It has a built in poisson distribution random generator, making my life easy and making the change a breeze.

The main point about the poisson distribution, that should address the concerns: The Poisson distribution is not symmetrical; it is skewed toward the infinity end.

I also found and fixed a bug in my stats calculation. It didn't change the results enough to invalidate my previous conclusion, but these results should be accurate.

So with that, I reran my tests from before.


Slowest Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 7.29973071787 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 8.33333333333X the block-find-speed

Slow Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 6.86225439815 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 4.0X the block-find-speed

Medium Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 6.00872764122 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 2.0X the block-find-speed

Fast Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 4.23694576719 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 1.0X the block-find-speed

Very Fast Coin

Worker 1 has: 7.69230769231 percent of the hash power
But worker 1 has: 0.0129950864524 percent of the profit
Over sample size of 100000
When worker1's average share-find-speed was: 0.5X the block-find-speed

As you can see, the results are very similar.



And of course, obligatory code so you know I am not full of shit and can try for yourself.

<sub>import random
import numpy as np

class worker():
    def __init__(self,hashrate):
        self.hashrate = hashrate
        self.sharesolvetime = 60 / hashrate
        self.shares = 0

class pool():
    def __init__(self,blockfindtime):
        self.blockfindtime = blockfindtime

pool1 = pool(500)
worker1 = worker(1)
worker2 = worker(12)
samplesize = 100000

for n in range(0,samplesize):
    clock = np.random.poisson(pool1.blockfindtime)
    clock1 = clock
    while clock1 > 0:
        sharesolve = np.random.poisson(worker1.sharesolvetime)
        if sharesolve > clock1:
            break
        else:
            worker1.shares = worker1.shares + 1
            clock1 = clock1 - sharesolve
    clock2 = clock
    while clock2 > 0:
        sharesolve = np.random.poisson(worker2.sharesolvetime)
        if sharesolve > clock2:
            break
        else:
            worker2.shares = worker2.shares + 1
            clock2 = clock2 - sharesolve
    
print "Worker 1 has: " + str((float(worker1.hashrate) / float(worker2.hashrate + worker1.hashrate)) * 100) + ' percent of the hash power'
print "But worker 1 has: " + str((float(worker1.shares) / float(worker2.shares + worker1.shares)) * 100) + ' percent of the profit'
print "Over sample size of " + str(samplesize)
print "When worker1's average share-find-speed was: " + str((float(pool1.blockfindtime) / float(worker1.sharesolvetime))) + 'X the block-find-speed'</sub>
    



If you want to run it yourself, you need numpy. http://www.numpy.org/

visit pools in my sig ... i have tuned the pplns to cover for exactly what you are talking about  i noticed this happening a long time ago and couldnt stand seeing the little miners get poor pplns score. my diff is set to 63 which will save everyone bandwidth but the pplns scoring i use will even what you were asking about.

If the block changes every 30 seconds on average, and you find a share every 30 seconds on average, and someone else finds a share every 5 seconds, how do you have the same chances?

You'll completely whiff on any given block just as often as you get one share.. meanwhile the other guy gets 6 in. If he has a little bad luck, he gets 5. You have a little bad luck? you get 0. But, it all evens out right? A little bad luck this time, a little good luck next time. Of course every time you have a little good luck, you don't get 2, you'll still get 1. He'll get 7

You each have a good block and a bad block. You have 1 share. He has 12.

1/12 is not the same ratio as the ratio as 1/6.

I think of a higher difficulty in a pool as solo-mining with a lower difficulty.  If you want to try to be lucky for a day as a GPU or Block Erupter miner, set your stratum difficulty to 128 or 256 and watch the variance kick in.

That's just not the case. He has the exact same chances per hashrate of finding a share before the block changes as a high hashrate miner. Just with a higher variance.

Because the block changed. We he finds one, its a whole new block by then. All his work on that last block will never be given credit.

Meanwhile, fast miner got 9 on the last block (or 8, or 10, whatever you like). Even if slow miner finds a block this time, fast miner probably got 8-12 shares again.

Over time, the block changing hurts the slow miner more.

I think you can still simulate their race in sequence by seeing how long each one takes to get something, and compare their "times" to see who won.

How is that a problem? The slow miner hasn't found a share yet. Whenever he does find one, he'll report it to the pool and get credited.

I also made one in the old thread, but again, Liquidfire didn't even answer. https://bitcointalksearch.org/topic/m.2988147


An anology with a die would is much better, it's simpler and more extensible.

When a block changes, every miner will have been working towards their next share.

But one share means different things for different miners. Losing one share for a slow miner is a bigger hit than for a very fast miner.

In the time the slow miner was working, getting nothing (yet), the fast miner already got work in, and got it accepted.

Higher ratio of recognized work / work actually done.

Look forward to your simulations...

but.. i think the argument is invalid.

Say the duration to the interruption is 20 seconds.

A 100 GH/s miner will have exactly 10x the odds of finding a share than 10 GH/s miner. irrespective of the share difficulty. Over time its the same distribution of rewards.

The argument sounds like saying : Pools using diff 1 shares is unfair, because I have done many hashes and didnt find a single diff 1 hash to submit. The work was done and went unrecognized.

I am pretty sure when you do the simulation, you will find that over time, at any difficulty, the 100GH/s miner would have submitted 10x of what 10 GH/s miner would have.

Going by h2odysee suggestion, try this. 2 processes generating random numbers between 1 and 1000000. Process A generates 1000 randoms a second, B does 10000 / sec. Log the output in one place if the number is 100 and another place if its below 10. run the processes many times for 10 second intervals.... im pretty sure both A and B will have similar ratio for the 2 difficulties... over time... i.e. B's ratio will be more consistent than A's but over time itll be the same.

So here's a sim I just made: http://pastebin.com/nud90qDK

After doing it, I realized that there's really not much to simulate. It's pretty trivial.

No matter how fast the block rate, I get the same result:

Shares per step: 0.01029

I thought about doing this. The problem is then I need to process the workers in a parallell fashion. Meaning run multiple threads/processes. This is because now both miners are racing each other. This complicates the code significantly, but also now you have introduced variables that you didn't have before, race conditions within the operating system.

I agree that would be a more accurate method, but to do it really properly you almost be better off just setting up a real pool and have real miners mine shares at different rates and take stats.

Minus my admittedly inefficient block/share time random generation (working on that), as long as you trust the probability it should be accurate, to the best of my knowledge. The block solve for each block is used by both workers on that block, so they can accurately be evaluated serially.

Edit: also, 2 workers is a really small sample in terms of solving a block, something that is in reality really consistent

In your simulation, I suggest you don't use any of those formulas, and just model mining as though it really is at the base level. Randomly generate a number between 1 and 100000, and if it's below 1000, it's a share. If it's below 10, it's a block.

I am in the process of updating my simulation script to include a more accurate distribution of block/share times.

I believe this to be a positive Poisson distribution at this time. I don't believe it will make much difference, but it needs to be addressed before some people will accept my work as proof.


For you new comers, the "con" argument to high diff is essentially this. Fast miners are more efficient at reporting their work, that is, they have less work go "unreported". Slower miners are less efficient at reporting their work, because the interruption of new blocks makes a larger percentage of their work go unrecognized. They will have been working longer without finding a share at block change.

Result: no change to overall pool profit. The effect is of the distribution if rewards.

The fastest miners are basically stealing some of the slowest miners hashrate, in terms of reward.

Now, I am no conpiracy theorist. However, it has crossed my mind that some of the people in this forum are some of the fastest miners. They might know about this effect, know it benefits them, and are therefore trying to keep the status quo. I am certainly not accusing anyone of that and I think it's unlikely that this is the case, just something to tuck away in the back of your minds.

mueslo
yes very guilty of oversimplification but the analogy is not so flawed as to miss the point for some of the misconceptions I am trying to clear up.
I hope my description simplifies the issues without preaching anything baseless. I'm not explaining how crypto mining works just the fairness aspect of pool difficulty.

I didn't think presenting the information in the form you have above was going to help the discussion at this point.

If you want to take the discussion further in to the a detailed description and minutia of mathematics please be my guest.
I was getting sick of the huge misunderstanding some people were labouring under.

A 20 GH/s miner is twice as likely to solve a diff 1 share than a 10 GH/s miner.
A 20 GH/s miner is twice as likely to solve a diff 10 share than a 10 GH/s miner.
A 20 GH/s miner is twice as likely to solve a diff 100 share than a 10 GH/s miner.

That's pretty flawed analogy by any standard. Those distributions have nothing whatsoever to do with what actually happens, they aren't scalable.

This is the probability distribution of zeros: P(n) = 2^(-n), where 32≤n≤256 the number of zeros.