Topic: Getting rid of pools: Proof of Collaborative Work - page 3. (Read 1861 times)

Hi, I have a stupid question, for sure I'm missing something,

I cannot see any reward for finalization block.
where is the incentive to to mine a finalization block?

NOTE that you overlooked my fix where ln(n) should instead be ln(1/mindiff).

I am not much of a probability theory expert, but for now, I'm persuaded about @tromp calculations:
For any minimum relative difficulty^* set for every share, mindiff, the average number of shares per (Finalized) block, n, would satisfy this formula : n*ln(n)*mindiff = 1

^* minimum relative difficulty is the ratio by which PoCW reduces the calculated network difficulty and is the least difficulty allowed for submitted shares.

it yields n * ln(n) = 1/mindiff and suggests a better than scalable dependency between the two. i.e. a decrease in mindiff (more utilization) causes a better than linear growth in the avaerage number of shares.

Notes:
1-  @tromp assumption about shares being not overly lucky enforces this formula even more (i.e. the average weight can be a bit higher, hence n is always a bit smaller )

2- The exact sum of n shares according to this protocol is 0.93 instead of 1 (there is one fixed %2 share for the finalized block itself) so the formula should be modified for n to satisfy n*ln(n)*mindiff = 0.95

I just did some trivial calculations using the latter formula
For mindiff being set to 10^-4, we will have n < 1,320  
For 10^-5 we have n<10,300
For  10^-6 we have n< 83,800

It is so encouraging: Setting difficulty for shares to a minimum of one million times easier than network difficulty we need only 83,800 shares per block as an average instead of 1320 for current 0.0001. Note that the difficulty is already reduced by a factor of 10,  as a result of decreased block time to one minute and we are talking about 10 million times utilization compared to currently proposed 100 thousand times.

And yet we don't have to decrease the mindiff (currently set to 10^-4) in such a strict way, instead we would prefer  moderate adjustments, an even more promising situation.

Based on this assessments, it is assertable that Proof of Collaborative Work, is scalable and can achieve its design goal,  despite constant growth in network hashrate and difficulty indexes by a better than linear increase in demand for computing and networking resources (and no increase in other resources). The design goal is keeping difficulty of shares low enough to help average and small miners in participating directly in the network without being hurt by  phenomenons such as mining variance or their inherent proximity disadvantage. Fixing one of the known flaws of PoW, mining pressure.

I guess we might start thinking about a self adjustment algorithm for mindiff (the minimum difficulty figure for issued shares).
No rush for this tho, the core proposal is open to change and it is just a long term consideration

This hypothetical algorithm should have features such as:

-Not being too dynamic. I think the adjustment shouldn't happen frequently, once every year I suggest.

-Not being linear. The increase in network hashrate is about introducing both new investment by miners and improved efficiency of mining hardware. Both factors, specially the latter, suggest that we don't have to keep too small facilities competitive artificially by subsidizing them. We are not Robin Hood and we shouldn't be.

So, our algorithm should "dump" the impact of difficulty increase instead of covering it.
It would help the network to upgrade smoothly.
A factor of 30% to 50% adjustment, as a result of 100% increase in target difficulty, seems more reasonable to me than just an exact proportional compensation for new difficulty.

I don't mind, as long as you consider the edits i made to fix some errors.

Solid. Will include this formula and the proof in the white paper, if you don't mind.

Consider a uniformly random real x in the interval [10^-4,1]
Its expected inverse is the integral of 1/x dx from 10^-4 to 1, which equals ln 1 - ln (10^-4) = ln(10^4).

Now if we scale this up by 10^4*T, where T is the target threshold, and assume that shares are not lucky enough to go below T, then the n hashes will be uniformly distributed in the interval [T, 10^4*T], and we get the formula above.

Thanks for the support.

As of your argument about hardware centralization being more dangerous without pools:

It is so tricky. This proposal is not an anti-pool or pool-resistant protocol, instead it is a fix for pooling pressure.

Iow, it does no prevent people to come together and start a pool , it just removes the obligation for them to join pools (and the big-pool-better-pool implications) the current situation for almost any PoW coin.

EDIT:
It is also interesting to consider the situation with Bitmain. No doubts this company has access to the biggest mining farms ever and yet Bitmain has Antpool and insists on having more and more people pointing their miners to their pool. Why? because it is always better to have more power and be safe against the variance and have a smooth luck statistics.

So, I would say after this fix, there would be not only no pressure toward pooling but also no incentive.

Interesting, appreciate it if you would share the logic beyond the formula. It would be very helpful. To be honest I have not done too much on it and my initial assumption about 4650 shares is very naive. I was just sure that the number won't be any higher for average number of shares per block.

Thank you so much for your contribution.

Interesting, you put a lot of thought into this proposal.  I would support it and see how it goes.  The goal is really hard to reach.  The idea would be to increase difficulty to scale up operations.  Pool mining can be damaging, but one guy with a huge operation can be worse if no one can pool together.

I calculated wrong.

n shares expect to accumulate about n*ln(n)*10^-4 in weight, so we expect
a little under 1400 shares to accumulate unit weight...

Yes, my mistake to call it NP-complete, it is NP.  In the context of this discussion, when we refer to hash functions, the PoW problem (like one you have suggested, a conditional hash generating problem) is what we usually mean, yet I should have been more precise.

This was posted in a chaotic atmosphere but the point is maintainable that, verifying shares (not the Prepared block or its counterpart in traditional PoW, block) is a trivial job, by definition. Because it needs just verifying an answer for a NP  problem.

Hash functions are not decision problems, so they cannot be NP-complete.
I could create a decision problem out of a hash function though.
Something relevant for mining would look like:

The set of of pairs (p,y) where
  p is a bitstring of length between 0 and 256,
  y is a 256 bit number,
  and there exists an 256-bit x with prefix p such that SHA256(x) < y

Such a problem is in NP.
But it would still not be NP-complete, since there is no way to reduce other NP problems to this one.

@anonymint

keep cool and remain focused, ...

unfortunately your last post was of no quality in terms of putting enough meals on the table, instead you are continuing your holly war (against what?) with inappropriate language, as usual.

Please take a break, think a while and either leave this discussion (as you promise repeatedly) or improve your attitude,

will be back 

It is 0.93 to be exceeded to be exact.

Yes. To be more exact, as the shares are randomly distributed in the range between 0.0001 up to 0.93,  the median would be 0.465.
Yet it is not the highest difficulty, just the median.

As you mentioned, it is more what traditional bitcoin is doing and I'm trying to fix. It is not collaborative and as both theoretically and experimentally  has been shown, is vulnerable to centralization. The same old winner-takes-all philosophy leaves no space for collaboration.

As of the 'overhead' issues, this has been discussed before. Shares are not like conventional blocks, they take a very negligible cpu time to be validated and network bandwidth to be propagated.

EDIT:
I have to take back my above calculations:  Some blocks may have as few as 2 shares and some may have as many as 9301 shares to satisfy the difficulty, this yields an average number of shares to be around 4650 for a large number of rounds. The highest share is (0.93) and the lowest will be 0.0001 no more indexes I've calculated and tried to calculate till now.

On further reflection, if you randomly accumulate shares of weight (fraction of required difficulty) >= 10^-4 until their sum weight exceeds 1, then the expected number of shares is 5000.

BUT, the expected highest weight among these shares is close to 0.5 !
(if you throw 5000 darts at a unit interval, you expect the smallest hit near 1/5000)

So rather than summarize a sum weight of 1 with an expected 5000 shares,
it appears way more efficient to just summarize a sum weight of roughly 0.5 with the SINGLE best share.
But now you're essentially back to the standard way of doing things. In the time it takes bitcoin to find a single share of weight >=1, the total accumulated weight of all shares is around 2.

All the overhead of share communication and accumulation is essentially wasted.

@anunymint
I appreciate the fact that you spend a considerable time on this subject, it is a good evidence for me to become even more convinced that:
1- You have good faith and as of trolling part of your writings, you just can't help it and I should be an order of magnitude more tolerant with you  
2- You are smart and have been around for a long time, a good choice for chewing a complicated proposal like PoCW. Again, more tolerant, as tolerant as possible and more ... I should repeat and keep it in mind  

Unlike what you suggest, ASICs won't be helpful in syncing block chain. Full nodes are not ASICs and don't utilise ASICs ever to validate a hash, they just compute that hash with their own cpu!
SHA256 and other hash functions, are NP-Complete problems: their solutions consume negligible time and resource to be verified, it is basic in "computer science"  

Good objection.
Shelling points (transition points from Preparation to Contribution and from the Finalization to the next round ) have %7 value cumulatively (%5 for first and %2 for the second point). It is low enough, yet it is not at stake, totally:

For the first %5 part, the hot zone (the miner and its neighboring peers) are highly incentivized to share it asap, because it is not finalized and practically worth nothing as it won't be appreciated if it doesn't get enough support finding its way to finalization. Note that neighbors are incentivized too, as if they want to join the dominating current, they need their shares to be finalized as soon as possible, it needs the Prepared Block to be Populated although it is not their's.

For the second Schelling point, the finalized block found event, with %2 percent block reward value, hesitating to relay the information is in a very high risk of being orphan by means of other competitors(for the lucky miner), and to be mining orphan shares/final blocks (for the peers).

I understand you have some feelings that more complicated scenarios could be feasible but I don't think so and until somebody has not presented such a scenario, we better not to be afraid of it.

I'm aware that you are obsessed with some kind of network being divided because you think selfish mining is a serious vulnerability and/or propagation delay is overwhelming.
Network wont be divided neither intentionally nor as a result of propagation delay, and if you are not satisfied with my assessment of propagation delay you should recall my secret weapon, incentivizing nodes to share their findings, as fast as possible to the extent that they will put it in high priority. They will dedicate more resources (both hardware and software) to the job.


Correct! Now if you would just internalize that thought, and understand that your point also applies to your reckless (presumptuous) overconfidence and enthusiasm.