Topic: Are pools more efficient? (Read 3256 times)

That seems sensible to me.  Long polling shouldn't be necessary for a local network at all (not that it hurts).  I would argue that, even with long polling, pool miners will have more of a stales problem than solo miners.

Just set your maximum scan time to 1 second. This means you'll lose a maximum of ~144 seconds of mining a day or 0.166%. You'll usually lose  more effort than that from pools due to other issues (stales, bugs, network latency).

Pools can't do this because all the users would slam them. But it's a perfectly reasonable thing when running on your own. (not that running pushpool with LP locally is much of an issue).

bitcoind doesn't broadcast out new work. Workers poll for new work when they run out or run low on work.

Long poll is a technique for letting a server push data to a client with HTTP which doesn't really support server push. This way the server can give new work to the miners when a block change happens which invalidates all the old work the miners have. Then miners immediately start on the new work instead of doing useless work on old data, finding proofs or work that are useless and sending them to the server only to have them get registered as rejected proofs of work, aka. "stales".

Unfortunately bitcoind doesn't support long poll or any other kind of push or notification to clients that anything has happened. But pools do.

Edit:
more about long polling in general: http://en.wikipedia.org/wiki/Comet_%28programming%29
more about bitcoin getwork with long poll: https://en.bitcoin.it/wiki/Getwork

Is that right?  I thought long polling was all about helping the mining software and bitcoind communicate and thereby reducing the proportion of invalid shares.  If both pieces of software are on the same private network such communication should be a non-issue.  To me, the interesting communication problem is the broadcasting of a new block to all existing bitcoind nodes.

Could someone explain long polling or give me a link?  Particularly I'd like to know whether or not it improves on bitcoind's current block broadcasting and, if so, how?

To sum up, 
Advanced pool software = more efficient / reliable
Mining in group = not more efficient.

Good points.  Never thought about it but lack of LP is going to cost you 1% to 5% depending on card speed. 
cgminer helps reduce that somewhat.  It does independent checking for block changes and when it detects a block change will empty the queue and request a new getwork().

Two things that make pools more efficient than a (naive) solo mining setup:

• Long polling (don't continue checking old "lottery tickets" after they expire, get new ones instead)
• Merged mining (get free namecoins while you work on making bitcoins)

If I was going to solo mine, I think I'd get some of the free pool software and set that up, even if it was just for myself. If you solo mine directly against the bitcoin program, you are losing money.

Yep.  You can think of mining like repeatedly rolling a die.  If you roll a 1 you generate a block (and score 50 BTC).  This sounds easy until you realise that the die has about (2^32 * [Bitcoin difficulty]) sides.  No matter how high the difficulty, you could still get a 1 on your first try!

A CPU thread rolls a die repeatedly but fairly quickly.  A GPU rolls more slowly but is able to roll hundreds or even thousands of dice at a time!

A pool works by giving you the usual dice to roll but paying you a small amount whenever you roll a number less than about [Bitcoin difficulty].

Okay, I just wanted to make sure I understood how it works.  Thanks.

There is a chance you could solve a block w/ a single hash.  There is also a chance you would solve every single block for the day with only an Atom CPU as your mining hardware.

There is always a chance (however remote).

I'm still in learning mode here.

So this means that technically, I could turn on gen=1 in bitcoin.conf so the client tries to generate bitcoins, and if I were extremely lucky, there's a tiny chance I could solve a block 5 minutes later, right?

Even IF you used the same wallet on each machine (stupid) rather than point each machine to a single wallet or install a unique wallet on each machine they won't have same RNG seed.  Things like HDD serial #, processor serial #, etc are used in the RNG thus they will generate different extra-nonces.  There may be a tiny amount of duplicated work (probably insignificant) as each instance may use try an extra-nonce already tried by another instance however it would be very small.

The addresse is already generated since the HDD is cloned

I ask because It would be rather common (yet stupid at this time) that someone would build 4 similar computer and clone the HDD sector by sector and start them all SOLO mining at once.

AFAIK the PRNG uses serial numbers on the hardware components, so even if the two computers have the same configuration, they will generate different wallet addresses and extraNonce.

No.

Three scenarios:

Two computers have different wallets.
The coinbase transaction contains the reward address so with different wallets they will always have different merkle tree roots


Two computers share the same wallet (one copy).
The coinbase also contains an "extra nonce" value which is different for each work request so no duplication.

Two computers have identical copies of the same wallet (very stupid to do)
If the two computers had a duplicated the wallet running simultaneously on both machines at the same time there could be duplicated work in theory.  Even here the amount would be insignificant.  The only duplicated work would be when one of the wallets picked randomly an extra-nonce already picked by the other copy (which it isn't aware of). GIven the small number of extra-nonce changes per block (600 seconds) the amount of work would be a rounding error.

This has to be the most illuminating post on how block are generated.

Scenario : Two similar PC, same hardware, same date, same cloned hard-drive, same cloned client, being started at the exact same time.

Would this result in the same work being done twice ?

Pools do not get more out of their combined computing power than the sum of their individual miners. Due to the way BitCoin is designed there is no need for miners to tell each-other which solutions they have tried.

The only reason pools exist is to reduce the variance of payouts.

Agreed.  This establishes why the following thought experiment doesn't apply to Bitcoin.


In this example, keeping track of failed guesses is very helpful because you will expect the number of failed guesses for a round to be much more than the number of different solutions (one in this case).  But with Bitcoin, for every block the number of failed hashes (worldwide) is utterly insignificant compared to the number of potential solutions.  Even if everyone were using the same pool and using random nonces there would be practically no efficiency loss due to repeated work.

What you're missing is that there are many correct solutions. The block header is 640 bits (some of them can't be easily controlled, but never mind that) so there are 2^640 potential solutions, which is indeed virtually infinite. Of those, there are 2^608/Difficulty correct solutions. So people still have a chance to find a correct solution (1/(2^32*Difficulty) per hash), but no chance to test the same solution someone else did (if they're randomizing properly).

Exactly.  The space is 2^256 so it can't be exhaustively searched.


Timestamp is just used to provide a record of the approximate time when block was solved.  You could change reward address but that would be complicated on the backend.  The coinbase field can hold an "extra nonce" which is just a pseudo-random value with no meaning.  Changing that will result in a different block header so the pool can ensure each miner is working on a different header.

More efficient pools use NTimeRolling which instructs the miner to locally increment the timestamp reusing all the same values.
So a miner will start with a timestamp of x.
try all nonces in nonce range (roughly 4 billion).
update timestamp
try all nonces in nonce range (roughly 4 billion).
...
until the NTimeRolling value expires.  This improves communication efficiency between pool and miner by cutting down on predictable block header changes. 

Thanks, that makes more sense to me. The key thing was that there was no certainty of finding a solution within a certain space.

What is the importance of the timestamp? Would there be any benefit to incrementing the reward address rather than incrementing the timestamp?

No because there is no defined space or more technically the defined space is constantly changing.

The input for a hash is:
merkle root hash (a hash of all transactions in block including the pool reward address so you can't cheat)
hash of the last block (prevents you from working on block "ahead of time" and also creates the "chain")
timestamp (current time)
nonce (a 32 bit number 2^23)

So a miner will try every nonce value.  That is defined space.  There are roughly 4 billion possibilities however at current difficulty (1.2 million) there is only a 1 in 1.2 million chance that any value in the nonce range will produce a value hash.  1,199,999 out of 1,200,000 complete nonce ranges (4 billion hashes) will produce no solution (on average).

So once that defined space is exausted your miner gets new work.  The simplest way to get new work is simply increment the timestamp and now you are working on an entire new solution space.  Each complete nonce range hashed gives you a 1 / (difficulty) chance of finding the solution.

Periodically the pool will update the other elements in block header.   When a block is found the header changes to reflect new "last block hash".  As transactions are added they change the merkle root hash.  To avoid 2 miners in same pool duplicating work the pool uses a value called "Extra nonce" in the coinbase transaction to ensure each miner is given a unique blockheader to work on at all times.

So wouldn't that imply that 100 computers working in a pool will solve a block faster than 100 computers working for themselves? Because the computers in the pool can all use the pool address and split up the possible solutions. 100 computers working by themselves would essentially be all independently trying to solve a much larger problem because of their unique addresses.

Your solutions/shares are specific to a set of headers for the block.  When you're solo mining, the headers include YOUR payout address, so any random nonce you try will produce a different hash, even though that same nonce/extraNonce has been tried by someone else.

In pool mining, the work is distributed so everybody is trying difference nonces without overlap.  That overhead is handled by the pool and its bitcoind.

How do they assign who will try which solutions so that there is no overlap?

No.

As far as i know no one ever redo the solutions already tried by someone else. Pool mining or solo mining you always try different solutions so there is no overlap and wasted time.

Only advantage of pool mining is that maybe they receive faster new blocks/when they find a new block they immediately start crunching for the next one while in solo you would have to first download it...

But to find a single solution from an "essentially infinite" space of solutions, you will have to sample a significant proportion of those "essentially infinite" solutions. That means it's not infinite at all. Probabilistically, you would expect to sample as many solutions as there are total possible solutions if you just randomly guess at it. If you keep randomly guessing a number between 1 and 10, you will expect to guess 10 times before you get it. You could possibly guess wrong twenty times in a row.

But if you split up the solutions and systematically eliminate them, then you would only expect to sample  half of them, actually (n+1)/2. If you systematically try to guess numbers between 1 and 10 without repeating the same guess, you will expect to guess 5.5 times before you get it. So if you are eliminating known wrong solutions you cut your work time in half. You will never guess more than 10 times. I was wondering if pools are doing this, or if the Bitcoin protocol somehow incorporated it automatically.  

Pools aren't "more efficient" in the way you mean. The search space of solutions is virtually infinite, so there is no chance of people duplicating work even without any coordination. Basically miners are trying random numbers until they get a hit, whether they're mining pool or solo. The average number of blocks found by a miner is the same in both cases.

What pools do is reduce the variance in a miner's payout. I discuss the basics of this in chapter 1 of Analysis of Bitcoin pooled mining reward systems.

Not if people properly set up failover pools. And other current problems of pools with respect to network robustness have solutions which will be implemented in the future.

That completely did not answer my question at all.

Finding Bitcoin blocks is a lottery.  You get a certain number of tickets every second, depending on your hardware speed.  Chances of winning this lottery is slim, but the winner gets 50 coins + fees of transactions in the block.

A pool is where many miners join forces and put all tickets they buy into a common pool.  When someone wins, the 50 coins and possibly the fees (some pool owners keep the fees) are shared among all participants based on how much each participant contributed.

A pool will typically find blocks more often than a solo miner, because the combined hardware power of the pool is so much larger than a single miner.  For each miner the result is the same in the long run, given that the pool owner takes no fees.  In practice solo mining is more profitable in the long run because a solo miner is only dependant on his own miner and network, and can even tolerate short network outages.  Fewer dependencies means fewer interruptions in mining.  Pools are down now and then, and even DDOSed quite often.

The Bitcoin network would be more robust if more people mined solo.  When pools are DDOSed, block rate slows down and transactions confirms slower as a result.

I'm not super technically into how bitcoin works so maybe I am being dense here. But the way I see it, if there is a brute force problem that everyone is trying to solve. It will be faster if you don't waste time redoing the solutions you have already tried once. So if one computer is working on a brute force problem, they are just randomly trying solutions. But if a group of computers are working on a problem, do they somehow distribute solutions to check or tell each-other which solutions they have tried so nobody is wasting time? Is this how pools work? Do they get more out of their combined computing power than the sum of individual miners?