Topic: Mining inefficiency due to discarded work - page 3. (Read 12335 times)

I see no problem here.


I don't know why miner uploaded same nonce twice, but it is fine that second attempt wasn't counted. Btw I'm expecting that nonce is rising from zero, so looks weird for me that nonces are mixed.


That's fine, isn't it?


I thought that miner is already doing this. At least Diablo's isn't stopping when 'block' is found. Don't forget that this is tweak only for pool, because with crunching real-difficulty hashes, second nonce cannot be valid bitcoin block in any case...


You should put it back. Otherwise you are crunching invalid work for long time (~ 30/600 seconds, so 5% of time is wasted). Plus pool now reject shares which cannot be valid bitcoin blocks, for more info read pool thread. So this isn't improvement in any case.


how did you calculated 20-30%? It isn't correct.

Yes, less often getwork update saves some resources, but will make mining much less effective.

Personally I don't understand why are you posting this, because I already answered all those questions in PMs...


Nothing or not any significant amount. Learn basics of statistics and visit pool stats page...

I've been asking m0mchill the same things in PM lately. It seems to me that multiple answers could be found for each 2^32 keyspace, and the likelihood that the first answer is the actual answer for the block is only '1 : The average number of answers found in a getwork() that results in answers', with some getwork() not resulting in answers at all.

This brings up the question; If some getwork() simply do not have answers, is this due to a 2^32 keyspace not being an actual equal portion of the block, or is this due to overlapping 2^32 getworks? ...and if it's overlapping, is that the cause of the invalid or stale?

i.e.:
I send a getwork() request and receive my 2^32 keyspace to work on.
XYZMiner belonging to another person gets another 2^32 keyspace in getwork
Do we share an overlap of 2^16 (arbitrary figure for the sake of example) in our respective keyspaces?

Meaning, am I getting invalid or stale because there are multiple people working on the same exact portions of keyspace? If so, isn't that an issue with the getwork patch?

I've also asked m0mchill about the askrate, and it seems his answer to why the client fixes the askrate is basically a "fuck it if it doesn't find it quick enough". Although, he speaks more eloquently than that.

He has also stated that, yes, we are ignoring large portions of the keyspace because we submit the first hash and ignore anything else in the keyspace, whether it's found in the first 10% of the keyspace, or the last 10% of the keyspace. He believes that this is trivial though, since you are moving on to another keyspace quick enough.

Here is a quote from my PM with him (m0mchill):

So, we're not searching the entire keyspace in a provided getwork. What if one of those possible answers you're ignoring is the answer to the block? You just fucked the pool out of a block.

I understand the premise that some getwork may not result in answers, but to the same extent, you could also get multiple valid answers (2, 5, 8, whatever) for a SINGLE getwork.

Assuming it is working as designed........that is indeed the question I'm raising.

So why is it when I process the ENTIRE getwork request (vs just finding the first answer and then moving on to the next getwork), I'm seeing a much different result, and quite frequently at that.

Notice that this getwork got 5 answers, and reported all of them to slush's server and were accepted.
29/01/2011 01:41:42, Getting new work.. [GW:29]
29/01/2011 01:41:57, 9d5069e9, accepted at 40.625% of getwork()
29/01/2011 01:41:58, 09fc7161, accepted at 43.75% of getwork()
29/01/2011 01:41:59, 27af8f3c, accepted at 46.875% of getwork()
29/01/2011 01:42:02, ad105798, accepted at 56.25% of getwork()
29/01/2011 01:42:12, 0e920ae8, accepted at 75.0% of getwork()

This sample shows that 3 answers were accepted, 1 invalid, then 1 more accepted.  Notice the Invalid answer is the same as the 1st accepted answer.
31/01/2011 06:46:34, Getting new work.. [GW:291]
31/01/2011 06:46:38, 41b15022, accepted at 6.25% of getwork()
31/01/2011 06:46:53, c597d2b5, accepted at 31.25% of getwork()
31/01/2011 06:46:59, 9babdadf, accepted at 50.0% of getwork()
31/01/2011 06:47:07, 41b15022, invalid or stale at 75.0% of getwork()
31/01/2011 06:47:11, 1ba08127, accepted at 87.5% of getwork()

This sample shows 4 getworks, all without a single answer, then just a single answer for 1 getwork.
31/01/2011 06:04:21, Getting new work.. [GW:212]
31/01/2011 06:04:54, Getting new work.. [GW:213]
31/01/2011 06:05:26, Getting new work.. [GW:214]
31/01/2011 06:05:59, Getting new work.. [GW:215]
31/01/2011 06:06:32, Getting new work.. [GW:216]
31/01/2011 06:06:43, d08c8f3d, accepted at 31.25% of getwork()
31/01/2011 06:07:04, Getting new work.. [GW:217]

These outputs are from a modified version of m0mchill's miner.

Changes include:
- Does not stop when first answer/hash/nonce is found.  It continues searching until all possible hashes have been searched.
- Removes the hard coded limit of 10 seconds for the askrate.  (My card does a single getwork in 30 seconds, so I just set the askrate to 3600 and forget it)

The advantages of these modifications are:
- The entire getwork request is processed, finding all possible answers to a getwork.
- Raises efficiency of getwork/answers to 100%.  Right now it's between 20-30%.
- Lowers the amount of requests and bandwidth used by pool servers.

Right now, I believe, huge portions of hashes in the getwork requests are not being processed because when just 1 answer is found, it moves on to the next getwork, or if the askrate has been reached, it then moves on to the next getwork.  Also, in comparing the results over time of the original miner and this modified version, they find the same amount of answers in the same amount of time (on average).  The original miner seems to generate about 260% more traffic for the pooled mining server and discards well over 60% of the getwork when a single answer is found, or when the askrate is reached.

SO....

If the chances are 1 in a zillion in finding the same answer for different getwork request, then how is this miner finding multiple answers for a SINGLE getwork, some are valid, some a DUPLICATES, how is this happening?

If all getwork request are unique, and possible answers are overlooked when the first answer is found, how does the pool/bitcoin handle searching the rest of that discarded hashes?  Does it?

Is this a problem with the miner, or the getwork patched version of bitcoin?

Have YOU searched the entire 2^32 possible hashes of a getwork before, or do you just trust the miner knows best when it stops short of checking all 2^32 possibilities?

If I'm correct in my findings, how many more blocks could the pool be finding by searching all of the 2^32 hashes per getwork?? 
How many blocks have we missed because we didn't search the entire getwork??

That's your food for thought.  Please think about this and provide good answers.
I'm expecting to release the modified client in the next week or so after I get the pre-compiled versions and sources wrapped up.

Looking forward to feedback on this one..

Assuming it is working as designed. Bugs throw out all bets.

Hashes are 256 bits. The odds of finding two random hashes the same would be much less than the odds of getting struck by lightning on the same day you buy a single ticket and win the lottery. I call that never.

If there are no duplicate getwork responses, then the probability of re-doing work is so low that it will almost certainly never happen to anyone as long as Bitcoin exists.

Duplicate getwork responses would cause you to do work that has already been done. However, slush's pool relies on Bitcoin to handle this, so there shouldn't be any difference between mining on your own and mining in a pool. If anything, the pool is more likely to produce duplicates, since Bitcoin was not designed to produce unique work for so many miners.

Ever?
Duplicates will never-ever-ever be found in the getwork searches?
Each getwork is unique, and no result from getwork A could possibly match getwork B?

Just trying to make sure I understand that is what you're saying.

False. there are zillions of possible blocks/hashes to find and no one ever duplicates any searches.

the advantage of pools is you get smaller rewards more frequently. the total rewards are on the average almost identical.

I am also interested to learn what is the difference between these two. I am guessing it would be more efficient to do pool mining resource wise, but it's just a guess since I don't know much about the mining strategy algorithm in each case.

Say I have several 5970s, if I let them mine solo, there's no coordination among these miners, I'd think there would be a some kind of wasted work because of duplicate/overlapped attempts on the same hashes, while the pool mining work can be distributed in a way to avoid or minimize this waste. True or false?

This is not a network communications problem.
I record my getwork request and answers.
The two getwork were ENTIRELY different getwork request, with different data and midstates, yet produced the same answers.

Hi FairUser,

'invalid or stale' for unique nonces can be explained by pool update. Pool now doesn't accept shares when new bitcoin block come (see pool thread for more). Now it works in the same way as standalone mining.

About same getworks - it looks like network troubles, which are more and more common. I hope that it will be solved by protocol change, which significantly reduce network communication. Will be introduced this week.

I keep seeing these "invalid or stale" a couple of times a day.

30/01/2011 15:26, bea41297, accepted
30/01/2011 15:26, bea41297, invalid or stale
30/01/2011 15:27, b9c26c57, accepted
30/01/2011 15:27, fb27e19a, invalid or stale

Notice the two lines at the top in bold.  
Could there possibly be more than one answer per getwork?
As you can see, I got the answer previously, then got the same answer again.
Then on the last line, it just came back invalid...probably cause someone else got the answer to the getwork before I did.

Is every getwork unique?  If so, how did I get the same answer twice?

UPDATE:  Got two more
30/01/2011 15:30, e67d85ec, invalid or stale
30/01/2011 15:36, f1f124d9, invalid or stale