Topic: Fascinating what Michael Deletes [BTCguild 90% luck and dropping] - page 2. (Read 4231 times)

It isn't a hardware fault or error.

If nonce 728937289 solves the block for a given unit of work and a particular hardware for a variety of reasons doesn't check nonce 728937289 it is not going to find a solution.  It is that simple.  I assume you are confused because you believe that hardware normally checks all nonces all the time.  Nothing could be further from the truth.  There are dozens of reasons why a nonce wouldn't be checked and no software is going to report that as an error.  HW errors as reported by cgminer and the like are the result of the hardware reporting that work A + nonce B produces a hash of particular difficulty and it doesn't.

It isn't a static range.  Say you have an ASIC with 64 cores the designer may decide to take the nonce range (2^32) and break it into 64 chunks.  It does this by assinging an offset to each core.  So all the cores get the same work, each one starts at a difference nonce value and they all increment 2^26.  However due to yields not all chips will have all 64 cores operating.  The seller may have designed it around 60 cores being "good enough" to achieve the hashrate.   So if 4 cores are "off" then that means millions of nonces will never even be attempted.  Most ASICs also work on dynamic load so as the hardware error rate and/or temp rise it shuts off the cores.   So the same nonces won't be covered all the time.

Still it goes way beyond just which nonces are checked.  Both the ASICs and the mining software queue work.  So what happens when a miner has 30 seconds worth of work queued up and you send him the "test work".  Are you going to hold the block for 30 seconds which would mean a >7% orphan rate?  What happens if due to latency the miner returns the proper solution but only after you have broadcast the block?  If he cheating or just slow or just had a lot queued up, or poor connectivity?   How are you going to avoid the attacker just sharing info between workers to detect the "test work" and always return those with 100% success?  

Also don't take this is as exhaustive I just don't feel like covering every possible scenario.  The only way to prevent these types of attacks is for the pool to know something the miner doesn't.  The current block hashing protocol doesn't make that possible.

You are correct that there isn't any observable difference between the 2 types of problems.  And in practice, I think it's extremely unlikely that someone with hardware that has catastrophic levels of false negative errors is unaware of the issue, so the intent is likely equivalent as well.

When you have people of very questionable competence designing silicon on crash schedules it's inevitable that serious hardware defects are going to be in play.  And with the money involved, if there is a way to externalize the cost of failure to pool participants people are going to choose that over eating the loss themselves.

I think it's an absolute must that stratum allow pools to send test jobs to validate hardware integrity in a blind way that allows detection of malicious withholding as well.  Without that, I expect the days of public pools are numbered, and with them all hope of even modest decentralization will go as well.

Can someone explain to me how is "block withholding attack" different from the "false negative" type of hardware fault? I mean in the observable symptoms, not in the intent.

The last time I looked into the miners source codes (late FPGA era,early bitfuries) I've only seen the "false positives" counted as "hardware errors": nonce comes up as gold, but software re-check shows that it isn't.

I haven't seen any published source code that does the opposite check for "false negatives": chip should find a golden nonce but hadn't.

From the skimming of the available hardware documentation I see that only Spondoolies has a BIST operation that allows to cheaply test for false negative. By "cheaply" I mean "cheaper than doing a regular, full scan".

Obviously this test has to be hardware specific, because e.g. bitfuries only test 768/1024 of the available nonce space. This would have to be even more complex to account for failed sub-engines in the multi-engine chips.

I think that Stratum Mining protocol can be extended to allow the client to send to the pool server the maps of the "blind spots" in the nonce space. Then the pool can sensibly proctor the tests meant to discover withholding attacks.

100% in agreement... Mining will collapse at the feet of the large farms unless the community pulls together as a large diversified pool or pools withing a large pool. Until then, we'll see large pools like BTCguild collapse.

Well yes a hard fork would imply that all nodes would need to update not just for mining but for validating the new block version as well.   Solo miners wouldn't be affected.  There would still be two values in the new block header but they would know both.  No need to hide info from yourself.

I was kind of trying to avoid that.


But I read the section you recommended, dyslexically, and with your above explanation I think I comprehend it now.

How would that impact solo mining?  I guess the bitcoind/qt would have to be reworked as well anyway.

Enough of my uninformed rambling.
Thanks,
Sam

How about reading the linked proposal that meni spent a lot of time developing and writing up?  Without a hard fork you are right there is no way to separate it, that is why a hard fork would be needed to support a method that prevents a miner from knowing if the hash solves a block.  It would change the blockheader such that a valid block is based on two different hash values and the miner only knows one.   The miner can compute the share difficulty, the pool can compute if it meets the difficulty for solving a block.  The pool withholds the second value from the miner which prevents the miner from knowing what the pool knows.

Probably or it could be revised to fit.  The actual ASICs really have no concept of what they are hashing just that they are hashing a blob of data and checking the output of the hashed blob.  So the size of the blob (and the fact that the trailing 32 bits are the incrementing nonce) can't change but the header can be extended in 256 bit increments by making the midstate the result of two blocks (hashing definition not bitcoin definition) of inputs instead of one.

As far as will people support a hard fork?  I don't know but if convinced it is a legit threat non-miners may see little downside in supporting it and potential devaluation of their bitcoins if they don't.   If nothing else I find it interesting because "if" this ends up being the Bitcoin killer, well there is already a solution for the altcoin that replaces bitcoin.

Checking a share difficulty IS checking if it meets or exceeds current difficulty.  I don't see how you could separate one from the other.  It would have to be same operation.

Sooo... if you don't know if you've solved a block then you wouldn't know what the difficulty of the share you submitted is so you wouldn't know if you were getting paid correctly.

If you could tell what your share diff is without the pool telling you then you would know if its a block solver or not

That is not correct.  The miner would know the exact difficulty of the share just not if the share solves a block or not.

p2pool is just as vulnerable to withholding.  Maybe even moreso, since you can't actually take actions (ban/freeze accounts) against somebody doing it.

The miner would know enough to know how many shares they submitted.  They would not know enough to know if they solved a block though.

The other question of course:  Would that proposed solution even work with existing ASICs, since you're changing the block header fields that are going to be hashed?  Not that it matters...nobody is going to hard fork bitcoin in order to preserve pools.

p2pool ftw.

M

It seems to me that the solution you recommended earlier in this thread would allow a pool to withhold vital information about the hash's that a miner submits.  The miner wouldn't know if he really found a block or not or even know what difficulty of the share was.  So a miner would have no way to verify if a pool was above board or not or even know how much he was getting paid for his hash's.

game on then

actually let me be a little clearer.... if there is a solution being held back because of the aversion of a hard fork then why would miners endure it for months and just say luck is luck??

ya, trust is at a all time high in these parts

There is a very good solution but it will require a hard fork.

well then the game is over for public pool mining with no innovation here..  if there was already a solution there wouldn't be a problem

I agree ckolivas.  I assumed this was fairly common knowledge but I think you are right that there are some incorrect assumptions on what work it takes to complete a specific difficulty share/block.

I think this is a misunderstanding of  most users that you must check the entire nonce range to be doing as much "work" which is wrong. It's purely the number of nonces you check, even if they're from different work.

Very little hardware actually checks the full nonce range. Many abort work on the first nonce they find while others have a matrix of nonces they're likely to check and miss others (there is nothing invalid about doing this, nor is there any way to make this sort of practice find less blocks).