Pages:
Author

Topic: Fascinating what Michael Deletes [BTCguild 90% luck and dropping] - page 3. (Read 4238 times)

donator
Activity: 1218
Merit: 1079
Gerald Davis
so if miners are not equal, perhaps pools should be segregated?  detection and enforcement would need to be worked out, but much more complex systems exist in this world

Not sure what you mean by equal.  There is no requirement to use all possible nonces.  It doesn't allow them to cheat or gain a disproportionate share of the reward.  It is also dynamic.  Two different ASIC chips from the same company many not cover the same portion of the nonce range.
sr. member
Activity: 462
Merit: 250
 Some ASIC chips don't attempt every value in nonce range.  If the winning solution uses a nonce of 0x18392740 and a particular ASIC is not checking that value the miner isn't going to return a solution.  You could send him the same work 1,000 times and he will NEVER return a solution.  


so if miners are not equal, perhaps pools should be segregated?  detection and enforcement would need to be worked out, but much more complex systems exist in this world
newbie
Activity: 26
Merit: 0
Well, like I said,  a forum post does not a system make.  

Nobody is confusing your post with a "system" but it is possible to that a proposal is flawed even at the conceptual level.  If you told me you were going to the moon by jumping high enough to break orbit, well I don't need to wait to see the engineering plans to tell you it isn't going to work.

Quote
but a sanity check of hardware is not an unheard of thing, and it's clearly needed now.

Your proposal isn't a sanity check, it is a ban randomly miners without any basis in the name of "doing something" and rack up even more losses due to orphans.

Once again.  It's a forum post with a high level approach.  Details need to be worked out.  If you want to write out the code, then I applaud you,  but I'm not interested in debating the finer points of the protocol.

Right now folks using btcguild are losing ~10 to 20% of their earnings to either malice or incompetence.   A sanity check that cost less 10/20% would be net positive.
donator
Activity: 1218
Merit: 1079
Gerald Davis
Well, like I said,  a forum post does not a system make.  

Nobody is confusing your post with a "system" but it is possible to that a proposal is flawed even at the conceptual level.  If you told me you were going to the moon by jumping high enough to break orbit, well I don't need to wait to see the engineering plans to tell you it isn't going to work.

Quote
but a sanity check of hardware is not an unheard of thing, and it's clearly needed now.

Your proposal isn't a sanity check, it is a ban randomly miners without any basis in the name of "doing something" and rack up even more losses due to orphans.
newbie
Activity: 26
Merit: 0
  Those miners wouldn't even have a way to know it is a solution to a block.

Well that is the point isn't it?

Clearly this would need to be flushed out and tested and instaban is simply rhetoric.    I'm not going to engineer the entire test here in a forum post,  but that's the top level pseudo code.  However any miner that fails the test should be considered suspect and perhaps subject to an additional test.   A miner with a 1% error rate would have a 1 in 1,000 chance of artificially failing 2 known good blocks.  Make it 3 tests and the odds raise to 1 in 100,000

You are still not getting it.  Some ASIC chips don't attempt every value in nonce range.  Some never do under any conditions, the multcore chips break the nonce range into "chunks" with a chunk going to each core.  They never have 100% of the cores active because nothing has 100% yield, some are turned on.  So if the nonce that solves the block is 7289372 and it is sent to core 2 but core 2 is disabled then that miners will NEVER return a solution for that work.

Of course you ignored the rest, how long do you wait? how much do you want to lose to orphans because you are waiting to publish a winning block and every second you delay the probability of another pool broadcasting a block and orphanning you increases.  What about a miners that due to latency returns the solution after you publish the winning block?

It isn't a viable system. 

Well, like I said,  a forum post does not a system make.   

This is a problem that has to be solved,  or pooled mining will die.   We aren't going to solve (or at least I'm not going to try) in this thread,   but a sanity check of hardware is not an unheard of thing, and it's clearly needed now.

Folks are not going to stick around while they watch 20% of their earnings evaporate.
donator
Activity: 1218
Merit: 1079
Gerald Davis
 Those miners wouldn't even have a way to know it is a solution to a block.

Well that is the point isn't it?

You misunderstand.  My point is that legit miner would NEVER find a solution for that particular unit of work.  It isn't due to fraud, it is due to the mechanics of mining.  So if a miner doesn't return a solution (in time) is it because he is withholding, is it because he legitimately found no solution, is it because due to queued up work he never even started working on that work until after you broadcasted the winning block (at which point he would flush all stale work)?

You don't know.  There is no way to know.

Quote
Clearly this would need to be flushed out and tested and instaban is simply rhetoric.    I'm not going to engineer the entire test here in a forum post,  but that's the top level pseudo code.

It doesn't work even at the top level code.  You are making incorrect assumptions about how much work is actually completed and returned.  The error rate wouldn't be 1% it would be more like 20%+.  Some ASIC chips don't attempt every value in nonce range.  If the winning solution uses a nonce of 0x18392740 and a particular ASIC is not checking that value the miner isn't going to return a solution.  You could send him the same work 1,000 times and he will NEVER return a solution.  

Redo your numbers with say 10% or 20%.  Also how long are you willing to wait to test all miners?  How much increased orphan losses are you willing to accept?  Is 100% luck with 20% orphan loss a good pool?

It is also trivial to bypass.  Attacker simply has more than one account and compares the work received between accounts.  Work is normally never duplicated so any duplicate work is a test.  Attacker returns 100% of test blocks and drops 100% of legit blocks.  Even at a smaller % tested the attacker can sniff the test blocks out with ease.  If you test 10% of users and attacker has hashpower spread across 30 accounts the probability that the test block won't be duplicated across his accounts is nil.
newbie
Activity: 26
Merit: 0
 Those miners wouldn't even have a way to know it is a solution to a block.

Well that is the point isn't it?

Clearly this would need to be flushed out and tested and instaban is simply rhetoric.    I'm not going to engineer the entire test here in a forum post,  but that's the top level pseudo code.  However any miner that fails the test should be considered suspect and perhaps subject to an additional test.   A miner with a 1% error rate would have a 1 in 1,000 chance of artificially failing 2 known good blocks.  Make it 3 tests and the odds raise to 1 in 100,000

All this aside.  It seems that pools now need a way to prove themselves reliable.   As much as I like BTCGuild, I have moved away because it is crystal clear that it is now tainted in some way that costs me money.
donator
Activity: 1218
Merit: 1079
Gerald Davis
I would think you could detect someone performing a block withholding attack on the pool through a simple test:

First identify your 'unluckiest' participants.  Then when a block is solved, immediately send the work unit for the block to those miners.  If they consistently fail to return the winning solution you know they are ripping you off.

There are a number of other attacks that could be detected with similar approaches.

What's troubling to me is the massive asymmetry of risks right now.  The miners on BTCguild have over $50M in hardware investments that is depreciating at 25% / month .  I'd be shocked if Micheal has $50k in total in hardware to run the pool.

My personal theory has been mentioned already:


Startup 42e1 spends $2M on design and masks to build out their $5M 2Ph/s mine only to discover that the design has a flaw that prevents it from finding nonces with difficulties over 4.2 billion.  It still hashes lots of results below that level, so instead of scrapping things they move all their machines off the internal pool to a big public pool.


There are many, many other reasons someone could be maliciously attacking a pool, and the stakes keep getting larger.  If you're operating a public pool and not actively developing defenses, you will be burned.

And personally if I was clearing $250k / month off my pool, I wouldn't care what little people think either, but I would pay somebody to care.

Actually,  You should send the the winning hash/nonce to ALL particpants.  Just that hash.  Every miner that doesn't return the winning block should instantly be blacklisted.

THis is very very minor tax,  but a huge payoff in preventing a withholding attack, or blocking bad hardware.

How long do you wait before you broadcast the block?  Every 6 seconds that passes there is a 1% chance that another pool solves a block and your block becomes orphaned.  The hardware error rate of miners is not zero.  You would rapidly blacklist yourself to a pool of zero.  What about miners who already have work queued up?  Are you going to wait multiple seconds and increase your orphan rate while still having false negatives and end up blacklisting good miners?  Some ASICs don't attempt every nonce.  Those miners wouldn't even have a way to know it is a solution to a block.
hero member
Activity: 560
Merit: 500
I would think you could detect someone performing a block withholding attack on the pool through a simple test:

First identify your 'unluckiest' participants.  Then when a block is solved, immediately send the work unit for the block to those miners.  If they consistently fail to return the winning solution you know they are ripping you off.

There are a number of other attacks that could be detected with similar approaches.

What's troubling to me is the massive asymmetry of risks right now.  The miners on BTCguild have over $50M in hardware investments that is depreciating at 25% / month .  I'd be shocked if Micheal has $50k in total in hardware to run the pool.

My personal theory has been mentioned already:


Startup 42e1 spends $2M on design and masks to build out their $5M 2Ph/s mine only to discover that the design has a flaw that prevents it from finding nonces with difficulties over 4.2 billion.  It still hashes lots of results below that level, so instead of scrapping things they move all their machines off the internal pool to a big public pool.


There are many, many other reasons someone could be maliciously attacking a pool, and the stakes keep getting larger.  If you're operating a public pool and not actively developing defenses, you will be burned.

And personally if I was clearing $250k / month off my pool, I wouldn't care what little people think either, but I would pay somebody to care.

Actually,  You should send the the winning hash/nonce to ALL particpants.  Just that hash.  Every miner that doesn't return the winning block should instantly be blacklisted.

THis is very very minor tax,  but a huge payoff in preventing a withholding attack, or blocking bad hardware.


It won't work. Our miners for example don't check all nonce range, and a lot of cgminer generated jobs get dropped. Also we use ntime-rolling differently from other pools. It's impossible to know what the miner will generate from stratum template.
newbie
Activity: 26
Merit: 0
I would think you could detect someone performing a block withholding attack on the pool through a simple test:

First identify your 'unluckiest' participants.  Then when a block is solved, immediately send the work unit for the block to those miners.  If they consistently fail to return the winning solution you know they are ripping you off.

There are a number of other attacks that could be detected with similar approaches.

What's troubling to me is the massive asymmetry of risks right now.  The miners on BTCguild have over $50M in hardware investments that is depreciating at 25% / month .  I'd be shocked if Micheal has $50k in total in hardware to run the pool.

My personal theory has been mentioned already:


Startup 42e1 spends $2M on design and masks to build out their $5M 2Ph/s mine only to discover that the design has a flaw that prevents it from finding nonces with difficulties over 4.2 billion.  It still hashes lots of results below that level, so instead of scrapping things they move all their machines off the internal pool to a big public pool.


There are many, many other reasons someone could be maliciously attacking a pool, and the stakes keep getting larger.  If you're operating a public pool and not actively developing defenses, you will be burned.

And personally if I was clearing $250k / month off my pool, I wouldn't care what little people think either, but I would pay somebody to care.

Actually,  You should send the the winning hash/nonce to ALL particpants.  Just that hash.  Every miner that doesn't return the winning block should instantly be blacklisted.

THis is very very minor tax,  but a huge payoff in preventing a withholding attack, or blocking bad hardware.
hero member
Activity: 560
Merit: 500
First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

Not necessarily.  If there is a bug in the hashing engine itself it could be possible.  Think about the shift operations in SHA-256.  You could have an incorrect implementation that produced good hashes in the last 32 bits (all that is tested in 1st gen approaches) up until you roll into hashes with zeros all the way up to 64 plus bits.  Then think about what chips were famous for generating invalids at outrageous rates.  :-)

If something like this is going on the actual details of how it happens is likely to be vastly more complicated that what I am saying.  But with hardware implementations anything is possible.
No sorry you don't understand sha256. It's a 32 bit operation on a fixed length string in mining and has NO concept of difficulty or zeroes.

Your failure to understand me doesn't mean I don't understand SHA-256.  I've implemented it in both software and hardware and my comments are based upon failure modes I worried about in the hardware implementation.

Let's just sit back and see what the next few days reveal.

You can easily fix this in cg-miner/miner driver. Give lower difficulty (let's say 43 leading zeroes). Even on 5TH miner it will give you less then 1 win per second. Then drop unneeded results from HW. You will get all the shares, even if the HW doesn't know how to compare "high" zeroes. 
hero member
Activity: 756
Merit: 501
First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

Not necessarily.  If there is a bug in the hashing engine itself it could be possible.  Think about the shift operations in SHA-256.  You could have an incorrect implementation that produced good hashes in the last 32 bits (all that is tested in 1st gen approaches) up until you roll into hashes with zeros all the way up to 64 plus bits.  Then think about what chips were famous for generating invalids at outrageous rates.  :-)

If something like this is going on the actual details of how it happens is likely to be vastly more complicated that what I am saying.  But with hardware implementations anything is possible.
No sorry you don't understand sha256. It's a 32 bit operation on a fixed length string in mining and has NO concept of difficulty or zeroes.

Your failure to understand me doesn't mean I don't understand SHA-256.  I've implemented it in both software and hardware and my comments are based upon failure modes I worried about in the hardware implementation.

Let's just sit back and see what the next few days reveal.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

Not necessarily.  If there is a bug in the hashing engine itself it could be possible.  Think about the shift operations in SHA-256.  You could have an incorrect implementation that produced good hashes in the last 32 bits (all that is tested in 1st gen approaches) up until you roll into hashes with zeros all the way up to 64 plus bits.  Then think about what chips were famous for generating invalids at outrageous rates.  :-)

If something like this is going on the actual details of how it happens is likely to be vastly more complicated that what I am saying.  But with hardware implementations anything is possible.
No sorry you don't understand sha256. It's a 32 bit operation on a fixed length string in mining and has NO concept of difficulty or zeroes.
hero member
Activity: 756
Merit: 501
First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.

Not necessarily.  If there is a bug in the hashing engine itself it could be possible.  Think about the shift operations in SHA-256.  You could have an incorrect implementation that produced good hashes in the last 32 bits (all that is tested in 1st gen approaches) up until you roll into hashes with zeros all the way up to 64 plus bits.  Then think about what chips were famous for generating invalids at outrageous rates.  :-)

If something like this is going on the actual details of how it happens is likely to be vastly more complicated that what I am saying.  But with hardware implementations anything is possible.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
First and even second generation asics had no diff support within the chips whatsoever so they just produced a hash and it was up to the software to decide what to do with the result so it cannot be the hardware in the case of avalon 1/2, bfl etc. 3rd generation hardware almost all has diff support on chip or in firmware and that's the only place that the hardware itself could be responsible.
hero member
Activity: 756
Merit: 501
Someone (meni or was it gmaxwell) proposed a solution to this any many other attacks against pools.  Going from memory (and likely butchering it) it involved sending blinded work to the miner such that the miner would know if a hash produced a valid share but not if it produced a valid block.  The pool using information kept from the miner could check all returned work and confirm which ones produced valid blocks.  Thus there would be no withholding attack possible (not even against PPS).  It would require a hard fork but I could see it spun as a security measure to prevent disruption of the vital proof of work component.  Maybe I can find the paper.

Found the paper (annoyingly it doesn't support copying, so I can't quote the relevant section).
https://bitcoil.co.il/pool_analysis.pdf
See Section 6.3.2 Oblivious Shares

A hard fork is definitely a bridge too far - there would be too many competing options to ever develop consensus on a solution that requires a hard fork.

I do think stratum is broken if it isn't possible for a pool to verify that the hardware it is connected to is functional and operating honestly.

Given what has been said today, I am thinking a certain first generation design known for having many bugs may be broken by current difficulty.  If so, the large users of that hardware had to be aware of the issue and choose to cheat others rather than deal with the problem.  They would have seen a degradation in solo-mining performance as the cliff approached since the probability of a share >1B is dramatically larger than the probability of finding a share between 1B and 4.2B.
legendary
Activity: 1750
Merit: 1007
Someone (meni or was it gmaxwell) proposed a solution to this any many other attacks against pools.  Going from memory (and likely butchering it) it involved sending blinded work to the miner such that the miner would know if a hash produced a valid share but not if it produced a valid block.  The pool using information kept from the miner could check all returned work and confirm which ones produced valid blocks.  Thus there would be no withholding attack possible (not even against PPS).  It would require a hard fork but I could see it spun as a security measure to prevent disruption of the vital proof of work component.  Maybe I can find the paper.

Yes, I remember something about it, I believe it was Meni.  The problem is, as you said, it would require a hardfork.  Even if we had proof that this type of attack was happening to explicitly kill off the ability for miners to pool together without incurring huge losses, it would ever be implemented.

If it becomes widespread, the obvious solution would be going to invitation based pooling where miners can be vetted before receiving payments.  The problem is this would completely kill the ability for small miners to join a pool since they would be too slow to prove that they were going to provide valid blocks.
donator
Activity: 1218
Merit: 1079
Gerald Davis
Someone (meni or was it gmaxwell) proposed a solution to this any many other attacks against pools.  Going from memory (and likely butchering it) it involved sending blinded work to the miner such that the miner would know if a hash produced a valid share but not if it produced a valid block.  The pool using information kept from the miner could check all returned work and confirm which ones produced valid blocks.  Thus there would be no withholding attack possible (not even against PPS).  It would require a hard fork but I could see it spun as a security measure to prevent disruption of the vital proof of work component.  Maybe I can find the paper.

Found the paper (annoyingly it doesn't support copying, so I can't quote the relevant section).
https://bitcoil.co.il/pool_analysis.pdf
See Section 6.3.2 Oblivious Shares
newbie
Activity: 35
Merit: 0
Unscrupulous commercial miners will destroy the public pools to eliminate competition from small players.

That would be counter productive as it would devalue bitcoin and destroy confidence in it.

That assumes that the goal of the commercial farms is to make a profit in bitcoin. That there is no ulterior motive for preserving the status quo commercial banking and money transferring systems, or for bringing their own crypto currency to the market. "See, bitcoin is broken, but here we have this massive farm and will use it to back up and secure bitcoin2."
legendary
Activity: 3586
Merit: 1098
Think for yourself
Unscrupulous commercial miners will destroy the public pools to eliminate competition from small players.

That would be counter productive as it would devalue bitcoin and destroy confidence in it.
Pages:
Jump to: