Pages:
Author

Topic: [ANN] Stratum mining protocol - ASIC ready - page 23. (Read 146083 times)

sr. member
Activity: 351
Merit: 250
Quite logically, the implementation should be that difficulty should be part of the work, however, previous work on the same block should also be allowed since it is to the miners disadvantage for the pool to throw away valid work ... obviously Tongue

The reason this is an issue is because of the way the protocol has been implemented (not necessarily designed). Cleanjobs was designed to tell the miner to drop all work, presumably because a new block has been found. BTC Guild is pushing cleanjobs=true on a difficulty change.

There would not be an issue if difficulty changes were pushed with a new job (around every thirty seconds and with cleanjobs=false and with pools associating the difficulty of each connection with the job). It is trivial to track the associated difficulty of a connection and a new work request (with cleanjobs = false) and to ensure the work submitted matches the difficulty associated with the connection at the time the job was pushed.

A workable solution for the initial 30 seconds, which would not result in share loss, is to push connection specific jobs on difficulty changes for new connections.

Clearly miners benefit from efficient pools and pools benefit from efficient miners. Clearly, no one wants to lose a block because of a difficulty change!

Before we jump to the solution, I think we need to understand the problem. Losing shares is a problem for miners. Perhaps someone else can articulate the problem of difficulty changes between new jobs for the pools?
legendary
Activity: 2576
Merit: 1186
BIP process will result in a new protocol that combines the capabilities of both.

OK quick a quick and easy question...

Bitcoind 0.7 has GetBlockTemplate, does that mean I can use my BFL 1.5 TH Mini rig to solo mine using only bitcoind and BFGminer?
I want to underscore this one point, USING ONLY bitcoind and BFGminer, no proxy, no pool, and no other software.

NOTE: a yes, but... is a NO.
Yes, this is supported by a beta BFGMiner branch and should be merged before the BFGMiner 3.0 release (which will introduce ASIC support).
hero member
Activity: 780
Merit: 510
Bitcoin - helping to end bankster enslavement.
BIP process will result in a new protocol that combines the capabilities of both.

OK quick a quick and easy question...

Bitcoind 0.7 has GetBlockTemplate, does that mean I can use my BFL 1.5 TH Mini rig to solo mine using only bitcoind and BFGminer?
I want to underscore this one point, USING ONLY bitcoind and BFGminer, no proxy, no pool, and no other software.

NOTE: a yes, but... is a NO.
legendary
Activity: 2576
Merit: 1186
Well, python is structured and strictly object oriented. Maybe your criticism is because you can't follow control flow and don't understand it ;-). I'm not sure if you're specifically refering my python code or general python code. But I'm using asynchronous framework (Twisted), which *is* hard to learn and hard to read for beginners, but it is really powerful for some kind of tasks...
My bad.  I might as well interject into a conversation of Mensa members.  But "Twisted" is what I would call Python and I'm not the only one. Yet, I must agree you can do things in a very small amount of code in Python.  But I still think it bites.  (pun intended) Smiley
Well, Twisted deviates from normal Python code quite a bit - almost bad enough that it's really its own language.
Also, Twisted doesn't work with any current Python versions (3.x).

So the war is on.  VHS (Stratum) vs BetaMax (GBT)

It does not matter who is better, only the protocol that will be used will be the winner.
Hopefully slush's recent steps toward putting Stratum through the BIP process will result in a new protocol that combines the capabilities of both.
hero member
Activity: 780
Merit: 510
Bitcoin - helping to end bankster enslavement.
Well, python is structured and strictly object oriented. Maybe your criticism is because you can't follow control flow and don't understand it ;-). I'm not sure if you're specifically refering my python code or general python code. But I'm using asynchronous framework (Twisted), which *is* hard to learn and hard to read for beginners, but it is really powerful for some kind of tasks...
My bad.  I might as well interject into a conversation of Mensa members.  But "Twisted" is what I would call Python and I'm not the only one. Yet, I must agree you can do things in a very small amount of code in Python.  But I still think it bites.  (pun intended) Smiley

So the war is on.  VHS (Stratum) vs BetaMax (GBT)

It does not matter who is better, only the protocol that will be used will be the winner.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
Which - if I'm reading the spec right - is quite hard to do with the current protocol. Even if you change the difficulty at the exact same moment as sending out a new work unit, the difficulty change applies to all the previous work units and miners are allowed to continue working on those at the new difficulty unless you invalidate all of them at the same time. Except that if you do that, you're making the miner throw away work they've done without submitting it. There's a mismatch between how the protocol wants you to handle difficulty changes and the way you actually need to handle them to prevent cheating by miners.
btcguild is the only stratum pool with variable diff, and it  sends out a clean work with every difficulty change meaning all work gets thrown out with each diff change. Even blocks and valid shares at the new diff get thrown out I might add.
Quite logically, the implementation should be that difficulty should be part of the work, however, previous work on the same block should also be allowed since it is to the miners disadvantage for the pool to throw away valid work ... obviously Tongue

This does, however, lead to the issue that pools may be silly enough to send a 1TH/s miner 1 difficulty work.
Yeah it seems that it is impossible for the pool software programmers to handle this properly ... the solution seems to be that it's the miners problem, not the pools problem, so the miner should lose work because of it ... yet the protocol clearly states that it's the pool that has absolute control of this.

I guess setting a time limit on how much old work is valid (yes there was a reason I made that comment at the end of the post: https://bitcointalksearch.org/topic/m.1287015 Tongue) and the pools will just have to put up with this time limit since they are unable to program a suitable solution without throwing away valid miners work.

Hmm - looking at the protocol in the light of this discussion, it really currently is a protocol that gives more power to the pool and takes it away from the miner.
The protocol, by definition, allows the pool to reject shares as it chooses and gives a way to do that, that the protocol defines as valid.
Giving the pool WAY less work to do, and passing that work reduction to the miner to do, yet the miner also loses rights about the pool accepting it's valid work ... hmm ... giving away rights and being made to do more work ... where have I heard that before Cheesy

The point of this protocol was a long term solution, but with getwork using roll-n-time and high difficulty shares, it won't be necessary quite yet.
A 1.5TH/s device can do ~233 nonce ranges a second, so with the roll-n-time limit set to 7000 as it is in cgminer at the moment, that's 30s worth of work ... so there is still some (small? amount) of time left before people are forced to give away rights and do more work ... and in the mean time there may be a better solution ...
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
Which - if I'm reading the spec right - is quite hard to do with the current protocol. Even if you change the difficulty at the exact same moment as sending out a new work unit, the difficulty change applies to all the previous work units and miners are allowed to continue working on those at the new difficulty unless you invalidate all of them at the same time. Except that if you do that, you're making the miner throw away work they've done without submitting it. There's a mismatch between how the protocol wants you to handle difficulty changes and the way you actually need to handle them to prevent cheating by miners.
btcguild is the only stratum pool with variable diff, and it  sends out a clean work with every difficulty change meaning all work gets thrown out with each diff change. Even blocks and valid shares at the new diff get thrown out I might add.
hero member
Activity: 686
Merit: 564
Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
Which - if I'm reading the spec right - is quite hard to do with the current protocol. Even if you change the difficulty at the exact same moment as sending out a new work unit, the difficulty change applies to all the previous work units and miners are allowed to continue working on those at the new difficulty unless you invalidate all of them at the same time. Except that if you do that, you're making the miner throw away work they've done without submitting it. There's a mismatch between how the protocol wants you to handle difficulty changes and the way you actually need to handle them to prevent cheating by miners.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
This has to be discussed many times,please read thread history and dont open this topic again.
Sweep it back under the carpet Smiley
legendary
Activity: 1386
Merit: 1097
This has to be discussed many times,please read thread history and dont open this topic again.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
BTC Guild forces you to submit new work when updating difficulty currently.  It won't accept work from previous work when you're adjusted because it keeps track of the miner's difficulty separate from the work sent to the miner.  With the current setup, if I accepted old work, you'd get credited at the new difficulty.  This opens up an exploit where a very fast miner can purposely withhold higher diff shares at first, knowing that they are going to force multiple difficulty increases in a short time, then submit them all for higher credit than what they should have been earning.

So basically, the protocol's flawed in exactly the way kano said it was - in order for it to work reliably and without opening pools up to exploits, difficulty changes need to be tied to work units rather than happening in the middle of a work unit.

It's not a flaw in the protocol, it's a flaw in my current implementation.  It was significantly easier to tie difficulty to the miner connection object, rather than to keep a list of the miner's recent difficulties per work unit.  If I did that, it would allow you to submit "old difficulty" work, and be credited at your old difficulty.

Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
... i.e. difficulty needs to be an attribute of the work ...
legendary
Activity: 1750
Merit: 1007
BTC Guild forces you to submit new work when updating difficulty currently.  It won't accept work from previous work when you're adjusted because it keeps track of the miner's difficulty separate from the work sent to the miner.  With the current setup, if I accepted old work, you'd get credited at the new difficulty.  This opens up an exploit where a very fast miner can purposely withhold higher diff shares at first, knowing that they are going to force multiple difficulty increases in a short time, then submit them all for higher credit than what they should have been earning.

So basically, the protocol's flawed in exactly the way kano said it was - in order for it to work reliably and without opening pools up to exploits, difficulty changes need to be tied to work units rather than happening in the middle of a work unit.

It's not a flaw in the protocol, it's a flaw in my current implementation.  It was significantly easier to tie difficulty to the miner connection object, rather than to keep a list of the miner's recent difficulties per work unit.  If I did that, it would allow you to submit "old difficulty" work, and be credited at your old difficulty.  I didn't do this because difficulty adjustments do not happen frequently enough for this to materially affect any miner unless they have a habit of reconnecting every 10 minutes.

I will be making that change to account for "stale olddifficulty" shares in one of the upcoming pool software updates.

Variable difficulty -has- to adjust upon new work without being opened up to exploitation, regardless of protocol used.  It's very simple for a fast miner to withhold higher difficulty shares when it knows that it's going to force a difficulty increase, then submit them for higher credit once the adjustment lands.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
To be fair, I had exactly the same discussion with Eleuthria but I had bigger fish to fry and stopped caring about it. He or slush can come and defend why they think it's ok.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
BTC Guild forces you to submit new work when updating difficulty currently.  It won't accept work from previous work when you're adjusted because it keeps track of the miner's difficulty separate from the work sent to the miner.  With the current setup, if I accepted old work, you'd get credited at the new difficulty.  This opens up an exploit where a very fast miner can purposely withhold higher diff shares at first, knowing that they are going to force multiple difficulty increases in a short time, then submit them all for higher credit than what they should have been earning.

So basically, the protocol's flawed in exactly the way kano said it was - in order for it to work reliably and without opening pools up to exploits, difficulty changes need to be tied to work units rather than happening in the middle of a work unit.
Yeah they already are doing this - to get around the problem in the design Tongue

https://bitcointalksearch.org/topic/m.1286637
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
I wouldn't care about it either. If one is implementing Stratum in the first place, including difficulty in each job really doesn't help any.
Just thought I'd quote this ...
hero member
Activity: 686
Merit: 564
BTC Guild forces you to submit new work when updating difficulty currently.  It won't accept work from previous work when you're adjusted because it keeps track of the miner's difficulty separate from the work sent to the miner.  With the current setup, if I accepted old work, you'd get credited at the new difficulty.  This opens up an exploit where a very fast miner can purposely withhold higher diff shares at first, knowing that they are going to force multiple difficulty increases in a short time, then submit them all for higher credit than what they should have been earning.

So basically, the protocol's flawed in exactly the way kano said it was - in order for it to work reliably and without opening pools up to exploits, difficulty changes need to be tied to work units rather than happening in the middle of a work unit.
legendary
Activity: 1386
Merit: 1097
I wish I could read python code, it's not structured code, it's not object oriented, it's not even assembly language.  I can't follow control flow, I can't understand variable usage, nothing.

Well, python is structured and strictly object oriented. Maybe your criticism is because you can't follow control flow and don't understand it ;-). I'm not sure if you're specifically refering my python code or general python code. But I'm using asynchronous framework (Twisted), which *is* hard to learn and hard to read for beginners, but it is really powerful for some kind of tasks...

Quote
So why do you guys use Python?

Because I can focus to the target while programming and code on high level. Btw I understand weaknesses of Python quite good (those are generally bottleneck of current CPython implementation, not problem of language itself) and currently I'm learning Scala. But this discussion is a bit OT here :-).
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
That's a very generous donation and I only implemented unwittingly while implementing stratum, but I gladly accept it  Wink

No problem glad to help out, just not sure what you ment did you implement GBT as well when implementing stratum?
No I mean the sticking to an IP.

I have not begun to implement GBT as I think it's not as comprehensive a solution as stratum and has some serious weaknesses for pooled mining. I am in discussions with jgarzik to try and improve GBT to make it better and address one of its biggest weaknesses, but really, I'd much rather all the pools just moved to stratum instead.
hero member
Activity: 780
Merit: 510
Bitcoin - helping to end bankster enslavement.
I've been looking over the python code Smiley which works pretty well actually, just need to add DB work.

I wish I could read python code, it's not structured code, it's not object oriented, it's not even assembly language.  I can't follow control flow, I can't understand variable usage, nothing.  I could learn it but, I don't believe computer languages should be harder to use for all developers at all levels.  This is my opinion and I understand I may be wrong, I believe I'm right.  Tongue

So why do you guys use Python?

I'd be willing to help an open source implementation of stratum.

So please virtually any other language except python. Smiley
I would pay for it if it's coded in Mono C#.  Smiley

That's a very generous donation and I only implemented unwittingly while implementing stratum, but I gladly accept it  Wink

No problem glad to help out, just not sure what you ment did you implement GBT as well when implementing stratum?
hero member
Activity: 556
Merit: 500
I'd be willing to help an open source implementation of stratum.

You can improve https://github.com/slush0/stratum or at least you can read how server side works...

I've been looking over the python code Smiley which works pretty well actually, just need to add DB work.
Pages:
Jump to: