[ANN] Stratum mining protocol - ASIC ready - page 11.

slush

legendary

Activity: 1386

Merit: 1097

Quote from: -ck on February 20, 2013, 08:29:19 PM

Well of course I like that (since I suggested it) but the only disadvantage I see is the pool really does not have a flag now in its parameters to tell the mining software whether it supports resume or not, so the only way to find out is if cgminer tries the command first. That's ok too, but the issue there is what the pool does in response to a command it does not recognise: some will drop the connection, and I'm not sure what others do. I need some comments from the pool ops please?

Pool should not close the connection on unknown message. Like this:

Code:

{"id": 1, "method": "mining.resume", "params": []}
{"error": [-3, "Method 'resume' not found for service 'mining'", null], "id": 1, "result": null}

Server should close connection only when the payload is corrupted (thus there's no way to identify messages and respond with standard error).

-ck

legendary

Activity: 4088

Merit: 1631

Ruu \o/

Quote from: slush on February 20, 2013, 07:54:38 PM

The first parameter is the rest of original Stratum idea; to be universal overlay protocol with some extended publish/subscribe features. To be honest, there's still no real use for initial meaning of the first parameter and the whole idea seems to be a bit overcomplicated.

So I'm offering to change meaning of the first parameter, let's call it session_id :-). I checked all known implementations (stratum proxy, poclbm, cgminer/bfgminer) and all three seems to completely ignore the first parameter. Using first argument as session_id seems to be fully backward compatible.

Any ideas?

Well of course I like that (since I suggested it) but the only disadvantage I see is the pool really does not have a flag now in its parameters to tell the mining software whether it supports resume or not, so the only way to find out is if cgminer tries the command first. That's ok too, but the issue there is what the pool does in response to a command it does not recognise: some will drop the connection, and I'm not sure what others do. I need some comments from the pool ops please?

slush

legendary

Activity: 1386

Merit: 1097

...and I'll also fix proxy to ignore any unknown parameters in subscribe notification. It wasn't intentional, I'm sorry for such kind of issue :-/.

slush

legendary

Activity: 1386

Merit: 1097

The first parameter is the rest of original Stratum idea; to be universal overlay protocol with some extended publish/subscribe features. To be honest, there's still no real use for initial meaning of the first parameter and the whole idea seems to be a bit overcomplicated.

So I'm offering to change meaning of the first parameter, let's call it session_id :-). I checked all known implementations (stratum proxy, poclbm, cgminer/bfgminer) and all three seems to completely ignore the first parameter. Using first argument as session_id seems to be fully backward compatible.

Any ideas?

-ck

legendary

Activity: 4088

Merit: 1631

Ruu \o/

Quote from: slush on February 20, 2013, 02:14:17 PM

Basically the idea is that miner remember the last difficulty given by the previous mining session and it sends mining.suggest_difficulty(difficulty) on the beginning of the next session (it may be sent before mining.subscribe or mining.resume, but it should not be a requirement). The response to the call is irrelevant for the miner, but if the pool accept the suggestion in any way, it will send standard mining.set_difficulty with new computed difficulty.

Obviously the difficulty can differ from pool to pool while switching between backup pools, because every pool can have different SPM target . The main goal is to give at least some approximation for the newly connected pool, so it won't start with difficulty 1 for 10 THash/s monster rig...

Ok, I can easily implement this.

Moving back to the resume support, since mining.resume breaks your proxy when implemented pool-side with the current format, and there will be lots of miners still using your proxy and this makes the pools not interested in supporting the feature, can I suggest a modification to a use of the mining.subscribe response?

SEND: {"id": 0, "method": "mining.subscribe", "params": []}
RECV: {"error": null, "id": 0, "result": [["mining.notify", "ae6812eb4cd7735a302a8a9dd95cf71f"], "f8019658", 4]}

Is it possible to use the existing unique subscription ID as the session ID with mining.resume instead? I know some pools use this for different reasons, so I'm not sure if that's possible, but just a suggestion. Alternatively, can you put the session ID somewhere else in the JSON response that does NOT break your proxy? While I have committed the changes to cgminer git with the existing format, there are no users currently depending on that format so we can and should change it.

slush

legendary

Activity: 1386

Merit: 1097

Quote from: eleuthria on February 20, 2013, 04:04:53 PM

However, I think simple minimum difficulty settings available through the frontend solve problems for large workers. And at least with my implementation, the pool will adjust very high speed workers to an appropriate difficulty before bandwidth becomes an issue. They'll see a spike at the very start, but with aggressive difficulty adjustments you can prevent it from being enough to saturate even on a very slow connection (3g/dial-up).

I agree that this will work, too, but it requires that user knows what's the appropriate difficulty for his miners. For the most users difficulty is simply abstract number.

In the end, pools will use some mixed solutions like
a) suggestion provided by mining software
b) minimum difficulty set by the user on pool profile
c) using higher default difficulty for unknown connections (will affect slow miners until the difficulty settle down, but it will help fast miners during startup).

eleuthria

legendary

Activity: 1750

Merit: 1007

Quote from: jamesg on February 20, 2013, 03:27:49 PM

Is it OK for a pool to remember a minimum diff per worker and when the worker reconnects set their diff to the remembered value?

The chain of events would be something like:

- mining.subscribe (default diff of the pool is given)
- mining.authorize (this is where the worker would be known and a remembered diff could be determined.)
- if remembered worker diff higher than default send mining.set_difficulty

Thoughts?

That is certainly one way a pool can handle it, and one way I've been considering for BTC Guild. However, I think simple minimum difficulty settings available through the frontend solve problems for large workers. And at least with my implementation, the pool will adjust very high speed workers to an appropriate difficulty before bandwidth becomes an issue. They'll see a spike at the very start, but with aggressive difficulty adjustments you can prevent it from being enough to saturate even on a very slow connection (3g/dial-up).

Quote from: jddebug on February 20, 2013, 04:01:32 PM

I use one worker at ozcoin and it sets each rig to a unique difficulty somehow.

Stratum by definition is per-connection difficulty. gigavps's suggestion would only affect pools which try to remember/re-instate difficulty on a after worker authorization.

jddebug

sr. member

Activity: 446

Merit: 250

Quote from: slush on February 20, 2013, 03:49:12 PM

No, except that slow devices using the same worker name as fast ones will have huge difficulty and it will take some time to settle on natural level.

I use one worker at ozcoin and it sets each rig to a unique difficulty somehow.

slush

legendary

Activity: 1386

Merit: 1097

No, except that slow devices using the same worker name as fast ones will have huge difficulty and it will take some time to settle on natural level.

jamesg

vip

Activity: 1358

Merit: 1000

AKA: gigavps

Quote from: slush on February 20, 2013, 03:40:53 PM

Quote from: jamesg on February 20, 2013, 03:27:49 PM

Thoughts?

This is one of possible solutions. Unfortunately many users use one worker accross all miners, because they were lazy to set up unique worker per miner...

Does the break any of the current rules for stratum?

slush

legendary

Activity: 1386

Merit: 1097

Quote from: jamesg on February 20, 2013, 03:27:49 PM

Thoughts?

This is one of possible solutions. Unfortunately many users use one worker accross all miners, because they were lazy to set up unique worker per miner...

jamesg

vip

Activity: 1358

Merit: 1000

AKA: gigavps

Is it OK for a pool to remember a minimum diff per worker and when the worker reconnects set their diff to the remembered value?

The chain of events would be something like:

- mining.subscribe (default diff of the pool is given)
- mining.authorize (this is where the worker would be known and a remembered diff could be determined.)
- if remembered worker diff higher than default send mining.set_difficulty

Thoughts?

slush

legendary

Activity: 1386

Merit: 1097

Basically the idea is that miner remember the last difficulty given by the previous mining session and it sends mining.suggest_difficulty(difficulty) on the beginning of the next session (it may be sent before mining.subscribe or mining.resume, but it should not be a requirement). The response to the call is irrelevant for the miner, but if the pool accept the suggestion in any way, it will send standard mining.set_difficulty with new computed difficulty.

Obviously the difficulty can differ from pool to pool while switching between backup pools, because every pool can have different SPM target . The main goal is to give at least some approximation for the newly connected pool, so it won't start with difficulty 1 for 10 THash/s monster rig...

-ck

legendary

Activity: 4088

Merit: 1631

Ruu \o/

Quote from: slush on February 20, 2013, 12:44:24 PM

Quote from: DrHaribo on February 20, 2013, 12:26:30 PM

Would it be a suggestion that the server can take under advisement and choose to ignore?

I suggested this already, but there was almost no response by miner developers (cgminer). In oposite to mining.resume, mining.suggest is trivial to implement on pool side or it can be ignored if pool doesn't understand it.

Just tell me what to do and I'd implement it, bugs and all.

jddebug

sr. member

Activity: 446

Merit: 250

Quote from: DrHaribo on February 20, 2013, 12:26:30 PM

Quote from: Bitsilver.us on February 20, 2013, 09:18:10 AM

I do see reasons to implement a difficulty setting in the miner

What are those reasons?

I had brought it up with Luke Jr. because I am mining at Eligius Diff 64 and I was having network problems. I'd be dropping to backup pool Diff 1 and then slowly climb to 64 with massive bandwidth usage in the meantime and then a while later I'd be back at Eligius at Diff 1 again with massive bandwidth usage while it made its way back to Diff 64 again. Back and forth with 168GH I was seeing the drawback to having to wait for the pool to figure out my Difficulty and I asked Luke Jr. if I could just set it myself.

Luke Jr. explained thats not possible with Stratum at this time. I have 10TH on order. If I had that hashrate at this time and I was dropping to and from backup pools it appears I would have a major problem and it would appear the pool would too.

Am I mistaken?

slush

legendary

Activity: 1386

Merit: 1097

Quote from: DrHaribo on February 20, 2013, 12:26:30 PM

Would it be a suggestion that the server can take under advisement and choose to ignore?

I suggested this already, but there was almost no response by miner developers (cgminer). In oposite to mining.resume, mining.suggest is trivial to implement on pool side or it can be ignored if pool doesn't understand it.

DrHaribo

legendary

Activity: 2730

Merit: 1034

Needs more jiggawatts

Quote from: Bitsilver.us on February 20, 2013, 09:18:10 AM

I do see reasons to implement a difficulty setting in the miner

What are those reasons?

How would it work? Would it be a suggestion that the server can take under advisement and choose to ignore?

Bitsilver.us

newbie

Activity: 14

Merit: 0

Quote from: jborkl on February 19, 2013, 05:30:23 PM

User defined Stratum diff is already available. So I don't see the rush to implement it into a miner.

I do see reasons to implement a difficulty setting in the miner and hope this bounces up the list of priorities for BFGMiner.

nathanrees19

full member

Activity: 196

Merit: 100

Quote

cgminer's implementation is

This unjustified assertion has no place in this thread.

Quote

Sadly, I've also found I cannot simply disable it based on client.get_version response - apparently requesting this information before initialization has completed breaks

It's normal for things to break if you hammer them with requests before initialization is finished.

Quote

The only solution I can think of to deal with this bug is to have newer clients send their version unsolicited before trying to subscribe.

This is surprisingly not a bad idea.

Quote

Another feature request (which I have to note has been specified in GBT for a year now)

There's no need to beat a dead horse here.

Quote

is the ability for miners to send a default difficulty/target to the server.

It should be up to the server to decide what rate of shares it can accept, but if an unsolicited client version is to be added, then perhaps a "suggested minimum difficulty" could be added as well.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: jborkl on February 19, 2013, 05:30:23 PM

Luke, I thought you were still using this full of bugs codebase.....?

Fixing bugs as I find them, but Eloipool still has to support cgminer for now.

Quote from: jborkl on February 19, 2013, 05:30:23 PM

Luke , have you submitted a support ticket to Con, so he can help you out?

Feel free to pass it on to him. The details on the mining.resume bug lie in send() succeeding even when the packets don't actually reach their destination, and cgminer leaking the sshare object since it never gets a response.
BFGMiner's codebase is too different from cgminer's in the relevant code, so if this gets fixed it can be the first cgminer-unique feature until I get a chance to write a version for BFGMiner. Wink

Topic: [ANN] Stratum mining protocol - ASIC ready - page 11. (Read 146096 times)