Topic: OFFICIAL CGMINER mining software thread for linux/win/osx/mips/arm/r-pi 4.11.0 - page 438. (Read 5805983 times)

See Eleuthria's cheat sheet:
https://www.btcguild.com/new_protocol.php

You're sending a zero sized  string for extranonce1

Trying to implement Stratum on my pool, but cgminer crashes when I connect to the server.

This is what happens, from the server perspective:

recv: {"id": 0, "method": "mining.subscribe", "params": []}
send: {"result":[["mining.notify","1"],"",4],"error":null,"id":0}
recv: {"id": 1, "method": "mining.authorize", "params": ["Test_Test", "Test"]}
send: {"result":true,"error":null,"id":1}

Then Windows says "cgminer.exe has stopped working"

Fuck off. I'm impatiently waiting to your open initiative for ASIC device-level protocol. I don't believe that somebody with your attitude is preparing proprietary protocol which must be adopted by the community to use your miners.


I'm also impatiently waiting to your open proposal for mining protocol which will be superior to Stratum. I implemented the protocol which fits my needs. I don't keep gun to any poolop's head to support Stratum as well. If you don't like Stratum, stay away from it.

I cannot resist your arrogant style. But fine, I'm calming down and I'll try to be constructive now...


Pretty vague sentence. Can you be more specific? Why you didn't report your problems?


I don't believe you wrote such long post before you even read my post addressing this issue. Btw protocol don't define *any* window, it's up to server side implementation.


I think that I understand it pretty good. Any evidence for your claim? Can you elaborate more on Stratum bottlenecks regards to ASICs and what's your solution then?


Large difficulty windows are in your head, not in the protocol. Protocol don't care about any windows, pool can enforce difficulty every second. It is completely up to pool implementation and there are NO limits defined in the protocol in this area.


How is this liimited by the protocol? For example, my vardiff code is rising the difficulty pretty aggressive and in the near future, pool even won't use diff1 on the beginning of the connection. Slower miners will wait a moment until the difficulty settle to lower values. Mining on diff1 is going to die in very near future anyway, I believe that default connection difficulty will skyrocket in few months.


Hm, still don't understand how this is related to the protocol and not to the pool implementation. Can you elaborate a bit more? I'm affraid you're mixing together possible problems of miner implementation, server implementation and the protocol...


There're much simpler solutions how to DoS the pool, than hoping with real 500 TH/s. But fine, every solution which will save pool from overloading is welcome. What's yours proposal?


Exactly my solution. Again, clearly possible with Stratum.


Correct, it doesn't address submitting old shares. AFAIK there's no easy solution how to check if the share has good difficulty until you validate it on the server. What's your proposal here? Even banning the IP won't solve the problem, it only saves a bit of pool CPU time, but validating the share is quite cheap anyway...

Inaba, I ask you to calm down your tone. If you have any problems with the solution, you can report them and offer the solution in inteligent way.

With regards to being aggressive, I am irritated with the Stratum protocol in general.  It was developed in secret and foisted upon the mining world as the second coming of mining.  That's all well and good, but it was flawed and basically every single problem we've run into so far with it would have been fixed if it was developed in the open with input from the pool operators.  Instead, I'm stuck having to implement a broken protocol and make changes/fixes as the protocol is fixed in a manner that should have happened from the start... so yes, it's very irritating.  But what's done is done and I have not posted much in that thread because there's no sense in bitching about it; it is what it is.

Personally, I don't care what solution I use to solve the problems my pool is facing, so long as it works and doesn't cause problems down the road like GW has (but who can blame Tycho et al for GW since nobody really saw this scale of mining back then).  I digress... the current implementation other pools (IE - pools that are not EMC) have with Stratum is the hysterisis methods being employed are there to fix the (former?) problem with Stratum decoupling the difficulty from the work.  There is absolutely no compelling reason to make a large window of fluctuating difficulty, other than the fact that if you don't (on Stratum) shares are incorrectly discarded.  It's not like it requires a large amount of additional work on the miner end to have difficulty changing with every submission or anything.  Additionally, changing difficulty more often alleviates future problems with fluctuating hashrates. 

That said, it's something I wanted to bring up at some point:  I don't think you (Slush) or any of the other pool ops quite understand what's about to happen with regards to ASICs.  With these large difficulty windows, you are going to get flooded with shares as large scale miners come on and offline.  Even with Gen1 ASICs, the problem is going to be a bit pronounced on the larger pools as multi terahash miners come and go.  With Gen2, Gen3, and so on into the future, the problem will only magnify to unmanageable levels.  Waiting to change difficulty for longer than is necessary will only serve to flood your server with unnecessary work returns, increasing your load and potential points of failure.  Stop and consider what happens when 250 TH across 20 workers decides to hop on and off your pool... or even better yet, what if you get someone in control of 500 TH that decides to screw with your pool by hopping on and off your pool in a manner that will keep the difficulty lower than it should be?  This is going to be a problem, either through malicious acts or through random chance - it's why I do not allow user selected difficulty on EMC and why I have difficulty changing fairly rapidly.  I don't want to get DoS'd either through intention or not simply because I have 500 TH submitting difficulty 32 shares for 5 minutes before they get bumped up, rinse and repeat.  There is no drawback to rapid difficulty change in the ASIC era, and artificially limiting that to accommodate a problem with the protocol instead of fixing the protocol is backwards thinking.

I'll review the changes you mentioned... from a cursory glance, it doesn't seem to address what happens when a worker submits a share from a previous work block that had a lower difficulty or does it?  Your post is a little unclear on that, so if you could clarify that, it would help and I will work to get that incorporated in EMC. 

Assuming that addresses the fundamental issue of a miner submitting a lower difficulty share that is still valid after the difficulty has increased, does that address the problems CGMiner is having with EMC and it's rapid difficulty changes, Con?   If not, can you elaborate on exactly what the problem is with CGMiner and EMC over Stratum?

con, have u ever considered using a PPA?  would help simpletons on ubuntu like me.

First incoming donated as promised. It ain't much, as I've only got a small setup - but it's the thought that counts eh?

Thanks again for all your work on this, great stuff.

Inaba, why so aggressive? I never meet you on #stratum channel, so I'm bit surprised with your sudden interest in "proper design" of the protocol.

Did you read https://bitcointalksearch.org/topic/m.1344456 ? AFAIK it fixes all previous problems with difficulty manipulation. I also didn't read any opinion to this by you, so I suppose you agree on this change. All existing miner software (cgminer, poclbm, proxy, I expect that bfgminer as well as it is derived from cgminer) is already compatible with this change, so feel free to implement it on your pool.

I only need to change documentation to cover this change. If we ever have days with 25 hours...

To be clear, again I agree difficulty should be tied in with work on stratum instead of the current implementation. However pools can do a lot to alleviating the harm that has by tolerating a certain amount of hysteresis before retargetting diff. I'm sure emc can be tweaked some more with this in mind.

You see - that's the correct version.

Please at least understand what you are talking about before making ludicrous posts that show how stupid you are.

This is the third time in here recently you've made posts that make you seem like a simpleton.
Here's two more:
https://bitcointalksearch.org/topic/m.1342045
https://bitcointalksearch.org/topic/m.1349309

You just post crap and never back up anything you say - and it is simply that - crap.
Coz you can't back it up.

There is a known issue with Stratum (that I brought up) that if you change difficulty, then the miner will lose a very small number of shares.
If you change it often, then of course that number increases.
Yes, this is my main issue with Stratum that I have brought up all over the place - feel free to copy me and help me get slush to accept the change.
But while your at it - at least understand what the fuck you are talking about

When I mine on OzCoin with Stratum I set difficulty fixed at 8 diff so there are no difficulty changes.
Works well.
Some days I only get a single Reject in 24 hours ...

--

There are also well known issues with GBT that you have been clearly explained.
Go fix them.
But no, your head is so far up your arse you don't hear anything anyone says.

Seriously (as I have said before) I wouldn't mind if there were 2 new competing protocols.
But at the moment there aren't - there's only one: Stratum - the other one is not worth considering.

Read thru this thread: https://bitcointalksearch.org/topic/ann-stratum-mining-protocol-asic-ready-108533

"Everone invloved" would be Conman, Kano, Luke, Slush, myself and pretty much anyone who has anything to do with development and implementation of Stratum.  What is your authority to make a claim?

Of course you aren't seeing the poof, because it shows up as a rejected share when it shouldn't be.  But how would you know?

"Acknowledged" by whom?
who specifically is "everyone involved"?

My personal mining numbers show the opposite as shown on the pools actual websites. The only magical POOF I'm seeing is the end of that cool aid you've been sipping on.

Yes, but as with other code that BFGMiner inherited from cgminer, I've fixed numerous bugs in it.

I'm not sure what that has to do with the fact that it works though? The problem is not with the Stratum code, it's with the Stratum protocol.  It's broken.

Excuse me but 99.99% of the stratum code in bfgminer is from cgminer

No, it's an acknowledged problem with Stratum.  The original Stratum design is flawed.  Difficulty is decoupled from work and that is simply an incorrect way to handle mining.  Difficulty and work are inseparable from a mining perspective.  The way Stratum handles is is entirely incorrect and needs to be addressed. This is not really in question, everyone involved pretty much agrees that something needs to be done about it, the only question is exactly what.

My implementation of variable difficulty is the original implementation of variable difficulty and has been working fine on both GW and GBT for months now. It works fine in CGminer on GW, it also works fine in GBT, Stratum and GW in BFGminer. The only implementation it does not work on is CGMiner with Stratum, but that's not really CGMiners fault as its' a design flaw in Stratum and Conman doesn't really have control over that.

I hope you realize that every other Stratum implementation throws away a bunch of valid work you are doing for the pool and it even throws away solved blocks if the conditions are just right because of the flawed difficulty design of Stratum.  So tell me which would you prefer?  Shares going POOF magically on your Stratum server of choice or you getting paid for your work?

Seems to be your implementation of variable difficulty with stratum. It works fine on all the other implementations that were not influenced by GBT.

The implementation of stratum and vardiff on OZCO is running flawlessly with cgminer

stratum and variable difficulty are 2 different pieces of code
Ozcoin implemented stratum then coded the vardiff- not sure what you consider "broken" but we have it working fine, maybe the vardiff you coded before implementing stratum is the issue?

cut/paste ...

2.9.5
An Xubuntu 11.04 x86_64 executable is in my github downloads called cgminer-2.9.5a
https://github.com/kanoi/cgminer/downloads
(it also works on Fedora 16 and 17)

For anyone who didn't realise, it's just the executable file to put in place of 'cgminer'
Nothing else needs changing
First get and extract the full binary release from ckolivas and then copy my file in place of 'cgminer'

No problems so far on my '2xGPU' or 'BFL+2xICA' (90 minutes so far)
GPUs (687Mh/s) on solo and BFL+ICAs (1.6GH/s) on OzCoin Stratum with fixed 8 diff
(MMQ is doing new code testing)

The same configure options as cvolivas' binary version
In case anyone was wondering:
CFLAGS="-O2 -W -Wall" ./autogen.sh --enable-icarus --enable-bitforce --enable-ztex --enable-modminer --enable-scrypt
make clean
make

--

(and yes I made a 2.9.4a but didn't post about it)

--

sharky112065's post above is actually (indirectly) about ASIC.

I've rewritten the MMQ driver (as I've said a few times) and the problems with windows drivers included a problem with libusbx so I switched to libusb (thanks for help from the libusb developer) and the last problem was gone.
So when I finally get my changes into the main git - we need to use libusb not libusbx on windows.
Those changes include early code for the up coming ASICs.