(OLD) BFGMiner: modular FPGA/GPU, GBT, Stratum, RPC, Avalon/Lnx/OpnWrt/PPA/W64 - page 6.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: aeris on February 23, 2013, 04:16:32 PM

Quote from: Luke-Jr on February 23, 2013, 03:46:03 PM

Is it possible BitMinter is vardiff and the others are not?

Could be a clue. I will check it.
But I have a low hashrate (<50MH), IMO even if vardiff, I can't be on a higher diff…

Hmm, yeah, it would be strange for 50 Mh to hit a higher diff.
Can you get it to behave more as expected by using older versions and/or --no-stratum? Maybe try --no-gbt --no-stratum both as well?

Quote from: aeris on February 23, 2013, 04:16:32 PM

Quote

The community-developed standard ("new bitcoin way") is getblocktemplate (GBT).
Stratum supports less functionality, was developed behind closed doors, and uses more bandwidth than GBT unless used in a way harmful to Bitcoin.

Oh, sorry for my mistake, this is bitcoin.cz which confusing me, because they push all the pool to stratum.

Ok for GBT, but is there any way in bfgminer to enable stratum only on some pools ?
--no-stratum disable it for all pools, but some need it (bitcoin.cz e.g.) !

--no-stratum only disables automatic switching from getwork/GBT.
You can still specify the stratum URI manually. eg stratum+tcp://host:port

aeris

newbie

Activity: 18

Merit: 0

Quote from: Luke-Jr on February 23, 2013, 03:46:03 PM

Is it possible BitMinter is vardiff and the others are not?

Could be a clue. I will check it.
But I have a low hashrate (<50MH), IMO even if vardiff, I can't be on a higher diff…

Quote

The community-developed standard ("new bitcoin way") is getblocktemplate (GBT).
Stratum supports less functionality, was developed behind closed doors, and uses more bandwidth than GBT unless used in a way harmful to Bitcoin.

Oh, sorry for my mistake, this is bitcoin.cz which confusing me, because they push all the pool to stratum.

Ok for GBT, but is there any way in bfgminer to enable stratum only on some pools ?
--no-stratum disable it for all pools, but some need it (bitcoin.cz e.g.) !

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: crazyates on February 23, 2013, 04:12:06 PM

Quote from: Luke-Jr on February 23, 2013, 03:21:47 PM

Quote from: crazyates on February 23, 2013, 03:09:32 PM

Quote from: Luke-Jr on February 23, 2013, 02:58:16 PM

BitMinter does support GBT, which is better - not sure why they keep switching people to stratum.

Your own comparison between GBT and Stratum shows Stratum using a fraction of the bandwidth. Any potential advantages of having a miner automatically download a copy of the transaction list is completely unused by every miner software out here, including yours, last time I checked.

I know GBT is your favorite, and that's fine, but please don't make universal statements like "GBT is better" when the evidence proves otherwise.

Only blind stratum uses less bandwidth. Secured stratum uses more, since it doesn't support compression.
While there aren't any special checks on transaction data (yet), just fetching it means someone could be checking it, and deters any possible abuse.
Put another way, are you going to rob a vending machine if your bosses are all standing right there, even if they're not specifically looking for foul play at the moment?

So I was right. At this time, choosing GBT over Stratum is just wasting bandwidth, with no tangible advantages in the here and now. Sure, it has potential, but until that potential turns into an actual increase in network security, you can't really complain about pools not choosing GBT, can you?

It's already actual, since pools can't know who is or isn't verifying what.

crazyates

legendary

Activity: 952

Merit: 1000

Quote from: Luke-Jr on February 23, 2013, 03:21:47 PM

Quote from: crazyates on February 23, 2013, 03:09:32 PM

Quote from: Luke-Jr on February 23, 2013, 02:58:16 PM

BitMinter does support GBT, which is better - not sure why they keep switching people to stratum.

Your own comparison between GBT and Stratum shows Stratum using a fraction of the bandwidth. Any potential advantages of having a miner automatically download a copy of the transaction list is completely unused by every miner software out here, including yours, last time I checked.

I know GBT is your favorite, and that's fine, but please don't make universal statements like "GBT is better" when the evidence proves otherwise.

Only blind stratum uses less bandwidth. Secured stratum uses more, since it doesn't support compression.
While there aren't any special checks on transaction data (yet), just fetching it means someone could be checking it, and deters any possible abuse.
Put another way, are you going to rob a vending machine if your bosses are all standing right there, even if they're not specifically looking for foul play at the moment?

So I was right. At this time, choosing GBT over Stratum is just wasting bandwidth, with no tangible advantages in the here and now. Sure, it has potential, but until that potential turns into an actual increase in network security, you can't really complain about pools not choosing GBT, can you?

jddebug

sr. member

Activity: 446

Merit: 250

Quote from: Luke-Jr on February 23, 2013, 03:21:47 PM

Quote from: crazyates on February 23, 2013, 03:09:32 PM

Quote from: Luke-Jr on February 23, 2013, 02:58:16 PM

BitMinter does support GBT, which is better - not sure why they keep switching people to stratum.

Your own comparison between GBT and Stratum shows Stratum using a fraction of the bandwidth. Any potential advantages of having a miner automatically download a copy of the transaction list is completely unused by every miner software out here, including yours, last time I checked.

I know GBT is your favorite, and that's fine, but please don't make universal statements like "GBT is better" when the evidence proves otherwise.

Only blind stratum uses less bandwidth. Secured stratum uses more, since it doesn't support compression.
While there aren't any special checks on transaction data (yet), just fetching it means someone could be checking it, and deters any possible abuse.
Put another way, are you going to rob a vending machine if your bosses are all standing right there, even if they're not specifically looking for foul play at the moment?

My guess is that if you add the transaction data checking that there would be a number of miners who would like to use it just to be part of Bitcoin security. I'd want to check it out at least.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: aeris on February 23, 2013, 03:38:23 PM

Quote from: Luke-Jr on February 23, 2013, 02:58:16 PM

These numbers aren't relevant in any meaningful way.
If your concern is shares, look at the 3rd hashrate
If your concern is bandwidth, look at E: (higher is better)

Bandwidth is not a problem for me.
And I notice a problem because the bitminter java client produces a lot of shares for the same hashrate than bfgminer (~5× more).
Same problem with bfgminer in load-balance mode, 50btc and bitcoin.cz produces 4× more shares than bitminter for the same number of work request and hashrate.

Is it possible BitMinter is vardiff and the others are not?

Quote from: aeris on February 23, 2013, 03:38:23 PM

Quote

Pretty normal for stratum pools.

bitcoin.cz is stratum too, and no such « work restart » on it…

That would be very odd.. are you sure you're connecting to it with stratum? Maybe the messages are just not as noticable since you're finding more shares between them?

Quote from: aeris on February 23, 2013, 03:38:23 PM

Quote

You can use --no-stratum to avoid it.

Stratum is the new bitcoin way for mining, why avoid it Huh

Some pool give you more payback when you use stratum instead of gwork, like bitcoin.cz.

The community-developed standard ("new bitcoin way") is getblocktemplate (GBT).
Stratum supports less functionality, was developed behind closed doors, and uses more bandwidth than GBT unless used in a way harmful to Bitcoin.
While it might with time become a new standard (slush recently started bringing it to the BIP standardization process), it still has a way to go before it can really compete with GBT.
Admittedly, knowing this doesn't do much for you while you mine on pools that force you to use stratum because they refuse to support standard GBT, but that is not the case with BitMinter Wink

aeris

newbie

Activity: 18

Merit: 0

Quote from: Luke-Jr on February 23, 2013, 02:58:16 PM

These numbers aren't relevant in any meaningful way.
If your concern is shares, look at the 3rd hashrate
If your concern is bandwidth, look at E: (higher is better)

Bandwidth is not a problem for me.
And I notice a problem because the bitminter java client produces a lot of shares for the same hashrate than bfgminer (~5× more).
Same problem with bfgminer in load-balance mode, 50btc and bitcoin.cz produces 4× more shares than bitminter for the same number of work request and hashrate.

Quote

Pretty normal for stratum pools.

bitcoin.cz is stratum too, and no such « work restart » on it…

Quote

You can use --no-stratum to avoid it.

Stratum is the new bitcoin way for mining, why avoid it Huh

Some pool give you more payback when you use stratum instead of gwork, like bitcoin.cz.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: crazyates on February 23, 2013, 03:09:32 PM

Quote from: Luke-Jr on February 23, 2013, 02:58:16 PM

BitMinter does support GBT, which is better - not sure why they keep switching people to stratum.

Your own comparison between GBT and Stratum shows Stratum using a fraction of the bandwidth. Any potential advantages of having a miner automatically download a copy of the transaction list is completely unused by every miner software out here, including yours, last time I checked.

I know GBT is your favorite, and that's fine, but please don't make universal statements like "GBT is better" when the evidence proves otherwise.

Only blind stratum uses less bandwidth. Secured stratum uses more, since it doesn't support compression.
While there aren't any special checks on transaction data (yet), just fetching it means someone could be checking it, and deters any possible abuse.
Put another way, are you going to rob a vending machine if your bosses are all standing right there, even if they're not specifically looking for foul play at the moment?

crazyates

legendary

Activity: 952

Merit: 1000

Quote from: Luke-Jr on February 23, 2013, 02:58:16 PM

BitMinter does support GBT, which is better - not sure why they keep switching people to stratum.

Your own comparison between GBT and Stratum shows Stratum using a fraction of the bandwidth. Any potential advantages of having a miner automatically download a copy of the transaction list is completely unused by every miner software out here, including yours, last time I checked.

I know GBT is your favorite, and that's fine, but please don't make universal statements like "GBT is better" when the evidence proves otherwise.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Code:

[quote author=aeris link=topic=78192.msg1553650#msg1553650 date=1361648097]I have some trouble with bfgminer on bitminter pool.
I have a very low shares production compare to bfgminer on other pools:
- bitminter : 656 Work Request, 4 Shares -> 6‰
- 50btc : 761 Work Request, 20 Shares -> 26‰
- bitcoin.cz : 1513 Work Request, 43 Shares -> 28‰[/quote]These numbers aren't relevant in any meaningful way.
If your concern is shares, look at the 3rd hashrate
If your concern is bandwidth, look at E: (higher is better)

Quote from: aeris on February 23, 2013, 02:34:57 PM

On log, I see a lot of « Stratum from pool 0 requested work restart » for bitminter, and no such trace on other pools:

Code:

[2013-02-23 20:22:04] Stratum from pool 0 requested work restart
 [2013-02-23 20:22:36] Stratum from pool 0 requested work restart
 [2013-02-23 20:23:07] Stratum from pool 0 requested work restart
 [2013-02-23 20:23:38] Stratum from pool 0 requested work restart
 [2013-02-23 20:24:09] Stratum from pool 0 requested work restart
 [2013-02-23 20:24:40] Stratum from pool 0 requested work restart
 [2013-02-23 20:25:11] Stratum from pool 0 requested work restart
 [2013-02-23 20:25:43] Stratum from pool 0 requested work restart
 [2013-02-23 20:26:14] Stratum from pool 0 requested work restart
 [2013-02-23 20:26:45] Stratum from pool 0 requested work restart
 [2013-02-23 20:27:16] Stratum from pool 0 requested work restart

After activate debug, seems each work restart occurs just after a get_transactions, which is an unknown method on bitminter :

Code:

 [2013-02-23 20:31:57] Unknown stratum msg: {"error": [-3, "Method 'get_transactions' not found for service 'mining'", null], "id": "txlist19ea", "result": null}
 [2013-02-23 20:31:58] Work stale due to work restart (32 != 33)
 [2013-02-23 20:31:58] Popping work from get queue to get work

Pretty normal for stratum pools. You can use --no-stratum to avoid it.
BitMinter does support GBT, which is better - not sure why they keep switching people to stratum.

aeris

newbie

Activity: 18

Merit: 0

Hello everybody and thanks a lot to Luke for this awesome miner.

I have some trouble with bfgminer on bitminter pool.
I have a very low shares production compare to bfgminer on other pools:
- bitminter : 656 Work Request, 4 Shares -> 6‰
- 50btc : 761 Work Request, 20 Shares -> 26‰
- bitcoin.cz : 1513 Work Request, 43 Shares -> 28‰

On log, I see a lot of « Stratum from pool 0 requested work restart » for bitminter, and no such trace on other pools:

Code:

[2013-02-23 20:22:04] Stratum from pool 0 requested work restart
 [2013-02-23 20:22:36] Stratum from pool 0 requested work restart
 [2013-02-23 20:23:07] Stratum from pool 0 requested work restart
 [2013-02-23 20:23:38] Stratum from pool 0 requested work restart
 [2013-02-23 20:24:09] Stratum from pool 0 requested work restart
 [2013-02-23 20:24:40] Stratum from pool 0 requested work restart
 [2013-02-23 20:25:11] Stratum from pool 0 requested work restart
 [2013-02-23 20:25:43] Stratum from pool 0 requested work restart
 [2013-02-23 20:26:14] Stratum from pool 0 requested work restart
 [2013-02-23 20:26:45] Stratum from pool 0 requested work restart
 [2013-02-23 20:27:16] Stratum from pool 0 requested work restart

After activate debug, seems each work restart occurs just after a get_transactions, which is an unknown method on bitminter :

Code:

 [2013-02-23 20:31:57] Unknown stratum msg: {"error": [-3, "Method 'get_transactions' not found for service 'mining'", null], "id": "txlist19ea", "result": null}
 [2013-02-23 20:31:58] Work stale due to work restart (32 != 33)
 [2013-02-23 20:31:58] Popping work from get queue to get work

So, am I good if I believe that this missing method is the source of bad performance ?
What I can do to regain perf on this pool ?

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: Epoch on February 23, 2013, 10:11:38 AM

Quote from: Luke-Jr on February 22, 2013, 10:25:55 PM

It seems to be failing over from a getwork/GBT pool to a stratum pool which has been suspended due to inactivity.
I can't reproduce your problem here, but I'm adding an extra check that a pool is working before switching to it, for 2.10.6; hopefully that will fix it.

Appreciated, thanks. It doesn't take long for me to run into this state (no more than a few hours at most) so I will, at the very least, be able to see if your fix improves the situation quickly enough.

If you can test 2.99.0, that does include this change.

Bitsilver.us

newbie

Activity: 14

Merit: 0

Quote from: Luke-Jr on February 22, 2013, 11:55:43 PM

I've uploaded some prerelease/alpha Windows builds of BFGMiner 3.0 for testing. ... Please verify your FPGAs/GPUs/pools/etc still work as expected.
Note the new --show-processors option to view your FPGAs at a more detailed level.

bfgminer-2.99.00 Win-64 been up about an hour now, nothing to report, everything is running well.

Epoch

legendary

Activity: 922

Merit: 1003

Quote from: Luke-Jr on February 22, 2013, 10:25:55 PM

It seems to be failing over from a getwork/GBT pool to a stratum pool which has been suspended due to inactivity.
I can't reproduce your problem here, but I'm adding an extra check that a pool is working before switching to it, for 2.10.6; hopefully that will fix it.

Appreciated, thanks. It doesn't take long for me to run into this state (no more than a few hours at most) so I will, at the very least, be able to see if your fix improves the situation quickly enough.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

I've uploaded some prerelease/alpha Windows builds of BFGMiner 3.0 for testing.
While they do support BitForce SC ASICs, this is completely untested so far (obviously).
There has been some major restructuring involved, so 3.0 will work differently both under and over the hood for FPGAs, especially BitForce and X6500 which have been ported to take advantage of the new device API.
37 files changed, 4325 insertions(+), 2993 deletions(-)
Please verify your FPGAs/GPUs/pools/etc still work as expected.
Note the new --show-processors option to view your FPGAs at a more detailed level.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: Epoch on February 22, 2013, 07:53:37 PM

Luke, I've managed to capture this WAIT state after 15 minutes of operation. Edited debug info here: http://pastebin.com/JYB7Lr9g

It seems to be failing over from a getwork/GBT pool to a stratum pool which has been suspended due to inactivity.
I can't reproduce your problem here, but I'm adding an extra check that a pool is working before switching to it, for 2.10.6; hopefully that will fix it.

Epoch

legendary

Activity: 922

Merit: 1003

Quote from: Luke-Jr on February 22, 2013, 02:42:12 PM

Quote from: Epoch on February 22, 2013, 02:20:48 PM

Running 2.10.5 under Win7/64, my Singles occasionally get into a WAIT state in which they stop hashing or submitting shares. The software is still running because I can see the regular messages appear (longpoll detected, requested work restart, etc.), but the Singles don't seem to get out of WAIT unless I manually switch pools via the (p)(s) command. I've had this happen when mining on bitminter and eclipse, using Stratum as well as the regular vardiff/LP ... so there doesn't seem to be a common theme.

Anecdotally I started encountering this sometime after Stratum support was added (several months ago); it has happened in all of the past few versions since then. I seem to be able to avoid the WAIT state by using the --no-stratum flag. But since I'm not hearing of anyone else having this issue, I'm sure it must be something unique to my setup ... but I have 3 separate clusters running off 3 separate PCs and I've seen the WAIT behavior on all of them.

I have several backup pools configured, and I'm wondering why bfgminer doesn't switch automatically to one of the backup pools instead of putting the miners into a WAIT state. Or is there a way to configure it to do this (I don't see anything obvious in the readme, but perhaps I missed something).

The only change --no-stratum makes is disabling automatic switching from GBT/getwork to stratum. If it works with that option, this means your problem is stratum-specific.
A debug log of this occurring would help:

Code:

bfgminer YOUR OPTIONS FIRST --debuglog 2>debug.log

Failover is currently still managed by code inherited from cgminer, which is rather slow to react to problems.
I plan to rewrite it eventually, but haven't got around to that yet.

Luke, I've managed to capture this WAIT state after 15 minutes of operation. Edited debug info here: http://pastebin.com/JYB7Lr9g

For this test run I used the following command line (windows .bat file; usernames and passwords obfuscated; 15 Singles in total):

Code:

bfgminer ^
-o http://us2.eclipsemc.com:8337 -u xxxxxxxx -p yyyyyyyy ^
-o http://mint.bitminter.com:8332 -u xxxxxxxx -p yyyyyyyy ^
-o http://us.ozco.in:3333 -u xxxxxxxx -p yyyyyyyy ^
-o http://api.bitcoin.cz:8332 -u xxxxxxxx -p yyyyyyyy ^
--disable-gpu --no-submit-stale ^
-S all ^
--debuglog 2>debug2.log

The run started at 13:51, and all of the Singles got into a WAIT state at around 16:07 from which they would not recover. At 16:06 they are all still happily mining at around 800mhps (search for the string "5s:"), but that starts to drop to 0 after 16:07.

I noticed that bfgminer was listing all of the Singles as 'WAIT' around 16:25 at which time I issued a (q) command to quite bfgminer and collect the debug info.

I hope this can help you track down where (and why) this situation happens. And as I also mentioned before, if I used the "--no-stratum" parameter, this situation never seems to occur.

Thanks in advance for the help.

Luke-Jr

legendary

Activity: 2576

Merit: 1186

Quote from: Epoch on February 22, 2013, 02:20:48 PM

Running 2.10.5 under Win7/64, my Singles occasionally get into a WAIT state in which they stop hashing or submitting shares. The software is still running because I can see the regular messages appear (longpoll detected, requested work restart, etc.), but the Singles don't seem to get out of WAIT unless I manually switch pools via the (p)(s) command. I've had this happen when mining on bitminter and eclipse, using Stratum as well as the regular vardiff/LP ... so there doesn't seem to be a common theme.

Anecdotally I started encountering this sometime after Stratum support was added (several months ago); it has happened in all of the past few versions since then. I seem to be able to avoid the WAIT state by using the --no-stratum flag. But since I'm not hearing of anyone else having this issue, I'm sure it must be something unique to my setup ... but I have 3 separate clusters running off 3 separate PCs and I've seen the WAIT behavior on all of them.

I have several backup pools configured, and I'm wondering why bfgminer doesn't switch automatically to one of the backup pools instead of putting the miners into a WAIT state. Or is there a way to configure it to do this (I don't see anything obvious in the readme, but perhaps I missed something).

The only change --no-stratum makes is disabling automatic switching from GBT/getwork to stratum. If it works with that option, this means your problem is stratum-specific.
A debug log of this occurring would help:

Code:

bfgminer YOUR OPTIONS FIRST --debuglog 2>debug.log

Failover is currently still managed by code inherited from cgminer, which is rather slow to react to problems.
I plan to rewrite it eventually, but haven't got around to that yet.

Epoch

legendary

Activity: 922

Merit: 1003

Running 2.10.5 under Win7/64, my Singles occasionally get into a WAIT state in which they stop hashing or submitting shares. The software is still running because I can see the regular messages appear (longpoll detected, requested work restart, etc.), but the Singles don't seem to get out of WAIT unless I manually switch pools via the (p)(s) command. I've had this happen when mining on bitminter and eclipse, using Stratum as well as the regular vardiff/LP ... so there doesn't seem to be a common theme.

Anecdotally I started encountering this sometime after Stratum support was added (several months ago); it has happened in all of the past few versions since then. I seem to be able to avoid the WAIT state by using the --no-stratum flag. But since I'm not hearing of anyone else having this issue, I'm sure it must be something unique to my setup ... but I have 3 separate clusters running off 3 separate PCs and I've seen the WAIT behavior on all of them.

I have several backup pools configured, and I'm wondering why bfgminer doesn't switch automatically to one of the backup pools instead of putting the miners into a WAIT state. Or is there a way to configure it to do this (I don't see anything obvious in the readme, but perhaps I missed something).

jddebug

sr. member

Activity: 446

Merit: 250

Quote from: BR0KK on February 20, 2013, 06:53:43 PM

Oh: I'm a windumb user, thats far out of my league .... I'll have a look at it but in general, i run into unexplainable problems when it comes to linux! I think it hates me Wink

Brokk, you and me seem to have had the same Linux experiences.

Topic: (OLD) BFGMiner: modular FPGA/GPU, GBT, Stratum, RPC, Avalon/Lnx/OpnWrt/PPA/W64 - page 6. (Read 260061 times)