Mining against bitcoind stratum issues

tom22022

jr. member

Activity: 33

Merit: 8

Everyone on this post was so helpful I wanted to reach back out and let you know my solution which is subject to change by any other good advice I receive here.

I went out and bought a raspberry pi 4 with 8gigs and then bought the Argon One M.2 case that holds a SSD drive. I bought the 1TB drive from Samsung. Its a really neat tiny but powerful setup. Then, I bought myself a Newpac USB miner based on the BM1387 chip. Its running ubuntu.

I really had to do a lot of trial and error to get this all to work right. I really like BFGMiner, but it will not work with the new Newpacs. There was some posts about a year ago from @luke-jr about getting it supported but it doesn't have it yet. The great features of BFGMiner from my point of view are the ability to use it as a stratum proxy for other miners as well as its ability to solo mine against the bitcoind higher versions. But it just doesn't support the miner I use.

That's where cgminer comes in. Cgminer is the only miner that supports the Newpac based on BM1387. But, cgminer will not work for GBT solo mining against modern versions of bidcoind greater than 0.19.

So what I ended up doing is running my bfgminer as a stratum proxy and then run my cgminer with stratum+tcp and point it to the bfgminer. Bingo works!

So that's my ugly setup!

I hope luke-jr reads this and decides to spend some time with the newpac drivers.

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: tom22022 on April 04, 2021, 12:02:54 PM

I guess this is where I am confused at, if I want to solo mine against bitcoind, which is the way to go if I don't want to downgrade to 0.19.1? Add it back into the bitcoind code and recompile (not sure exactly how) --or-- remove code from miner client and recompile?

Thanks

I wouldn't want to mess with Bitcoin codes if that is the case though I'm sure that they've only modified that part and didn't cause any issues otherwise.

I've found this awhile back: https://bitcointalksearch.org/topic/m.56584362. I didn't test his code but it makes the field optional. My solution for my own fork was to just remove it completely but the modification as referenced would work fine as well.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: ranochigo on April 04, 2021, 08:52:17 AM

Quote from: tom22022 on April 03, 2021, 05:53:53 PM

When you say add it at the end, you mean compile it back in?

Yes. As GBT expects coinbase aux, you can just include it as an empty field again. It'll be easier to ensure minimal downtime as you won't have to waste your time to manage each of your mining instances. Alternatively, remove it from your mining client so it doesn't throw an error.

I guess this is where I am confused at, if I want to solo mine against bitcoind, which is the way to go if I don't want to downgrade to 0.19.1? Add it back into the bitcoind code and recompile (not sure exactly how) --or-- remove code from miner client and recompile?

Thanks

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: tom22022 on April 03, 2021, 05:53:53 PM

When you say add it at the end, you mean compile it back in?

Yes. As GBT expects coinbase aux, you can just include it as an empty field again. It'll be easier to ensure minimal downtime as you won't have to waste your time to manage each of your mining instances. Alternatively, remove it from your mining client so it doesn't throw an error.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

Quote from: tom22022 on April 03, 2021, 05:53:53 PM

When you say add it at the end, you mean compile it back in?

The PR mentions that the coinbaseflags was unused anyway so it must have had some default value set. So you'd either patch the RPC call in Bitcoin Core to return this key and value in the JSON or you paste whatever default value Core was using into cgminer code without fetching it from an RPC call.

If you ask me I'd say it's a lot easier to just fill in the missing value in cgminer.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: ranochigo on March 28, 2021, 06:28:23 PM

Quote from: tom22022 on March 28, 2021, 12:47:48 PM

Thanks. My linux bitcoind server is almost done syncing, will look into 0.19.1 when it finishes. My question though would be how do mining pools do this?

Most of them probably don't rely completely on Bitcoin Core and builds the block using their own implementation. Most mining pool don't use GBT but stratum instead.

Unless you're able to see how the pool functions, you probably won't know if the problem exists for them and what their remedy is. Since the error exists due to the fact that they expect the coinbase flags, they can just add it at the end before feeding it to the miners.

When you say add it at the end, you mean compile it back in?

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: tom22022 on March 28, 2021, 12:47:48 PM

Thanks. My linux bitcoind server is almost done syncing, will look into 0.19.1 when it finishes. My question though would be how do mining pools do this?

Most of them probably don't rely completely on Bitcoin Core and builds the block using their own implementation. Most mining pool don't use GBT but stratum instead.

Unless you're able to see how the pool functions, you probably won't know if the problem exists for them and what their remedy is. Since the error exists due to the fact that they expect the coinbase flags, they can just add it at the end before feeding it to the miners.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: ranochigo on March 28, 2021, 11:13:06 AM

Quote from: tom22022 on March 28, 2021, 10:23:33 AM

Thanks for the information. I read through that link but cannot figure out if this is a bug or intentional?

Intentional, the mining client should modify their client to remove the flags as it isn't in use anyways. I think neither BFGminer nor CGminers are in active development anymore?

Quote from: tom22022 on March 28, 2021, 10:23:33 AM

Do you know if 0.19.1 can be successfully dropped into an existing .bitcoin profile as to not have to download the whole blockchain again?

There isn't any (only the peers.dat will be discarded, if I'm not wrong) compatibility issues, if that is what you mean.

Thanks. My linux bitcoind server is almost done syncing, will look into 0.19.1 when it finishes. My question though would be how do mining pools do this?

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: tom22022 on March 28, 2021, 10:23:33 AM

Thanks for the information. I read through that link but cannot figure out if this is a bug or intentional?

Intentional, the mining client should modify their client to remove the flags as it isn't in use anyways. I think neither BFGminer nor CGminers are in active development anymore?

Quote from: tom22022 on March 28, 2021, 10:23:33 AM

Do you know if 0.19.1 can be successfully dropped into an existing .bitcoin profile as to not have to download the whole blockchain again?

There isn't any (only the peers.dat will be discarded, if I'm not wrong) compatibility issues, if that is what you mean.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: ranochigo on March 28, 2021, 09:50:52 AM

Just a little insight: I was interested in using my spare unused 2Pacs for some solomining. It was working with 0.19.1 but it broke after updating to 0.20.0 and I was looking at the code and figured out that this pull request was causing the problem[1]. If you want to mine with 0.21.0/0.20.X instead, then you'll probably have to either modify your mining client somehow to ignore that missing part in the scriptsig and compile it.

SoloMining with 0.19.1 is perfectly fine, if you don't want to complicate things.

[1] https://github.com/bitcoin/bitcoin/pull/17519

Thanks for the information. I read through that link but cannot figure out if this is a bug or intentional?

Do you know if 0.19.1 can be successfully dropped into an existing .bitcoin profile as to not have to download the whole blockchain again?

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Just a little insight: I was interested in using my spare unused 2Pacs for some solomining. It was working with 0.19.1 but it broke after updating to 0.20.0 and I was looking at the code and figured out that this pull request was causing the problem[1]. If you want to mine with 0.21.0/0.20.X instead, then you'll probably have to either modify your mining client somehow to ignore that missing part in the scriptsig and compile it.

SoloMining with 0.19.1 is perfectly fine, if you don't want to complicate things.

[1] https://github.com/bitcoin/bitcoin/pull/17519

tom22022

jr. member

Activity: 33

Merit: 8

I think I made some progress here but still error messages. I was looking at the bitcoin-qt windows server debug.log and ran into the following entry-

Code:

WARNING: option -rpcallowip was specified without -rpcbind; this doesn't usually make sense

So I googled and ran across an alternate bitcoin.conf with the addition of rpcbind which I tried.

Code:

server=1
listen=1
daemon=1
rpcuser=tom11011
rpcpassword=openup11
rpcbind=127.0.0.1
rpcbind=192.168.0.198
rpcallowip=127.0.0.1
rpcallowip=192.168.0.0/24
rpcport=8332

From a remote linux computer I ran the following against my windows bitcoin-qt computer.

./cgminer -o http://192.168.0.198:8332 -u tom -p secret --btc-address XXXXX

I am definitely seeing something new now.

Code:

[1970-01-05 14:36:37.723] Started cgminer 4.11.1
[1970-01-05 14:36:37.723] Probing for an alive pool
[1970-01-05 14:36:39.523] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:39.523] Block change for http://192.168.0.198:8332 detection via getblockcount polling
[1970-01-05 14:36:39.531] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:40.328] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:40.335] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:41.059] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:41.067] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:42.042] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:42.049] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:42.755] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:42.762] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:43.451] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:43.459] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:44.179] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:44.186] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:45.013] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:45.052] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:45.554] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:45.905] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:45.912] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:46.636] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:46.652] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:47.370] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:47.378] Block height change to 676707 detected on pool 0
^C [1970-01-05 14:36:47.627] Shutdown signal received.

[1970-01-05 14:35:43.185] Started at
[1970-01-05 14:35:43.185] Pool: http://192.168.0.198:8332
[1970-01-05 14:35:43.185] Runtime: 110 hrs : 35 mins : 43 secs
[1970-01-05 14:35:43.186] Average hashrate: 0.0 Mhash/s
[1970-01-05 14:35:43.186] Solved blocks: 0
[1970-01-05 14:35:43.186] Best share difficulty: 0
[1970-01-05 14:35:43.186] Share submissions: 0
[1970-01-05 14:35:43.186] Accepted shares: 0
[1970-01-05 14:35:43.186] Rejected shares: 0
[1970-01-05 14:35:43.186] Accepted difficulty shares: 0
[1970-01-05 14:35:43.186] Rejected difficulty shares: 0
[1970-01-05 14:35:43.186] Hardware errors: 0
[1970-01-05 14:35:43.186] Utility (accepted shares / min): 0.00/min
[1970-01-05 14:35:43.186] Work Utility (diff1 shares solved / min): 0.00/min

[1970-01-05 14:35:43.186] Stale submissions discarded due to new blocks: 0
[1970-01-05 14:35:43.186] Unable to get work from server occasions: 0
[1970-01-05 14:35:43.186] Work items generated locally: 0
[1970-01-05 14:35:43.186] Submitting work remotely delay occasions: 0
[1970-01-05 14:35:43.186] New blocks detected on network: 0

[1970-01-05 14:35:43.186] Summary of per device statistics:

[1970-01-05 14:35:43.186]

not sure why the time date shows weird

root@server1:/usr/src/cgminer# date
Sun 28 Mar 2021 02:33:32 PM UTC

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 27, 2021, 07:58:59 PM

Quote from: tom22022 on March 27, 2021, 06:16:27 PM

When I try this against my bitcoin-qt install on windows which is fully sync'd, I get an error message
No Stratum, GBT or Solo support in pool 0 http://192.168.0.198:8332 unable to use

I'm not sure what's missing.

bitcoin-qt does not start the rpc server by default. You can enable that by adding server=1 to your bitcoin.conf.

It's in there, here's my conf on the windows server.

Code:

server=1
listen=1
daemon=1
rpcuser=tom
rpcpassword=secret
rpcallowip=192.168.0.0/24
rpcport=8332

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: tom22022 on March 27, 2021, 06:16:27 PM

When I try this against my bitcoin-qt install on windows which is fully sync'd, I get an error message
No Stratum, GBT or Solo support in pool 0 http://192.168.0.198:8332 unable to use

I'm not sure what's missing.

bitcoin-qt does not start the rpc server by default. You can enable that by adding server=1 to your bitcoin.conf.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 27, 2021, 05:27:46 PM

Quote from: tom22022 on March 27, 2021, 03:51:14 PM

Is bitcoind the same as bitcoin-qt?

Yes. They use the same code, just that bitcoin-qt has additional code to show a GUI.

Quote from: tom22022 on March 27, 2021, 03:51:14 PM

And is getblocktemplate accessed through http:// instead of stratum+tcp://

Yes. getblocktemplate uses JSON-RPC over http. Stratum is a custom protocol. bitcoind does not implement stratum.

When I try this against my bitcoin-qt install on windows which is fully sync'd, I get an error message
No Stratum, GBT or Solo support in pool 0 http://192.168.0.198:8332 unable to use

I'm not sure what's missing.

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: tom22022 on March 27, 2021, 03:51:14 PM

Is bitcoind the same as bitcoin-qt?

Yes. They use the same code, just that bitcoin-qt has additional code to show a GUI.

Quote from: tom22022 on March 27, 2021, 03:51:14 PM

And is getblocktemplate accessed through http:// instead of stratum+tcp://

Yes. getblocktemplate uses JSON-RPC over http. Stratum is a custom protocol. bitcoind does not implement stratum.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 27, 2021, 03:04:34 PM

Quote from: tom22022 on March 27, 2021, 01:47:53 PM

I guess it is not fully synced yet. I looked at debug.log at it is only about 3/4 of the way through at block 492500. Do you think it will work when fully synced? I tried this with a fully synced bitcoin-qt last week and it did not work.

bitcoind must be synced for getblocktemplate to become available.

Is bitcoind the same as bitcoin-qt? And is getblocktemplate accessed through http:// instead of stratum+tcp://

Thank you.

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: tom22022 on March 27, 2021, 01:47:53 PM

I guess it is not fully synced yet. I looked at debug.log at it is only about 3/4 of the way through at block 492500. Do you think it will work when fully synced? I tried this with a fully synced bitcoin-qt last week and it did not work.

bitcoind must be synced for getblocktemplate to become available.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 26, 2021, 08:41:25 PM

Is bitcoind synced?

I guess it is not fully synced yet. I looked at debug.log at it is only about 3/4 of the way through at block 492500. Do you think it will work when fully synced? I tried this with a fully synced bitcoin-qt last week and it did not work.

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Is bitcoind synced?

Topic: Mining against bitcoind stratum issues (Read 490 times)