Mining against bitcoind stratum issues

tom22022

jr. member

Activity: 33

Merit: 8

Everyone on this post was so helpful I wanted to reach back out and let you know my solution which is subject to change by any other good advice I receive here.

I went out and bought a raspberry pi 4 with 8gigs and then bought the Argon One M.2 case that holds a SSD drive. I bought the 1TB drive from Samsung. Its a really neat tiny but powerful setup. Then, I bought myself a Newpac USB miner based on the BM1387 chip. Its running ubuntu.

I really had to do a lot of trial and error to get this all to work right. I really like BFGMiner, but it will not work with the new Newpacs. There was some posts about a year ago from @luke-jr about getting it supported but it doesn't have it yet. The great features of BFGMiner from my point of view are the ability to use it as a stratum proxy for other miners as well as its ability to solo mine against the bitcoind higher versions. But it just doesn't support the miner I use.

That's where cgminer comes in. Cgminer is the only miner that supports the Newpac based on BM1387. But, cgminer will not work for GBT solo mining against modern versions of bidcoind greater than 0.19.

So what I ended up doing is running my bfgminer as a stratum proxy and then run my cgminer with stratum+tcp and point it to the bfgminer. Bingo works!

So that's my ugly setup!

I hope luke-jr reads this and decides to spend some time with the newpac drivers.

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: tom22022 on April 04, 2021, 12:02:54 PM

I guess this is where I am confused at, if I want to solo mine against bitcoind, which is the way to go if I don't want to downgrade to 0.19.1? Add it back into the bitcoind code and recompile (not sure exactly how) --or-- remove code from miner client and recompile?

Thanks

I wouldn't want to mess with Bitcoin codes if that is the case though I'm sure that they've only modified that part and didn't cause any issues otherwise.

I've found this awhile back: https://bitcointalksearch.org/topic/m.56584362. I didn't test his code but it makes the field optional. My solution for my own fork was to just remove it completely but the modification as referenced would work fine as well.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: ranochigo on April 04, 2021, 08:52:17 AM

Quote from: tom22022 on April 03, 2021, 05:53:53 PM

When you say add it at the end, you mean compile it back in?

Yes. As GBT expects coinbase aux, you can just include it as an empty field again. It'll be easier to ensure minimal downtime as you won't have to waste your time to manage each of your mining instances. Alternatively, remove it from your mining client so it doesn't throw an error.

I guess this is where I am confused at, if I want to solo mine against bitcoind, which is the way to go if I don't want to downgrade to 0.19.1? Add it back into the bitcoind code and recompile (not sure exactly how) --or-- remove code from miner client and recompile?

Thanks

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: tom22022 on April 03, 2021, 05:53:53 PM

When you say add it at the end, you mean compile it back in?

Yes. As GBT expects coinbase aux, you can just include it as an empty field again. It'll be easier to ensure minimal downtime as you won't have to waste your time to manage each of your mining instances. Alternatively, remove it from your mining client so it doesn't throw an error.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

Quote from: tom22022 on April 03, 2021, 05:53:53 PM

When you say add it at the end, you mean compile it back in?

The PR mentions that the coinbaseflags was unused anyway so it must have had some default value set. So you'd either patch the RPC call in Bitcoin Core to return this key and value in the JSON or you paste whatever default value Core was using into cgminer code without fetching it from an RPC call.

If you ask me I'd say it's a lot easier to just fill in the missing value in cgminer.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: ranochigo on March 28, 2021, 06:28:23 PM

Quote from: tom22022 on March 28, 2021, 12:47:48 PM

Thanks. My linux bitcoind server is almost done syncing, will look into 0.19.1 when it finishes. My question though would be how do mining pools do this?

Most of them probably don't rely completely on Bitcoin Core and builds the block using their own implementation. Most mining pool don't use GBT but stratum instead.

Unless you're able to see how the pool functions, you probably won't know if the problem exists for them and what their remedy is. Since the error exists due to the fact that they expect the coinbase flags, they can just add it at the end before feeding it to the miners.

When you say add it at the end, you mean compile it back in?

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: tom22022 on March 28, 2021, 12:47:48 PM

Thanks. My linux bitcoind server is almost done syncing, will look into 0.19.1 when it finishes. My question though would be how do mining pools do this?

Most of them probably don't rely completely on Bitcoin Core and builds the block using their own implementation. Most mining pool don't use GBT but stratum instead.

Unless you're able to see how the pool functions, you probably won't know if the problem exists for them and what their remedy is. Since the error exists due to the fact that they expect the coinbase flags, they can just add it at the end before feeding it to the miners.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: ranochigo on March 28, 2021, 11:13:06 AM

Quote from: tom22022 on March 28, 2021, 10:23:33 AM

Thanks for the information. I read through that link but cannot figure out if this is a bug or intentional?

Intentional, the mining client should modify their client to remove the flags as it isn't in use anyways. I think neither BFGminer nor CGminers are in active development anymore?

Quote from: tom22022 on March 28, 2021, 10:23:33 AM

Do you know if 0.19.1 can be successfully dropped into an existing .bitcoin profile as to not have to download the whole blockchain again?

There isn't any (only the peers.dat will be discarded, if I'm not wrong) compatibility issues, if that is what you mean.

Thanks. My linux bitcoind server is almost done syncing, will look into 0.19.1 when it finishes. My question though would be how do mining pools do this?

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Quote from: tom22022 on March 28, 2021, 10:23:33 AM

Thanks for the information. I read through that link but cannot figure out if this is a bug or intentional?

Intentional, the mining client should modify their client to remove the flags as it isn't in use anyways. I think neither BFGminer nor CGminers are in active development anymore?

Quote from: tom22022 on March 28, 2021, 10:23:33 AM

Do you know if 0.19.1 can be successfully dropped into an existing .bitcoin profile as to not have to download the whole blockchain again?

There isn't any (only the peers.dat will be discarded, if I'm not wrong) compatibility issues, if that is what you mean.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: ranochigo on March 28, 2021, 09:50:52 AM

Just a little insight: I was interested in using my spare unused 2Pacs for some solomining. It was working with 0.19.1 but it broke after updating to 0.20.0 and I was looking at the code and figured out that this pull request was causing the problem[1]. If you want to mine with 0.21.0/0.20.X instead, then you'll probably have to either modify your mining client somehow to ignore that missing part in the scriptsig and compile it.

SoloMining with 0.19.1 is perfectly fine, if you don't want to complicate things.

[1] https://github.com/bitcoin/bitcoin/pull/17519

Thanks for the information. I read through that link but cannot figure out if this is a bug or intentional?

Do you know if 0.19.1 can be successfully dropped into an existing .bitcoin profile as to not have to download the whole blockchain again?

ranochigo

legendary

Activity: 3038

Merit: 4418

Crypto Swap Exchange

Just a little insight: I was interested in using my spare unused 2Pacs for some solomining. It was working with 0.19.1 but it broke after updating to 0.20.0 and I was looking at the code and figured out that this pull request was causing the problem[1]. If you want to mine with 0.21.0/0.20.X instead, then you'll probably have to either modify your mining client somehow to ignore that missing part in the scriptsig and compile it.

SoloMining with 0.19.1 is perfectly fine, if you don't want to complicate things.

[1] https://github.com/bitcoin/bitcoin/pull/17519

tom22022

jr. member

Activity: 33

Merit: 8

I think I made some progress here but still error messages. I was looking at the bitcoin-qt windows server debug.log and ran into the following entry-

Code:

WARNING: option -rpcallowip was specified without -rpcbind; this doesn't usually make sense

So I googled and ran across an alternate bitcoin.conf with the addition of rpcbind which I tried.

Code:

server=1
listen=1
daemon=1
rpcuser=tom11011
rpcpassword=openup11
rpcbind=127.0.0.1
rpcbind=192.168.0.198
rpcallowip=127.0.0.1
rpcallowip=192.168.0.0/24
rpcport=8332

From a remote linux computer I ran the following against my windows bitcoin-qt computer.

./cgminer -o http://192.168.0.198:8332 -u tom -p secret --btc-address XXXXX

I am definitely seeing something new now.

Code:

[1970-01-05 14:36:37.723] Started cgminer 4.11.1
[1970-01-05 14:36:37.723] Probing for an alive pool
[1970-01-05 14:36:39.523] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:39.523] Block change for http://192.168.0.198:8332 detection via getblockcount polling
[1970-01-05 14:36:39.531] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:40.328] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:40.335] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:41.059] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:41.067] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:42.042] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:42.049] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:42.755] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:42.762] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:43.451] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:43.459] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:44.179] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:44.186] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:45.013] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:45.052] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:45.554] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:45.905] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:45.912] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:46.636] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:46.652] Block height change to 676707 detected on pool 0
[1970-01-05 14:36:47.370] Pool 0 JSON failed to decode GBT
[1970-01-05 14:36:47.378] Block height change to 676707 detected on pool 0
^C [1970-01-05 14:36:47.627] Shutdown signal received.

[1970-01-05 14:35:43.185] Started at
[1970-01-05 14:35:43.185] Pool: http://192.168.0.198:8332
[1970-01-05 14:35:43.185] Runtime: 110 hrs : 35 mins : 43 secs
[1970-01-05 14:35:43.186] Average hashrate: 0.0 Mhash/s
[1970-01-05 14:35:43.186] Solved blocks: 0
[1970-01-05 14:35:43.186] Best share difficulty: 0
[1970-01-05 14:35:43.186] Share submissions: 0
[1970-01-05 14:35:43.186] Accepted shares: 0
[1970-01-05 14:35:43.186] Rejected shares: 0
[1970-01-05 14:35:43.186] Accepted difficulty shares: 0
[1970-01-05 14:35:43.186] Rejected difficulty shares: 0
[1970-01-05 14:35:43.186] Hardware errors: 0
[1970-01-05 14:35:43.186] Utility (accepted shares / min): 0.00/min
[1970-01-05 14:35:43.186] Work Utility (diff1 shares solved / min): 0.00/min

[1970-01-05 14:35:43.186] Stale submissions discarded due to new blocks: 0
[1970-01-05 14:35:43.186] Unable to get work from server occasions: 0
[1970-01-05 14:35:43.186] Work items generated locally: 0
[1970-01-05 14:35:43.186] Submitting work remotely delay occasions: 0
[1970-01-05 14:35:43.186] New blocks detected on network: 0

[1970-01-05 14:35:43.186] Summary of per device statistics:

[1970-01-05 14:35:43.186]

not sure why the time date shows weird

root@server1:/usr/src/cgminer# date
Sun 28 Mar 2021 02:33:32 PM UTC

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 27, 2021, 07:58:59 PM

Quote from: tom22022 on March 27, 2021, 06:16:27 PM

When I try this against my bitcoin-qt install on windows which is fully sync'd, I get an error message
No Stratum, GBT or Solo support in pool 0 http://192.168.0.198:8332 unable to use

I'm not sure what's missing.

bitcoin-qt does not start the rpc server by default. You can enable that by adding server=1 to your bitcoin.conf.

It's in there, here's my conf on the windows server.

Code:

server=1
listen=1
daemon=1
rpcuser=tom
rpcpassword=secret
rpcallowip=192.168.0.0/24
rpcport=8332

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: tom22022 on March 27, 2021, 06:16:27 PM

When I try this against my bitcoin-qt install on windows which is fully sync'd, I get an error message
No Stratum, GBT or Solo support in pool 0 http://192.168.0.198:8332 unable to use

I'm not sure what's missing.

bitcoin-qt does not start the rpc server by default. You can enable that by adding server=1 to your bitcoin.conf.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 27, 2021, 05:27:46 PM

Quote from: tom22022 on March 27, 2021, 03:51:14 PM

Is bitcoind the same as bitcoin-qt?

Yes. They use the same code, just that bitcoin-qt has additional code to show a GUI.

Quote from: tom22022 on March 27, 2021, 03:51:14 PM

And is getblocktemplate accessed through http:// instead of stratum+tcp://

Yes. getblocktemplate uses JSON-RPC over http. Stratum is a custom protocol. bitcoind does not implement stratum.

When I try this against my bitcoin-qt install on windows which is fully sync'd, I get an error message
No Stratum, GBT or Solo support in pool 0 http://192.168.0.198:8332 unable to use

I'm not sure what's missing.

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: tom22022 on March 27, 2021, 03:51:14 PM

Is bitcoind the same as bitcoin-qt?

Yes. They use the same code, just that bitcoin-qt has additional code to show a GUI.

Quote from: tom22022 on March 27, 2021, 03:51:14 PM

And is getblocktemplate accessed through http:// instead of stratum+tcp://

Yes. getblocktemplate uses JSON-RPC over http. Stratum is a custom protocol. bitcoind does not implement stratum.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 27, 2021, 03:04:34 PM

Quote from: tom22022 on March 27, 2021, 01:47:53 PM

I guess it is not fully synced yet. I looked at debug.log at it is only about 3/4 of the way through at block 492500. Do you think it will work when fully synced? I tried this with a fully synced bitcoin-qt last week and it did not work.

bitcoind must be synced for getblocktemplate to become available.

Is bitcoind the same as bitcoin-qt? And is getblocktemplate accessed through http:// instead of stratum+tcp://

Thank you.

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Quote from: tom22022 on March 27, 2021, 01:47:53 PM

I guess it is not fully synced yet. I looked at debug.log at it is only about 3/4 of the way through at block 492500. Do you think it will work when fully synced? I tried this with a fully synced bitcoin-qt last week and it did not work.

bitcoind must be synced for getblocktemplate to become available.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 26, 2021, 08:41:25 PM

Is bitcoind synced?

I guess it is not fully synced yet. I looked at debug.log at it is only about 3/4 of the way through at block 492500. Do you think it will work when fully synced? I tried this with a fully synced bitcoin-qt last week and it did not work.

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Is bitcoind synced?

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 26, 2021, 07:48:37 PM

Don't specify --stratum-port. Do not specify that the mining software should be using the stratum protocol. If anything, try to see if you can specify that it should use getblocktemplate (frequently shortened as gbt).

The version of bitcoind doesn't really matter. Bitcoind never supported stratum so stop trying to use stratum with it.

On the bitcoind server, I ran it again without the --stratum-port
slightly different error
[2021-03-27 01:17:09] Pool 0 slow/down or URL or credentials invalid
[2021-03-27 01:17:09] No servers were found that could be used to get work from
.
[2021-03-27 01:17:09] Please check the details from the list below of the serve
rs you have input
[2021-03-27 01:17:09] Most likely you have input the wrong URL, forgotten to ad
d a port, or have not set up workers
[2021-03-27 01:17:09] Pool: 0 URL: http://127.0.0.1:8332 User: tom Pass
word: secret
[2021-03-27 01:17:09] Press any key to exit, or BFGMiner will try again in 15s

I am not sure how to specify gbt but ran across this on luke-jr github site-

Code:

SOLO MINING

BFGMiner supports solo mining with any GBT-compatible bitcoin node (such as
bitcoind). To use this mode, you need to specify the URL of your bitcoind node
using the usual pool options (--url, --userpass, etc), and the --generate-to
option to specify the Bitcoin address you wish to receive the block rewards
mined. When you run Bitcoin Core on the same computer as your miner, the pool
itself will be automatically configured for you (on the default goal). Please be
aware that solo mining via GBT is at this time only supported for Bitcoin.

IMPORTANT: If you are solo mining with more than one instance of BFGMiner (or
any other software) per payout address, you must also specify data using the
--coinbase-sig option to ensure each miner is working on unique work. Note
that this data will be publicly seen if your miner finds a block using any
GBT-enabled pool, even when not solo mining (such as failover).

If your bitcoin node does not support longpolling (for example, bitcoind 0.8.x),
you should consider setting up a failover pool to provide you with block
notifications. Note that solo mining does not use shares, so BFGMiner's adjusted
hashrate (third column) may suddenly drop to zero if a block you submit is
rejected; this does not indicate that it has stopped mining.

Example solo mining usage:

bfgminer -o http://localhost:8332 -u username -p password \
--generate-to 1QATWksNFGeUJCWBrN4g6hGM178Lovm7Wh \
--coinbase-sig "rig1: This is Joe's block!"

If you want to solo mine on multiple GBT-compatible Bitcoin blockchains, you can
specify --generate-to multiple times with a goal name prefix followed by a
colon. Note that at this time, the coinbase sig is always shared across all
goals/pools.

Example multi-blockchain solo mining usage:

bfgminer -o http://localhost:8332 -u username -p password \
--generate-to 1QATWksNFGeUJCWBrN4g6hGM178Lovm7Wh \
-o http://localhost:7221 -u user2 -p password --pool-goal mychain \
--generate-to mychain:1QATWksNFGeUJCWBrN4g6hGM178Lovm7Wh \
--coinbase-sig "rig1: This is Joe's block!"

I tried the --generate-to switch but still get the same errors regarding can't find the pool.

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

Don't specify --stratum-port. Do not specify that the mining software should be using the stratum protocol. If anything, try to see if you can specify that it should use getblocktemplate (frequently shortened as gbt).

The version of bitcoind doesn't really matter. Bitcoind never supported stratum so stop trying to use stratum with it.

tom22022

jr. member

Activity: 33

Merit: 8

Quote from: achow101 on March 26, 2021, 05:25:34 PM

bitcoind is not a stratum server, so the stratum protocol will not work with it. The only mining protocol that bitcoind supports is getblocktemplate. If you change the url from stratum+tcp to http, your mining software should automatically try getblocktemplate and that should work.

hmm didn't seem to work.

On the bitcoind server I changed

./bfgminer -o stratum+tcp://127.0.0.1:8332 --stratum-port=3333 -u tom -p secret

-to-

./bfgminer -o http://127.0.0.1:8332 --stratum-port=3333 -u tom -p secret

but received a similar error.

For cgminer, similar error as well.

./cgminer -o stratum+tcp://192.168.0.179:3333 -u tom -p secret --btc-address 1E1pz9GoUVEvEEoLHm4VDXEph13yZBbXGX
and directly
./cgminer -o http://192.168.0.179:8332 -u tom -p secret --btc-address 1E1pz9GoUVEvEEoLHm4VDXEph13yZBbXGX

Maybe its a version of bitcoind that is older than 0.21.0 that this works?

achow101

staff

Activity: 3458

Merit: 6793

Just writing some code

bitcoind is not a stratum server, so the stratum protocol will not work with it. The only mining protocol that bitcoind supports is getblocktemplate. If you change the url from stratum+tcp to http, your mining software should automatically try getblocktemplate and that should work.

BitMaxz

legendary

Activity: 3472

Merit: 3217

Playbet.io - Crypto Casino and Sportsbook

I don't know exactly how to mine solo in full node I tried that before but no success.

Can you try to run your own pool instead?
You can follow the guide from this thread below.
- https://bitcointalksearch.org/topic/dont-pay-ultimate-crypto-server-installer-script-setup-your-pool-in-minutes-4980079

You can also use this video below as your guide and reference

- https://www.youtube.com/watch?v=MKb9YKIAltE
- https://www.youtube.com/watch?v=qE0rhfJ1g2k

Once you setup add a coin like Bitcoin pointing to your own full node.

Or follow Xavatar to automate the setup from here https://www.xavatar.com/tuto-installing-crypto-mining-pool-yiimp-on-ubuntu/
Just use translator to read it.

tom22022

jr. member

Activity: 33

Merit: 8

Trying to be my own bitcoin mining pool for fun and lotto. I realize the odds and the need for asics.

I have installed bitcoind to ubuntu linux server and sync'd the block chain current. I then proceeded to create a wallet and an address.

Tried to mine with cgminer from another computer on the same lan but received the following error-

Code:

root@server1:/usr/src/cgminer# ./cgminer -o stratum+tcp://192.168.0.179:8332 -u tom -p secret --btc-address 1E1pz9GoUVEvEEoLHm4VDXEph13yZBbXGX
[1970-01-03 18:55:33.904] Started cgminer 4.11.1
[1970-01-03 18:55:33.904] Probing for an alive pool
[1970-01-03 18:56:33.914] No servers were found that could be used to get work from.
[1970-01-03 18:56:33.914] Please check the details from the list below of the servers you have input
[1970-01-03 18:56:33.914] Most likely you have input the wrong URL, forgotten to add a port, or have not set up workers
[1970-01-03 18:56:33.914] Pool: 0 URL: stratum+tcp://192.168.0.179:8332 User: tom Password: secret
[1970-01-03 18:56:33.914] No servers could be used! Exiting.

192.168.0.179 is the ip of the bitcoind server.
I start bitcoind with this ./bin/bitcoind -daemon
Here is what my bitcoin.conf looks like, I have verified it is being read with the debug.log file.

Code:

server=1
rpcuser=tom
rpcpassword=secret
rpcallowip=192.168.0.0/24
rpcport=8332

I started googling and there was very little information to be found other than the following-

1.) In the recent past, this kind of support was removed from bitcoind
2.) People made it work by installing bfgminer on the same server as bitcoind to act as a proxy.

I tried #2.

Code:

./bfgminer --stratum-port=3333

Pool setup failed

Added a little more

Code:

./bfgminer -o stratum+tcp://127.0.0.1:8332 --stratum-port=3333 -u tom -p secret

[2021-03-26 18:59:07] Most likely you have input the wrong URL, forgotten to ad
d a port, or have not set up workers
[2021-03-26 18:59:07] Pool: 0 URL: stratum+tcp://127.0.0.1:8332 User: tom Password: secret
[2021-03-26 18:59:07] Press any key to exit, or BFGMiner will try again in 15s.

[2021-03-26 18:59:23] No servers were found that could be used to get work from
.
[2021-03-26 18:59:23] Please check the details from the list below of the serve
rs you have input

Can anyone help me out to fix this? Or maybe I am asking the wrong question and the right question is how to setup my own mining pool? Or maybe there is a better blockchain software other than bitcoind?

My goal would be to host my own wallet and blockchain without relying on a third party. How do mining pools do it? I know someone here knows how to do this!

Thanks

Topic: Mining against bitcoind stratum issues (Read 492 times)