Topic: [Guide] Solo mine testnet bitcoins with cgminer, Bitcoin Core, and a Compac F (Read 1896 times)

All getting a bit off-topic here...  

Perhaps best to export these S9 questions to a separate thread?

At least, i don't remember any change or update since it's created on 2012. Meanwhile Stratum V2 exist, although AFAIK the adaption is very slow.


There's possibility some mining pool may use Bitcoin Core's GBT to create block they'll mine. But Stratum doesn't know or care how it's created.


At a glance, your config seems to be right. Have you tried checking debug.log file created by Bitcoin Core? It may give some hints why public-pool unable connect to Bitcoin Core.

Hi ABCbits,

Sorry jumping on your thread updates here.

Is GBT outdated? I didn't think it was. Perhaps I've misunderstood something all this time but isn't Stratum just a bridge between Bitcoin core GBT and pool services?

Thanks

G.

You're welcome. I've spent days chasing typo's before it can get real frustrating.

Good you got it working.

Thank you sooooooooooo very much for this! I spent days trying and trying and couldn't find what I missed. It works now.

Try your local net address instead of what I think you have mistyped.

127.0.0.1 not 172.0.0.1

I bought a second hand red compac F miner, and followed your instructions. I need help to get it working.

Bitcoin Core, cgminer and the compac F are installed on the same system.

I use this:



I think I can skip this because the code on github is updated already:
.
I use this (last comma removed, thanks n0nce)

It doesn't work. I get:
A minute later:
2 minutes later:

Why can't cgminer connect to Bitcoin Core? What do I have to change?

The changes aren't that difficult to do if you're careful patching them into the correct locations. If I recall most of it is in cgminer.c and there's only a handfull of functions that are affected.

I'm no expert on driving GIT otherwise I'd create my own branch with the adaptations already done.

I don't know if Kano is interested in porting in the GBT mods from cmmodtools, they released these updates quite a long time ago and its never made it back into the master branch.

Nice, I wonder if kano would be OK bringing those changes upstream.

At least for me, @kano branch of cgminer is the one that I use whenever I build cgminer after CKolivas stopped development.

Hopefully there is only one branch with all the required changes.

A bash script to mine coins on derived addresses, to improve privacy:
Of course, instead of bitcoin-cli, the same trick can be used to mine on mining pools. However, note that many pools require producing at least N shares before paying anything, so make sure you produce enough, before you switch to some other address. But: as far as I understand it, that method of switching addresses seems to be good enough, right?

I also thought about mining on silent payment addresses, but I have to think about deriving it properly, because existing clients don't understand "sp1q" addresses.

I have something of my own creation a somewhat overclocked (725Mhz) BM1397 that averages 480GH/s, its using about 18Watts at the DC port including the power for the fan and the RaspberryPI Zero 2W thats controlling it.

Presently its pointing at CK pool, but I had it running on my local node for about a week without any issue.

Updating Kano's CGMiner is a bit of a chore but its achievable provided you follow the differences carefully.

Good luck!

Thanks for the reply, I am going to get a bitaxe supra I think and do this method. What is giving you 500G/H? How much power?

Goodluck!

I don't think the nodes connected to other nodes is any different to any other machine connecting to a server elsewhere on the net, it still has to traverse the same gateways, routers, switches and other network infrastructure.

Bitcoind looks like it handles all of the node to node connections automatically anyway. In that respect it looks no different from any other DNS lookup or TCP/IP communication.

Presently I just can't see anything to support the claim that mining solo to a pool is any better than mining directly to bitcoind beyond the fact that someone else has taken the time to set-up and maintain the pool and all the backend side of it reducing the initial technical hassles.

Once I have a couple more mining boards built, I'll point them at my bitcoind server and let them run indefinately, if by chance (and it is a remote chance) they hit a block I'll celebrate that here!

Cheers

G.

Aside from latency, i also hear claim that node run by mining pool connected to other node which belong to different mining pool and node run by major company. Anyway, i agree the biggest reason to use them is we don't have to setup and maintain stuff.

I know, alledgedly pools are supposed to have lower latency.

TBH I'm not so sure given I have a low latency GBit fibre connection to the internet I cant see why it'd be any less effective than a server in a data centre somewhere that only has a 400Mbit capped link tied to it. Unless the other bitcoin nodes are located in the same data centre, the latency between nodes would be similar.

Anyway, this was purely a fun experiment and technical exercise to see if I could get it to work.

I merged the changes from https://github.com/cmmodtools/cgminer into the last maintained version of cgminer 4.12.1 https://github.com/kanoi/cgminer/ and getting it to build, that was technical exercise 1.

Then the challenge was getting bitcoind running and accepting RPC connections for testnet, that worked ok then moving over to mainnet.

Once I'd figured that the named block in the config file was [main] not [MainNet] or [mainnet] it all started to work as it did for testnet, although it did take nearly 3 days to sync the blockchain.

Its been running overnight and statistically its performing similar to that of a pool looking at the volume of data exchanged and the current best share.

There's nothing presently that makes me think solo mining to a pool has any advantage, of course I dont know how things would perform should it hit a block and if the block would successfully propagate. Like I said it could be a very long time before I found out.


Never the less with that said, I guess it's easier to rely on someone else's technical expertise in setting up the bitcoind and maintaining the back-end of things.

Aside from low hashrate, you probably better use solo mining pool (such as ckpool or kano) which supposed to offer better block propagation speed.

I know I'm late to the party with this one but, for mainnet you have to change the [test] to [main] as well as removing testnet=1


Anyone's guess as to if it'll actually mine a valid block, but it does seem to be running.


With 500GH/s pointed at it, I guess I'll be waiting a while to find out.  

Actually, I got maybe 6 or 7 'regular difficulty' (67.108.864 - only 30% lower than what it should probably be) blocks in just a couple of days now. I have to check the numbers, but it seems very unlikely to be this lucky this often.

I believe so, too. Even a CompacF takes just a fraction of a second.
Actually, one reason that might explain how I'm able to get almost every single 1-difficulty block is that 'regular' miners connect to a pool, meanwhile with nullama's method, we connect to our own full nodes. This should surely result in a smaller delay.

I'll run some numbers as soon as I can; likelihood of mining a 67M block (normal distribution), as well as ratio of 1-diff blocks I was able to scoop up vs. the total number in a given period of time.

There's no point in trying, if it's once in a million (or maybe billion). That means it will probably never happen, and that's what makes CPU mining futile.

At difficulty 1, getting the data from your pool may already take more time than an ASIC miner needs to find the next block.

Well, your gains will be always proportional to your power, in the long-term perspective. Which means, CPU-based miners will lose that battle very, very often, but once per N blocks, they could win. Or rather, to make it realistic: a pool of CPU miners could win.

Also, for that reason I left mining testnet3, and tried merged-mining with mainnet difficulty, but in a model, where each miner has its own difficulty, and is always rewarded every 10 minutes. Because one of the problems with testnet3 is that so much power is "wasted" into mining worthless coins. In that case, merged-mining fits better, because you can get coins that are almost worthless, but not quite. And you can easily test that kind of things inside LN, where normally you would have to pay, but if you are a miner, you can use your computing power as your currency. Also, because normally nodes don't want to pay you, what you can focus on is cheaper transactions: a node can give you some discount, or process some transactions for free, if you show your shares.

It is simple: if you have any mining power, then mathematically, you can always hit a block. Your blocks are as good as produced by any huge miner. So, instead of hitting some block every 10 minutes, you can hit it every day, every week, or even every month. But it also means, once per year you can hit three blocks in a row, or something like that. But then, you should not fall into gambler's fallacy, and think that you will hit the next block in the same day, if you were extremely lucky for a while.

It is not something you can fix. Your calculations are correct. But the very low probability, close to zero, does not mean that something will not happen. Even if you reach zero in your calculations, you could still encounter such situation in the real world.

There we go! Another block found, using the R909 actually. I like how it can quickly be set up without worrying about external fans and fine tuning any power settings. ^{[Review coming... soonTM?]}

Sadly the block reward is quickly dwindling in Testnet already.. +0.044tBTC this time.

Edit: Another one, an hour later! +‎0.02996310 tBTC
Edit 2: I got the immediately following block, too, at full difficulty.

With my previous calculations, that is very unlikely. I have 1.7TH/s, so:

I'm basically expecting to need 2 days to mine a block at this difficulty, but did it in 3 minutes. Pretty neat.

Edit 3: I found another difficulty=1 block, as well as another at 67108864. How is that possible? I need to double-check my calculations...

Not at the current difficulty. People are mining blocks every 2 minutes; there is realistically no chance for a 20 min time period without blocks.

I know, that's what this whole thread is about and I've been scooping up tons of 1-difficulty blocks months ago, myself!

That is not true; LoyceV has shown it in an experiment and we've calculated it on paper, too. ASICs solve these blocks in a fraction of a second, meanwhile a (regular / not highest-end) CPU can need minutes or more; no advantage in internet speed will be enough to offset that.

That's incorrect.

Despite being worthless, people are still mining testnet. So I expect at least some of them to continue mining when there's no block reward left. That also means total fees can be under the dust limit. And even if they don't, with low difficulty just one miner is enough to continue the chain. We may see 1000 block 99% attacks once in a while.

Testnet coins already seem to be quite scarce, my guess is many are lost. After all, there's not much of a reason to be careful with worthless coins. That means testing in the future will have to happen with smaller and smaller amounts.

For that reason, it is very unlikely that testnet3 will ever be killed. Because now, it is the only network that is quite close to mainnet, and where you can test, what will happen when the basic block reward will be very small. It is also probably the first network, that will reach zero block reward, and then a lot of people will carefully observe, what will happen next.

By the way, testnet coins are worthless by definition. So, if nothing else will happen, then testnet3 will finally reach a state, where the basic block reward will truly represent the real value of those coins. In general, testnet3 could have zero supply, but some coins are needed as a spam protection.

Well, if you use other test networks, then you can easily get much more coins. But as far as I know, testnet3 is the only truly decentralized test network, that we currently have. Signet is centralized, because of signet challenge. Regtest is centralized, because you can create a blockstorm on CPU, so it cannot be used online. So, for better or worse, we are left with testnet3, because only in this network you can test everything, including mining. But of course, it could be improved, and some improvements were proposed, so we will see, how it will evolve.

You can always have some 20-minute block with the minimal difficulty. After 20 minutes, there are two options:
1. Some block with difficulty one will be accepted.
2. Some block with the real difficulty will be accepted.
And even if you cannot produce some strong block, there is always a chance to produce some weak block. It can be reorged or not, it depends if someone will try to attack, and if you will be well-connected or not. Because when it comes to weak blocks, everyone can produce them, by using CPU, GPU, FPGA, ASIC or whatever. So, in that case, the winner is the one who will spread it faster to the rest of the network, and who will not be reorged. In those cases, CPUs are more important than other hardware, because they are used to send blocks, so it is a competition between nodes, and not between miners.

Oh wow, that makes total sense! I never thought about that. Very interesting. It doesn't sound intended; more like a bug that should be fixed. On the other hand, I feel like nobody really cares too much about Testnet.

Edit: Since this happened before, I guess there will be no changes made!

I'm aware of this, yes. There were even discussions about it here on Bitcointalk a while back..

---

Unfortunately, for now this means we won't have 20-minute block times in a while, i.e. no solo-mined tBTC. As the base difficulty has already exceeded 16M a few days ago. I expect it to take about 2-3 more epochs to reach normal levels again.

It came from several blocks per second, see Interesting behaivor of TESTNET difficulty.

This caused it:

And this made it last:

---

---

Note that there's not much left to mine on testnet: the block reward is now 0.024, about 300 times lower than Bitcoin mainnet.

Today I noticed something weird on the Testnet blockchain. We are getting blocks about every 2 minutes; difficulty is way too low. According to https://mempool.space/testnet, the difficulty was already raised in the last difficulty adjustment and it will be raised again today by 300%.

Was there a huge dip of hashpower a few weeks or months ago that tanked the difficulty or did someone point an enormous amount of miners to Testnet recently?

When a stale block is mined, it is not included in the longest version of the blockchain, and is therefore not valid.

Basically this can be due to network problems, by the time you distribute your block and it is spread among enough nodes, it may have been overtaken by another block or become invalid in the meantime. That's why when you mine solo it's very important to have the lowest possible network latency (we often say less than 100ms with the solo pool to be ok)

Please note that even if you see a reward for a stale block, you will never be able to spend it because the coins will be invalid. If I'm not mistaken, the coins are returned to the mempool
Fortunately most of the pools have a low rate of orphan and stale blocks, another problem is that the big pools accept to mine empty blocks without transaction, which is anything but good for the BTC network...

so you have a block with only coinbase transaction. what can make it stalled after 5 seconds? and what means stalled? yeah for educational purpose it would be not bad to make it splitted from broadcast unit. It can be done like in openssl ( openssl.exe ) where you can make stand-alone utils instead

What transaction? You want to write a miner which doesn't automatically submit a new block to the blockchain, but requires you to do it manually? That would be hilarious. You would mine so many stale blocks. I mean, a correct block at difficulty 1 will be stale in a matter of seconds, if not less. My ASIC alone, generates around 100 valid blocks per second as soon as the difficulty drops.

Furthermore, while you can submit raw transactions with some wallets, you cannot submit blocks without running Bitcoin Core.

Thanks all for your kind help and support.
I tried cmmodtools and got the same errors, then I tried phaenomenon. Same errors. Then finally I tried vthoang and it compiled perfectly. I couldn't figure what was causing the error.

i think you'd overcomplicated it really. it could be 400 lines cpp single file( with 50 lines go to simd switch/case) where you broadcast tx later yourself with a wallet.

As mentioned earlier, you're not sharing here the full log of errors, so it's hard to help with what you posted only.

And based on what you did post, it looks like you're not following the instructions exactly, as it seems like you're using a different repository (cgminer_phaenomenon instead of cmmodtools).

So, first, try following the guide correctly(instead of doing it wrong multiple times), and then if that doesn't work for you, post detailed logs.

The part of the log output showing the actual issue is missing. You can post the whole log in a code block here or if you feel unsure if there is any PII in it, send it to me via DM.

Hi,

I cannot install the cgminer on my Ubuntu 20.04.5 LTS. I keep on getting this error message:

make[2]: *** [Makefile:999: cgminer-cgminer.o] Error 1
make[2]: Leaving directory '/home/user/bitcoin_testnet/cgminer_phaenomenon'
make[1]: *** [Makefile:1894: all-recursive] Error 1
make[1]: Leaving directory '/home/user/bitcoin_testnet/cgminer_phaenomenon'
make: *** [Makefile:808: all] Error 2

I have installed and updated all the libraries and tried to do the whole process over and over but I get the same end result.

Any thoughts on this issue?

As far as I could see, I never mined a block.

I assume that's not a problem, Linux multitasking should be able to handle this. And I used nice for the miner.

Thanks! I'll PM you the details (and let's switch back to the correct thread).

Usually in parallel programming (multi-thread) when one thread finds the answer it breaks out of it and sends a "signal" to other threads to break too. Which means after finding the new block at least part of CPU has to be freed to be used for the broadcasting process.

You found my messages! Nice! I've got to plant some more (on a new address) then..

Sure, let's do it. Sorry, only just revisited this thread again.

Stale has less to do with mining and more to do with propagation. If you have all the cores screaming running the SHA did you have enough CPU power left over to do the rest and get it out to the world. It's not a lot of computing power, but if you don't have any to spare it could cause issues. The other half of stale is networking. How many nodes is your node talking to? If there is a lag there and more nodes see the other block then you loose.

Also, it could be other miners deliberately causing reorgs. I don't see the point on testnet, but if I have enough power I can just ignore some blocks figuring I will get 2 or 3 while the rest of the network gets 1. That would just be a waste, but I can see people messing around to see if they can do it.

-Dave

It's a quad core Xeon E3-1270 V2 @ 3.50GHz. That's why I'm wondering if I did something wrong somewhere.

That's 50,000 times faster than my CPU mining! Would you be interested to try? I still have a synced testnet blockchain running, and could give you access. At low difficulty, I wonder if I made a mistake for not getting anything in days.

Glad to see people now solo mining testnet bitcoins with this guide 

Interesting; I would have expected a CPU / GPU setup to work fine, too then. At difficulty 1, you need a fast network latency in my opinion, but also a block candidate immediately. In case the ASIC is 100x faster, that could play a bigger role than what I anticipated.
Again; I got consecutive blocks a few times, so I guess that a couple hundred GH suffice for consistent blocks rewards.

Yes It is also displayed by mempool.space.
Oddly enough; someone just sent me ‎0.00017893 tBTC, which was no coinbase transaction. Typo? Never happened on mainnet to me, in all these years.

---

I mined:
Block 2411861, Block 2411862 and Block 2411863 in a row.

What needs to be changed for this to work on main net? Remove testnet=1 ?

Nullama, your guide worked great (besides the few points I raised)! Amazing; I hadn't expected to mine multiple blocks per day.
In fact, I managed to grab almost 30 blocks in 24h. Sitting at roughly 0.9tBTC right now.
Even consecutive blocks a few times... I must have very good network latency or something. Only a single Compac F hashing at just under 300GH/s.

It was an awesome experience to solo-mine again (and hit blocks!) and I'll definitely do it again if I need testnet coins and / or for validating a miner / miner driver software.

PS: Check last 24h of blocks for hidden message!

A few recommendations and fixes:

(1) Just add datadir=/home/user/bitcoin_testnet/data to the conf file, as well. Then you won't have to specify it through CLI anymore.

(2) There is a typo here. It should be --enable-gekko; otherwise it will compile just fine, but without Gekko support and you won't even notice.

(3) Cgminer complains about the comma at the end of the last line ("failover-only" : true,); it has to be removed to work.

(4) For anyone unsure how to do this:

I just posted a new guide that doesn't require an ASIC. It only uses the CPU and/or the GPU of a normal computer, so you should be able to test it.

It's a simple text argument, the same as the btc-address argument you have to give in the config file. Just add another line with that.

Just try it out, that's the whole point of the testnet  

That was the motivation for me to do this in the first place 

A couple of things you might want to do:

Keep the log of your first block:


And you can of course write something into the coinbase of the block you found, just add this option to the config file:

That's great to know! I wasn't aware of this rule. Will spin a Compac F up to mine some testnet BTC, then..
I've never hit a solo block, so this should be a pretty cool experience.

Thanks for writing this guide. Although people who currently don't have ASIC better use Signet or Regtest, unless they also want to try their luck with solo mining on Bitcoin mainnet.


Not at some blocks, but after 20 minutes since last block mined. That might be why you mine more block than @NotATether expected.

Nicely detailed guide!

The difficulty was in the millions:


But apparently testnet3 resets diff to 1 at some blocks, as those are the ones I hit:

A few blocks overnight? That means the testnet difficulty is extremely low. Exactly what was the testnet difficulty when you wrote this guide?

I'm sure this will help a bunch of people who are setting up testnet bitcoin mining farms.

If you need some testnet bitcoins, you can easily get them by solo mining. Some other guides I've found in the web are outdated, so I made this one to work with the latest software/hardware.

You'll need:

• Latest version of Bitcoin Core(v23.0 at the time of writing)
• Special build of cgminer that allows solo mining, available from https://github.com/cmmodtools/cgminer with a small update from this guide.
• Ideally an ASIC miner. You can use any miner that works with cgminer, even the USB miner from GekkoScience, the Compac F:
  

To make things simple, for this guide I am going to assume you have two computers, one running a Compac F miner and another one running Bitcoin core. It shouldn't matter if you're running both on the same computer or if you have a different miner.

Step 1: Install and run Bitcoin Core

We're going to use the folder ~/bitcoin_testnet to save everything so that it's easy to remove once you're done and to keep things simple. You can of course use a different path if you want.

Inside ~/bitcoin_testnet create a folder called data:


Download the Bitcoin Core for your platform into ~/bitcoin_testnet/ and extract it there.

Now let's prepare the configuration file. Create a text file named bitcoin_testnet.conf in ~/bitcoin_testnet/ and put these content in it:


The thing that you need to understand here is that you're setting up the Bitcoin node to run in testnet, and you're defining an RPC port(5000 in this case, can be anything), user and password, and whitelisting a specific IP to connect to your node(YOUR_MINER_IP). You'll need these details and your Bitcoin node IP later when connecting from cgminer.

You can now start running your Bitcoin node by doing the following(make sure to change user to your actual user:


If you want, you can keep this running in the background with screen:


You can then see what's happening with:


To detach the screen, simply press Ctrl-A and then Ctrl-D. You'll be back in the console, and the command will continue running in the background.

Step 2: Install and run cgminer

We're going to use an updated repo of cgminer, this one has the latest changes from kano so that we can use the Compac F, plus some fixes that allow solo mining. Of course you might want to change the options to match your environment or miner.


Now, in theory we should be ready, but we need one extra change. Open the file cgminer.c with your favorite text editor and change this line:


To this:


Basically we need to also consider the first symbol of the testnet address, which is either n or m. OK, now run make again:

And now cgminer is compiled and ready. Let's create the configuration file:

Create a text file named ~/bitcoin_testnet/cgminer_testnet.conf with these contents:


If you don't have a testnet btc address yet, note that Bitcoin Core now doesn't generate a wallet by default any more so you will need to create one. You can read about it in createwallet. Then you should be able to getnewaddress. Make sure to use the legacy format as that's what cgminer is expecting. You address should start with n or m.

Alternatively, you can simply use the GUI. To open the GUI, first stop the bitcoind process (Ctrl-C) and then run bitcoin-qt instead. There you'll be able to easily create wallets and generate addresses by clicking around.

And now you can simply run cgminer like this:


You can of course also run it in the background with screen, or run this at startup, etc. It will depend on your particular setup. If you're on a pi for example, you could create a bash shell executable (~/start_mining.sh) with that code and then simply add this to /etc/rc.local:


That's just one example, you can choose to run it however you prefer.

There it is, it should now say:


If you leave it overnight you should hit a few blocks with the Compac F USB miner.

Happy testnet solo mining!