Topic: Do mining pools directly connect one to another? ("Miner intranet") (Read 1101 times)

Thanks for sharing that; I do find it interesting. And disappointing (in regards to its discussion); Bitcoin's current community just can't help turning a perfectly viable and reasonable proposal into yet another political slugfest.

Still, I remain hopeful that one day, reason will prevail and the community manages to find common ground and move on to better and greater things for Bitcoin. This "debate" has become too much of a colossal waste of time, energy, and resources.

But yeah, we are taking this thread a little off the rails from its original topic.


Me too. It'd be interesting to see how Unlimited would manage to wrangle different pockets of the network into agreeing on a universally-accepted block size and mitigate, if not eliminate, the significant risk of frequent hard forks and chain reorganizations posed by Emergent Consensus. If Unlimited manages to develop and deploy a viable solution, I believe the blockchain ecosystem as a whole would greatly benefit from this.


I concur.

Its a bit off topic here, but I have advocated some time ago for a BIP-100 based solution (here) with a similar mechanism you describe (credits to Jeff Garzik, Upal and DooMAD). In this proposal every difficulty retargeting (or a slower timeframe) miners can vote to increase the maximum block size by a small percentage (5% were proposed)  but adding a upper "security limit" to prevent a situation where malicious miners (or other users) could spam the network contiuously to achieve a to big block size. Maybe you find it interesting.

I agree to your view about Bitcoin Unlimited's EC, I don't like it. I would like to see it as an altcoin though, simply to see if and how it works.


Good point, I agree.


I share that observation. I have one hope here: that pressure from altcoins (e.g. from Litecoin adopting Segwit and Lightning first, Ethereum with Raiden, there could also be an altcoin adopting Extension Blocks) could calm the debate a bit in front of a potentially more dangerous enemy than an alternative implementation (a "Flippening"), and unite the community again, at least to the degree that Segwit could be accepted - as a soft fork or an hard fork. Let's see what happens.

Me too. I'm also for flexible block sizes that scale accordingly to meet the demands of the network, but within reason, within reasonable technical limits, and based on a reference framework so that there would be less chaos in the network as a whole coming to a consensus on which block size to use. I'd propose adjusting the block size along similar mechanisms as currently employed by the network in adjusting the network difficulty — i.e., adjust the block size once every two weeks based on the demand from the last two weeks. I believe this to be a far safer solution that what Emergent Consensus proposes; Emergent Consensus has the potential to unleash chaos in that different pockets of the network may end up mining different chains because these different pockets came to different consensus regarding block sizes, all before the network as a whole managed to catch up and settle on a universally-accepted block size. Hard forks and chain reorganizations may run rampant in such a case.

Furthermore, I believe that changes to the block size should come after fixing the many bugs and issues found in the current system, especially that of transaction malleability and the network's current inefficient use of block space. Increasing the block size without fixing these bugs and without optimizing block space usage essentially means amplifying the presence and effects of those bugs, and increasing wasted block space in proportion to useful block space. (In contrast, efficiency improvements to block space usage reduces wasted block space in proportion to useful block space). The net total would essentially be a buggier but bigger block, and a bigger but more wasteful (in terms of storage space) block.

TL;DR: Fix bugs and optimize block space usage before making changes to block size limits.


Yup, classic game theory at work here.


Yeah, I haven't yet found any evidence to suggest that he's a large miner or a pool operator or both. Still, anything's possible. He seems to be a developer though, and may have contributed a fair bit to the development of Bitcoin. Nevertheless, even though I am for SegWit and would like to see it activated as soon as possible, I would not support his proposal. It does way more harm than good to the network, and I fully agree with what achow101 said in response to Carlton Banks' proposal:



You're right, FIBRE would pretty much render Banks' proposal useless, since the mining pools are essentially in a virtual private network of their own. And yes, if the pools decide to go Banks' route, then FIBRE would essentially become a mirror of what would happen on the main P2P network if his proposal goes mainstream.


Yup, and it would be a massive problem, since that is essentially an actual hard fork. The two different factions would be mining their respective versions of the blockchain and broadcasting it to the network. SegWit nodes reject the Unlimited blockchain, and Unlimited nodes reject the SegWit blockchain. A hard fork would have occurred at that point.


I ask myself that too. And the more I immerse myself in the Bitcoin community, the clearer and uglier it becomes to me: pride. The two factions have become so resolutely stubborn in their respective beliefs, that they have blinded themselves to the merits in each other's proposals (no matter how beneficial to the network), and more devastatingly, to the damage that they themselves are inflicting on the Bitcoin ecosystem — they refuse to take any responsibility for their own actions, and instead prefer to blame the other party for any damage that is inflicted on the Bitcoin ecosystem as a result of their actions. They have essentially, in my opinion, reduced themselves to comparing whose dicks are bigger (there are no women in the leading development teams). They are making pathetic fools out of themselves in front of the entire world, and the response has been damning, to say the least.


Likewise, d5000.

That would confirm my fears that an UASF in a situation of disagreement between miners and "economically important nodes" is, in reality, a kind of hard fork - only that the initiative to fork away is passed to the nodes not running the newest version of the client. If Segwit was a regular hard fork, in a situation of disagreement between economy and miner majority the clients running the newest Core version would have to initiate the fork, while in an UASF the initiative to fork (=being the first not strictly following the chain with most PoW) is given to the non-upgrading nodes which then can be blamed for the chain split.

(I may sound like a Big Blocker here, but actually I'm a Segwit supporter, but not at all costs.)

I conclude from your answers that nodes have actually some power to guide miners' (& pools') decisions and incentive them to not change the rules, but it is a bit limited because other pools also in most cases fill that role because they have similar incentives to invalidate "rule-breaking" behaviours like the economic nodes. But in situations of polarization between relatively small changes (like Segwit), "economically important nodes' opinion" should matter a bit at least, but it's an "informal" power, not a power granted by the protocol (as far as I understand).


Here I've found the proposal. I think the user who proposed it is not a big miner nor a pool operator, but I may be wrong.

Well, here I think "if it's possible, it will happen" and therefore it should be viewed as a legit possibility for nodes to support a softfork proposal and be included in the considerations regarding consensus. But effectively Fibre or any other possibilities for the pools to directly connect one to another would make ineffective these efforts for regular nodes.

Like you said, it's a bit different if a mining pool (or various pools) is using that kind of software - they could actually really do change the incentive structure.


The problem is if there is a large proportion of pools acting that way - e.g. all Segwit-supporting pools blocking the BU nodes, and all BU-supporting pools doing the opposite. From my limited understanding that could also possibly lead to a hard fork.

So actually, all forms of updating the rules without hard fork develop into hard forks if there is no supermajority agreement for the proposal. Then I ask myself, why not simply do the hard fork and give the big blocker side an incentive to agree (e.g. a compromise solution like Segwit2MB?) to get the supermajority?

PS: Thanks for the links! It's good to refresh basic knowledge about Bitcoin's consensus sometimes ...

Yup, that's one way to put it.


True, but only if the rule change is a hard fork — i.e., the original blockchain permanently splits into two. And only if the hard fork does not have the backing of the economic majority — i.e., the new blockchain would have little, if any, economic value. So yes, miners would therefore be incentivized to mine the blockchain backed by greater economic value (in this case, the original blockchain), since it would yield more significant revenue for them. And yes, for the new blockchain (the one without the backing of the economic majority) to survive, miners seeking to preserve it and subsequently orphan the original blockchain would need both "destructive intentions" and really deep pockets; they would need to generate an extraordinary amount of hashing power not just to mount a 50%+1 attack, but to preserve that 50%+1 advantage against miners who would be incentivized into mining the original blockchain.


You are correct. Unless the soft fork — whether user-activated or otherwise — is backed by the majority of hashing power, it would most likely end up being the shortest chain and eventually get orphaned by the network. Since both upgraded nodes and non-upgraded nodes see both soft-fork blocks and non-soft-fork blocks as valid — i.e., non-upgraded nodes will accept as valid all the same blocks as upgraded nodes do — both upgraded nodes and non-upgraded nodes will accept the chain with the most proof-of-work as the best valid blockchain. The soft-fork version of the blockchain would therefore be competing for acceptance by the network with the non-soft-fork version of the blockchain. Since, in this scenario, 70% — and therefore the majority — of hashing power is mining non-soft-fork blocks, the non-soft-fork version of the blockchain wins.

However, in this scenario, since the UASF was backed by 70% of the economy, the 30% of miners who stuck with the UASF version of the blockchain would arguably be incentivized to continue mining this chain, since robust economic backing means significant revenue for them. This "orphaned" blockchain would therefore still be alive and active. In this case, a hard fork has occurred — i.e., two separate active blockchains have permanently diverged from the original chain.


I'm currently unaware of such a proposal to essentially sabotage the efforts of non-SegWit miners. In any case, I wouldn't call it an "incentive" to get those miners to signal for SegWit; I would call it coercion. Such efforts are to be condemned, pro-SegWit and non-SegWit alike.

As far as I understand it, FIBRE is merely a backhaul between mining pools. It is up to the mining pools to propagate the blocks as they see fit; the FIBRE network itself does not force the pools to propagate every block at every opportunity as quickly as possible. FIBRE's utility and value depend on mining pools being incentivized to propagate newfound blocks as quickly as possible so that they can mine on top of them as soon as possible.

Therefore, FIBRE in and of itself does not prevent malicious miners from withholding newfound blocks from pools, or delaying propagation of blocks from pools, that do not support their (malicious miners) views.

Nevertheless, I do not foresee such a proposal going very far. If a pool — no matter how big it is — decides to coerce other pools into playing by its own rules, the other pools would most likely blacklist the malicious pool on both the FIBRE network and the main Bitcoin P2P network. The malicious pool would therefore end up shooting itself in both feet; it now would be unable to utilize the FIBRE network to obtain new blocks as quickly as possible, and it would also find itself with a more limited set of mainnet nodes to obtain blocks from. Not to mention the backlash it would receive from the mining community at large; I foresee a not-insignificant number of miners leaving the pool in protest, further crippling its mining operations.

Thank you for clarifying. So basically you say that full nodes enforce the "hard" consensus rules (like total coin supply).

I could counter this affirmation by saying that if these rules are changed then Bitcoin would lose value almost instantly so any mining pool that depends on revenue would mine the chain with the correct consensus rules to orphan the chain with the changed rules and the "malicious pool" would have no chance anyway. For such a radical change really getting 50%+1, the participating mining pools would need to have destructive intentions (e.g. "banksters" trying to destroy bitcoin, miners bribed by short sellers ...). So I consider that scenario extremely unlikely.

From the individual node's point of view, I agree that a full node is a protection from many attacks, so it's always better to run one - that's actually a very strong argument against "too big blocks".

The UASF scenario is interesting:


Let's observe in detail that case of a disagreement between miners and "economically important" users. First the users would activate the soft fork and most miners ignore it. Let's say after that deadline 70% of the economic nodes and 30% of the miners run the UASF-compatible client, while 70% of miners and 30% of economic nodes would run the non-UASF client.

So the "UASF blocks" would have to be mined by the 30% UASF-accepting miners. But couldn't the 70% non-UASF-supporting miners simply orphan every block or couple of blocks mined by these 30% that support the UASF rules, using their large hashrate majority? OK, here we're going into the details of UASF, so if you want, you can point me to a thread or post where this was discussed.

For me, another scenario was of interest: There was an idea to create a Core-based client that could choose to delay or even refuse the propagation of blocks coming from miners that have a header with certain characteristics (e.g. that don't signal for a soft fork proposal like Segwit) to give them incentives to signal for the softfork. In this case, I think the Fibre network would practically invalidate that idea because no pool would depend on a fast propagation via the P2P network if all other pools are connected to Fibre.

Hmm, I do see your point; point taken.

Then again, if 50%+1 of Bitcoin's total hashing power is redirected to an alternative blockchain that is not backed by the majority of economic full nodes, this chain would arguably be essentially worthless in economic value. Miners would therefore arguably be wasting their resources mining a blockchain that does not yield much, if any, useful value to them; they would essentially be mining at a significant financial loss. Unless these miners can find a way to bring the economic majority onto their blockchain, it would be financially unsustainable for them to remain on this blockchain, given the significant costs that mining resources require. I would argue, then, that these economic full nodes do wield significant power in incentivizing miners to preserve the blockchain according to their (economic majority's) consensus rules.

Furthermore, on a microscopic level, there is also a case to be made for full nodes protecting the interests of the end users who rely on them for their economic activity. These users are assured that their bitcoins are locked in the blockchain of their choice, protecting them from being dragged down an alternative blockchain against their will (which is what would happen if these end users were to, for example, rely on SPV wallets instead, which blindly follow the blockchain with the most proof-of-work, regardless of whether this blockchain follows Bitcoin's current consensus rules). Again, if the majority of end users were to rely on full nodes in this way (i.e., for most, if not all, of their economic activity), miners would therefore most likely be incentivized to play by the rules as enforced by this majority. End users of full nodes can therefore be assured that their bitcoins are safe, and that any and all transactions they make and receive are valid according to the consensus rules of their choice.


I'm currently unaware of reliable research that has been carried out on this issue. However, as far as my limited understanding of this scenario goes, nothing much will change if 50%+1, or more, of Bitcoin's total hashing power refuse to follow a user-activated soft fork (UASF). Firstly, such a soft fork is "user-activated," so miners do not have any say in whether or not such a fork be activated. Secondly, it is a soft fork, not a hard fork. Therefore, non-UASF-conforming miners would still be able to follow this blockchain and mine valid blocks on this blockchain, but just not blocks that follow the UASF's revised rules. Full nodes that receive these miners' non-UASF-conforming blocks would still see these blocks as valid and propagate them accordingly.

Thank you - so that's what they mean when they talk about "Fibre". Yes, that was what I searched for - if it's totally private or public is not so important, the crucial fact would be that they're directly connected.


As far as I understand, that would only be the case if most pools do not collude - so a competing pool can mine an alternative chain that the economic nodes (exchanges, big holders etc.) reject and that one then becomes the main chain. But in this case the economic nodes' intervention is not strictly necessary, because if the other pools don't agree to the consensus change, then they would mine the alternative chain anyway.

The problem is if pools with >51% of the hashrate collude on a change that's controversial but not universally rejected by economic nodes, e.g. if they reject a planned UASF. But I must look further into that scenario. If this case has been studied, I'll be grateful for any links.


I have no problem if it's moved to that subforum. So if a moderator sees this (s)he's free to move it.

I'm unaware of private "mining pool intranets." That said, it wouldn't be too far-fetched to assume that one, or several, already exist(s) among the major mining pools. However, this is merely speculation on my part; I don't have any evidence to back up this claim.

There is the Relay Network though. As far as I am able to understand it, it's a separate backhaul for miners and/or mining pools to shuttle blocks among each other as fast as possible, avoiding potential bottlenecks on the main Bitcoin P2P network. It's open to all miners.


I wouldn't go so far as to say that non-mining full nodes' validation work is "basically useless." Full nodes, whether connected to miners or not, still play a massively important role in ensuring that miners' newly created blocks follow Bitcoin's consensus rules, therefore ensuring the integrity of Bitcoin's blockchain.

P.S.: This topic may be more suited to the "Pools" sub-forum. Just a suggestion .

Does someone know if it's common practice that mining pools connect directly from one to another to get new blocks faster?

This question came out in another thread about the scaling debate. Some think that there is a kind of "miner intranet" where mining pools connect directly to the nodes of other known pools to get new blocks faster and get and advantage in finding the next block.

This behaviour could be a bit problematic because then "non-mining users" would have much less importance for the network because they are never consulted for blocks, only perhaps for new transactions. So their validation work is basically useless (unless there is a catastrophic event where they could serve as a backup).

As I suppose here in this subforum are the mining experts watching, I would be grateful to know if a study about that behaviour was carried out or if there are signs or even evidence for that.

If this topic has been discussed in other threads feel free to point me to it and I'll close this one.