Topic: Incorporating the p2pool concept into Bitcoin - page 4. (Read 17439 times)

More overall hashrate is better, even if the providers are deluded (or "charitable"  ). This is the crux of the proof of work security model. The bigger the network hashrate, the higher expense there is to take control of it. Simple. So we shouldn't be forcing a technical impediment that prevents otherwise useful and economic mining equipment from being a part of the network.

Also, if the p2pool codebase is to be useful and open source, multiple IP's must be able to connect to a single p2pool node. These factors will allow some form of centralised pools to continue despite any protocol changes to enforce p2pool only mining. Current p2pool allows people the freedom to create publicly accessible nodes with their own rules about payouts and service fees. So that capability would have to be removed to satisfy your proposal. We'd be better off if we can convince people to mine with p2pool on the basis of how profitable and how easy it is to set up and use.

I really appreciate all of the responses here.

So to summarize what I'm learning: p2pool is a fairly sound system in concept, but its weaknesses are that (1) it doesn't currently have the visibility or level of appeal to gain the attention it might deserve, and (2) even if it gained more attention and adoption, it probably wouldn't be able to compete with well-run centralized pools.

Does that sound about right?

So, if Bitcoin's current proof-of-work reward system encourages centralized pools to emerge and flourish (which are a threat to the system), is there really nothing better we can do than wait and watch the system fall apart in time? Surely there has to be a better solution...

If p2pool faces (from what I'm hearing) an uphill battle to gain any substantial adoption in its current form, then what about the original question I asked:

Would it be technically feasible to incorporate the concept behind P2Pool into Bitcoin itself [a forced change by the protocol]?


Carlton mentioned:

To me, seeing some miners forced out as a consequence (due to difficulty or hardware limitations) seems like a price worth paying for the network's survival.

Maybe I'm missing something obvious, but wouldn't such a change ultimately solve both of the issues (adoption and being competitive with centralized pools)?

For the sake of argument, suppose ALL mining had to be done through this built-in decentralized pool (or else the official clients wouldn't accept the blocks)... would it be possible or advantageous for traditional centralized pools to even exist on top of that? Are there technical reasons (other than perhaps being a pain to implement in a non-breaking and safe way) that such a change just wouldn't work?

I agree with gmaxwell's statements, and I'll also add that in the existing Bitcoin system p2pool will never be strictly competitive even if it were cost-free to run because p2pool will always have a higher orphan rate vs. well-run centralized pools. We're fortunate that the difference in profit is currently small enough that it's lost in the noise of variance, but increased importance of transaction fees and a larger - especially uncapped - blocksize will change that for the worse in the medium to long term. Additionally in the short-term efforts that improve block relaying efficiency by transmitting transaction lists rather than full transactions, and even just dedicated connections between large pools, will also harm p2pool by making the orphan rates of large pools less than it.

But it's not all doom and gloom you know: sure p2pool and Bitcoin itself have some fundamental flaws, but the impact of those flaws won't be fully felt until some time in the future. For now we can learn a lot about crypto-currencies and when the flaws do become critical the next system will have a chance at fixing them. I don't think efforts to improve p2pool are going to have anywhere near the impact people think they will, but they aren't going to be wasted either.

There is an argument that the official client should do even less rather than more.

Adding p2pool functionality to the main client just adds more code that needs to be maintained.  Most people who download the client don't use it to mine, so it is extra code which doesn't add functionality.

Having p2pool as a (semi-)official piece of software would be better, with hosting on the official site.  It keeps the 2 projects separated and they can interface via RPC (or bitcoin protocol)

Integrating into the main client codebase (as basic CPU mining was/is) would have a major advantage over the current situation: the resource management in the p2pool code could be optimised to run more efficiently, and so make p2pool closer to being able to run on the less capable hardware that many people use for mining nodes (the raspberry pi and router class of computer). And the main client is already compiled for the range of hardware that p2pool typically runs on, so the argument about throwing the cross-platform runtime away would be pretty much void. But I wonder whether "closer" is all that would be achieved with such an effort at this point in time, I'm sure someone who knows the p2pool design and codebase would be better placed to judge, but my instinct is that these cheap machines would still be out of the question. And even if it did work, it would still be pretty marginal. The main client really struggles on the 512Mb RAM raspberry pi, although I've only tried that with 0.8.1, but it was painful enough to convince me not to try again.

Given a bit more time, this situation could change. As has already been mentioned, transaction fees are only going to increase as a proportion of the block reward in future. When 12.5 BTC blocks become a reality in 36+ months, possibly coinciding with a consistently higher transaction fees portion of the block reward due to network usage uptake, we could see a real change in miners attitudes towards their valuing of the fees as part of their mining efforts. Further centralising and/or corporate encroachment of the mining market could contribute to the same effect, and so priorities look set to shift when you consider the likely direction of change in these factors (although the reward halving is the only factor with any certainty). Given these assumptions and the timescale on which they could become realised, a $30 raspberry pi type device could be more than capable of running p2pool and bitcoind. Possibly even without a great deal of performance compromise in comparison to a server or desktop box.

The heart of the issue is that the design model for that current solution is functional and useful, but also has it's limits. If the hashing network were forced to switch to the current p2pool solution overnight, the share difficulty would be pushed up so high that those with low-end hashrates would feel tempted to switch off their rig. That's just inherent to the model, the dynamics make it less attractive to the lower hashrated miners, who just so happen to be the majority. So we're left with the unfortunate situation that the market is making a fairly well-informed choice right now. Sure, it makes some sense for those at the margins of breaking into the sharechain to keep using p2pool, but if you're getting shares less than half the time you're mining, you're making a more rational choice in picking a pool where low difficulty shares are accounted for. If luck goes against you in the (uncertain) window of opportunity you have left for your current mining setup, p2pool looks like a gamble, and so you choose the conservative and reliable option instead.


I don't know whether (in our present situation of a 15 GB blockchain) it might be an idea to consider increasing the length of the sharechain to achieve a lower share diffculty. The rationale being that people aren't as pushed for hard disk space as they are for hashing power.  The downloading of the present sharechain doesn't take so long in the present system, and when you consider that the sharechain size is a static number, it seems like this could change the overall balance in p2pools favour. Doubling or tripling the sharechain to establish a share difficulty of a half or a third it's present difficulty would be my suggestion. I realise this also increases the number of days that the sharechain window is spread over, so the cost of uncertainty in the face of longer term commitment might work against the idea. So maybe advertise a staged increase over time. I think the present scheme has amply demonstrated it's longevity (and it's potential market share) by how consistently p2pool has appeared in the mining pool charts. If more miners were encouraged to use it, the problem with spending 7 or 10 days to get to your hashrate-commensurate position in the sharechain might be more easy to sell than it once was. People's understanding of p2pool mining dynamics have only improved since p2pool first started, and so the argument for a longer sharechain may well be more persuasive now than when the whole concept was unfamiliar.

Thanks Mike! Glad to hear someone on the core team is paying attention and concerned.


I'm willing to put in the effort - not sure how much I can contribute directly to p2pool development, but I'm becoming more and more interested in creating a community effort to draw attention to this and allocate resources intelligently. There is still time, but people need to take preemptive action, not wait until everything is on fire.

Got out of bed the wrong side this morning Gregory?

Yes, the situation with mining centralisation is pretty dire. P2Pool suffers badly from a lack of professionalism, to the extent that p2pool.org actually is now being squatted by something that isn't actually p2pool.

But it's not all bad! p2pool has proven that the underlying protocols and concepts can work. Decentralisation is always a lot harder than centralisation so we should not be surprised that things tend to centralise over time without big efforts.

This is a specific case of a more general problem, which is that the community lacks any systems for crowdfunding of decentralised infrastructure/the commons, beyond the Foundation, which at the moment is not primarily doing technical investment. So the decentralised infrastructures end up being immature and generally offputting (e.g. a website that is just a wiki), and that puts people off because they just bought expensive equipment from a professional ASIC producer and want to feel like the whole toolchain is equally commercial.

If you want to work on fixing this, IMHO the right place to start is by improving the bitcoin.org website. Create a section for miners and put best practices and advice there, make videos showing how to configure p2pool, heck, as p2pool screwed up their own website why not give it a section of bitcoin.org and work with Forrest to make that the official website? There are no other competing decentralised pools after all.

Integrating it with the Bitcoin Core software is probably less important than just making it generally easier to set up and run. I doubt the cost of a full node is all that big a deal compared to the cost of the ASIC miners themselves, lots of people run them on a cheap VPS.

Anyway, there's lots of low hanging fruit here to push p2pool forward. But someone is going to have to make it their primary project, for a long time. When I decided I wanted to build a decentralised mobile/Android wallet, it took about 2 years until it was competitive with more centralised solutions. So these can be big commitments.

P2pool simply uses to much resources.  If you use a pool, you can run mining software on a raspberry pi or a basic router.

Perhaps the two ideas could be used in combination (a protocol change along with a fund temporarily subsidizing miners or pools that adopt/support the change). I'm personally most interested in a technical solution to the problem - hence my posting to this board.

My only issue with a fund alone is that it seems to leave loopholes for people to game the system (e.g. pools staying "small" but secretly colluding). For that reason, I believe the solution needs to be incorporated into the software/protocol itself.

I've mentioned this in a separate thread but again it was in General Discussion so people didn't take it too seriously.

I have a theory that a Decentralization Fund run entirely from community donations could use its resources cleverly enough to create incentives for miners to switch away from large pools. It could be possible to subsidize small mining pool operators in such a way that mining pool operators are encouraged to cap their hashrate at 20-25% for fear of mass defection to a subsidized pool.

There are many ways to potentially structure such an incentive system but I believe it would not be as expensive as it might appear at first glance (a hard upper bound is around 200,000 BTC, but in reality it could be several orders of magnitude less). Even preparing bounties for pool operators to implement obvious incentives like merged mining and low fees is a fantastic start. Or, make bounties to improve p2pool.

A big part of the problem is simply that GHash and BTCGuild are more convenient and more miner-friendly than p2pool.

It would be almost trivial to implement— just bundle it, if nothing else. But I suspect the ship has already sailed for this.

We now have miners with hundreds of thousands of dollars of equipment which run it off a raspberry pi. Who send their coins directly to coinbase to be sold. Who have never used a Bitcoin client of any kind (except for the coinbase webwallet), certainly not a full node, and they have no concept of why they'd want to.  The name they trust most in mining is operator of their chosen pool— who could be robbing them blind, but maybe isn't— who has a financial interest to the tune of— say— >$700,000/month in keeping miners on their pool, and who tells them they don't need to worry about things, and who is believed because far too many people— including you— overly fixate on "51%" and ignore the fact that someone who controls 25% hashpower can reorg 6 confirms with 5% success or 2 with 31% success.

Never mind that the question of parties with large hashpower using it to harm other Bitcoin users is not hypothetical, as its recently been done— achieved huge profits in the process— and seems to have resulted in no negative consequences for the involved pool, just a lot of victim blaming, and some months delayed promises that the responsible parties have been sacked.

Perhaps many miners could be moved to running something p2pool like if doing so was easy, but just running a Bitcoin node is no longer so easy that it can be treated as costless, with now gigabytes of space wasted by pointless dust-scale messaging transactions. Transactions that the Bitcoin users didn't care about because they weren't running nodes and because many people had a monetary interest in being able to wastefully use the systems resources in that manner.

In any case, I don't think the problems you're facing are technical. The problem is that participants in the system don't know or care. I think the problem is also that to some extent people who should know better are not paying attention to the mining ecosystem and don't realize what a mess things are, and some who do are tempering their statements because saying "Hey everyone, the Bitcoin security assumptions are basically invalid in the current environment" too loudly may be adverse to the value of their holdings.

If you can figure how to educate people on the subject in a world where people have multimillion dollar a year income streams that depend on hashers not being educated and while other people own hundreds of millions of dollars of Bitcoin whos value might be eroded if the concerns become too wide spread— then I think progress could be made. Otherwise?  ::shrugs::

[I don't even intend to suggest intentional misconduct due to the monetary interest, only: "It is difficult to get a man to understand something when his salary depends upon his not understanding it."]

In another thread (https://bitcointalksearch.org/topic/how-are-large-mining-pools-not-a-threat-393815), we've been discussing the so-called "51% attack." The point I've made there is that regardless of your personal opinions about the profitability or motivation for such an attack, the vulnerability still exists and is particularly worrisome in the context of large, consolidating mining pools.

The original Bitcoin paper mentions the following:


My interpretation of the proof-of-work concept (reading between the lines) is that it is intended to distribute power among those who legitimately control and benefit from that hashing power. The problem with most mining pools in practice is that the pool operators can ultimately control the hashing power of pool members (obviously they don't have direct control of the the members' hardware, but with respect to the members' contribution to the blockchain, they do in fact have 100% control).

Putting it another way, I find it much, much less likely that the majority of the network's hashing power would be "dishonest" (willing to collude and carry out an attack) if each of the nodes were truly controlled by their owners as opposed to mining within a pool. Mining pools obviously offer some benefits to their members (over mining individually), but in effect, they undermine the goal of spreading the network's power over a large number of independent disinterested contributors.


So here's my 2-part question:

1. Would it be technically feasible to incorporate the concept behind P2Pool into Bitcoin itself? Obviously, it would require changes to Bitcoin clients and to the Bitcoin protocol. In effect, every mined block would be required to come from the decentralized "pool" and every contributor of hashing power would be rewarded - directly - for their share of contribution.

2. Do you think such a solution would help in reducing the chance of a 51% attack from occurring? Of course, I realize that due to the decentralized and democratic nature of Bitcoin, the possibility for a 51% attack would probably NEVER be fully eradicated, but perhaps by removing the utility of mining pools, the level of control currently held by pool operators would be redistributed to those who actually own/operate mining resources.


I would appreciate respectful and thoughtful responses the the questions above. Please, I'd ask you to refrain from making statements like "no rational pool is going to destroy/attack the network on which they rely to succeed". I'm not looking for arguments about why we "don't need a solution to the 51% problem". I'm interested in a solution that will help protect the democratic and decentralized objectives of Bitcoin regardless of whether you think these threats are rational, probable, or profitable.