Topic: Incorporating the p2pool concept into Bitcoin - page 2. (Read 17439 times)

I don't think you understood what I was talking about wrt hardware changes. I wasn't suggesting a change to the controllers at all.

I understand what you're saying. If we're just talking about variance, then yes, I agree variance is a problem, especially if it forces a large percentage of miners to discontinue mining. But when it comes to controller hardware/software considerations, I don't think the variance argument really applies, because if tomorrow the protocol/software necessitated a slight upgrade in controller hardware in order to keep current mining hardware functional, I believe most would upgrade very quickly (if the choice is between turning your mining investment into a paperweight and spending a few bucks to upgrade the controller hardware, it's not a tough decision). The cost of controller hardware is and likely always will be a tiny fraction of the total cost of a competitive mining rig. For that reason, I still believe that the solution should not necessarily cater to any specific hardware configuration.

I envision p2pool being hierarchical for it to succeed: sharechain > sub-pools > nodes > miners
randomly connecting to nodes/sub-pools solves the high variance problem of bitcoin. Miners are no longer incentivized to join a pool/node with the most hashrate, since they should be automatically balanced.
However, doing so raises the trust issue that TierNolan described and to which he proposes an incremental trust level solution.
I believe this is the way to go for p2pool to go mainstream. People just access the network, start mining, no cares given unless they want to.

I was discussing the issues with incorporating sub-pools into the protocol with Riclas.

The problem with having sub-pools is how to manage trust.

When a sub-pool connects to the network, it gets paid all the block rewards.  The pool operator is then responsible for actually making the payout.

This could be handled by a complex system of reputation tokens and fraud proofs.

But, a simple way of finding reliable nodes would be to do a few tests.  There would be some risk initially.

The miner connects to a random sub-pool and does 0.01BTC worth of mining.  If the node sends him the 0.01BTC, then he does another 0.01BTC worth of mining for that node.

The miner could do solo mining for any additional hash rate it had.

The payment channels system could be used here too.  The server puts 0.001BTC in a channel and then uses it to pay the miner as hashes arrive.

The channel size could be expanded as the miner does more work.

The rule could be that nodes which are hitting a share more than once an hour act just mine directly.  Nodes which are hitting less than one share per hour would search for super-nodes.  Nodes could indicate if they are super-nodes on connect and maybe include it in any address messages.

People will use marginally profitable hardware while the exchange rate keeps it that way. A 100% p2pool network would give more than just the marginally profitable too much variance to tolerate, I think I quoted > 1TH/s.

To put it another way, let's fast forward 10 years to a time when (if Bitcoin is still around) all of the mining hardware you're referring to is completely useless for any kind of profitable mining. The state-of-the-art mining rigs dwarf all of today's machines in comparison. Are you going to be fighting the same battle then, arguing that since some miners from years ago relied on obsolete microcontrollers for controlling their mining hardware, development should still cater to those as a least common denominator?

Because, as you put it earlier, "more overall hashrate is better." Apparently that assertion trumps all other concerns.

How could the network possibly deal with those people shutting down obsolete mining rigs? /sarcasm

(I honestly don't mean any personal disrespect in this conversation - I'm just finding it difficult to understand the rationale for catering to any specific hardware configuration, especially a hardware configuration that seems so… misguided and cheap).

Yes, I understand the difference between a controller (which admittedly can run on very low-end hardware) and hashing power. My point is, if you were using a $35 piece of hardware to control a nuclear reactor (yes, I'm exaggerating a bit to make a point), and the reactor - or rather the entire energy grid - were at risk of meltdown without a very minor, known-working hardware upgrade (compared to the reactor) for the controller to support the software fix, would you be wasting time trying to figure out how to somehow re-architect the software to run on the old hardware just to save a couple of bucks?

Do you really think that forcing/encouraging that minimal upgrade would disrupt the current (non-existent) "balance of power" on the network?

No, I don't think that's comparing apples to oranges. If anything, I think the current mining situation is even more ridiculous given the fact that (assuming the network survives), all of the current hashing power mining behind those cheap controllers is going to become worthless in a year (or less) anyway.

Go ahead and spin your wheels trying to save a small subset of current mining hardware while you watch the network crash as all those ignorant miners  who purchased pi-controlled ASICs point their hashing power at "benevolent" centralized pools who would NEVER do anything harmful (because the miners don't know better and/or they can't run something better on their controllers). Don't complain that we didn't have enough time/resources to fix the problem. Talk about missing the forest for the trees...

Er, you're kind of comparing apples to oranges there. I'm not sure that I made any case for using anything but the most basic device as a controller. Comparing a cheap controller with the amount of hashing per unit is missing the point.


You're making a strangely exaggerated argument. Which is it to be, that we should have low priced miners, overpriced miners, or that bitcoin is dead?

There is twisty logic mayhem in your post. I'm not convinced you understand mining or miner hardware too well.

I guess all I can do is echo gmaxwell's sentiment:



In your mind, mining can only either be controlled by a $2 microprocessor, or "the most dense and voluminous concentration of hashing power in a unit... requiring 240 volt electricity supplies, and thousands of watts of heat exhaustion"? Really, nothing in between?

I find it difficult to comprehend how someone could make a case for delaying or hindering adoption of a critical, network-saving change just for the sake of ridiculous hardware constraints like that mentioned above... unless there is a conflict of interest involved or some kind of financial incentive to profit from the deployment of such devices.

I... simply don't have the words to describe my dismay at this kind of flawed reasoning. If the only thing standing between Bitcoin's success and failure are a bunch of overpriced machines controlled by Raspberry Pis (engineered, by the way, to be underpowered and save *pennies* for the sake of bringing technology education to the less fortunate), then I'm sorry, Bitcoin is dead. End of story. We're talking about securing a global decentralized cryptocurrency, not learning to type "print('Hello, World!')".

Well, I'm not lying to you when I say that the p2pool developer changed the system to make it possible to use the type of hardware design that the first ASICs used. People tried using AsicMiner blades and Avalons on p2pool when they first came out in Spring 2013, and they didn't work. Forrestv changed the share interval and it's overall period to allow for it. So there's one way that hardware has dictated the direction of p2pool's design. Kind of pretty significant really, there was a lengthy hand-off period between the old scheme and the new scheme, after which the nodes running the old scheme were cut off from everyone running the 30sec/72 hour version.

The current range of hardware can only perform with the characteristics that it has been designed with, and I'm not sure to what extent that was to simplify the development, the design, or to help conform to standards in the protocol (the mining protocol changed shortly before ASICs, to enable their efficient usage of network bandwidth, and I believe we have two standards to replace the old CPU and GPU proficient standard)

What the challenges are with changing the hardware to send and return solution attempts in shorter batches, I'm not entirely clear on. So I'm putting it out there.


When you think about the health of the mining network overall, you've got to consider the influence the players have. The big centralised pools get consulted to secure their cooperation with any issues on the network that require emergency measures (blockchain rollbacks, emergency patches being the issues that come to mind). Forcing small players out of the network to "decentralise" may only end up further consolidating the influence of the big players. In the growing climate for big private mining firms, both now and in the near future, protecting the share of the small players is important. What you're suggesting decreases the incentive to mining hardware manufacturers for attempting to produce anything but the most dense and voluminous concentration of hashing power in a unit. What if all mining devices ended up requiring 240 volt electricity supplies, and thousands of watts of heat exhaustion? So much for vires in numeris.

We should do all we can to diversify the mining network to keep everybody pulling in the same direction, as the mining network is such a fundamental part of bitcoin's security and integrity. A small number of large firms with self run mega-mines puts too much decision making power in too few hands. There are many potential scenarios for that kind of situation to cause setbacks and imbalances. We should be aiming to make the network with a low barrier to entry, so that even very small players can have a stake in the gains and the rules that create the supply of this money. Promoting an environment that allows large firms to corner the market could put us all in a position not unlike the way money has been issued in the past. I became involved with cryptocurrency in part because it represented  an opportunity to change that situation for the better.

Sorry for potentially misunderstanding the problem, but why is hardware - in any way - dictating the direction of software development here? Bitcoin itself is built to scale with changing hardware capabilities by automatically adjusting difficulty as needed. What makes a decentralized mining pool so different conceptually?

Why do you worry about forcing out a subset of current miners who will be forced out with time (and increasing difficulty) anyway? If all miners were forced onto p2pool tomorrow and people with less than (as an example) 1TH/s rigs are no longer competitive, would the remainder of the hashing power distribution really be any less diverse in terms of control than the current situation with GHash.IO? It seems to me that with a bit of time, everyone currently mining is going to be "forced out" anyway due to the competitive forces of hardware advancement. If a software change temporarily accelerates that process in an unbiased way for everyone on the network, how is that bad? The only way I can see it mattering significantly is if the % of miners forced out is very high (e.g., a majority of all miners) causing a slightly "unfair" advantage for those with access to the best hardware (since the marginal cost for additional ASIC hardware could buy a slightly bigger piece of the pie than it could previously), but that was true when the first ASIC rigs became available anyway. The market (in my opinion) has and will adjust if there is any such economic advantage for certain classes of mining hardware.

Remember that increased usage of p2pool will only increase payout variance to most miners, the opposite dynamic to that of joining a large centralised pool.


To reiterate, if all miners were forced onto p2pool tomorrow, the share difficulty would be pushed up so high that I suspect many lower hashing rigs would quit due to the uncertainty of getting a payout (people with less than > 1TH/s rigs could be mining for months with zero payout, which is a risk they won't tolerate in an environment with rising block difficulty and 24/7 multi-100 watt electricity usage)

My brief tug of war with Tier Nolan should illustrate one important area of improvement to investigate: hardware design. The current generation of hardware comprises many ASIC chips working in a single unit, interfacing with a low cost computing device to run the mining software that schedules and feeds the work to the chips. Some aspect of these current designs makes it necessary to only return the results of work in batches that last ~30 seconds. Forrestv overhauled the share interval and the length of the PPLNS window that p2pool uses just in order to accommodate this type of hardware. When GPUs and FPGAs were the dominant mining devices available, 10 second share interval spread over 24 hours was fine. This was changed to 30 second share interval spread over 72 hours for the typical ASIC.

Can we get ASIC designers to produce chips or PCB layouts that make <10 second share intervals possible once again? The dust is beginning to settle in the ASIC node geometry wars, so they'll need some different direction to go in if they wish to continue to innovate.

It sounds like there may be some interest in bringing p2pool (or a similar concept) closer to the reference Bitcoin implementation (whether that be a change in the protocol, bundling p2pool with the reference client, or simply giving p2pool a stronger online presence in connection with bitcoin.org).

I'd like to try and gauge interest in the various approaches to solving this problem, and perhaps (if it hasn't been done already) formalize some kind of plan of action.

The possible directions I'm seeing are (and these are not mutually exclusive):

• More discussion leading to a formal BIP submission with changes to the Bitcoin protocol and reference client. Then, I guess we wait and hope that either the core team can get to accepting and implementing that BIP sooner rather than later (understanding that there are numerous accepted BIPs whose priority seem to be uncertain), or someone else can step up to the challenge... makes we wonder though how many developers outside the core team there really are that have the expertise, familiarity with Bitcoin, and incentive to contribute such a fundamental change in the timeline needed.
• More discussion about improvements to p2pool (as the separate piece of software it is now) attempting to address any technical shortcomings of p2pool (scaling, speed, hardware requirements, etc).
• More discussion about improving the non-technical issues of p2pool - public awareness, user experience improvements, etc.
• Discussing ways to add resources/velocity to the speedy development of one or more of the solutions above (in the form of crowdfunding, offering bounties, raising developer awareness, etc).

So my question is: how can we best contribute to this issue being solved effectively in the quickest timeframe possible?

What do the core developers feel is most needed at this point? Can we all try and reach some kind of consensus as to how this should be addressed? I feel like unless we can (most of us anyway) concentrate our efforts on one of these things, we are wasting time and resources by chasing a number of different proposals. In theory (with open-source software like Bitcoin), a number of separate contributing groups can work on different solutions to the same problem, and I suppose the best of many potential solutions would naturally emerge, but in the case of Bitcoin, I get the feeling that the number of developers in a position to make these kinds of changes is somewhat limited, and we don't have the time to wait for a solution to just roll around "naturally".

Any thoughts or opinions on how best to move forward to address this concern?

I'm afraid that if we don't act quickly, these discussions will merely become artifacts of a failed cryptocurrency replaced by something better - all because we couldn't fix these kinds of issues fast enough.

This is the only one I know of.  The admin of the pool commented in the other p2pool thread and his pool thread is here

That stats suggest that nobody is mining bitcoin on it though.  If a miner uses his system, they would only get the same effect as mining using p2pool.

As I said, if p2pool gets more hashing power, the minimum difficulty increases.  Pools like that act as a 2nd buffer.

P2pool has the potential to act as a backbone for lots of small pools (say 0.1% of total hashing power each).  By grouping together, they get low variance without centralising control of the system.

Normally, large pools have an advantage over small pools due to smaller pool variance.

Are any pools known to be mining against p2pool at present?

The total number of shares is equal to 3 blocks worth of shares (in terms of difficulty).

If the average difficulty per share is lower, you get more shares included (up to the 3 day limit).


If the difficulty is to high, then smaller miners would only be on the chain sometimes.


One of the proposals is sub-pools.  Rather than mining against the main network, a mining pool could mine against p2pool.

A smaller pool inherently has higher variance, since it is hard to find bitcoin blocks.  Mining against p2pool gives better variance for smaller pools due to the lower share difficulty.

Thinking about it, if you did have a 1024 PPLNS system by default, so that the last 1024 shares were paid, and miners could increase their difficulty and get paid etc etc.., would it not be the case that all the miners would lower themselves down to the smallest miner able to get on the chain ? So that, in the ideal case, each miner still only got 1 share per 1024.

This way you would at least have 1024 miners/mining pools visible on the chain ? Rather than the 10 or so currently available on Bitcoin.

Not sure how important decreasing the difficulty of the individual shares actually is.

The weakest miner able to get on the chain would set the hashrate that the others would have to emulate by increasing their difficulty.

Having 1024 mining pools able to get on the chain is better than less than 1024. No ?

Ahh.. yes I see now.. A miner can only decrease the overall difficulty of the chain as a proportion of his own hash rate vs the rest of the network.. If he has half the hash power, he can decrease the overall difficulty by a half.

He can bring himself down to the level of the smaller miners..

..TierNolan!! 

Okay, you've convinced me. I wasn't accounting for the relationship between the block difficulty and the share difficulty.


This is true, there's a guy on this forum who was using a 32 GB RAM disk for p2pool. I don't know how he will justify the expense to add the RAM he will need in the future, but he maybe didn't buy the RAM for a p2pool node in the first place (and the cost per GB seems to drop regularly too)

I was responding to Carlton Banks.  He felt that increasing the length of the share window would decrease the difficulty.


1% bonus would probably be enough.

Any miner who is getting more than 1 share an hour should definitely increase his difficulty.


No, because "small" miners would still be on lower difficulty.  A miner with 1000X the power of the other 1000 would get half the shares.

If he set his difficulty to 1000, then he would get 1 share while the rest got 1000 shares between them.  The difficulty would be halved to being things back to one share every 30 seconds.


The suggestion in the document from the donation thread was to simply charge per share.  The charge could be something like 1% of the expected payout per median share.