Topic: why isn't p2pool more popular than it is? (Read 6934 times)

I had been using P2Pool for a long time but my internet is shit so my getblock latency is high and I was getting too many stales. Switched back to Eclipse until I find a solution. If there is one. The internet around here is shit even though my town claimed to have put fiber in back in the early 2000's. I'm pretty sure they just buried some twine and embezzled the money.

Reposting from the p2Pool thread:

well, i can believe that stratum is causing problems after observing its slowness.  this would cause much more problems on p2pool then anywhere else.

cgminer was set to realtime priority (and also minerd affinity was 6 of 8 cores) before anyone mentions that

Should be interesting to see the figures after this block hits...those numbers are about to take a huge hit (they only update when a block is found).

Based on the fact that 'random' and 'trend' are usually unrelated
Especially as the sample grows.

Anyway, I guess looking at the graph now, vs last time, it has indeed jumped up again (to almost expected) then had another slope down after that.
The line I am looking at is of course the red one (30 day moving average)

From Nov 28 to Jan 20 was indeed an obvious trend down. However, since it is supposed to be a 30 day average, you'd expect a drop somewhere 30 days after the spike that pushed it up so high on the 28th Nov - certainly by the 28th Dec
The line from Dec 20 to Dec 24 might suggest it was ~30days before that.
The jump back up by Jan 24 may indeed show the exact opposite - that the drop ~24 Dec was simply an unlucky couple of days that move out of the average by Jan 24.

Anyway, look at the 7 day average (green), it does indeed jump up an down all over the place as would be expected of some random statistic.

But lastly looking at the 90 day average, that has indeed been a clear downward trend since 18 Dec ... and I'd say 2.5 months of that would be considered unlikely ... no maths provided

As my tutor at uni used to write: PLEASE SHOW WORKING.

The statement: "the probability of producing a downward trend for a few months isn't very likely at all" as it stands is simply not true - for example if average luck at the start of a trend was very high, a downward trend would not be unexpected. Saying "it isn't likely" doesn't make it so.

You need to define "trend" better.

I was good at math, I just couldn't be bothered with it.......girls & beer were more my thing.....lol

Well, in my case it wasn't a forecast, it was a simple, and reasonable, interpretation of a graph generated from a large sample.
The LUCK graph STILL looks like a downward trend.

Now you can say the probability of being within a range is likely, but the probability of producing a downward trend for a few months isn't very likely at all.

As to why it has that downward trend? Well that's another question all together that of course I have no idea what the answer is.

... but ... I never did well at stats in university coz I hated it

Organofcorti,

Please, lighten up a bit. Didn't you see the smileys?

It was a good guess though eh?

Or was it an accurate guess?   lol

Forecast, lol. It was a guess and giggle at best.

One can really only forecast things of which one has some understanding. Sorry if that seems harsh, but I'd really prefer you present your uninformed opinions as exactly what they are. Please don't give them the weight of an understanding you don't have.

Could be taking my life by my own hands here........breathing life back into this thread....lol

Kano, it seems we were pretty accurate with our forecast:

http://p2pool.info/luck/     and still falling...also hearing murmurings in the ranks.

This is tragic, really. I want the old p2pool back!!

Still, the times -they are a changing.......(singalong)

No idea - I don't have easy access to the pool's data. Fireduck was going to look into making a csv available, but nothing came of it.

does HHTT have worse luck than bitcoinpool?

well, i was using the stratum miner thing and i found 3 of those blocks. 

at 8000mhash

in like 10 days

and i got about 10 btc instead of 75 =/

P2Pool

I'm just saying that the graph shows an interesting decline.
Even if every point along that graph is within expected luck, what is the probability of actually getting that graph as it looks?
It does look like a somewhat linear luck drop over 60 days.

But of course I've no idea if it is even possible to determine that probability (the reason for the )

The probability of getting a graph like that for 60 days is obviously not 100%
So either it's common to expect luck to drop in that manner (I doubt it) or even possible that there is something systematically wrong with the production of the graph (no idea) or ... ?

Generally, or in p2Pool's case?

Also when you say "probability" do you mean for an arbitrary number of rounds or an arbitrary timeframe?

LOL!! Pretty darn good I reckon mate......

So ... what's the probability of a luck graph showing a pretty close to linear drop for 60 days?

The End 

.. and why it's extremely important to be an informed miner, and understand luck better. Luck doesn't trend, and I can't think of any way p2Pool's luck could be affected negatively - unless the data itself was wrong.


Why, thank you - and I encourage you to post questions rather than statements when there's something you don't understand. It's the only way to learn.

I agree completely, but do you see my point? Someone who doesn't understand it will obviously go with the pool who's luck "appeared" to be good. It's a very natural thing to do, and an easy mistake to make.

(great info you do by the the way).

tl;dr: unless you have some way of assessing luck, don't worry about it. It's unlikely to ever be abnormal, and if you think it is let me know and I'll usually be able to give you an idea of how unusual the unlucky streak is.


I understand why you think that the graphs are showing something they aren't. There are ways to provide a "luck chart" that is menaingful even if you don't understand "luck", for example:



Shares per round / difficulty is a measure of luck - the higher it is, the less was earned and luck is worse. I think the p2Pool luck charts are 1/(Shares per round / difficulty)*100 to make it a bit more intuitive.

On the chart, the jagged line is the cumulative mean of (Shares per round / difficulty). As long as the line is in the shaded area, the shares per round / difficulty is within a 95% confidence interval for the data, and luck is in the expected range.

I'm not going to post an up to date chart right now, but the line is certainly in the shaded area.

As for


There are lots of pools with luck tending up or down. It's normal for pools as long as it's in an appropriate range. It's something you just have to accept about mining - luck will go up and down, and very rarely will it be outside of a normal range.

Another chart:



The dots are a measure of the weekly average luck. Higher dots are worse luck, lower dots are better luck. BitcoinPool for example has had abysmal luck. p2Pools luck doesn't look unusual.

LOL!!  Nomnom.....that's a first for me...

Organofcorti,

You are very knowledgeable in the the bitcoin field, I know that. I have read many of your posts and learned a great deal from you, the work you put into the community is second to none - I bow to your bitcoin wisdom.

However, if somebody with less knowledge/experience than you, or I for that matter, were researching which pool to join and came across that luck graph - they would definitely have second thoughts about joining that pool - rightly or wrongly and for what ever reason. Maybe calling it a "luck" graph is a little misleading to some people, but not me. I understand it completely, and am not sure what made you presume I didn't. Given the choice between a pool who's "luck" was steady/rising and a pool who's "luck" was on a 60 day permanent decline - and still falling - what pool would the average potential miner choose do you think? My answer was from a general point of view, not mine and mine alone, as the question was a general question.

I'm sorry if my answer has offended anyone, that was never my intention. Graphs are there to provide information, aid research, help people decide what pools to join and aid in eliminating problems. If you believe everything is hunky dory with that graph, that's fine, but I can assure you that many more people will be put off by it, rightly or wrongly. That was my point.

So, I stand by my answer.

He's not a troll. He even said so.

dont feed the trolls

If you don't understand "pool luck" and you don't want to learn about it, please stop mentioning it.

Spiccioli,

I don't think you understand me. I am a fan of P2pool, and I too have been mining there since 2012 up until very recently. I am simply answering a good question with what I believe to be a reasonable answer, while also making it clear that I have a lot to learn. I don't see how the luck chart link can be described as FUD at all, it's a link that is viewable for everyone to see. Also, it's common knowledge that the stale/reject rate it significantly higher on P2pool than most other pools. This alone puts many people off mining there. I am not in the business of posting FUD for no reason, only facts and giving my honest opinion to a very good question.

Nobody wants P2pool to be successful more than me, believe me.

PatMan,

please stop spreading FUD here and there!

p2pool luck is OK, it is called variance, I've been mining on p2pool since january 2012, there have been longer unlucky periods and even longer lucky ones.

I do agree that it takes a little more work to setup and babysit it that it takes for other pools... but that's the fun of it!

You're effectively mining solo when mining on p2pool... and like the whole bitcoin thing, with great power comes great responsability!

spiccioli.

Thanks for the 1st link, i really didn't know about that stuff till now.

Question:     Why isn't p2pool more popular than it is?

Answer:    1) http://p2pool.info/luck/

               2) Rejects, stales, DOA rate.

               3) Loads of tech stuff I don't understand, but my guess is there's something amiss with the code.

How bizarre...

Nice beginning thanks for the warm feelings... As you may have noticed I replied to Kano who complained about not having received any reply to this kind of problems, not your system in particular. The purpose of my post was just to tell that not everyone experience this kind of problems and stipulate on probable causes based on the specifics of my setup. Solving your problems might or might not be possible using what I wrote.


I suggested that the bitcoind version matters too. Stable releases are far slower than the current development tree.


I don't know about the rest of the network just that my efficiency hovers around 105% vs the network and that I earn more than I would on a 100%PPS pool distributing transaction fees by ~5%. Currently I'm at 2.6% DOA and 106.6% efficiency.
I don't know what the problems are with your setup but given your first sentence I'm not willing to take the time reading your previous posts. I'll let you find out for yourself.

ofc, it could be solved by giving the lucky person that 'solves' the block the entirety of the transaction fees.  then (nearly) everyone has incentive to include everything they can get.

most pools are colo /dedicated servers on robust networks,  so including transactions isnt as big a deal (because all the other pools will receive your block quickly as well)

if you don't know about my system, you probably shouldnt be commenting at all.

mine is run off of a RAM drive, and even on 200kb blocks, it rarely goes above .15s on the 'GetBlockTemplate Latency' ...

alas, that's not what's causing my shares with 200kb of transactions to become orphans

did you notice this recent run of blocks we had?  overall orphan/DOA rate down to 8%?  coincidence, im sure.

It's bitcoind. The option reflects the protocol: as long as a block is valid it can be generated. Blocks without transactions are valid, removing the option would only lead to a fork if mining empty blocks is more profitable.

The only way to make the situation better is to remove the incentive of ignoring transactions by making it nearly free (or even profitable) to include transactions in mined blocks. The last p2pool hard fork was designed to help in this respect (transactions are pre-forwarded in the P2Pool network to minimize the block publishing latency when one is found: orphans should be less frequent than on most pools this way). I'm not even sure that the parameters I posted are optimal (as long as I don't believe I hurt the network I'm not motivated to test new values though).

Seems like forrestv needs to remove that option to restrict block size.

Unless it's a bitcoind thing.  Either way, the author should remove that option.

M

Yes, I was just explaining the background behind "transaction fees are required" (in the long term). In theory, we should have some time left with free-of-charge transactions. In practice, DoS attacks like SatoshiDice (which don't help Bitcoin adoption) are abusing the network and current fee rules to crowd out more legitimate transactions (which makes Bitcoin adoption more difficult, since now people need fees before we've reached critical mass).

Don't know about your setups but on my system with a bitcoind from git HEAD and its data on SSD:
leads to an average latency of 0.05s for p2pool<->bitcoind and very low stales. Maybe people with high stales that only go away with blockmaxsize=0 are using a slow version of bitcoind or a slow disk.
If I understand my parameters well, I limit a block to 100kB (1/5 of the max and about the current average block size if I'm not mistaken), reserve a small place for old 0-fee transactions that aren't yet included in a block and ask for 0.0005 BTC to include a TX right-away. These are the recommended value I stumbled upon when configuring p2pool some time ago. I haven't tested other values since then so if you think it's not contributing enough, please advise and I'll report any change.

I tried to point this out in the p2pool thread but only got silly replies ... so I gave up.

I guess that is yet another answer to the question this thread asks ...

https://bitcointalksearch.org/topic/m.1423255

https://bitcointalksearch.org/topic/m.1423312

Which is a shame, as mining is about verifying transactions for the bitcoin network.

like this:

with maxblocksize at 0, i get much fewer orphans than with maxblocksize at 300,000.   it doesn't matter if my efficiency is (bordering) on 100% with maxblocksize at 300,000.  when it's at 0, it's 110% or more.  the *possible* .5 or so more from fees (of which i'd get around 4 or 5% (ed: more like 3%)) doesn't come close to making that up...   it doesn't take high level maths to figure out that you're worse off by including transactions

If ASIC's work with p2pool I will be adding 2.4TH to the network as soon as my first order arrives.

Why not using a MtGox deposit address. It is free to receive and withdraw so you can combine your dust outputs to bigger ones

Syke is just a troll, simply put him on ignore. Despite the fact that there is already a public script who auto updates p2pool in combination with cron.

The block solver gets a 0.5% bonus while the rest of the 99.5% is split to all the miners. I still don't know what you're talking about with these orphans, there hasn't been an orphaned block since just over 2 weeks ago. If you're talking about shares, 5% is perfectly within the acceptable range.

How is the block 'bonus' calculated for the person who solves a block?  Maybe if the person that solved it got *all* the transaction fees, it would encourage people to include the transactions?

I think it may already work something similar to this, but only a portion of the transaction fees?

Right now I have my block size limit set to 30000 because of all the orphans I was getting with the 200kb block sizes.... but, if the person that solves the block got all the fees included in that block, it'd be worth the extra 5% orphans..

Is it any different from making sure the conventional pool is working?

I just saw someone griping the pool he was using didn't tell him his miner was offline for 10 days.  I saw another one who was complaining the stats are wrong.  It seems to me the upkeep is the same between conventional and unconventional.

M

My biggest complaint about p2pool is the maintenance required to keep a p2pool node up and running. That said, the advantages to p2pool are enough to keep me on it.

The next halving is in a little under 4 years, right?  Isn't that plenty of time to figure this out?

M

Thanks for the explaination.

They're optional right now because blocks have subsidies; Bitcoin also benefits from being the cheapest way to spend money online, which encourages its adoption and drives the market price upward (which is beneficial to the miners of course too). As designed, Bitcoin block subsidies will eventually drop below the break-even point for miners, at which point mining will be dependent on the transaction fees more immediately - though the longer we can afford to provide confirmations without a fee, the more likely Bitcoin's successful adoption is.

Hmm.  If transaction fees are required, why does the client default to not providing any?  I know more than once I've accidentally sent a payment, all from p2pool, without a transaction fee because that setting wasn't on.  Never had a problem with them going through quickly or getting confirmations either.

Not that I'm advocating sending transactions without fees, just stating they aren't as required as people make them out to be.

M

This is not a planned feature of 0.8. While it will only use an unspent tx database, it still needs to download (and at present, store) the full blockchain. Just FYI.

Transaction fees are a very important part of Bitcoin, so I would NOT advocate mucking with it.

I don't know what you mean by "unspent tx".  Whats your definition of an unspent transaction?
Sam

Edit: forgot one little word

Maybe we can propose a change to the fee rules that put less fees on transactions that reduce the number of unspent tx. With bitcoin 0.8, it'll be even more important (bitcoin nodes can download only a list of unspent tx, which reduces the blockchain size to something like 200MB).

From what I've seen on the threads for the other pools.. none of them are fully stable.

M

Another piece of software to set up and maintain on my different platforms. I think the increased earnings will go to waste with the downtime involved when upgrading. So I'm sticking with a regular stable pool with port 80 support, and using a backup pool in case of netsplits or attacks...

What you say is true.  However, for the last 90+ days, the 90 day history has been > 105%.  Yes, for at least the last 180 days it's been better than average.

M

p2pool's luck is just that:  luck.  The smaller the pool size, the more varied that is.  90 days is hardly enough time for measuring it given it makes up less than 3% of the network.  For almost a year the p2pool luck was abysmal and anybody using it would've been better off with any decent sized pool.



It's hard to give a real world example, but you'll have a very hard time ever sending 13 BTC with 325 different inputs.  In that kind of situation, you'd be better off sending your inputs into a separate wallet with larger chunks (keeping the transactions small while doing so).  Do that a few times, then wait for those to mature, then use those to make even larger chunks (keeping transaction size small while doing so) again once they've matured enough to not require a fee.  Note:  This will not work if your initial inputs aren't big/old enough to avoid transaction fees.  Each 0.04 input would have to be 25 days (I think) old prior to being merged.

The "bag of pennies" scenario is a significant problem for small miners on p2pool.  Not so bad for larger ones.  But the larger p2pool becomes, the worse this problem is.  It's a hidden fee for using p2pool.  You get your money faster than some pools, though there are many pools that don't require waiting for blocks to mature (or PPS where you don't even have to wait for a block).  However, the larger p2pool becomes, the more frequently you'll be forced to pay txfees when spending coins.



p2pool's concept is great, but the payment method and share chain prevent it from becoming a major "pool".  Unless it gets turned into a series of supernodes which people mine through, which would end up being worse than the current pool system due to high stale rates using a remote node that has longpolls every 10 seconds.

If the outgoing connections it picks are good, sure.

I'm considering having my local node just have 1 outgoing connection, to my nogleg server.

(figured I'd edit this to say that I'm going to shut the local p2pool down right now and maybe rework it later.  got an incoming connection now requesting shares that's saturating my upstream, giving me like 30% DOA and 500ms ping times)

That's a very helpful explanation.

So at 4 Bit Cents per block my 13 BTC example would have 325 inputs.  Would that really be only a 0.0005 BTC fee?

Does anyone have a real world example?
Thanks,
Sam

Don't forget that by default, there is a fee attached to mining with p2pool unless you disable it. I'm not advocating or passing judgement on that fact, just pointing out that people seem to forget or not even know the fee is there.

With 500MHs/s your average reward should be around 0.04BTC per block found. You can pack several inputs in a transaction with a 0.0005 BTC fee so your loss from fees is less than 1% when you use p2pool mined coins.

So if the alternative is a pool with more than 1% fees you're automatically better off with p2pool.

Currently the fees earned by block mining are around 1-2%, so if the alternatives don't pay you network fees, that's more you lose vs p2pool.
And lastly, my 2 months performance on p2pool is 107,5% vs expected 100%PPS so p2pool seems to have a higher than average reward than most pools. I didn't even count some small downtimes in that period so it's a low estimate. This matches what http://p2pool.info/ reports: constant higher luck.

One explanation may be that p2pool is by its nature better connected to the bitcoin network than most pools: all P2Pool nodes quickly broadcast a p2pool found block on the bitcoin network which may lower orphan rate.

Seriously? 

M

Yes. More inputs make a transaction bigger which will make it more likely to require a transaction fee to get it confirmed.

OK, I'll try to restate again

I mine 500Mhs/s on p2pool and get a bunch of small payments per block and get 13 BTC and pre order a Jalapeno.

I mine another 13 BTC, say on Deepbit, with a 1 BTC threshold to a different wallet and pre order another Jalapeno.  Will the 13 BTC from p2pool require higher transaction fee to spend because they comprise a bunch of small input transactions?
Thanks,
Sam

I'm not sure what you're asking.  If you're saying "if I spend 0.25 BTC at a time, instead of waiting until I have 1 BTC to spend, and I pay 'proper' transaction fees for each 0.25 BTC", then yes, you'll spend more in transaction fees.  AFAIK, there aren't fees associated with mining.

M

Cool!


That's all well and good, but it doesn't answer my statement/question.  I'll attempt to restate.

Spending your coins mined "in increments" on p2pool will have a higher transaction fee associated with it than coins received, say, 1 Bitcoin at a time.

Is that statement correct?  Or not?

Thanks,
Sam

My bandwidth usage was about the same before and after the incoming connections popped up. I also noticed a pretty big delay in normal internet browsing while it was running (my upstream bandwidth got maxed out pretty regularly).

I'm coming to the realization that incoming connections don't have to be there.  I have mine limited down to 3 to prevent stales and is working just fine.  If bandwidth is an issue, don't turn on port forwarding.

M

Eligius has been pretty stable for a while, and offers most of p2pool's features without its limitations (high variance, high bandwidth/CPU, more maintenance, more vulnerable to DDoS, etc).

It takes less bandwidth to DDoS every p2pool node, than to DDoS some of the non-p2p pools. A number of these non-p2p pools also support decentralized mining.

They're worse, in fact, since they are centralized. While GBT's bandwidth usage is reasonable for ordinary Bitcoin mining, it's massive for p2pool's 10 second blockchain, so not a viable option there (yet).

I literally just updated it yesterday to try it out with Stratum. So yes, I did.

you probably didnt update p2pool...

The massive amount of bandwidth usage certainly doesn't help things. I tried p2pool again yesterday and racked up 1GB in just over 12 hours (even with only ~3 incoming connections). I can't really spare 1.5-2GB a day on a 150GB monthly limit.

It's not like conventional pools aren't having problems.  I'm seeing them left and right.

But well said.


M

The honest answer is a combination of effort required to get it up and running and stay in sync with p2pool changes, profit margins being lower due to the altered block nature and higher effective rates of lost work, and the intermittent issues that seem to repeatedly plague p2pool during development. Most people use pools to avoid the rolling release/change cycle at the pool end since that adds yet another potential point of failure on top of getting your client right. Setting up 3 or 4 regular pools with failover is mostly a fire and forget thing for miners by comparison.

Actually I do want an answer.  

Yes, incoming mining payments are in increments.  The only difference between this and a conventional pool is the conventional pool sits on it until it's matured, and usually will hold it until your requested threshold is reached.  With p2pool, you receive it as it happens, and it matures in your wallet.  No transaction fees for receiving it, and you can spend it when you want, with whatever transaction fees you'd pay normally.

You also get a reward for finding a block.  There was an optional 0.5% author fee to keep development going - v10 he bumped it up to 1.0%.  It is optional - you can override it.

It also has stratum support.

I'm really not seeing the allure of conventional pools over this.

Oh, BFL singles don't work on it.  Those people have to stick with conventional pools.

M

EDIT: Forgot to mention PPS.  No PPS on p2pool.  I can see why PPS users would like a conventional pool.  All others though...

I doubt that you really want an answer to that question.

But one possibility is maybe because you get paid allot of small amounts instead of meeting a threshold and getting paid a larger amount in one lump sum.

When you then spend your coins you could end up paying a large transaction fee, correct?  Or maybe I'm all wrong about that?
Sam

I can't help but wonder why p2pool isn't more popular than it is?  From what I'm seeing in the various pool threads, most, if not all pools, keep having problems.   Either namecoin crashes, hardware issues, or DDOS attacks.  Why not use p2pool?

Anyone who can manage setting up a mining farm, large or small, shouldn't have any difficulty setting up a p2pool node.  Then you can do your own namecoin merged mining if you wish (most pools seem to be dropping namecoin support), but most importantly, you're then part of a decentralized pool.  If someone gets DDOSed, that node goes down, but the rest keeps working, unlike a conventional pool.

Yes, there are public p2pool nodes.  I don't suggest using them unless you have a really small operation, as they are no better off than normal public pools.

P2pool ftw!  https://bitcointalksearch.org/topic/1500-th-p2pool-decentralized-dos-resistant-hop-proof-pool-18313

M