Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 619. (Read 2591920 times)
November 20, 2012, 12:37:54 AM
Updated to v9. It seems I'm always one of the last ones to move over (because I'm not as active on these forums anymore). Could I possibly get someone to shoot me an email when a new binary is released? That way I don't get left behind, and we can move the transition to the next version along?
November 19, 2012, 04:25:18 PM
Last git is killer for litecoin pool
http://pastebin.com/EGbXXAps
Nodes are disconnected over and over.
http://pastebin.com/EGbXXAps
Nodes are disconnected over and over.
November 19, 2012, 02:18:35 PM
I see a big bold warning on the client that says this is a test build and should not be used for mining.
I am curious if others are mining with this anyway. I, also, tried the latest and then went back to the stable version after seeing this message. Mostly because I have used git-versions of bitcoind a lot in the past and they generally never had warnings like this. So I took the presence of this warning to be an indication that it was especially unwise to mine with it. But does anyone actually know if it has serious issues or if the message is mostly about prudence?
November 19, 2012, 01:48:48 PM
for you guys running the latest git versions of bitcoind, are you building these locally or is there someone trustworthy doing nightly builds?
Once you make compiling machine up and running it is not a problem to compile new git yourself even everyday Code:
cd bitcoin
git pull
cd src
make -f makefile.unix bitcoind
strip bitcoind
git pull
cd src
make -f makefile.unix bitcoind
strip bitcoind
Code:
make -f makefile.unix -j2 USE_UPNP= bitcoind
You will also need some dependencies. On Ubuntu 12.04, I use:
Code:
sudo apt-get install -y git-core build-essential libssl-dev libboost-all-dev libdb5.1-dev libdb5.1++-dev libgtk2.0-dev
November 19, 2012, 11:42:07 AM
for you guys running the latest git versions of bitcoind, are you building these locally or is there someone trustworthy doing nightly builds?
Once you make compiling machine up and running it is not a problem to compile new git yourself even everyday Code:
cd bitcoin
git pull
cd src
make -f makefile.unix bitcoind
strip bitcoind
git pull
cd src
make -f makefile.unix bitcoind
strip bitcoind
November 19, 2012, 09:57:37 AM
for you guys running the latest git versions of bitcoind, are you building these locally or is there someone trustworthy doing nightly builds?
November 18, 2012, 07:24:48 PM
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.
What might be underestimated too is the amount of work from nodes which stop mining after stale shares.
- p2pool.info luck,
- my efficiency,
- the average tx fee percentage.
This is why I'm back mining on p2pool: hashpower promises at least 105% PPS when leasing and falls short due to high rejects, I compared my actual PPS over several weeks and stats are largely in favor of p2pool for me. I left p2pool because I had I/O issues and likes to test novelties, but now bitcoind and p2pool are both on SSDs and they are just flying (at least when I'm not rewiring the whole apartment...).
My first SSD should be arriving soon. I'm seriously hoping it decreases my stale rate - like you, I think it's I/O related.
M
Have you got latest bitcoind from git? It's much faster than stable version, with new database. With latest bitcoind from git my GetWork Latency dropped about 5-10x times.
I managed to get ubuntu up and running a vm, and managed to get bitcoin up and running without too much of the usual linux shenanigans. However, I see a big bold warning on the client that says this is a test build and should not be used for mining. How are you supposed to test it if you can't use it in prod? I'm sure not pointing GPUs to test.
M
November 18, 2012, 06:05:35 PM
forrestv, please check my comment on last commit
https://github.com/forrestv/p2pool/commit/daa3ba3dfed955caceef49438f6d0067c7ea99a1
https://github.com/forrestv/p2pool/commit/daa3ba3dfed955caceef49438f6d0067c7ea99a1
November 18, 2012, 05:55:57 PM
Nice explanation from forrestv on how p2pool is counting stales on #p2pool channel:
Conclusion for kano, stales don't explain the current 110%, variance is probably the biggest part and p2pool has an advantage vs other pools too: it broadcasts its blocks through a much larger set of bitcoind nodes. When we had latency problems we had quite frequent orphan blocks (and probably bad luck too) and averaged ~90% PPS. Now we rarely see orphans compared to what we saw at the time.
Code:
23:05 < forrestv> gyver, assuming the share chain is operating normally (no major
forks), the pool hash rate estimate is unbiased except for it being
a little lower than true due to people leaving before telling
everyone they got a stale by finding another share
23:06 < forrestv> each share has a flag that says either "i don't have any uncounted
dead/stale shares", "i have 1 uncounted dead share", or "i have 1
uncounted stale share"
23:07 < forrestv> your node keeps counts of how many dead/stale shares you've mined
and how many times you've successfully produced a share with one of
those flags
23:07 < forrestv> and it keeps producing shares with one of those flags until you've
notified everyone of your past dead/stale shares
I can live with the lower estimation. Stales to report, according to my experience from both the network as a whole and my nodes, are mostly between 6 and 12%. So on average we lose between 6% and 12% of the hashrate of people leaving p2pool (assuming their node isn't coming back later) for the average period between their last p2pool-valid shares. The unrealistic (and provably "not happening") worst case scenario is when everybody mining on p2pool would create nodes that would only mine for a period matching their own expected interval between shares. That could explain a lower estimate of hashrate matching an artificial 110% PPS result.forks), the pool hash rate estimate is unbiased except for it being
a little lower than true due to people leaving before telling
everyone they got a stale by finding another share
23:06 < forrestv> each share has a flag that says either "i don't have any uncounted
dead/stale shares", "i have 1 uncounted dead share", or "i have 1
uncounted stale share"
23:07 < forrestv> your node keeps counts of how many dead/stale shares you've mined
and how many times you've successfully produced a share with one of
those flags
23:07 < forrestv> and it keeps producing shares with one of those flags until you've
notified everyone of your past dead/stale shares
Conclusion for kano, stales don't explain the current 110%, variance is probably the biggest part and p2pool has an advantage vs other pools too: it broadcasts its blocks through a much larger set of bitcoind nodes. When we had latency problems we had quite frequent orphan blocks (and probably bad luck too) and averaged ~90% PPS. Now we rarely see orphans compared to what we saw at the time.
November 18, 2012, 05:36:42 PM
If you have always seen above 100% PPS then you are missing including something.
Block+Fees average is currently 100.36% according to blockchain.info
Not sure which stat you are referring to (whole network over a period of time ?).Block+Fees average is currently 100.36% according to blockchain.info
Anyway, my computations seem fine to me. That's not rocket science to compute expected income from one's hashrate and current difficulty and compare it to actual income over a large enough period (at least 4 weeks in my case). I have a whole spreadsheet dedicated to my mining activites and one sheet is dedicated to expected/actual mining income: I dutifully enter each mining related transaction and compute expected income for each day (with appropriate adjustments on the days difficulty changes).
If my results seems too good to you on p2pool, you miss one perfectly reasonable explanation. It could simply be luck the PPLNS system doesn't use 1-difficulty shares but far more difficult shares which introduce variance in your expected income (I've seen at least +/- 15%) for a block found. So I may just have been lucky when p2pool found blocks and unlucky when it didn't. But as I said my results are matching with my reported efficiency (between 102 and 106%, was mostly 105 and is around 104 recently), p2pool.info reported luck (~110%) and the 100.36% you report for block+fees (in fact I used 100.25%).
November 18, 2012, 08:03:42 AM
In case you didn't see my post, I wrote a .Net "widget" for Windows that allows you to monitor your miners on p2pool, eclipse, ozcoin, and 50btc. More pools are coming as time permits (bitminter is next).
https://bitcointalksearch.org/topic/mpoolmonitor-42-monitors-most-pools-idle-worker-notification-blockchaininfo-86502
M
https://bitcointalksearch.org/topic/mpoolmonitor-42-monitors-most-pools-idle-worker-notification-blockchaininfo-86502
M
November 18, 2012, 06:14:31 AM
Remember to
Code:
strip bitcoind
November 18, 2012, 04:16:46 AM
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.
What might be underestimated too is the amount of work from nodes which stop mining after stale shares.
- p2pool.info luck,
- my efficiency,
- the average tx fee percentage.
This is why I'm back mining on p2pool: hashpower promises at least 105% PPS when leasing and falls short due to high rejects, I compared my actual PPS over several weeks and stats are largely in favor of p2pool for me. I left p2pool because I had I/O issues and likes to test novelties, but now bitcoind and p2pool are both on SSDs and they are just flying (at least when I'm not rewiring the whole apartment...).
My first SSD should be arriving soon. I'm seriously hoping it decreases my stale rate - like you, I think it's I/O related.
M
Have you got latest bitcoind from git? It's much faster than stable version, with new database. With latest bitcoind from git my GetWork Latency dropped about 5-10x times.
I'll try it on my server. Sounds risky for my main wallet, but main wallet isn't used for mining.
Thanks!
M
That's are my results, on Intel Atom 230 1.6GHz HT + 1 GB RAM:
It was after switching from bitcoind 0.7.1 to newest from git
November 17, 2012, 09:10:43 PM
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.
What might be underestimated too is the amount of work from nodes which stop mining after stale shares.
- p2pool.info luck,
- my efficiency,
- the average tx fee percentage.
This is why I'm back mining on p2pool: hashpower promises at least 105% PPS when leasing and falls short due to high rejects, I compared my actual PPS over several weeks and stats are largely in favor of p2pool for me. I left p2pool because I had I/O issues and likes to test novelties, but now bitcoind and p2pool are both on SSDs and they are just flying (at least when I'm not rewiring the whole apartment...).
My first SSD should be arriving soon. I'm seriously hoping it decreases my stale rate - like you, I think it's I/O related.
M
Have you got latest bitcoind from git? It's much faster than stable version, with new database. With latest bitcoind from git my GetWork Latency dropped about 5-10x times.
I'll try it on my server. Sounds risky for my main wallet, but main wallet isn't used for mining.
Thanks!
M
November 17, 2012, 08:46:39 PM
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.
What might be underestimated too is the amount of work from nodes which stop mining after stale shares.
- p2pool.info luck,
- my efficiency,
- the average tx fee percentage.
This is why I'm back mining on p2pool: hashpower promises at least 105% PPS when leasing and falls short due to high rejects, I compared my actual PPS over several weeks and stats are largely in favor of p2pool for me. I left p2pool because I had I/O issues and likes to test novelties, but now bitcoind and p2pool are both on SSDs and they are just flying (at least when I'm not rewiring the whole apartment...).
My first SSD should be arriving soon. I'm seriously hoping it decreases my stale rate - like you, I think it's I/O related.
M
Have you got latest bitcoind from git? It's much faster than stable version, with new database. With latest bitcoind from git my GetWork Latency dropped about 5-10x times.
November 17, 2012, 08:20:37 PM
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.
What might be underestimated too is the amount of work from nodes which stop mining after stale shares.
...
Block+Fees average is currently 100.36% according to blockchain.info
Thus you must be missing counting some work somewhere ...
November 17, 2012, 12:34:24 PM
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.
What might be underestimated too is the amount of work from nodes which stop mining after stale shares.
- p2pool.info luck,
- my efficiency,
- the average tx fee percentage.
This is why I'm back mining on p2pool: hashpower promises at least 105% PPS when leasing and falls short due to high rejects, I compared my actual PPS over several weeks and stats are largely in favor of p2pool for me. I left p2pool because I had I/O issues and likes to test novelties, but now bitcoind and p2pool are both on SSDs and they are just flying (at least when I'm not rewiring the whole apartment...).
My first SSD should be arriving soon. I'm seriously hoping it decreases my stale rate - like you, I think it's I/O related.
M
November 17, 2012, 12:16:50 PM
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.
What might be underestimated too is the amount of work from nodes which stop mining after stale shares.
- p2pool.info luck,
- my efficiency,
- the average tx fee percentage.
This is why I'm back mining on p2pool: hashpower promises at least 105% PPS when leasing and falls short due to high rejects, I compared my actual PPS over several weeks and stats are largely in favor of p2pool for me. I left p2pool because I had I/O issues and likes to test novelties, but now bitcoind and p2pool are both on SSDs and they are just flying (at least when I'm not rewiring the whole apartment...).
November 17, 2012, 11:25:25 AM
So back to my original statement then.
The pool hash rate reported is below the actual hash rate of the miners (like all pools)
However, since p2pool has 60x the number of LP's compared to other pools, the actual hash rate is quite a bit higher than reported - whereas with a normal pool it shouldn't be more than about 1% higher (I typically get <0.4% for GPU/BFL/ICA and ~0.7% for MMQ)
Yet in those ignored shares, if there is a valid Block, it will of course get through, so rather than the number of blocks being representative of the number of valid shares, it is in fact representative of the number of all work generated by all p2pool miners which is of course higher than the reported hash rate.
Work done during that time is counted as DOA shares, as cabin said:The pool hash rate reported is below the actual hash rate of the miners (like all pools)
However, since p2pool has 60x the number of LP's compared to other pools, the actual hash rate is quite a bit higher than reported - whereas with a normal pool it shouldn't be more than about 1% higher (I typically get <0.4% for GPU/BFL/ICA and ~0.7% for MMQ)
Yet in those ignored shares, if there is a valid Block, it will of course get through, so rather than the number of blocks being representative of the number of valid shares, it is in fact representative of the number of all work generated by all p2pool miners which is of course higher than the reported hash rate.
One thing that might mitigate some of this is that p2pool does send a header that tells the miner to submit all work, even if it is considered dead right after a long poll. This work should show up in the number of DOA shares the pool reports. So we might be counting most of the hash rate that others pools miss right after a long poll.
So when people are reporting that p2pool is getting 110% luck consistently (which of course isn't possible - the probability of getting 110% over even 1 month is excessively unlikely) they are in fact not comparing the correct numbers.
Yes p2pool may well now be averaging 100% as expected, but those > 100% numbers are the result of not comparing the correct numbers
No, the probability of us getting >110% luck for a given month is 21% (assuming we should get 2.3 blocks/day and a month has 30 days).Yes p2pool may well now be averaging 100% as expected, but those > 100% numbers are the result of not comparing the correct numbers
Code:
In[23]:= expected = 2.3*30;
In[24]:= 1 - CDF[PoissonDistribution[expected], expected*1.1] // N
Out[24]= 0.214626
In[24]:= 1 - CDF[PoissonDistribution[expected], expected*1.1] // N
Out[24]= 0.214626
Gyver, not quite. Share have a flag that indicates whether the node that generated it got a stale share in the past, and from those, nodes can compute the stale share percentage and then the total hash rate with an expression like: unstale_hash_rate / (1 - pool_stale_proportion)
Is it just a flag or a counter? We have whole network stats for DOA and Orphans too, are there 2 flags/counters?With a counter the stale estimation should be pretty good. If it's a flag, multiple stales between shares would be miscounted as an unique stale.
What might be underestimated too is the amount of work from nodes which stop mining after stale shares.
November 17, 2012, 09:59:45 AM
The correlation coefficient between hashrate per round and shares per round on D is 0.08. That means no significant correlation at all. Hashrate does not correlate with round length in a linear way. Plotting one against the other doesn't really show any pattern at all. It looks like a cloud of mosquitos.
Unfortunately the data only goes back to August - is there access to all rounds somewhere?
Unfortunately the data only goes back to August - is there access to all rounds somewhere?
If you are getting them from my site, you have to add a querystring parameter to get all data since I changed the main page to only show 90 days worth to improve routine performance:
http://p2pool.info/blocks?all=true
Jump to: