Topic: [1500 TH] p2pool: Decentralized, DoS-resistant, Hop-Proof pool - page 629. (Read 2591920 times)
November 02, 2012, 10:32:01 AM
What does "Punishing share for 'Block-stale detected!" mean? I gather it happens after a new block appears on the network.. but what is the punishment?
November 01, 2012, 10:34:21 PM
I'm running Win7 64bit with 12gb of RAM and a 3 core 3.0ghz processor and P2Pool is always losing connection to bitcoind. What can I do to keep bitcoin running properly?
edit I created a symbolic link to the Bitcoin folder in appdata and moved the blockchain stuff to a faster drive in my computer. I'll see if this helps.
I have linux VM w/2GB ram running two p2pools and 5 xCoind`s w/o any trouble edit I created a symbolic link to the Bitcoin folder in appdata and moved the blockchain stuff to a faster drive in my computer. I'll see if this helps.
Try exclude bitcoin and p2pool directory form virus scanning.
Excluding from the virus scanner seems to have helped. It must have been rescanning the whole blockchain every time a new block was found.
November 01, 2012, 03:52:53 PM
Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
You could easily roll something in the coinbase script, yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical.
Why can't it do it every time it completes a nonce range?
From the point of view of the merkle tree, it's only one side that needs to be recalculated (unless you are adding more transactions or using different transactions)
Again I don't see what I am missing here that makes this impossible.
Given what else was said and assuming the other issues can be worked out stratum may be a better choice then, mostly because of bandwidth. GBT and Stratum both offer some modification to the coinbase (GBT through "coinbase/append" flag and Stratum through how the coinbase is always built to identify each miner).
Maybe it would be worth adding a "transaction/compact" flag to the GBT request and allow it to respond with just the merkle branches instead of the full transactions. If my logic isn't too far off that would close the gab between GBT and Stratum based on bandwidth usage.
If you implement it in p2pool, p2pool itself is also adding roll-n-time to the miner on top of what it is doing so the miner doesn't have to get work every nonce range.
Stratum block building is done by the miner, in p2pool it is done by the p2pool software only - so yeah the miner wont know the p2pool rule to add to Stratum block building.
So results so far is that p2pool is forced to use the roll-n-time hack - it has no other way to give out long term work to the miner with it's current design.
... you could throw some BTC to ckolivas to request a change to cgminer to support the p2pool rule addition to the stratum generation process ...
... an optional flag that p2pool sends to say it needs to add that extra change to the coinbase transaction generation
(Edit: of course a larger nonce in the block header would solve that
) 
November 01, 2012, 02:46:13 PM
Might as well just use Stratum in that case, as you've negated the main benefit of GBT (decentralization).
As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool
But if you're just using Stratum for communication between p2pool and your mining software, centralization isn't a problem.As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool
If you're mining on someone else's P2Pool instance then you're already becoming more centralized.
Not if it was using GBT. But obviously with p2pool's 10 second altchain blocks, the protocol used by GBT today is unreasonable.It does defeat some of the ideological niceties of GBT, but it's still an optional flag (Stratum has similar from what I've heard, though it's off by default, instead of on by default with this proposed compact flag).
No, it's currently not possible to use GBT in a centralized setup. Stratum has recently added a similar feature, but it still has overhead for decentralized use. 
November 01, 2012, 02:30:26 PM
Maybe it would be worth adding a "transaction/compact" flag to the GBT request and allow it to respond with just the merkle branches instead of the full transactions. If my logic isn't too far off that would close the gab between GBT and Stratum based on bandwidth usage.
Might as well just use Stratum in that case, as you've negated the main benefit of GBT (decentralization).As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool
It does defeat some of the ideological niceties of GBT, but it's still an optional flag (Stratum has similar from what I've heard, though it's off by default, instead of on by default with this proposed compact flag).
November 01, 2012, 02:06:05 PM
Might as well just use Stratum in that case, as you've negated the main benefit of GBT (decentralization).
As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool
But if you're just using Stratum for communication between p2pool and your mining software, centralization isn't a problem. As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool
November 01, 2012, 02:03:23 PM
Maybe it would be worth adding a "transaction/compact" flag to the GBT request and allow it to respond with just the merkle branches instead of the full transactions. If my logic isn't too far off that would close the gab between GBT and Stratum based on bandwidth usage.
Might as well just use Stratum in that case, as you've negated the main benefit of GBT (decentralization).As long as we're talking p2pool though, might as well use another pool entirely, since decentralization is the main benefit of p2pool
November 01, 2012, 01:42:55 PM
Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
You could easily roll something in the coinbase script, yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical.
Why can't it do it every time it completes a nonce range?
From the point of view of the merkle tree, it's only one side that needs to be recalculated (unless you are adding more transactions or using different transactions)
Again I don't see what I am missing here that makes this impossible.
Given what else was said and assuming the other issues can be worked out stratum may be a better choice then, mostly because of bandwidth. GBT and Stratum both offer some modification to the coinbase (GBT through "coinbase/append" flag and Stratum through how the coinbase is always built to identify each miner).
Maybe it would be worth adding a "transaction/compact" flag to the GBT request and allow it to respond with just the merkle branches instead of the full transactions. If my logic isn't too far off that would close the gab between GBT and Stratum based on bandwidth usage.
November 01, 2012, 09:09:09 AM
Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
You could easily roll something in the coinbase script, yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical.
Why can't it do it every time it completes a nonce range?
From the point of view of the merkle tree, it's only one side that needs to be recalculated (unless you are adding more transactions or using different transactions)
Again I don't see what I am missing here that makes this impossible.
November 01, 2012, 08:46:42 AM
Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
You could easily roll something in the coinbase script, yes, but it wouldn't result in a valid p2pool share. The miner would need to be smart enough to update the data in the last txout, which is impractical.
November 01, 2012, 08:40:36 AM
Um - it's the actual coinbase field itself - the same place where you put all that other random data.
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
If you can randomly throw merged mining data it there, you should be able to add a 4 byte fields in there that you can roll instead of the ntime field.
I'm not sure what I'm missing here?
November 01, 2012, 08:25:56 AM
The earlier argument (^) about remote miners not working was a bit of a farce, as shown by the above numbers.
The thing to focus on in order to make sure things work is the timestamp rolling support in ASIC mining software. We need to ensure that it can roll ahead of the current time (and potentially backwards, though that would require an extension to getwork).
...
Sorry - I don't get the point of this comment.The thing to focus on in order to make sure things work is the timestamp rolling support in ASIC mining software. We need to ensure that it can roll ahead of the current time (and potentially backwards, though that would require an extension to getwork).
...
Why do you need rolling the time?
Unless I've completely missed your point, you don't need to roll the time with Stratum or GBT, you roll a value in the coinbase instead, named (IMO badly) the secondary nonce instead of screwing with the block timestamp (which is a hack) unnecessary
P2Pool can't handle simply altering the coinbase script. P2Pool internally has a structure that describes how to recreate the coinbase transaction, including the coinbase script, but that can't be changed because the hash of that structure is stuck in the last txout. P2Pool uses it to create short, self-contained proofs of work by storing the SHA256 midstate with shares and computing the
generation transaction's hash using that midstate and the hash of that internal structure.
TL;DR: If the coinbase script is altered, the hash in the last txout of the generation transaction has to be changed too.
However, I discovered yesterday that Stratum is flexible enough to roll any data in the coinbase transaction at all, though, so rolling something in the last txout script is probably possible (though it would require a change to P2Pool's rules).
November 01, 2012, 06:52:38 AM
It requires sending the entire block template (potentially up to 1 MB) to miners every 10 seconds, which is definitely impractical for remote miners. An extension to GBT that allowed only sending the merkle root would avoid this, though.
Sending only the merkle root, whether it be via getwork, Stratum, or some new GBT extension, makes the pool centralized, and kinda defeats the entire point of p2pool... Perhaps it would make sense for "remote" p2pool servers to be the first to move forward with mandatory-miner-provides-the-transactions GBT?Are you arguing that GBT isn't centralised because you HAVE to receive the full transaction list?
That optionally receiving it isn't good enough? Seriously?
Yeah forrestv it seems that you need to take his hidden agenda comments with a grain of salt.
His agenda is to promote GBT no matter what problems there are with it, and to use dictionary arguments to avoid transparency.
November 01, 2012, 06:43:28 AM
The main problem I see coming is that even the slowest announced ASIC (BFL's Jalapeno) can go faster than one getwork per second. Most devices are a lot faster. Pulling the work away from the pools and closer to the actual device is a decent looking way to deal with that. I'm not aware of any huge issues with timestamp rolling as long as they can get enough getwork blocks, especially after a longpoll when they need more new pieces of work to start working on the new block (small surges of getworks every 10 seconds when a block comes out seems excessive).
On my desktop, P2Pool can supply ~130 getworks/second, or enough work for 520 GH/s (130/s * 4 GH), without any timestamp rolling. With timestamp rolling one minute backwards and forwards, it can supply 62 TH/s (520 GH/s * 120) of work, or 480 GH/s (1/s * 4 GH * 120) of work at a rate of one getwork per second.
I don't think ASICs will need any special support. P2Pool can provide getwork results fast enough for hundreds of GH/s (from a normal computer) and could be optimized for more. In addition, any timestamp rolling multiplies that.
What about mining on a remote node? It seems like ASICs could kill P2P Mining.The thing to focus on in order to make sure things work is the timestamp rolling support in ASIC mining software. We need to ensure that it can roll ahead of the current time (and potentially backwards, though that would require an extension to getwork).
...
Why do you need rolling the time?
Unless I've completely missed your point, you don't need to roll the time with Stratum or GBT, you roll a value in the coinbase instead, named (IMO badly) the secondary nonce instead of screwing with the block timestamp (which is a hack) unnecessary
October 31, 2012, 11:15:19 PM
The main problem I see coming is that even the slowest announced ASIC (BFL's Jalapeno) can go faster than one getwork per second. Most devices are a lot faster. Pulling the work away from the pools and closer to the actual device is a decent looking way to deal with that. I'm not aware of any huge issues with timestamp rolling as long as they can get enough getwork blocks, especially after a longpoll when they need more new pieces of work to start working on the new block (small surges of getworks every 10 seconds when a block comes out seems excessive).
On my desktop, P2Pool can supply ~130 getworks/second, or enough work for 520 GH/s (130/s * 4 GH), without any timestamp rolling. With timestamp rolling one minute backwards and forwards, it can supply 62 TH/s (520 GH/s * 120) of work, or 480 GH/s (1/s * 4 GH * 120) of work at a rate of one getwork per second.
I don't think ASICs will need any special support. P2Pool can provide getwork results fast enough for hundreds of GH/s (from a normal computer) and could be optimized for more. In addition, any timestamp rolling multiplies that.
What about mining on a remote node? It seems like ASICs could kill P2P Mining.The thing to focus on in order to make sure things work is the timestamp rolling support in ASIC mining software. We need to ensure that it can roll ahead of the current time (and potentially backwards, though that would require an extension to getwork).
I looked at GBT, and I don't see it improving anything. It requires sending the entire block template (potentially up to 1 MB) to miners every 10 seconds, which is definitely impractical for remote miners. An extension to GBT that allowed only sending the merkle root would avoid this, though.
It may be easy enough to just give out work that has a 1 minute old timestamp and just allow 2 minutes of rolltime.
October 31, 2012, 08:44:15 PM
It requires sending the entire block template (potentially up to 1 MB) to miners every 10 seconds, which is definitely impractical for remote miners. An extension to GBT that allowed only sending the merkle root would avoid this, though.
Sending only the merkle root, whether it be via getwork, Stratum, or some new GBT extension, makes the pool centralized, and kinda defeats the entire point of p2pool... Perhaps it would make sense for "remote" p2pool servers to be the first to move forward with mandatory-miner-provides-the-transactions GBT? 
October 31, 2012, 08:27:45 PM
I looked at GBT, and I don't see it improving anything. It requires sending the entire block template (potentially up to 1 MB) to miners every 10 seconds, which is definitely impractical for remote miners. An extension to GBT that allowed only sending the merkle root would avoid this, though.
Yes we're debating that very point at length in the GBT thread at the moment, as I do not see the advantage of that either, only potential disadvantage of increasing network bandwidth, or worse, encouraging not perpetuating transactions. Stratum on the other hand uses merkle branches to negate this effect entirely and the bandwidth is virtually static regardless of the miner's hashrate or transaction count. October 31, 2012, 08:07:26 PM
The main problem I see coming is that even the slowest announced ASIC (BFL's Jalapeno) can go faster than one getwork per second. Most devices are a lot faster. Pulling the work away from the pools and closer to the actual device is a decent looking way to deal with that. I'm not aware of any huge issues with timestamp rolling as long as they can get enough getwork blocks, especially after a longpoll when they need more new pieces of work to start working on the new block (small surges of getworks every 10 seconds when a block comes out seems excessive).
On my desktop, P2Pool can supply ~130 getworks/second, or enough work for 520 GH/s (130/s * 4 GH), without any timestamp rolling. With timestamp rolling one minute backwards and forwards, it can supply 62 TH/s (520 GH/s * 120) of work, or 480 GH/s (1/s * 4 GH * 120) of work at a rate of one getwork per second.
I don't think ASICs will need any special support. P2Pool can provide getwork results fast enough for hundreds of GH/s (from a normal computer) and could be optimized for more. In addition, any timestamp rolling multiplies that.
What about mining on a remote node? It seems like ASICs could kill P2P Mining.The thing to focus on in order to make sure things work is the timestamp rolling support in ASIC mining software. We need to ensure that it can roll ahead of the current time (and potentially backwards, though that would require an extension to getwork).
I looked at GBT, and I don't see it improving anything. It requires sending the entire block template (potentially up to 1 MB) to miners every 10 seconds, which is definitely impractical for remote miners. An extension to GBT that allowed only sending the merkle root would avoid this, though.
October 31, 2012, 07:47:34 PM
It's kind of hard to test with something that you're not even sure exists... 
Anyway, with sane nTime rolling and adaptive pseudoshare targets, getworks can provide more than enough work to remote hosts with very low bandwidth. Everything for nTime rolling is already there (the HTTP header), and adaptive targets are disabled temporarily, but I'll re-enable them for the next release.
I really don't see any issue with getwork. Why is GBT necessary?
P2Pool can handle reward halving.
EDIT: Getwork does have some problems with timestamp rolling. Depending on how ASIC miners handle it, it could work, but I'll start working on GBT support to allow timestamp rolling.
The main problem I see coming is that even the slowest announced ASIC (BFL's Jalapeno) can go faster than one getwork per second. Most devices are a lot faster. Pulling the work away from the pools and closer to the actual device is a decent looking way to deal with that. I'm not aware of any huge issues with timestamp rolling as long as they can get enough getwork blocks, especially after a longpoll when they need more new pieces of work to start working on the new block (small surges of getworks every 10 seconds when a block comes out seems excessive). Anyway, with sane nTime rolling and adaptive pseudoshare targets, getworks can provide more than enough work to remote hosts with very low bandwidth. Everything for nTime rolling is already there (the HTTP header), and adaptive targets are disabled temporarily, but I'll re-enable them for the next release.
How hard would it be to implement GBT for pyraminingP2Pool? Seems like that would mitigate a few issues for remote miners anyway.
While I'm asking, is the P2Pool codebase setup for the block reward halving already? (I'd imagine so, but it gets messy otherwise).
Edit: Pyramining is already setup how they want to and will probably continue in the same. GBT support for p2pool was what I meant to ask about.
While I'm asking, is the P2Pool codebase setup for the block reward halving already? (I'd imagine so, but it gets messy otherwise).
Edit: Pyramining is already setup how they want to and will probably continue in the same. GBT support for p2pool was what I meant to ask about.
I really don't see any issue with getwork. Why is GBT necessary?
P2Pool can handle reward halving.
EDIT: Getwork does have some problems with timestamp rolling. Depending on how ASIC miners handle it, it could work, but I'll start working on GBT support to allow timestamp rolling.
October 31, 2012, 07:41:49 PM
I don't think ASICs will need any special support. P2Pool can provide getwork results fast enough for hundreds of GH/s (from a normal computer) and could be optimized for more. In addition, any timestamp rolling multiplies that.
What about mining on a remote node? It seems like ASICs could kill P2P Mining.Any serious P2Pool user, one serious enough to have ASICs, should definitely have their own P2Pool node...
Because of course no one uses a remote p2pool node for a failover backup. /sarcasm
Sounds like no real testing has been done to verify it will not be an issue.
I'm starting to get the feeling that you are sticking your head in the sand and hoping that everything will just work.
Well... a while ago, pyramining briefly switched over to my public node while they were working on some stuff.
I ended up having between 200-400GH/s at the time, and everything seemed to go just fine with my bandwidth, so p2pmining should be ok with the added bandwidth of the ASIC traffic.
If p2pmining would let you use higher diff shares, or maybe they can set up a high-hash node too. They're pretty much doing their own sub-share-chain, so maybe they will do something to accommodate the ASICs.
Not sure if they would tho. It sounded like they came into existence mainly to cater to the smaller miner who needed lower-diff shares.
-- Smoov
ps: Krak is in one of the outlying cities/towns that don't have a lot of infastructure to spread around. Rural-ish kind of area. That's where his main bandwidth bottleneck is. They get priced accordingly. Too much demand, not enough supply. So they get capped more.
I ended up having between 200-400GH/s at the time, and everything seemed to go just fine with my bandwidth, so p2pmining should be ok with the added bandwidth of the ASIC traffic.
If p2pmining would let you use higher diff shares, or maybe they can set up a high-hash node too. They're pretty much doing their own sub-share-chain, so maybe they will do something to accommodate the ASICs.
Not sure if they would tho. It sounded like they came into existence mainly to cater to the smaller miner who needed lower-diff shares.
-- Smoov
ps: Krak is in one of the outlying cities/towns that don't have a lot of infastructure to spread around. Rural-ish kind of area. That's where his main bandwidth bottleneck is. They get priced accordingly. Too much demand, not enough supply. So they get capped more.
For a quick answer. But the longer answer is still that getwork is anticipated not to work as well with ASICs. Hence the several new implementations out there to address this GBT and Stratum are the main two that I've seen.
Jump to: