Topic: Decentralized mining protocol standard: getblocktemplate (ASIC ready!) - page 8. (Read 32344 times)
October 20, 2012, 06:11:22 PM
I do intend to improve BFGMiner's GBT implementation such that it transparently combines pooled and solo mining, at least.
October 20, 2012, 05:10:49 PM
How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?
That's an idea... have to consult with Luke on that.I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me.
GBT is pretty scalable for now.Between using Stratum and GBT at the moment that clearly says to use Stratum
October 20, 2012, 04:54:51 PM
No, it's quite alright. I have enough to do myself as well and I'm not in the business of helping pool designs. I'll implement it as is and let the mining community determine which pools and with what protocols they prefer.
October 20, 2012, 04:39:43 PM
How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?
That's an idea... have to consult with Luke on that.I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me.
GBT is pretty scalable for now. October 20, 2012, 04:37:43 PM
slush, not a bad idea but I'm actually not questioning stratum here since I obviously think it's the better protocol for pooled mining as I've already implemented it in cgminer. It's just that this is the GBT thread, I have been asked to include GBT support as well and I'm not really happy with the protocol as is.
October 20, 2012, 04:27:05 PM
How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?
How about adding an extension to Stratum that has some kind of mode that sends "source transactions" for given job?
In that way, paranoid users will have a chance to introspect jobs generated by Stratum pools and common users which don't care will have optimized mining protocol.
And yes, I'm working on it already. Important thing in this extension is that list of transactions can be broadcasted "lazily" (=few miliseconds after job updates), because that list is not important for mining itself. So it won't hurt efficiency of mining, but still give a tool for online block template monitoring.
October 20, 2012, 04:24:32 PM
How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?
That's an idea... have to consult with Luke on that.
I also wanted to point out that if we (or someone anyway) doesn't build a scalable solution per pooled mining with an integrated node... then nobody will be able to include it in the POS machines that will eventually be developed, and missing the chance for millions of additional nodes seems sad to me.
October 20, 2012, 04:05:19 PM
Sigh. In that case, with p2pool being 400 GH of a 20TH network, then GBT is only going to benefit 2% of the miners. It's pretty obvious by now you can't force people to do what you think is best. If anything, GBT with non-p2pool pooled mining will make it more likely that pools will not want to pass on lots of transactions and will be worse than getwork.
How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?
How about adding an extension to GBT that has some kind of mode that sends a "suggested merkle root" for pooled mining?
October 20, 2012, 12:23:33 PM
1) In general, everybody should be running a full node, just to support the network. It's not a requirement, obviously, but we need as many full nodes as possible to keep the network resistant to Sybil attacks and such.
2) p2pool is the preferred solution for the community, with centralized pools being second best. Centralized pools make it much easier to be a miner, at the cost of centralizing network power in the hands of the pool operator.
October 20, 2012, 02:05:49 AM
The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.
Suddenly mining includes bitcoin network transaction and block processing.
... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.
Suddenly mining includes bitcoin network transaction and block processing.
... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.
Complication like "staying online" is complicated?
This is so much of a non-issue, it's just silly.
Miners SHOULD be running a node even if they aren't mining on it. Depending on the pool for information that can be pulled from bitcoind isn't smart. Having to trust the pool for transaction and block processing isn't smart either. Those add unneeded complexity to an already overcomplicated system.
In fact I think pools in general come at mining form the wrong direction. You don't need to develop a way to hand out work where you send it to the miner and they report back with the completed work - what you ought to be doing is making work assignment system where each miner generates, validates and submits the work for the pool - while transmitting just the nonce (or even better a nonce range completed) back to the pool.
Every miner would include a msg from them to the pool stating they mined the block. And now you've got decentralized and much lower bandwidth. And you aren't trusting the pool for what it may or may not be doing with your hash rate.
And this is just off the top of my head --- start thinking about 'how it should be' instead of 'how it is right now' --- and start developing in that direction also.
The most important is yet again someone telling everyone how they should be mining.
"how it should be"
I'm glad you feel that is how it should be.
"Miners SHOULD be running a node"
Again, glad you feel that's how it should be.
Lucky for everyone else, they have freedom of choice and are not under your control.
Even luckier for you - you can do this already for yourself today - without trying to force everyone else to do it.
--
Meanwhile, your first statement is the only one in this conversation that is "just silly"
Ignoring the blatantly obvious issues and pretending to simplify to a stupid statement of "staying online"
bitcoind cannot be used except for solo mining (or p2pool) - as I mentioned, there needs work done to support a combination of pool GBT mining and bitcoind to supply the transactions.
i.e. miners implementing bitcoind yet again is just a problem waiting to happen.
... and that adds complexity, it doesn't reduce it at all.
Do you actually understand what txn and block processing entails?
Then of course there's the fact that you need to run a high performance bitcoind to ensure maximum work success - yeah everyone has one of those ... you think that may be what most pools spend a lot of effort doing?
As for bandwidth, if you really want to reduce it, how is adding a bitcoind into the equation going to do that?
Go stratum - problem solved
October 20, 2012, 01:12:50 AM
Sorry but a lot of miners mine without their mining rigs being connected to a bitcoind instance. Yes there may be one somewhere in their network, or possibly not at all and they only fire up their wallet when they want to on a separate network. What you think is ideal is asking people to do more than they currently do. They currently use pools and they trust them already. The community is very vigilant with respect to pools misbehaving. The importance of the miner doing it all themselves is grossly exaggerated by proponents of GBT. Pools will continue to do as much as they already do. It's unrealistic to expect otherwise.
October 20, 2012, 01:08:37 AM
The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.
Suddenly mining includes bitcoin network transaction and block processing.
... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.
Suddenly mining includes bitcoin network transaction and block processing.
... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.
Complication like "staying online" is complicated?
This is so much of a non-issue, it's just silly.
Miners SHOULD be running a node even if they aren't mining on it. Depending on the pool for information that can be pulled from bitcoind isn't smart. Having to trust the pool for transaction and block processing isn't smart either. Those add unneeded complexity to an already overcomplicated system.
In fact I think pools in general come at mining form the wrong direction. You don't need to develop a way to hand out work where you send it to the miner and they report back with the completed work - what you ought to be doing is making work assignment system where each miner generates, validates and submits the work for the pool - while transmitting just the nonce (or even better a nonce range completed) back to the pool.
Every miner would include a msg from them to the pool stating they mined the block. And now you've got decentralized and much lower bandwidth. And you aren't trusting the pool for what it may or may not be doing with your hash rate.
And this is just off the top of my head --- start thinking about 'how it should be' instead of 'how it is right now' --- and start developing in that direction also.
October 20, 2012, 12:20:08 AM
So what are you anticipating the pools vs miners will be doing with transactions in pooled mining (ignoring p2pool)?
Transactions don't need to make it to the miners, just the pool server. The pool server can handle most of the merkle building bits, and verify the returned work from the miner.
The miner runs through nonce.
If miner needs more work, it runs through ntime range.
If miner needs more work, it increments extranonce.
That will keep your ASIC miner mining through the next century ;p
In which case, we're back to my original comment... the miners need all the transactions every so often or they're doing bitcoin a disservice.
October 20, 2012, 12:08:54 AM
So what are you anticipating the pools vs miners will be doing with transactions in pooled mining (ignoring p2pool)?
Transactions don't need to make it to the miners, just the pool server. The pool server can handle most of the merkle building bits, and verify the returned work from the miner.
The miner runs through nonce.
If miner needs more work, it runs through ntime range.
If miner needs more work, it increments extranonce.
That will keep your ASIC miner mining through the next century ;p
October 19, 2012, 11:00:28 PM
The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.
No, that is not correct at all.
The pool server may supply sufficient information to modify extranonce in the coinbase txn, and update the merkle root. Nothing more.
...
October 19, 2012, 09:57:20 PM
What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.
Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves. See https://en.bitcoin.it/wiki/BIP_0023
I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security.
But don't dismiss GBT based on faulty "full mode only" assumptions.
October 19, 2012, 08:18:31 PM
The problem with not receiving the transactions is that the miner program must have a full bitcoin connection to the network.
No, that is not correct at all.
The pool server may supply sufficient information to modify extranonce in the coinbase txn, and update the merkle root. Nothing more.
And even that's going over the absolute minimum requirement of simply a block header, with ntime rolling supported.
October 19, 2012, 08:03:24 PM
What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.
Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves. See https://en.bitcoin.it/wiki/BIP_0023
I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security.
But don't dismiss GBT based on faulty "full mode only" assumptions.
Suddenly mining includes bitcoin network transaction and block processing.
... and anyone with half an inkling of understanding of what that entails can see the issues of having to deal with all the complications that entails and then also the importance of the bitcoin network performance of the miner ... which before this point was zero.
The experiment of the issues with this already exists
P2Pool.The only really acceptable solution to this problem, in my opinion, would be to have bitcoind handle that fully for a GBT miner - not have yet another bitcoin network implementation added onto every miner.
But this still has the issue of the reliability of the local bitcoin network connection and the problems that can cause (that don't exist in the current implementations other than p2pool and solo mining)
October 19, 2012, 07:17:16 PM
What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them.
Unless something has changed, getblocktemplate should include a mode that skips sending the transactions themselves. See https://en.bitcoin.it/wiki/BIP_0023
I agree it is unlikely that non-p2pool miners will use the full bore GBT + all transactions, even if it is better for network security.
But don't dismiss GBT based on faulty "full mode only" assumptions.
October 19, 2012, 07:09:05 PM
An empty (coinbase-only block) StratumMP work payload is 399 bytes. An empty GBT work payload based on your example is 737 bytes.
StratumMP work payloads increase 67 bytes per power of 2 transactions. GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address).
For StratumMP, a block containing 257-512 transactions would require an extra 603 bytes, for a total of 1002 bytes. GBT would require approximately 128,000 bytes.
Using a more common example, lets say each block has 100 transactions on average. StratumMP would be 868 bytes per work payload. The GBT equivalent would be approximately 26,000 bytes.
Similarly, work submission payloads are quite different. GBT would require 501 bytes per share submission in your example. StratumMP requires only 109.
None of the above factors in the added overhead of wrapping GBT into HTTP Requests.
What a mess. With payloads increasing linearly with transaction count in GBT that makes each getwork equivalent much bigger than plain getwork already is the longer a block takes to solve. As bitcoin becomes more popular and transaction counts increase, gbt becomes less and less efficient. I'd hazard a guess that it may even be less efficient than plain getwork at some stage unless pools start ignoring transactions... there's a disincentive to include them. StratumMP work payloads increase 67 bytes per power of 2 transactions. GBT work payloads increase by the size of the transactions being included in a block (roughly 250 bytes for a normal tx with 1 input, 1 payout address, and 1 change address).
For StratumMP, a block containing 257-512 transactions would require an extra 603 bytes, for a total of 1002 bytes. GBT would require approximately 128,000 bytes.
Using a more common example, lets say each block has 100 transactions on average. StratumMP would be 868 bytes per work payload. The GBT equivalent would be approximately 26,000 bytes.
Similarly, work submission payloads are quite different. GBT would require 501 bytes per share submission in your example. StratumMP requires only 109.
None of the above factors in the added overhead of wrapping GBT into HTTP Requests.
Jump to: