Topic: Handle much larger MH/s rigs : simply increase the nonce size - page 2. (Read 10083 times)
kano is right. I did encounter this bandwidth issue, its somehow the same annoyance for hardware developers as it was diff1 and multi GHs miners network bandwidth consumption for pool operators. Unfortunately, solving hardware bandwidth issues is not as simple as a workaround in software, most of the time it requires changing design and adding extra costs.
Quote
So ... where is extra nonce referred to in the original Bitcoin document by Satoshi? Oh it isn't.
The same place scripts are referenced as well as locktime and nbits.Have even ever read "Bitcoin: A Peer-to-Peer Electronic Cash System"?
… The paper was written something like a year before the publication of the system. It covers the core concepts but omits things such as the script system, which is easily the most complicated part of a full node implementation. Since the paper covers _very_ little of the actual implementation, including a number of critical design decisions (I provided some examples of uncovered things), it would have been surprising if it _had_ mentioned extranonce.
Quote
Yes I guess for someone like you it is difficult to only associate BIP 22 with the thread title.
I'm not sure how you could associate the thread title with BIP 34 or 35, but that confusion is (in your opinion) 'having a clue'
Because:I'm not sure how you could associate the thread title with BIP 34 or 35, but that confusion is (in your opinion) 'having a clue'
(1) None of them were about ASIC mining. So I had to assume you were confused, and given that you are confused all bets are off.
(2) BIP 22 is primarily documenting (and cleaning up) an API we've had since long before 0.7; we merged it from forrestv in Septemberish 2011. So your talk about new in 0.7 more or less precluded it.
Generating a merkle root is exactly the same operation as the ASIC is specifically designed for already (double SHA256). No need to embed a CPU on the ASIC (an ASIC is a CPU, but one for a very specific purpose only).
That doesn't mean they should - it can be done at any stage (in the pool, in the computer controlling the asic, in some intermediate controller chip, ... I don't care). The point is that this is not hard; it's only different from how the current infrastructure works.
While the merkle root does use double SHA256, the optimizations are different. Where dsha(h) = SHA256(SHA256(h)), the merkle root is dsha(dsha(A+B) + dsha(C+C)) (for three transactions, one of which is the generation transaction). And the block hash is a highly optimized (for fewer operations which leads to quicker completion and higher speeds) down to precomputing the first round of SHA256, and running through one of two rounds of the inner iteration of SHA256 (the other being precomputed in the previous step) and the first 94% of the second iteration (the last 4 rounds don't matter).That doesn't mean they should - it can be done at any stage (in the pool, in the computer controlling the asic, in some intermediate controller chip, ... I don't care). The point is that this is not hard; it's only different from how the current infrastructure works.
TL;DR on that - merkle roots are a normal hash, insert data get answer. Block hashing wants a hash with a certain property and tries over an over with slightly different starting conditions to get a "special" end result. One cares about getting an answer for a specific input, one wants a specific output from a generic form of the input.
Also, because of several factors, it's a lot harder to make the merkle root generation in silicon (I'm sure it can be done, there's just not as much point, you can make them much more easily than hashing them. A few hundred hashes versus a few billion).
Also, an ASIC is by defenition Application specific. While you could make one to generate merkle roots via serial input, it'd be a lot of work for not a lot of gain. A CPU just does whatever you tell it to do. Generally on an embedded device like a mining rig a firmware update (more generally, a change in software) could completely change what the CPU is doing (for instance change how the merkle root is generated based on a new BIP), but the ASIC is stuck doing the exact same thing that it's orignally hardwired to do until it breaks.
Again to suggest a mining rig with fairly simple setup of plugging into ethernet and power cords (and configuring to a pool) could use a microprocessor to do all the work (talking to the network, deciding what transactions to include) EXCEPT crunching blocks which would be left to the ASIC part of the chip.
I have nothing to add, I'm just quoting this gem for the future reference.
Generating a merkle root is exactly the same operation as the ASIC is specifically designed for already (double SHA256). No need to embed a CPU on the ASIC (an ASIC is a CPU, but one for a very specific purpose only).
That doesn't mean they should - it can be done at any stage (in the pool, in the computer controlling the asic, in some intermediate controller chip, ... I don't care). The point is that this is not hard; it's only different from how the current infrastructure works.
That doesn't mean they should - it can be done at any stage (in the pool, in the computer controlling the asic, in some intermediate controller chip, ... I don't care). The point is that this is not hard; it's only different from how the current infrastructure works.
Generating a merkle root is exactly the same operation as the ASIC is specifically designed for already (double SHA256). No need to embed a CPU on the ASIC (an ASIC is a CPU, but one for a very specific purpose only).
That doesn't mean they should - it can be done at any stage (in the pool, in the computer controlling the asic, in some intermediate controller chip, ... I don't care). The point is that this is not hard; it's only different from how the current infrastructure works.
That doesn't mean they should - it can be done at any stage (in the pool, in the computer controlling the asic, in some intermediate controller chip, ... I don't care). The point is that this is not hard; it's only different from how the current infrastructure works.
...
Quote it - coz it isn't there.Quote
So ... where is extra nonce referred to in the original Bitcoin document by Satoshi? Oh it isn't.
The same place scripts are referenced as well as locktime and nbits.Have even ever read "Bitcoin: A Peer-to-Peer Electronic Cash System"?
I don't know the code well enough to reference it, but here's a reference by Theymos on the wiki from December 2010, which at the very least predates anything fast on the network. Per http://bitcoin.sipa.be/ the network back then was under 100GH/s.
So yes, people have been worrying about 2^32 not being enough for a long time, however if you're able to spin the merkle root yourself it's not a problem. Or if you're able to get a new one with relatively low latency (ie. Not from a remote web server, ie. pool) every time, it's okay and you can hash away on it.
Nothing about spinning new merkle roots has to be in silicon, a small microprocessor (CPUs on my ASICs?!?!?) would work fine, and probably exists there for other reasons, although it may not have enough power to do anything useful like generate new merkle roots.
Kano, you're confused because you're only in Bitcoin for "free money". Someday, you'll need to realize "free money" is not really a goal for Bitcoin as a whole.
... lulz reality please.Your desire to control bitcoin and your self-righteous intolerance have long ago blinded you to the FUD you regularly spread.
BIP 22 doesn't have anything at all to do with the block validity rules, which is the one thing this is all about. It's about a client-specific way of making mining software interact with the daemon (but it can, and hopefully will, be adopted by other software too). You could replace the BIP22 implementation in the daemon you're using with any equivalent, and nothing would break - nobody would even notice - as long as you patch the software that interacts with it as well.
BIP 34 however does concern block validity rules. It's not directly related to mining, so maybe it is harder to see why it is relevant in this discussion.
BIP 34 however does concern block validity rules. It's not directly related to mining, so maybe it is harder to see why it is relevant in this discussion.
...
Quote it - coz it isn't there.Quote
So ... where is extra nonce referred to in the original Bitcoin document by Satoshi? Oh it isn't.
The same place scripts are referenced as well as locktime and nbits.Have even ever read "Bitcoin: A Peer-to-Peer Electronic Cash System"?
Quote
Quote
There were that many BIPs added in 0.7.0 that you couldn't work out which one I was referring to?
It's awfully hard to work out what you're talking about when you haven't the foggiest clue about what you're talking about yourself.I'm not sure how you could associate the thread title with BIP 34 or 35, but that confusion is (in your opinion) 'having a clue'
So I guess there's no point discussing this any further with someone as 'clued' in as you are
Voting for fixing root cause instead of patching problems...
IMHO, Kano argue about something interesting, about seeing much much farter in the future than we are used to.. all must learn from history..
meanwhile, I doubt ASICs or "miner will" are a reason to rush things..
Must be well planned, if concidered needed.. As I like to say :
What do we want ?
Evidence based change !
When do we want it ?
After peer review !
was my 2 satoshi !
IMHO, Kano argue about something interesting, about seeing much much farter in the future than we are used to.. all must learn from history..
meanwhile, I doubt ASICs or "miner will" are a reason to rush things..
Must be well planned, if concidered needed.. As I like to say :
What do we want ?
Evidence based change !
When do we want it ?
After peer review !
was my 2 satoshi !
Oh that's right, GBT related changes didn't happen in bitcoind - I forgot.
HUH? GBT is a rename of the poorly named getmemorypool which we've had for a long time— much longer than anyone was talking about 1TH/s mining devices— it's used by pool daemons and p2pool. Mostly for dynamic coinbase creation (e.g. to do fancy payouts) and because bitcoind coinbase creation is somewhat slow (it's synchronous, single threaded, and basically completely unoptimized). The changes made in 0.7 beyond the rename were some extra fields to remove a potential DOS attack where some of the limits couldn't be properly enforced when the caller does dynamic transaction selection, support for the height in the coinbase, and some general cleanups to make it more 'designed' than accreted. BIP 22 makes no mention of 'asic', nor do the pull requests for the changes, and I don't believe asics _ever_ came up in discussion related to it around the reference client's development.To the extent that GBT is useful for higher speed miners getmemorypool was absolutely as useful.
Quote
So ... where is extra nonce referred to in the original Bitcoin document by Satoshi? Oh it isn't.
The same place scripts are referenced as well as locktime and nbits.Quote
There were that many BIPs added in 0.7.0 that you couldn't work out which one I was referring to?
It's awfully hard to work out what you're talking about when you haven't the foggiest clue about what you're talking about yourself.
Quote from: slush
How about that huge possible optimizations on wire protocol between computer and device? Why not to optimize this at first time? You told that current wire protocols has problems with handling few block headers per second. If that's true, then you can get many orders of magnitude in performance just by optimizing it, without breaking anything in bitcoin.
Quote from: slush
USB 2.0 has teoretical speed 480 Mbit/s. Block header has ~200 bytes. I know I'm over-simplifying it now, but teoretically it is possible to transfer up to 300000 block headers per second to the miner even with prehistoric USB 2.0.
Maybe we should focus to fixing protocols in these devices than fix protocol in bitcoin network? Why should device for $30k USD use serial port communication designed in the middle of 20th century?
Maybe we should focus to fixing protocols in these devices than fix protocol in bitcoin network? Why should device for $30k USD use serial port communication designed in the middle of 20th century?
Once again, bolded especially for kano. Please respond to this.
Quote
If you are dealing with work at a nonce range level, then for each nonce range you are passing work to the device.
How many iterations is possible for one nonce range? 2^32? So 80 bytes per 2^32 hashes? Is that around 230 low-level "jobs" (nonce ranges) per one terahash per second? Isn't there teoretical wire limit 300000 nonce ranges per second for USB 2.0?
Quote
FPGA's that use serial over USB have the beginnings of this issue - as per: BFL, Icarus, ModMiner, Cainsmore.
... and the 2 companies that will be releasing their ASIC first are 2 of those, BFL and ModMiner, with Icarus not far behind.
... and the 2 companies that will be releasing their ASIC first are 2 of those, BFL and ModMiner, with Icarus not far behind.
Serial over USB is that obsolete technology which I was refering in my previous post. Looks like *there* is a space for improvements, *not* in bitcoin protocol.
Quote
The work arounds suggested imply implementing merlke root processing and coinbase generation processing in silicon ...
Nobody is proposing this.
Quote
Would it 'always' work or would it only work with current implementations and need changing in the not too distant future?
Yes. There're already technologies for feeding Petahashes per second to ASIC device. However device manufacturers will probably need to drop obsolete wire protocols and use something faster than Serial over USB.
Quote
All these problems and questions are actually resolved by simply increasing the size of the nonce field.
The cause of the problem, and the correct andsimplesthardest solution to the problem.
The cause of the problem, and the correct and
Let me fix it for you.
Quote
The only argument against it is a hard-fork
It looks to me that you live in some alternate universe. Hard fork is definitely possible and there can be scenarios when it become necessary, but ASIC miners are not that case.
*headdesk*
Kano, you're confused because you're only in Bitcoin for "free money". Someday, you'll need to realize "free money" is not really a goal for Bitcoin as a whole.
So if nonce size isn't too small - why are we changing bitcoin at all to support ASIC?
We are not and have not.
Quote
Quote
Why is this stupid BIP in 0.7.0 that's been forced on everyone, that also removed functionality from bitcoind, necessary?
Wtf are you talking about.
There were that many BIPs added in 0.7.0 that you couldn't work out which one I was referring to?
Oh well.
Quote
Quote
Why have people called sticking more junk in the coinbase an 'extra' nonce? What? They gave it the wrong name?
The current nonce is big enough? It's not necessary?
The current nonce is big enough? It's not necessary?
Extranonce has been part of the design since day one. It is the only way to make your coinbase transactions have unique IDs other than constantly changing to a new public key whenever you update the candidate block. It's fundamental and is required for things independent of the nonce size.
So ... where is extra nonce referred to in the original Bitcoin document by Satoshi? Oh it isn't.
I'd guess you like the Roll ntime hack also?
Quote
Quote
Coz when you get a block header to hash, you can only hash 2^32 times before needing to change it.
Why does that restriction exist?
Why does that restriction exist?
A _4 billion to 1_ work factor is pretty reasonable. There are a couple tradeoffs here. Header size is utterly critical for SPV nodes of various kinds, and even adding a single byte to the header increases the size by 1.25%. Having too much workfactor ratio between header only work and block update work encourages miners to build devices which never process transactions: Cheaper to just build a constant root with no transactions and increment the header until you solve a block. On a fresh design I may have given a bit more space to nonce, sure. But there is no need to change it now.
Read previous post
Quote
Quote
It's gonna work fine without any changes right?
Oh ... no, that's not correct
Oh ... no, that's not correct
Yes, in fact it will
Since they are hacks anyway, just throw them away then.
Quote
Quote
Where IS the restriction at the moment?
That we're not permitted to throw you in a volcano.
It would be a little strange a device that can process 4 billion hashes faster than you can give it the 32 bytes to hash.
+1
As I have already said, no company would be silly enough to do Stratum or the other thing in silicon
Stop spreading FUD. Nobody is talking about implementing Stratum/GBT directly in chip.
How about that huge possible optimizations on wire protocol between computer and device? Why not to optimize this at first time? You told that current wire protocols has problems with handling few block headers per second. If that's true, then you can get many orders of magnitude in performance just by optimizing it, without breaking anything in bitcoin.
Oh, I forgot, it is because you need drama :-P.
Firstly - show me where I said "few block headers per second" - at least quote correctly ...
If you are dealing with work at a nonce range level, then for each nonce range you are passing work to the device.
If the transfer of the ~60-80bytes of hash data to the device is even within an order of magnitude of the device in processing that data, then you have a problem.
FPGA's that use serial over USB have the beginnings of this issue - as per: BFL, Icarus, ModMiner, Cainsmore.
... and the 2 companies that will be releasing their ASIC first are 2 of those, BFL and ModMiner, with Icarus not far behind.
The work arounds suggested imply implementing merlke root processing and coinbase generation processing in silicon ... as I have already explained ... but since it was a little difficult for you to understand I've repeated it. Maybe you are at the top of the first curve
Anyway, yet again, a work around to resolve the problem that the nonce size is too small.
However, how generic is that work around if it was implemented in silicon?
Would it 'always' work or would it only work with current implementations and need changing in the not too distant future?
Quote me: "ROI"
All these problems and questions are actually resolved by simply increasing the size of the nonce field.
The cause of the problem, and the correct and simplest solution to the problem.
The only argument against it is a hard-fork - which can be planned as a future change to resolve the problem before it becomes a problem.
Yeah a little forward thinking goes a long way.
All this melodrama about hard-forks being impossible - well when the first one comes along that is required 'now', then clearly that will be the end of BTC since everyone's argument is that it is impossible do a hard-fork ... ... which in this case is to fix the problem at the root cause - that then solves all related issues with one change - and a simple implementation everywhere.
As I'll repeat as I've implied before, this really is a good example of people failing to learn from the past.
In this case, a well know one, MSDOS and the early x86 architecture, where the design was based on an assumption that turned out to be an obvious underestimate not far down the track, and for years hacks were used to work around the problem because it was considered too difficult to fix it properly ... until it was fixed properly by removing the original constraints.
So if nonce size isn't too small - why are we changing bitcoin at all to support ASIC?
We are not and have not.
Quote
Why is this stupid BIP in 0.7.0 that's been forced on everyone, that also removed functionality from bitcoind, necessary?
Wtf are you talking about.
Quote
Why have people called sticking more junk in the coinbase an 'extra' nonce? What? They gave it the wrong name?
The current nonce is big enough? It's not necessary?
The current nonce is big enough? It's not necessary?
Extranonce has been part of the design since day one. It was the only way to make your coinbase transactions have unique IDs other than constantly changing to a new public key whenever you update the candidate block. It's fundamental and is required for things independent of the nonce size.
Quote
Coz when you get a block header to hash, you can only hash 2^32 times before needing to change it.
Why does that restriction exist?
Why does that restriction exist?
A _4 billion to 1_ work factor is pretty reasonable. There are a couple tradeoffs here. Header size is utterly critical for SPV nodes of various kinds, and even adding a single byte to the header increases the size by 1.25%. Having too much workfactor ratio between header only work and block update work encourages miners to build devices which never process transactions: Cheaper to just build a constant root with no transactions and increment the header until you solve a block. On a fresh design I may have given a bit more space to nonce, sure. But there is no need to change it now.
Quote
It's gonna work fine without any changes right?
Oh ... no, that's not correct
Oh ... no, that's not correct
Yes, in fact it will.
Quote
Yes I know the soft fork in April was done badly, but just because it was fucked up then, doesn't mean soft or hard forks can't be done.
Huh? The P2SH deployment went pretty smoothly. About the only bad thing that came out of it was that we discovered Bitcoin had a bug in handling large reorgs with lots of transactions. Timed less well— with large miners unable to reorg because of it— this could have been pretty fatal for the network. But since it only really impacted non-mining nodes (plus the tiny minority of non-updated miners) due to fork being invalid P2SH it was harmless. It's very fortunate, because I'm not sure we would have discovered that BDB issue even by now wthout it.
Quote
Where IS the restriction at the moment?
That we're not permitted to throw you into a volcano.
It would be a little strange a device that can process 4 billion hashes faster than you can give it the 32 bytes to hash.
+1
As I have already said, no company would be silly enough to do Stratum or the other thing in silicon
Stop spreading FUD. Nobody is talking about implementing Stratum/GBT directly in chip.
How about that huge possible optimizations on wire protocol between computer and device? Why not to optimize this at first time? You told that current wire protocols has problems with handling few block headers per second. If that's true, then you can get many orders of magnitude in performance just by optimizing it, without breaking anything in bitcoin.
Oh, I forgot, it is because you need drama :-P.
GBT was necessary even before ASIC, because the real problem isn't the nonce range being too small, it's that pooled mining today tends to centralize control too much. The primary key feature of GBT is decentralization; that it happens to also solve the ASIC problem came as a nice side-effect.
A perceived problem that even MOST of the network disagrees with you on.MOST of the network mines on pools - count the % of network using P2Pool or solomining and you will see your argument that GBT solves some control problem holds no water at all.
If the time to talk to the device is of the same order of magnitude as how long the device takes to process the work, then there is a problem.
Your sig says 54Gh/s, which would consume 12.5 merkle roots per second. That's 12.5 * 32 = 402 bytes per second.
It would be a little strange a device that can process 4 billion hashes faster than you can give it the 32 bytes to hash.
I'm sorry, maybe I'm missing something. The CPU increments the coinbase extra nonce and recalculates the merkle branch to make new merkle roots to send to the ASIC. Only the merkle roots need to go over the wire.
GBT was necessary even before ASIC, because the real problem isn't the nonce range being too small, it's that pooled mining today tends to centralize control too much. The primary key feature of GBT is decentralization; that it happens to also solve the ASIC problem came as a nice side-effect.
A perceived problem that even MOST of the network disagrees with you on.MOST of the network mines on pools - count the % of network using P2Pool or solomining and you will see your argument that GBT solves some control problem holds no water at all.
Jump to: