Topic: why cgminer increases "nonce2" and "ntime" every 5 seconds ? (Read 421 times)
Bitfury are rolled cores, so multiple clocks per calculation. Bitmain's always used unrolled cores, which take up a lot more die space but do the whole hash in a single cycle. BM1384 has 55 cores, 1387 has 114 cores.
Fuzzy, the S17 chip has 672 cores. Interestingly, the chip actually has four blocks of 168 cores each built in series within the die.
Interesting - I'd swear I've come across infor with it being over 8k of them. The Bitfury's in the Apollo has 4k of them per Bitfury's spec and the A1 (2014) had 1024 cores. Most odd.
Fuzzy, the S17 chip has 672 cores. Interestingly, the chip actually has four blocks of 168 cores each built in series within the die.
That would make sense then how it can do hundreds of GH/s with a clock of only hundreds of MHz
Fuzzy, the S17 chip has 672 cores. Interestingly, the chip actually has four blocks of 168 cores each built in series within the die.
'nonce' has nothing to do with this part, cgminer has nothing to do with the nonce ranges.
if cgminer has absolutely nothing to do with the nonce then who does? does the hardware miner have a routine to run through the nonce range - is it triggered by cgminer in some way? weird
A) As Fuzzy mentioned above
B) How work is queued in the physical miner.
b1) You can't always divide 2^32 evenly with the number of chips and the number of cores.
So some values may not be used because of this.
b2) If we are talking about a small miner, there may be no work queue inside the hardware, so the miner must have it's work refreshed by cgminer, before it reaches 2^32, so it doesn't repeat doing the same work again from the start.
Thus. timing here will mean that all sub-ranges that are divided up among the chips, would rarely if ever complete.
Be aware of the obvious fact that you don't need to mine the full range.
It makes absolutely no statistical difference if you stop a range before it completes and start again from the bottom with new work.
As I mentioned
It does not matter what the asic nonce range is. The only point of interest is for the asics to test a high % of the range to minimise work I/O to the asic.
Quote
It’s kinda crazy to think about; the fastest CPUs out there wouldn’t even be able to increment the nonce as fast as an ASIC, much less do any actual hashing
Huge difference between any CPU/GPU and a mining ASIC -- the number of cores in them. A top-end CPU might have what, 48 cores in it?The s17 mining ASIC has 8192 cores in the 1 chip and each core handles 1 part of the range being ran through so with each clock cycle a block of up to 8192 values are processed at 1 time. Add in the fact that most large miners will have 100-200 ASICs in them and that's how they achieve such incredible performance.
It should be noted that it is rare for all of the cores to be usable and often a chip will have many dead ones. That is part of the difference between the fastest (and most expensive) miners and their slightly slower and cheaper brethren - the fastest/most expensive are using chips that are binned (selected) as most functional cores and those make up a rather small % of usable dies from each wafer. The lower number of perfect/near-perfect ones allows them to be priced at a premium.
if cgminer has absolutely nothing to do with the nonce then who does? does the hardware miner have a routine to run through the nonce range - is it triggered by cgminer in some way? weird
The mining chip (ASIC) increments the nonce after each hash. Hundreds of billions of times a second.
AFAIK it starts at 0 and goes up to 2^32 unless you configure it to start somewhere else. This takes less than a second.
It’s kinda crazy to think about; the fastest CPUs out there wouldn’t even be able to increment the nonce as fast as an ASIC, much less do any actual hashing.
'nonce' has nothing to do with this part, cgminer has nothing to do with the nonce ranges.
if cgminer has absolutely nothing to do with the nonce then who does? does the hardware miner have a routine to run through the nonce range - is it triggered by cgminer in some way? weird
Alas there seems to be a lot of misunderstanding reading the code that says exactly what it does.
While the code shows it pretty straight forwardly, if you read it, what mining 101 on an asic does is:
Receive work from the pool every 30 seconds (that's up to the pool, but 30 seconds is optimal)
During that 30 seconds, that single pool work is used over and over again to generate work for the asics.
How that 'over and over' is done is by modifying 'nonce2' to generate a unique merkleroot to generating a unique block header.
Then hashing that block header doing the first half of the double sha256 then passing that to the asics with a job id.
'nonce' has nothing to do with this part, cgminer has nothing to do with the nonce ranges.
A CPU core can hash about 30 million times a second, so generating 10's or 100's or even thousands of work items isn't that hard to do quickly.
Each work it generates, will give the asics up to 4billion hashes to do, so as long as the asics aren't 4billion times faster than the time it takes cgminer to produce a single work item, all is good.
N.B. a single work item is not one hash, read the code if you want to work out what it does.
Using the work sent to them, the asics run the nonce value some % of the way from 0 to 4billion doing the 2nd half of the double sha256 for each nonce value, and if they find a hash with a difficulty of 1 or more, they return the job id and the nonce value that produced the hash.
cgminer then tests this returned data with the work job it should match, and sends it to the pool if the difficulty meets the pool's requirements.
It does not matter what the asic nonce range is. The only point of interest is for the asics to test a high % of the range to minimise work I/O to the asic.
(hashing a nonce range in CPU would be beyond ridiculously slow, millions of times slower than a current asic, so no one would be silly enough to do that ...)
That's the basic 101 version of it.
Multi midstates, asicboost, and fpga work generation (like in an avalon) requires different effort, to effectively get similar results.
While the code shows it pretty straight forwardly, if you read it, what mining 101 on an asic does is:
Receive work from the pool every 30 seconds (that's up to the pool, but 30 seconds is optimal)
During that 30 seconds, that single pool work is used over and over again to generate work for the asics.
How that 'over and over' is done is by modifying 'nonce2' to generate a unique merkleroot to generating a unique block header.
Then hashing that block header doing the first half of the double sha256 then passing that to the asics with a job id.
'nonce' has nothing to do with this part, cgminer has nothing to do with the nonce ranges.
A CPU core can hash about 30 million times a second, so generating 10's or 100's or even thousands of work items isn't that hard to do quickly.
Each work it generates, will give the asics up to 4billion hashes to do, so as long as the asics aren't 4billion times faster than the time it takes cgminer to produce a single work item, all is good.
N.B. a single work item is not one hash, read the code if you want to work out what it does.
Using the work sent to them, the asics run the nonce value some % of the way from 0 to 4billion doing the 2nd half of the double sha256 for each nonce value, and if they find a hash with a difficulty of 1 or more, they return the job id and the nonce value that produced the hash.
cgminer then tests this returned data with the work job it should match, and sends it to the pool if the difficulty meets the pool's requirements.
It does not matter what the asic nonce range is. The only point of interest is for the asics to test a high % of the range to minimise work I/O to the asic.
(hashing a nonce range in CPU would be beyond ridiculously slow, millions of times slower than a current asic, so no one would be silly enough to do that ...)
That's the basic 101 version of it.
Multi midstates, asicboost, and fpga work generation (like in an avalon) requires different effort, to effectively get similar results.
Thanks for this great explanation and linking to the document.
Does this explain why nonce2 traversal is so slow in comparison to nonce, because nonce2 cycling is kept in CPU ? Would it be possible to send nonce2 to the ASIC chip just like as with nonce so the traversal would take place on the much faster ASIC chip?
Just as an example:
- set nonce to a fixed value e.g. 123
- generate midstate work for nonce2 (instead of nonce)
- let the ASIC chip traverse through 2^64-1 cycles for nonce2 and have the hash reported back to cgminer
How can that be done ?
Does this explain why nonce2 traversal is so slow in comparison to nonce, because nonce2 cycling is kept in CPU ? Would it be possible to send nonce2 to the ASIC chip just like as with nonce so the traversal would take place on the much faster ASIC chip?
Just as an example:
- set nonce to a fixed value e.g. 123
- generate midstate work for nonce2 (instead of nonce)
- let the ASIC chip traverse through 2^64-1 cycles for nonce2 and have the hash reported back to cgminer
How can that be done ?
2. cgminer does not cycle nonce from 0 to 4.29G. It just sends a lot of block header data to ASIC chips. ASIC chips or hardware do cycle nonces by digital chips and frequency pulse to check and find qualified nonces. ASIC hardware do filter out qualified nonces. So, ASIC hash header data super fast.
Does that mean that cgminer has no control at all about the current nonce which should be tested? As cgminer sends a block header to the ASIC miner, the ASIC hardware would need to know how such a block header is constructed and to derive the part where the nonce is located, wouldn't it? Otherwise the ASIC miner would only see a very long string (blockheader) without knowing which bits to change for trying for a valid hashes.You don't really want cgminer, which runs on a relatively slow CPU to have to increment the nonce and communicate it to ASIC. That's happening much too fast. Chips like the Bitmain BM1387 or the BM1397 (like in your Compac F) have a set of address pins. If I understand this right, the address pins are used when you have several chips in a chain they can each be set to work on a different nonce range, so as not to overlap.
cgminer doesn't exactly send a block header to the mining chip. It's like a partially computed hash of the block called a midstate. Kano has a description of this here; https://kano.is/BMMMP63278.pdf. All of the bytes are sent in fixed positions to the mining chip, so it knows what goes where. Things start to get pretty crazy (to me) when cgminer sends multiple midstates to the ASIC, which is called ASICboost.
Very well explained, thank you.
1. The nonce2 is only used to create next new local block header data. When a new block header come in, nonce2 is initialzed to 0 in cgminer, e.g. parse_notify() or gbt_decoode(). The nonce2 is 64 bits integer (very large integer). The nonce2 only affects merkleroot value in local block header data. Huge numbers of local block header data are created and are given to ASIC chips for chips to find out some nonces which produce a hash value with last 8 digits 0.
hash value: xxxx....x????00000000.
Let's assume nonce2 would be also an unsigned 32 bit integer and the ASIC miner very powerful with a superior hash rate. What would happen when nonce2 range is exhausted before a new block header arrived ?hash value: xxxx....x????00000000.
2. cgminer does not cycle nonce from 0 to 4.29G. It just sends a lot of block header data to ASIC chips. ASIC chips or hardware do cycle nonces by digital chips and frequency pulse to check and find qualified nonces. ASIC hardware do filter out qualified nonces. So, ASIC hash header data super fast.
Does that mean that cgminer has no control at all about the current nonce which should be tested? As cgminer sends a block header to the ASIC miner, the ASIC hardware would need to know how such a block header is constructed and to derive the part where the nonce is located, wouldn't it? Otherwise the ASIC miner would only see a very long string (blockheader) without knowing which bits to change for trying for a valid hashes.3. Qualified or found nonces, then, are read by cgminer from ASIC chips. Then, cgminer verify those nonces with the pool's difficulty. Then, good nonces are sent to the pool for further verification by pool with network's difficulty.
So cgminer only sends a block header to the ASIC hardware and the ASIC miner itself cycles through the 32bit nonce range? Does the ASIC miner use a random cycle or incrementing the values?
After I read your posts, there are missing points.
1. The nonce2 is only used to create next new local block header data. When a new block header come in, nonce2 is initialzed to 0 in cgminer, e.g. parse_notify() or gbt_decoode(). The nonce2 is 64 bits integer (very large integer). The nonce2 only affects merkleroot value in local block header data. Huge numbers of local block header data are created and are given to ASIC chips for chips to find out some nonces which produce a hash value with last 8 digits 0.
hash value: xxxx....x????00000000.
2. cgminer does not cycle nonce from 0 to 4.29G. It just sends a lot of block header data to ASIC chips. ASIC chips or hardware do cycle nonces by digital chips and frequency pulse to check and find qualified nonces. ASIC hardware do filter out qualified nonces. So, ASIC hash header data super fast.
3. Qualified or found nonces, then, are read by cgminer from ASIC chips. Then, cgminer verify those nonces with the pool's difficulty. Then, good nonces are sent to the pool for further verification by pool with network's difficulty.
1. The nonce2 is only used to create next new local block header data. When a new block header come in, nonce2 is initialzed to 0 in cgminer, e.g. parse_notify() or gbt_decoode(). The nonce2 is 64 bits integer (very large integer). The nonce2 only affects merkleroot value in local block header data. Huge numbers of local block header data are created and are given to ASIC chips for chips to find out some nonces which produce a hash value with last 8 digits 0.
hash value: xxxx....x????00000000.
2. cgminer does not cycle nonce from 0 to 4.29G. It just sends a lot of block header data to ASIC chips. ASIC chips or hardware do cycle nonces by digital chips and frequency pulse to check and find qualified nonces. ASIC hardware do filter out qualified nonces. So, ASIC hash header data super fast.
3. Qualified or found nonces, then, are read by cgminer from ASIC chips. Then, cgminer verify those nonces with the pool's difficulty. Then, good nonces are sent to the pool for further verification by pool with network's difficulty.
I have changed the status bar to see which nonce currently was processed but I get "0" displayed all the time for the nonce. I also tried %u which will give similar error.
To be honest, I haven't looked that deep into cgminer yet; if nonce is always 0, I suppose it's possible that it only rotates through nonce2 instead - not sure. Maybe we can summon @kano; he knows for sure.
I did so 
But as I said, I am even struggling in getting the nonce value displayed.
I have changed the status bar to see which nonce currently was processed but I get "0" displayed all the time for the nonce. I also tried %u which will give similar error. I am not a developer, can only read some code. I don't think that the format %"PRIu32" or %u is the cause for the issue getting nonce=0 displayed. But I rather suspect that the reason is that in this code section the nonce was not defined at all or not intended to be used. I think that the nonce far before has been sent to the mining hardware as (scheduled/queed) work. Therefore the display of 0 in the output, kinda ?
But as I said, I am even struggling in getting the nonce value displayed.Code:
if (opt_debug) {
char *header, *merkle_hash;
header = bin2hex(work->data, 112);
merkle_hash = bin2hex((const unsigned char *)merkle_root, 32);
applog(LOG_DEBUG, "Generated GBT solo merkle %s", merkle_hash);
applog(LOG_DEBUG, "Generated GBT solo header %s", header);
//applog(LOG_DEBUG, "Work nonce2 %"PRIu64" ntime %s", work->nonce2, work->ntime);
applog(LOG_DEBUG, "Work nonce=%"PRIu32" nonce2=%"PRIu64" ntime=%s", work->nonce, work->nonce2, work->ntime);
free(header);
free(merkle_hash);
}
I have changed the status bar to see which nonce currently was processed but I get "0" displayed all the time for the nonce. I also tried %u which will give similar error. I am not a developer, can only read some code. I don't think that the format %"PRIu32" or %u is the cause for the issue getting nonce=0 displayed. But I rather suspect that the reason is that in this code section the nonce was not defined at all or not intended to be used. I think that the nonce far before has been sent to the mining hardware as (scheduled/queed) work. Therefore the display of 0 in the output, kinda ?
how ?
Add debug prints! 
Something like this:
Code:
applog(LOG_DEBUG, "Hello World!");
Then pipe stdout to a file. If that's not good enough, you will need to open a file and write to it instead, but that will be slightly more complicated.
I am trying to understand and see live what exactly happens in the mining process. Yes, I did "freeze" ntime for that educational purpose in the example above. I did not freeze nonce2 but I increased the timer from default to 5sec to testwise 15sec, just to see if my changes took effect. So far so good, however I am still far away from my goal.
What I like to see is:
- all the nonces that cgminer sends to the hardware for work, I like to see how this process is threaded and queued by cgminer
- the exact shares that are generated and sent to the hardware
- the exact nonce, nonce2 and ntime value that were used for each particular hash step and the result for it
Generally spoken, I'm looking to more infos that I miss with the --debug option of cgminer. Currently I'm stuck even trying to display simply the nonce values and the particular hash result of one single step. I am aware that this could output very high amount of lines and generate a log with huge size, but I'm interested in one complete cycle through this:
- ntime should be freezed at all, irrelevant for me, same for merkle root
- cycling through the whole nonce range and when nonce reached the upper limit the nonce should freeze its value or set to zero and freeze its value (no more incrementing)
- then it should cycle through the whole nonce2 range (nonce is fixed at 0)
- when nonce2 is exhausted cgminer should exit
I wanna see ALL the hash results in a convenient output view (e.g table), so I can scroll through the log file and see each single step including the particular values for nonce and nonce2 and what hashed they generated.
note: timestamps are just random examples picked by me, I did not exaclty calculate how long it takes
how ?
What I like to see is:
- all the nonces that cgminer sends to the hardware for work, I like to see how this process is threaded and queued by cgminer
- the exact shares that are generated and sent to the hardware
- the exact nonce, nonce2 and ntime value that were used for each particular hash step and the result for it
Generally spoken, I'm looking to more infos that I miss with the --debug option of cgminer. Currently I'm stuck even trying to display simply the nonce values and the particular hash result of one single step. I am aware that this could output very high amount of lines and generate a log with huge size, but I'm interested in one complete cycle through this:
- ntime should be freezed at all, irrelevant for me, same for merkle root
- cycling through the whole nonce range and when nonce reached the upper limit the nonce should freeze its value or set to zero and freeze its value (no more incrementing)
- then it should cycle through the whole nonce2 range (nonce is fixed at 0)
- when nonce2 is exhausted cgminer should exit
I wanna see ALL the hash results in a convenient output view (e.g table), so I can scroll through the log file and see each single step including the particular values for nonce and nonce2 and what hashed they generated.
Code:
timestamp nonce nonce2 hash
===========================================================
2022/11/07 15:33:10.789 0 0 0c82ba8v......
2022/11/07 15:33:10.790 1 0 f891a9c5......
2022/11/07 15:33:10.789 0 0 10d9e66a......
2022/11/07 15:33:10.789 0 0 4e9ff1c4b......
2022/11/07 15:33:10.789 0 0 004b20b3......
2022/11/07 15:33:10.789 0 0 0123abcd......
...
2022/11/07 15:33:10.871 3891853 0 e8c361c0......
2022/11/07 15:33:10.871 3891854 0 8a003dfb1......
...
2022/11/07 15:33:10.894 4294967295 0 910af3c314...
nonce exhausted, upper limit reached
...
2022/11/07 15:33:11.048 0 0 7016c6c1.....
2022/11/07 15:33:11.048 0 1 3d8eeaf0.....
2022/11/07 15:33:11.048 0 2 610c843a.....
2022/11/07 15:33:11.982 0 591857164 851758cc.....
2022/11/07 15:33:15.615 0 17178918541 58a7aa9b....
2022/11/07 15:33:17:195 0 18446744073709551615 84e701471...
nonce2 exhausted, upper limit reached
exit
how ?
First of all, this is wrong:
Sure? I am talking about the nonce2the nonce2 range which is from 0-18446744073709551615
From your other topic, it appears to me that you just removed the timer (or set it to a very long number) that resets the nonce to 0 and creates a new block template with a new timestamp. Is that what you're doing?
Nope. That is what cgminer seems to do by default. After the 5sec (default) new ntime is generated, nonces are resetted to zero and the process is started over.Quote
Started with nonce2=0
[2022-11-06 10:31:03.924] Work nonce2 0 ntime 63677a79
Ended with nonce2=15478
[2022-11-06 10:36:03.922] Work nonce2 15478 ntime 63677a79
[2022-11-06 10:31:03.924] Work nonce2 0 ntime 63677a79
Ended with nonce2=15478
[2022-11-06 10:36:03.922] Work nonce2 15478 ntime 63677a79
Now, what about my suggestion that the variable may be wrapping around? That's something to consider when working with large numbers / too large numbers for the variable types in use.
Hi jack1cryptotalk007,
yes, correct. I am working on gen_solo_work because I'm not using a pool, I'm mining on my own bitcoind node. I'm aware of the reason nonce and nonce2 are used. It is very well explained here and here.
Sure? I am talking about the nonce2 (extranonce in coinbase) and not the original nonce. If nonce2 would be incremented, where would be the upper limit of the range ? When I decrement nonce2 in cgminer it starts at 18446744073709551615 so that's why I think it is the upper limit of nonce2.
Note: Not to confuse with Extranonce1 vs. Extranonce2
yes, correct. I am working on gen_solo_work because I'm not using a pool, I'm mining on my own bitcoind node. I'm aware of the reason nonce and nonce2 are used. It is very well explained here and here.
First of all, this is wrong:
the nonce2 range which is from 0-18446744073709551615
Sure? I am talking about the nonce2 (extranonce in coinbase) and not the original nonce. If nonce2 would be incremented, where would be the upper limit of the range ? When I decrement nonce2 in cgminer it starts at 18446744073709551615 so that's why I think it is the upper limit of nonce2.
Note: Not to confuse with Extranonce1 vs. Extranonce2
From your other topic, it appears to me that you just removed the timer (or set it to a very long number) that resets the nonce to 0 and creates a new block template with a new timestamp. Is that what you're doing?
Nope. That is what cgminer seems to do by default. After the 5sec (default) new ntime is generated, nonces are resetted to zero and the process is started over.
In cgminer, any new block or any update of cuurent block will create a new "ntime" when block information is received. About 60 second, block update information is coming from pool.
see parse_notify() function.
In cgminer, nonce2 is used to create more local block header data for more hashing and is used for generating new merkleroot in local header data. it changes very quick. see gen_stratum_work or gen_solo_work funct.
It is used in submit work to pool and pool uses it to re-generate block header for verifying if block header is resolved.
see parse_notify() function.
In cgminer, nonce2 is used to create more local block header data for more hashing and is used for generating new merkleroot in local header data. it changes very quick. see gen_stratum_work or gen_solo_work funct.
It is used in submit work to pool and pool uses it to re-generate block header for verifying if block header is resolved.
Jump to: