Pages:
Author

Topic: [ANN] Stratum mining protocol - ASIC ready - page 18. (Read 146083 times)

legendary
Activity: 1386
Merit: 1097
December 11, 2012, 04:17:43 PM
No one cares if ntime is exact to the second, as long as it's roughly right.

ntime should reflect current time. If there's another way of creating unique data header for hashing, it should be prioritized over ntime rolling.

However ntime rolling is still possible, so don't worry, everything will be fine :-).
hero member
Activity: 563
Merit: 500
December 11, 2012, 03:52:12 PM
You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?

Why only once a second?  An ASIC is going to run through all nonces in far less than a second - why can't it roll ntime rapidly at least for a few seconds?  No one cares if ntime is exact to the second, as long as it's roughly right.
hero member
Activity: 563
Merit: 500
December 11, 2012, 03:44:40 PM
Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.

I'd just taken that as a given - it would seem to be madness not to do that in the ASIC.
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
December 10, 2012, 07:19:01 PM
Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.
... and hopefully every ASIC that does this will also return the amount of work actually done inside the device ...

I hate estimates Tongue
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
December 10, 2012, 07:02:28 PM
Early on I spoke to some of the ASIC-to-be manufacturers and suggested they do target testing for higher diffs internally as well, so I expect at least some of them to take a target as well That and as Eleuthria said, it's still insignificant CPU to test them anyway.
legendary
Activity: 1750
Merit: 1007
December 10, 2012, 06:55:55 PM
Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

That's not quite all the processing though.

I don't know if you send a target difficulty to the ASIC to work on, but at the moment FPGA's work at difficulty one, don't they?
So cgminer has to take these diff 1 nonces and rehash them to find their actual difficulty, and compare them to the target.
If ASICs don't have a settable target, if could add quite a bit more of a load to the CPU.

Validating the difficulty of a solution should only require 2 SHA256 passes and a simple evaluation to determine difficulty.  It would be significantly less work than preparing the work to be sent to the ASIC in the first place.
legendary
Activity: 1795
Merit: 1208
This is not OK.
December 10, 2012, 06:51:44 PM
Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

That's not quite all the processing though.

I don't know if you send a target difficulty to the ASIC to work on, but at the moment FPGA's work at difficulty one, don't they?
So cgminer has to take these diff 1 nonces and rehash them to find their actual difficulty, and compare them to the target.
If ASICs don't have a settable target, it could add quite a bit more of a load to the CPU.
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
December 10, 2012, 06:44:17 PM
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed Smiley

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)
You can still roll the time once per second, starting the second nonce field all over again. It really isn't going to be an issue unless some dramatic new technology that doesn't even exist yet is thought up and developed and manufactured and released in the decades ahead. Quantum computers anyone?
legendary
Activity: 4592
Merit: 1851
Linux since 1997 RedHat 4
December 10, 2012, 06:21:51 PM
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed Smiley

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)
roll-n-time is only needed to change the data being hashed when you run out of the nonce range with getwork

Stratum uses the coinbase transaction to resolve this, so the n-time field is simply set to whatever the current time is.

For 1 second of time (n-time=now) you can generate an amount of work greater than ever will be possible to be done in 1 second by any device and thus there is no need to roll the n-time field

As devices hash faster, you simply increase the size of the 'secondary nonce' field in the coinbase

From the point of view of CPUs not being able to hash enough merkle trees - of course as hashing devices get faster you can also expect CPUs to get faster - but implementing something 'like Stratum' in the MCU is not really that big an issue and will resolve it anyway.

Of course at some point in the FAR distant future this will pose a problem, since cycling the 'secondary nonce' in the coinbase does not guarantee to generate unique block headers to hash - the merkle root generated from the transactions is a hash function and thus can produce duplicate output for different input - since the input size is of course MUCH greater than the output size
In fact there is a very small chance of people already hashing the same block header twice - but it is extremely small - the only way around that is how I've suggested increasing the size of the nonce field in the block header - that was rejected

Edit: of course this FAR DISTANT FUTURE problem is present (and exactly the same) in GBT also
hero member
Activity: 563
Merit: 500
December 10, 2012, 05:02:52 PM
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.

Yeah, OK, you're right.  So assuming (for ease of maths) 2^10 transaction per block, you need 10 hashes per Merkle root, so you need a host CPU capable of 3.75kH/s.  Hell, my first gen EEEPC would do that more than comfortably (although it wouldn't surprise me if that's 10% CPU or so - I'm sure enough to make the CPU fan come on, even if only at low speed Smiley

Ok, so you've convinced me, we've got a little time before ASICs are fast enough to worry about this.... :-)

roy

ETA: But in 2 years time, when we have $99 600GH/s coffee warmers, then I'm going to be asking for rollntime support :-)
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
December 10, 2012, 04:53:01 PM
Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.
Well actually I don't think it would even register in CPU usage even at 1.5TH of getworks with the largest asic device currently planned. It's only 375 getworks per second. It takes almost no CPU to generate that many.
hero member
Activity: 563
Merit: 500
December 10, 2012, 04:47:02 PM
Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.

Yes, that's an interesting point.  All the more reason to make use of that support rather than having the host it's connected to calculating Merkle roots at a high rate of knots.

ETA: if cgminer running on my EEEPC 701 driving a couple of ASIC miners can keep the CPU load low enough that the CPU fan doesn't come on, I'd be very happy.  My guess is that it would need the miner to roll ntime to accomplish this.
hero member
Activity: 563
Merit: 500
December 10, 2012, 04:44:21 PM
Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.

Hmm, are you saying that the need with getwork for the pool to explicitly permit rollntime (and to tell the miner how much it's allowed to roll ntime) is just a quirk of how traditional pools generate work?  And that the assumption is that a Stratum pool will just accept any valid block, regardless of the value of the ntime field within the block header (within reason)?

roy
-ck
legendary
Activity: 4088
Merit: 1631
Ruu \o/
December 10, 2012, 04:40:37 PM
Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit (by default, it can be increased and the spec supports it very easily).  However, Stratum also does support ntime rolling, the official spec is that ntime should not be increased faster than real-time.

nTime rolling is a hack solution to extend getworks, and if used improperly can be extremely bad for the network if a large number of blocks were produced and pushed the mean-time of recent blocks into the future.

Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.
Some ASICs I suspect will have rolltime built into their mining protocol since their development would have started a long time ago (theoretically), when rolltime was all the rage.
legendary
Activity: 1386
Merit: 1097
December 10, 2012, 04:29:39 PM
Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit (by default, it can be increased and the spec supports it very easily).  However, Stratum also does support ntime rolling, the official spec is that ntime should not be increased faster than real-time.

nTime rolling is a hack solution to extend getworks, and if used improperly can be extremely bad for the network if a large number of blocks were produced and pushed the mean-time of recent blocks into the future.

Stratum protocol don't need to mention ntime rolling, because block header for hashing is produced directly on the miner, not by the pool. But re: Roy's concern about ASIC miners - yes, ntime rolling in ASICs will be still possible with Stratum.
hero member
Activity: 563
Merit: 500
December 10, 2012, 04:22:26 PM
Stratum does not need ntime rolling because a single stratum work unit is 4.2 billion times larger than a standard getwork unit

Yes, but it increases the size of the work unit at the expense of complexity.  Incrementing ntime extranonce and computing a new Merkle root is something that probably can't easily done in a hardware miner without increasing hardware cost, hence requiring work (and communication) by the host that the hardware miner is attached to everytime the nonce wraps.  On the other hand, ntime rolling could trivially be done in hardware (in the MCU, or even in future designs in the ASIC) massively reducing the work done by the host.

My thought is that you would do both: you'd incrememnt the nonce, naturally (this is done in hardware in the FPGA or ASIC).  Then when the nonce wraps round, you increment ntime (which could be done in the hardware miner's microcontroller, or longer term, in the ASIC).  Once ntime has been rolled by some configurable amount (30 seconds, 60 seconds, whatever), then ntime is reset, the host PC incremements extranonce, calculates a new work unit, and gives this work unit to the hardware miner.

Bottom line: the work unit for the protocol may be 4.2 billion times larger, but the work unit for the hardware miner will probably be unchanged in the short term.  ntime rolling is a far easier step for the hardware to take then full merkle branch support in the hardware.
legendary
Activity: 1750
Merit: 1007
December 10, 2012, 04:09:34 PM
hero member
Activity: 563
Merit: 500
December 10, 2012, 04:02:32 PM
I have a question re Stratum, namely: does Stratum support rollntime? - I skimmed the spec and couldn't see a mention (and apologies if this has already been discussed and my search missed it).

What I'm thinking is this:  a 60GH/s miner (which we have every expectation is going to be pretty run-of-the mill once ASICs hit) is going to consume 14 work units per second, and it's reasonable to assume that they will continue to get faster over time.  Over time, as mining hardware improves, then on low-end hardware the CPU load on the mining host will start to become more significant, particularly if hosting a large farm of ASIC miners.  The USB I/O load may start to become significant, too, particularly if the mining host is actually an old laptop that only does USB 1.

ntime rolling can trivially be implemented in the hardware miner (even if in the microcontroller rather than in the ASIC), and doing so would reduce by almost two orders of magnitude the number of work units that have to be generated by the mining host and communicated to the hardware miners.

Longer term, I imagine that the microcontroller in ASIC miners will roll extranonce itself, but the extra CPU and memory requirements placed on the microcontroller, not to mention the extra firmware complexity, probably means this won't be a popular approach with hardware manufacturers until it becomes unavoidable.

Even once we get to that point, I can imagine that as ASIC cores get faster, and smaller (so one microcontroller potentially driving more of them) then hardware designers might opt for an architecture where ntime is incremented in the ASIC, with the microcontroller only rolling extranonce and calculating the Merkle root, just to reduce the CPU and IO requirements within the hardware miner.

This is all just me thinking aloud, and it might be that we really are convinced that the usefulness of ntime rolling is so far away that it's just not worth thinking about, but I just thought I'd throw it out there....

Thoughts?

roy
legendary
Activity: 1386
Merit: 1097
December 09, 2012, 05:58:11 AM
i think a many files will better than 1 file, the file per block (with the worker shares ) will reasonable ?
new block started - new file created? block found - file closed.

csv (or json) format for the stored data will great

This is completely outside of the project scope. It's more like job for miner itself. Please check documentation of your miner, some of them (cgminer) has some statistics and reporting already implemented.
legendary
Activity: 1386
Merit: 1097
December 09, 2012, 05:56:13 AM
ps aux via crontab every few mins?

That will work, because crash of the proxy won't affect the watchdog itself.
Pages:
Jump to: