Author

Topic: [ANN] SpreadCoin | Decentralize Everything (decentralized blockexplorer coming) - page 224. (Read 790391 times)

legendary
Activity: 1484
Merit: 1007
spreadcoin.info
That looks like he's standing in front of an ASICs with vents. Has Mr.Spread already created an ASICs for SpreadX11?

Yep!
It's a hamster dung powered wooden ASIC alright.
He created it himself, just with the stuff that was lying around...  Huh
member
Activity: 81
Merit: 1002
It was only the wind.

If it was an error, it wouldn't get the right hash. It's correct.

I meant to say "bug", not error, ofcourse.


Not a bug - if I change/remove it, wrong hashes.
legendary
Activity: 1456
Merit: 1000
That looks like he's standing in front of an ASICs with vents. Has Mr.Spread already created an ASICs for SpreadX11?
legendary
Activity: 1484
Merit: 1007
spreadcoin.info
Maybe I can shortcut the memory usage!

I told Mr. Spread about your idea.

He LOL'ed and...



Who is Mr. Spread?
member
Activity: 81
Merit: 1002
It was only the wind.
Mining optimization meaning the current algo?

Wait, let's first further analyze this.

So you are saying we have about 3000 unnecessary SHA256 calculations per second, right?

On what GPU with what settings are you getting those values?


Not per second. PER HASH. I'm looking at the OpenCL code, it's right there.

ok, I'm only looking at the in-wallet CPU miner code, probably it's just an error in the OpenCL version?

If it was an error, it wouldn't get the right hash. It's correct.
legendary
Activity: 1456
Merit: 1000
I think its a good idea, on paper. Tx's are given priority, the rest is there to make mining equal. With near full blocks, mining should be easy.

Perhaps we should cut the block sizes down to 1 tx  Grin

1tx blocks, moving up to 1mb blocks by next year. Maybe we should just organise hard forks every 6 months.
legendary
Activity: 1484
Merit: 1007
spreadcoin.info
What I don't get is why the AMD miner is so much worse than nvidia; and it has problems too.

I can't really judge it (yet), but it's probably badly implemented.

It's largely SPH code... really, really bad. About the same as the original darkcoin-mod.

EDIT: Idea! What's the padding made out of? Maybe I can shortcut the memory usage!

padding is constructed starting with a seed/copy of the 32 bytes of previousHashBlock (or hashPrevBlock as it is called in the code), and then there are a few bitshifts (if necessary) and then tens of thousands of multiplications while we are filling the space (moving backwards thru the block).
But we basically start with just those 32 bytes, and everything is derived from them.

Padding starts here:

https://github.com/spreadcoin/spreadcoin/blob/master/src/main.cpp#L1511

Code:
   while (BlockData.size() % 4 != 0)
    BlockData << uint8_t(7);

    // Fill rest of the buffer to ensure that there is no incentive to mine small blocks without transactions.
    uint32_t *pFillBegin = (uint32_t*)&BlockData[BlockData.size()];
    uint32_t *pFillEnd = (uint32_t*)&BlockData[MAX_BLOCK_SIZE];
    uint32_t *pFillFooter = std::max(pFillBegin, pFillEnd - 8);

    memcpy(pFillFooter, &hashPrevBlock, (pFillEnd - pFillFooter)*4);
    for (uint32_t *pI = pFillFooter; pI < pFillEnd; pI++)
        *pI |= 1;

    for (uint32_t *pI = pFillFooter - 1; pI >= pFillBegin; pI--)
        pI[0] = pI[3]*pI[7];

    BlockData.forsed_resize(MAX_BLOCK_SIZE);

First thing we do is fill up the Block from the left (right after the tx-section) with a few 0x07 bytes (only if necessary), just so that the current size of the blockdata size (header + txs) is exactly divisible by 4.
Maybe size is already divisible modulo 4, so we don't need to add any such bytes.

Then we define pointers pFillBegin, pFillEnd and pFillFooter, and then copy hashPrevBlock to the end (last 32 bytes) of this MAX_BLOCK_SIZE block, and then fill all the empty bytes in between moving backwards, 4 bytes per iteration.

Oh, and before we start we also turning these 32 bytes (or the 8 x 4 byte integers it consists of)  "ODD", by doing this *pI |= 1 operation on them, so that they are not divisible by 2 anymore).
Then we just iterate backwards, in 4 byte steps, always taking pI[3]*pI[7] and writing the multiplication result into pI[0], we do that, until we reach the transaction section (or those 0x07 bytes we created earlier (if they were necessary))...

That's about it.

So this large padding section doesn't have any regularity or repetition if you were expecting that.

It's pretty messy & chaotic data.  Cheesy

Mr. Spread really wants your GPU to double-SHA256 a 200Kbyte datastructure. All the time.
member
Activity: 81
Merit: 1002
It was only the wind.
Why over 3000 SHA256 hashes were used, I'll never know. What's the URL for Slack?

Yeah, I'm sure mr.spread's mining algo can be optimized.
The reason we use a little SHA256 in our algo is because the miner has to provide a hashWholeBlock value (to prove he himself had first knowledge about the block content, called POK "Proof Of Knowledge" in the sourcecode),
which ofcourse is derived thru a common double SHA256, and which has to be recalculated whenever ANYTHING in the block changes (timestamp, nonce (but only every 64 nonces), txs, etc...)

It's explained in mr. spread's whitepaper : http://www.spreadcoin.info/downloads/SpreadCoin-WhitePaper.pdf
(which admittedly is a little bit cryptic and not using the "clearest" english... I'm working on a more readable whitepaper version when servicenodes have progressed a little bit)

No, I know that. What I don't know is why you need 3000 goddamned iterations.

I'm not sure about that either.

Mr. Spread uses the hashPrevBlock (which is known and stays the same while we hash) to fill up the padding he's using to construct a complete MAX SIZE block which is then used to create the hashWholeBlock value.
So you probably assume that it would be enough if we derived the hashWholeBlock just once after a miner has found a solution (since hashWholeBlock is not part of the actual POW).

But I'm not sure if we are not missing something important here.

I wouldn't mind if a mining specialist like you takes a close look at it.  Wink


Nope. There is ZERO reason to hash the same data that many times. It's barbaric overkill.

Maybe a deliberately crippled miner? I don't like to suggest it but I can't see any other reason behind it.

For what purpose? It's not like you can remove them and get the right answer. It's just stupid.
member
Activity: 81
Merit: 1002
It was only the wind.
Mining optimization meaning the current algo?

Wait, let's first further analyze this.

So you are saying we have about 3000 unnecessary SHA256 calculations per second, right?

On what GPU with what settings are you getting those values?


Not per second. PER HASH. I'm looking at the OpenCL code, it's right there.
legendary
Activity: 1484
Merit: 1007
spreadcoin.info
What I don't get is why the AMD miner is so much worse than nvidia; and it has problems too.

I can't really judge it (yet), but it's probably badly implemented.
legendary
Activity: 1456
Merit: 1000
I'm baffled as to why it's done the way it is. There's just a TON of data in there being hashed by SHA256, far larger than any block SPR has...

Yes, the idea is that for hashWholeBlock we need to fill a complete MAX_BLOCK_SIZE block with data (that's why we have the "padding").

MAX_BLOCK_SIZE for spreadcoin is 200 Kbyte. https://github.com/spreadcoin/spreadcoin/blob/master/src/main.h#L35

So you have the header (less than 100 bytes) , followed by a few tx (<1 KByte or more), and then a giant padding (199 KBytes) to fill up the rest.

(This large pseudoblock is only existing temporarily in memory BTW, just to create the hashWholeBlock hash.)

From the whitepaper:

Quote
Padding ensures that there is no incentive to mine empty blocks without transactions

So the way I understand this, it means that padding calculations are more numerous the fewer transactions you get into your block.
Vice versa: The bigger the block already is (more txs), the less padding has to be added/calculated.

So in a way, padding is horrible on efficiency since we are mostly having 1 transaction blocks these days.
When this changes, then padding will not carry as much weight as it does now.

And yes, we are ALWAYS doing a double SHA256 calculation on a MAX_BLOCK_SIZE block (200 KByte).
This size doesn't change from block to block, it's always the maximum.

Why Mr. Spread did it this way?
I'd have to guess, but probably to deliberately give GPUs a hard time?

Wasn't he playing the "CPU-Only-coin" - card for some time?



Possibly.

But maybe its to give everyone equal weight. If you think about it, everyone effectively processes a full block. There is no incentive to choose more efficient blocks to mine to move on to the next one quickly.

I suppose if there is an incentive to fill blocks quickly, it sounds like it makes it more profitable to process blocks with transactions vs. padding.

What I don't get is why the AMD miner is so much worse than nvidia; and it has problems too.
legendary
Activity: 1484
Merit: 1007
spreadcoin.info
I'm baffled as to why it's done the way it is. There's just a TON of data in there being hashed by SHA256, far larger than any block SPR has...

Yes, the idea is that to calculate hashWholeBlock we need to fill a complete MAX_BLOCK_SIZE block with data (that's why we have the "padding") and then Double-SHA256 it.

MAX_BLOCK_SIZE for spreadcoin is 200 Kbyte. https://github.com/spreadcoin/spreadcoin/blob/master/src/main.h#L35

So you have the header (less than 100 bytes) , followed by a few tx (<1 KByte or more), and then a giant padding (199 KBytes) to fill up the rest.

(This large pseudoblock is only existing temporarily in memory BTW, just to create the hashWholeBlock hash.)

From the whitepaper:

Quote
Padding ensures that there is no incentive to mine empty blocks without transactions

So the way I understand this, it means that padding calculations are more numerous the fewer transactions you get into your block.
Vice versa: The bigger the block already is (more txs), the less padding has to be added/calculated.

So in a way, padding is horrible on efficiency since we are mostly having 1 transaction blocks these days.
When this changes, then padding will not carry as much weight as it does now.

But I don't think that the padding calculations are that heavy anyway, but I need to do some benchmarks and look into OpenCL / Cuda.

And yes, we are ALWAYS doing a double SHA256 calculation on a MAX_BLOCK_SIZE block (200 KByte).
This size doesn't change from block to block, it's always the maximum.

Why Mr. Spread did it this way?
I'd have to guess, but probably to deliberately give GPUs a hard time?

Wasn't he playing the "CPU-Only-coin" - card for some time?

legendary
Activity: 1484
Merit: 1007
spreadcoin.info
Girino's IS better, and as I said, this is not a crippling of the miner, it's just WTF.

Can you be a little bit more specific please?

You said words like: WTF, Insane, Barbaric and Ooops, so now I am not sure anymore I understand the point you are trying to make.

Nobody doubts that the miner's efficiency can be improved, and everybody knows that SpreadX11 is pretty exotic *

So... what gives?

Are you just baffled by it? In a good/bad way?  Wink

*(adds MinerSignature to the blockheader, and constructs MAX SIZE block by padding the previousBlockHash (this means many thousand iterations, depending on how "empty" the block is, but those are just simple multiplications and bit operations, no hashes) so that a hashWholeBlock value can be added to the header, so as to prove that whoever found a block solution was also the same person to sign the coinbase and know about the whole blocks content, before any pool did.).
member
Activity: 81
Merit: 1002
It was only the wind.
Why over 3000 SHA256 hashes were used, I'll never know. What's the URL for Slack?

Yeah, I'm sure mr.spread's mining algo can be optimized.
The reason we use a little SHA256 in our algo is because the miner has to provide a hashWholeBlock value (to prove he himself had first knowledge about the block content, called POK "Proof Of Knowledge" in the sourcecode),
which ofcourse is derived thru a common double SHA256, and which has to be recalculated whenever ANYTHING in the block changes (timestamp, nonce (but only every 64 nonces), txs, etc...)

It's explained in mr. spread's whitepaper : http://www.spreadcoin.info/downloads/SpreadCoin-WhitePaper.pdf
(which admittedly is a little bit cryptic and not using the "clearest" english... I'm working on a more readable whitepaper version when servicenodes have progressed a little bit)

No, I know that. What I don't know is why you need 3000 goddamned iterations.

I'm not sure about that either.

Mr. Spread uses the hashPrevBlock (which is known and stays the same while we hash) to fill up the padding he's using to construct a complete MAX SIZE block which is then used to create the hashWholeBlock value.
So you probably assume that it would be enough if we derived the hashWholeBlock just once after a miner has found a solution (since hashWholeBlock is not part of the actual POW).

But I'm not sure if we are not missing something important here.

I wouldn't mind if a mining specialist like you takes a close look at it.  Wink


Nope. There is ZERO reason to hash the same data that many times. It's barbaric overkill.

I'm currently occupied with servicenode development, and I was going to look into mining optimization only after that.
But if you want to give it a try and optimize the mining code, you are highly welcomed, and I'm sure the community is going to reward your efforts.

BTW, nobody doubts that mr. spread's code can be optimized. He left us in freaking mid-air.


Mining optimization meaning the current algo?
legendary
Activity: 1526
Merit: 1001
Crypto since 2014
Mr. Spread first created an AMD gpu miner, around 20th Nov 2014:

https://bitcointalksearch.org/topic/m.9606917

And then during the next month it was decided that girino's GPU miner was "the better one" ?

https://github.com/girino/spreadcoinx11-sgminer

Anyway...
Ah so it might be easier to modify Mr. Spread's miner.
Anyway Girino has some explaining to do.

Girino's IS better, and as I said, this is not a crippling of the miner, it's just WTF.
It's not crippling? Ok that's good, I thought it was.
Does the hashing of all that data slow it down very much?
legendary
Activity: 2870
Merit: 1091
--- ChainWorks Industries ---
Looks like I oopsed, they're not hashing the same thing over 3k times, they're hashing an INSANE amount of data. What the fuck is all of this...?

i never used the amd miner wolf - ever ...

it never compiled properly for me - and never worked properly for me ...

thefarm ( nvidia based ) was the only thing that was working well based on sp's spreadminer ( which was obviously based on ccminer-tsiv ) and nonce-pool spreadcoin private pool ...

so if you are building an amd miner - could you also see ( if its part of your scope with the build ) if it can be integrated with the same sgminer as x11 / quark / lyra2rev2 / neoscrypt? ...

i am in the office again - so im back on irc ... i have some updates about farmamd ...

i have a softspot for spreadcoin ( and ftc for that matter ) but have never been happy about the algo itself ...

it has always been a difficult algo to implement on theafrm due to changing miners constantly whenever i wanted to accumulate ( spreadminer as opposed to ccminer-spmod ) - and im not the only one that has had this particular issue ... so an integration of the algo in the same sgminer would be a HUGE advantage ...

#crysx
member
Activity: 81
Merit: 1002
It was only the wind.
Why over 3000 SHA256 hashes were used, I'll never know. What's the URL for Slack?

Yeah, I'm sure mr.spread's mining algo can be optimized.
The reason we use a little SHA256 in our algo is because the miner has to provide a hashWholeBlock value (to prove he himself had first knowledge about the block content, called POK "Proof Of Knowledge" in the sourcecode),
which ofcourse is derived thru a common double SHA256, and which has to be recalculated whenever ANYTHING in the block changes (timestamp, nonce (but only every 64 nonces), txs, etc...)

It's explained in mr. spread's whitepaper : http://www.spreadcoin.info/downloads/SpreadCoin-WhitePaper.pdf
(which admittedly is a little bit cryptic and not using the "clearest" english... I'm working on a more readable whitepaper version when servicenodes have progressed a little bit)

No, I know that. What I don't know is why you need 3000 goddamned iterations.

I'm not sure about that either.

Mr. Spread uses the hashPrevBlock (which is known and stays the same while we hash) to fill up the padding he's using to construct a complete MAX SIZE block which is then used to create the hashWholeBlock value.
So you probably assume that it would be enough if we derived the hashWholeBlock just once after a miner has found a solution (since hashWholeBlock is not part of the actual POW).

But I'm not sure if we are not missing something important here.

I wouldn't mind if a mining specialist like you takes a close look at it.  Wink


Nope. There is ZERO reason to hash the same data that many times. It's barbaric overkill.
legendary
Activity: 1526
Merit: 1001
Crypto since 2014
Mr. Spread first created an AMD gpu miner, around 20th Nov 2014:

https://bitcointalksearch.org/topic/m.9606917

And then during the next month it was decided that girino's GPU miner was "the better one" ?

https://github.com/girino/spreadcoinx11-sgminer

Anyway...
Ah so it might be easier to modify Mr. Spread's miner.
Anyway Girino has some explaining to do.
legendary
Activity: 1484
Merit: 1007
spreadcoin.info
Mr. Spread first created an AMD gpu miner, around 20th Nov 2014:

https://bitcointalksearch.org/topic/m.9606917

And then during the next month it was decided that girino's GPU miner was "the better one" ?

https://github.com/girino/spreadcoinx11-sgminer

Anyway...
member
Activity: 81
Merit: 1002
It was only the wind.
Why over 3000 SHA256 hashes were used, I'll never know. What's the URL for Slack?

Yeah, I'm sure mr.spread's mining algo can be optimized.
The reason we use a little SHA256 in our algo is because the miner has to provide a hashWholeBlock value (to prove he himself had first knowledge about the block content, called POK "Proof Of Knowledge" in the sourcecode),
which ofcourse is derived thru a common double SHA256, and which has to be recalculated whenever ANYTHING in the block changes (timestamp, nonce (but only every 64 nonces), txs, etc...)

It's explained in mr. spread's whitepaper : http://www.spreadcoin.info/downloads/SpreadCoin-WhitePaper.pdf
(which admittedly is a little bit cryptic and not using the "clearest" english... I'm working on a more readable whitepaper version when servicenodes have progressed a little bit)

No, I know that. What I don't know is why you need 3000 goddamned iterations.
Jump to: