Author

Topic: [ANN][BURST] Burst | Efficient HDD Mining | New 1.2.3 Fork block 92000 - page 461. (Read 2171057 times)

hero member
Activity: 527
Merit: 500
Hmmm maybe I'm not fully understanding this. Looking at the read numbers in Resource Monitor, on the Intel setup, it's reading the drives at about 16Mb/s, there is no way it could finish the entire drive in 106s at that speed. So mining doesn't read the whole drive, just parts of it?
PoC reads 1 4096th of the data each block, or more simply 256MB / TB.

PoC2 will most likely be around 16MB / TB

So PoC2 will lighten hardware requirements extensively? Will PoC2 require a replot?

I imagine it will require a replot, since it entails a different way of storing nonces.
You did get it correctly with regards to optimizing and you should optimize your plots. Before I optimized mine you could really hear the drives seeking like mad Cheesy

Unfortunately I cannot recall now who already did raspberry pi tests in this thread. That member came to the conclusion that it is good for, at most, 8tb. IIRC he did plot on another intel machine.
legendary
Activity: 1932
Merit: 1042
https://locktrip.com/?refId=40964

Mmmm... I was thinking to try mining with raspberry pi b+ to save electricity, and I don't it is possible now after reading you post above. I think that the reason why I did not see anyone mine burstcoin with raspberry pi yet...

Why don't you try, likr a proof-of-concept....?  Smiley

It would be quite interesting to know how a raspberry would do, especially as we call BURST "enegy-friendly".

I'm already planning a test Wink I'll keep ya posted.


If crowetic already planned to start the raspberry test, and since I do not have a raspberry on hand yet, then I will wait for your proof-of-concept,  Smiley


I think that a raspberry pi has to low cpu-capacities for calculating those hashes. Maybe it'll work for <1TB, but not more. You also need a lot of cpu-power for the USB-Port (data transfer).

i think raspberry,
here is intendend to mine.
not to plot files...

to plot file you need  high end CPU o a good GPU.
i think 280X.
but with the latest CPU [i7 haswell] plotter i can see the performance are similar...
and more...
if you use CPU plotter you can rise stagger size to optimize plot...
newbie
Activity: 50
Merit: 0

Mmmm... I was thinking to try mining with raspberry pi b+ to save electricity, and I don't it is possible now after reading you post above. I think that the reason why I did not see anyone mine burstcoin with raspberry pi yet...

Why don't you try, likr a proof-of-concept....?  Smiley

It would be quite interesting to know how a raspberry would do, especially as we call BURST "enegy-friendly".

I'm already planning a test Wink I'll keep ya posted.


If crowetic already planned to start the raspberry test, and since I do not have a raspberry on hand yet, then I will wait for your proof-of-concept,  Smiley


I think that a raspberry pi has to low cpu-capacities for calculating those hashes. Maybe it'll work for <1TB, but not more. You also need a lot of cpu-power for the USB-Port (data transfer).
sr. member
Activity: 423
Merit: 250
Hmmm maybe I'm not fully understanding this. Looking at the read numbers in Resource Monitor, on the Intel setup, it's reading the drives at about 16Mb/s, there is no way it could finish the entire drive in 106s at that speed. So mining doesn't read the whole drive, just parts of it?
PoC reads 1 4096th of the data each block, or more simply 256MB / TB.

PoC2 will most likely be around 16MB / TB

So PoC2 will lighten hardware requirements extensively? Will PoC2 require a replot?
legendary
Activity: 1932
Merit: 1042
https://locktrip.com/?refId=40964
Hmmm maybe I'm not fully understanding this. Looking at the read numbers in Resource Monitor, on the Intel setup, it's reading the drives at about 16Mb/s, there is no way it could finish the entire drive in 106s at that speed. So mining doesn't read the whole drive, just parts of it?
PoC reads 1 4096th of the data each block, or more simply 256MB / TB.

PoC2 will most likely be around 16MB / TB

hi dev!!!

can you explain in a few words,
how can ensure transaction/blockchain consistency with 1/16 of data?

and then... for the future

we will have this scenario?

1) burst --> POC
2) burts2[new coin] -->POC2

or we will have this scenario?

1) only burst with a new implementation from POC --> POC2?


thanks

PoC1 and 2 will be storing different things. PoC1 is storing a bunch of hashes which are used each block, but PoC2 will be storing nonces that meet certain conditions. PoC1 stores 64bytes / scoop / nonce, but for PoC2 only 4 bytes / scoop / nonce is needed since the nonces can be stored as 4 byte differences between them to save space.

PoC2 will be added to burst, and you'll be able to use either. At some point in the future PoC1 might have to be disabled, although that will not happen soon.

If you're interested in more in-depth details as how PoC2 will work, or why it is better, you might want to read this post https://bitcointalksearch.org/topic/m.10222877

Thank you for the link dev!
I'm going to read it!

Have a nice day!
sr. member
Activity: 280
Merit: 250
Hmmm maybe I'm not fully understanding this. Looking at the read numbers in Resource Monitor, on the Intel setup, it's reading the drives at about 16Mb/s, there is no way it could finish the entire drive in 106s at that speed. So mining doesn't read the whole drive, just parts of it?
PoC reads 1 4096th of the data each block, or more simply 256MB / TB.

PoC2 will most likely be around 16MB / TB

hi dev!!!

can you explain in a few words,
how can ensure transaction/blockchain consistency with 1/16 of data?

and then... for the future

we will have this scenario?

1) burst --> POC
2) burts2[new coin] -->POC2

or we will have this scenario?

1) only burst with a new implementation from POC --> POC2?


thanks

PoC1 and 2 will be storing different things. PoC1 is storing a bunch of hashes which are used each block, but PoC2 will be storing nonces that meet certain conditions. PoC1 stores 64bytes / scoop / nonce, but for PoC2 only 4 bytes / scoop / nonce is needed since the nonces can be stored as 4 byte differences between them to save space.

PoC2 will be added to burst, and you'll be able to use either. At some point in the future PoC1 might have to be disabled, although that will not happen soon.

If you're interested in more in-depth details as how PoC2 will work, or why it is better, you might want to read this post https://bitcointalksearch.org/topic/m.10222877
legendary
Activity: 1932
Merit: 1042
https://locktrip.com/?refId=40964
Hmmm maybe I'm not fully understanding this. Looking at the read numbers in Resource Monitor, on the Intel setup, it's reading the drives at about 16Mb/s, there is no way it could finish the entire drive in 106s at that speed. So mining doesn't read the whole drive, just parts of it?
PoC reads 1 4096th of the data each block, or more simply 256MB / TB.

PoC2 will most likely be around 16MB / TB

hi dev!!!

can you explain in a few words,
how can ensure transaction/blockchain consistency with 1/16 of data?

and then... for the future

we will have this scenario?

1) burst --> POC
2) burts2[new coin] -->POC2

or we will have this scenario?

1) only burst with a new implementation from POC --> POC2?


thanks
hero member
Activity: 714
Merit: 500

Mmmm... I was thinking to try mining with raspberry pi b+ to save electricity, and I don't it is possible now after reading you post above. I think that the reason why I did not see anyone mine burstcoin with raspberry pi yet...

Why don't you try, likr a proof-of-concept....?  Smiley

It would be quite interesting to know how a raspberry would do, especially as we call BURST "enegy-friendly".

I'm already planning a test Wink I'll keep ya posted.


I bet that it's good for about 2.5TB of optimized Plots in roughly 35-45 seconds. A Beagle Bone Black would be interesting to see also, better RAM and CPU but still that poor little USB 2.0 bottleneck.

 I look forward to seeing the results
Good-Luck

If the USB 2.0 is the main issue, then maybe we can try banana pi with sata port, or maybe look for sata module for raspberry pi...
hero member
Activity: 714
Merit: 500

Mmmm... I was thinking to try mining with raspberry pi b+ to save electricity, and I don't it is possible now after reading you post above. I think that the reason why I did not see anyone mine burstcoin with raspberry pi yet...

Why don't you try, likr a proof-of-concept....?  Smiley

It would be quite interesting to know how a raspberry would do, especially as we call BURST "enegy-friendly".

I'm already planning a test Wink I'll keep ya posted.


If crowetic already planned to start the raspberry test, and since I do not have a raspberry on hand yet, then I will wait for your proof-of-concept,  Smiley
hero member
Activity: 1426
Merit: 506
Hmmm maybe I'm not fully understanding this. Looking at the read numbers in Resource Monitor, on the Intel setup, it's reading the drives at about 16Mb/s, there is no way it could finish the entire drive in 106s at that speed. So mining doesn't read the whole drive, just parts of it?
PoC reads 1 4096th of the data each block, or more simply 256MB / TB.

PoC2 will most likely be around 16MB / TB
Will PoC2 be adapted to Burst or are you planning on created a completely new coin?
sr. member
Activity: 280
Merit: 250
Hmmm maybe I'm not fully understanding this. Looking at the read numbers in Resource Monitor, on the Intel setup, it's reading the drives at about 16Mb/s, there is no way it could finish the entire drive in 106s at that speed. So mining doesn't read the whole drive, just parts of it?
PoC reads 1 4096th of the data each block, or more simply 256MB / TB.

PoC2 will most likely be around 16MB / TB
sr. member
Activity: 423
Merit: 250
Not sure if that's in response to me Merick, but the weird thing is when I watch the transfer rates, it never gets anywhere near the drive sequential speeds. The drives aren't being maxed out on random seeks or the utilization would be much higher. It seems like it's entirely CPU bottlenecked.

On my Intel rig, I removed all but one drive and got it down to 42s. So 42s for 5TB, 106s for 25TB.

Can someone explain what optimizing does? From what I understand it reorders the plot so it's all sequential reads instead of any seeks? Doesn't the plotter do this when you plot? I only have one plot per drive and it doesn't seem like there are a lot of random seeks happening. Would/how would this help me?
full member
Activity: 129
Merit: 100
I just did some calculations on my bottleneck and my issue is related to the PCIe 1.1 slot that I have my 4 port SATA III card installed in.

PCIe 1.1 has a max transfer rate of 250MB/s  split that between 4 drives and each drive can at most feed teh CPU data at 62.5MB/sec.  If I use itop in Linux to watch my I/O % rate I can see that my I/O goes from 50%-60% maybe a touch more sometimes.  This tells me that my CPU is waiting on data, bu it is pegged at 100% on both cores.... maybe its a miner optimization issue I'm note sure.

So, in my case under ideal conditions I am guessing that my system will peak out at 11.25TB to reach the 45 second marker.

62.5MB/sec max read,  256MB needs to be read per 1TB plot

4 seconds to read 1TB plot.

45sec / 4sec/TB

11.25TB can be scanned in that 45sec  for my crappy old Optiplex 380.

This is just quick 2:30am napkin math but it goes along with the current convo that mining burst is not just about having massive amounts of storage.

Or, maybe it's late and I am just missing something...  When I get to 11.25TB I will find out Smiley

sources:
http://blog.scoutapp.com/articles/2011/02/10/understanding-disk-i-o-when-should-you-be-worried
http://www.tested.com/tech/457440-theoretical-vs-actual-bandwidth-pci-express-and-thunderbolt/

Good-Luck
sr. member
Activity: 423
Merit: 250
Yeah, more I look at it, the more it seems like you need a pretty beefy processor when you start tacking on the TBs. I added more memory and it definitely helped (I'll check the auto pagefile thing too), but it seems to have hit a wall.

I'm currently trying out a low end Intel I also have and it's better, but it's still not good enough. I'm at 25TB in 106s (last five finished plotting today).

Are there any plans for a GPU assisted miner? It was said a few pages back it's not needed, but it sorta seems like it is. I can imagine the low end GPUs would be put to good use. Based on the block height chart it seems like you want to get in under 45s, lower the better.

I didn't know you lost your chance at the block if a new block is issued before you finish reading the last, something that probably should be a explained somewhere. It's pretty fundamental and important, but since PoC is new, I had no idea that was a thing.

Also should be listed somewhere that plotting also takes a decently beefy CPU, even while using a GPU plotter. I ran into a wall around 60k nonces a second using the GPU plotter. No matter the number of HDs or GPUs I threw at it, it wouldn't go higher.


Hmmm maybe I'm not fully understanding this. Looking at the read numbers in Resource Monitor, on the Intel setup, it's reading the drives at about 16Mb/s, there is no way it could finish the entire drive in 106s at that speed. So mining doesn't read the whole drive, just parts of it?
full member
Activity: 129
Merit: 100

Mmmm... I was thinking to try mining with raspberry pi b+ to save electricity, and I don't it is possible now after reading you post above. I think that the reason why I did not see anyone mine burstcoin with raspberry pi yet...

Why don't you try, likr a proof-of-concept....?  Smiley

It would be quite interesting to know how a raspberry would do, especially as we call BURST "enegy-friendly".

I'm already planning a test Wink I'll keep ya posted.


I bet that it's good for about 2.5TB of optimized Plots in roughly 35-45 seconds. A Beagle Bone Black would be interesting to see also, better RAM and CPU but still that poor little USB 2.0 bottleneck.

 I look forward to seeing the results
Good-Luck
legendary
Activity: 2282
Merit: 1072
https://crowetic.com | https://qortal.org

Mmmm... I was thinking to try mining with raspberry pi b+ to save electricity, and I don't it is possible now after reading you post above. I think that the reason why I did not see anyone mine burstcoin with raspberry pi yet...

Why don't you try, likr a proof-of-concept....?  Smiley

It would be quite interesting to know how a raspberry would do, especially as we call BURST "enegy-friendly".

I'm already planning a test Wink I'll keep ya posted.
sr. member
Activity: 462
Merit: 250

Mmmm... I was thinking to try mining with raspberry pi b+ to save electricity, and I don't it is possible now after reading you post above. I think that the reason why I did not see anyone mine burstcoin with raspberry pi yet...

Why don't you try, likr a proof-of-concept....?  Smiley

It would be quite interesting to know how a raspberry would do, especially as we call BURST "enegy-friendly".
hero member
Activity: 714
Merit: 500

I cannot find the specs on that CPU, is this a laptop?

When you say your at 2min for reading a block, does that mean it takes you 2min to scan your plot file?

I compared a AMD A4-4000 to my old Intel E5800 and the specs are sorta similar so I'm guessing your processor is similar to mine.

Everyone keeps saying that hard disk space is all that matters, but this is very misleading.  You also need a CPU that can process the data and a decent hard drive connection interface to get the data from the drive and into the CPU.  The hard-drive connection is more important that the CPU.

What is your stagger size on that 5TB plot?  If 8GB ram I am guessing your stagger is not greater than 10000.  What is your hard-drive connection?  if your reading this through USB 2.0 your max on paper read rate will be 60MB/s but take into account overhead and a low plot stagger and your probably looking at ~30MB/s.

I believe that 256MB/TB of plot size is read during each block (dev post on POC vs POCv2), if this is so then at 30MB/s it would take a USB 2.0 ~43 seconds to read the plot.  If your using windows go to the performance monitor --> Disks and watch the read rate for your drive, you can do the same thing in Linux.  This will give you an idea of were you are.

With the CPU I compared your's to I am able to read ~8TB of plots across 5 drives in about 35 seconds.  This speed will increase once I optimize my larger 3TB plots.

Reading a plot that is optimized will utilize your drives sequential read specifications.  Reading a plot that has a low stagger will utilize your drives random read specifications.  I higher stagger will move closer to optimized levels.  I have not created a graph showing this relationship so I do not know the cut-off point but there is a significant difference between the two ends of the scale.

Download a utility like CrystalDiskMark http://crystalmark.info/software/CrystalDiskMark/index-e.html or HDTune and see what your drive is capable of, especially the sequential read indicator, this will be the max you can read with an optimized plot.  Doing this will help pin point the bottle neck, and I am guessing it is not the CPU but a combination of hard drive connection type and low stagger.

* Edit * I just re-read your post, I missed the 4 x 5TB part.  Sorry.  But still basic benchmark testing to find bottle necks still applies.

Good-Luck

Mmmm... I was thinking to try mining with raspberry pi b+ to save electricity, and I don't it is possible now after reading you post above. I think that the reason why I did not see anyone mine burstcoin with raspberry pi yet...
full member
Activity: 129
Merit: 100
I saw him a few times in IRC I thought last week,  he showed up after his name was mentioned 3 times.... Kinda like Beatle-Juice.

The next post will be Uray  Tongue
|
|
V
legendary
Activity: 1932
Merit: 1042
https://locktrip.com/?refId=40964
Uray seems to have disappeared, strange it was right after my declining his offer to purchase all of his projects. Hoping he's in good health, and okay.

I hope uray is ok.
I mine over is pool from he start Singapore servers and then EU servers.
Great developer!
But....

The he disappear...

i hope he is fine.

Come back soon uray!
Jump to: