It's official, the on-GPU bloom filter code is not working on Nvidia Tesla K80. 
So what do u want me to put on those GPU k80 servers ?
Topic: Large Bitcoin Collider (Collision Finders Pool) - page 27. (Read 193404 times)
So what do u want me to put on those GPU k80 servers ?
It's official, the on-GPU bloom filter code is not working on Nvidia Tesla K80.
To be more precise, it's the simple single "printf" that's crashing the app.
The one single printf which shows - on rare occasions - a find. e.g.
I will probably make the on-GPU bloom selectable by command line option to navigate around this problem. Until then, if you have a K80 (which unfortunately is true for most of the cloud machines), the old GPU gnerator is working...
If you have newer machines like Maxwell or Pascal architectures and a Nvidia 375 or newer driver, chances are very good it will work for you. On the other hand, its quite probable if you have a Kepler GPU (or older), that the new code will bite you.
Rico
To be more precise, it's the simple single "printf" that's crashing the app.
The one single printf which shows - on rare occasions - a find. e.g.
Code:
2d17543d32448acc7a1c43c5f72cd5be459ab302:u:priv:0000000000000000000000000000000000000000000000000000000000000001 + 0x5e
02e62151191a931d51cdc513a86d4bf5694f4e51:c:priv:0000000000000000000000000000000000000000000000000000000000000001 + 0x65
9d74ffdb31068ca2a1feb8e34830635c0647d714:u:priv:00000000000000000000000000000000000000000000000000000000000f9001 + 0xf8c
3d6871076780446bd46fc564b0c443e1fd415beb:c:priv:00000000000000000000000000000000000000000000000000000000000f9001 + 0xf8c
I will probably make the on-GPU bloom selectable by command line option to navigate around this problem. Until then, if you have a K80 (which unfortunately is true for most of the cloud machines), the old GPU gnerator is working...
If you have newer machines like Maxwell or Pascal architectures and a Nvidia 375 or newer driver, chances are very good it will work for you. On the other hand, its quite probable if you have a Kepler GPU (or older), that the new code will bite you.
Rico
ubuntu@instance-5:~/collider$ ./LBC -u
Finished update run - system up to date.
ubuntu@instance-5:~/collider$ ./LBC -x
Testing mode. Using page 0, turning off looping.
Benchmark info not found - benchmarking... done.
Your maximum speed is 511909 keys/s per CPU core.
Finished update run - system up to date.
ubuntu@instance-5:~/collider$ ./LBC -x
Testing mode. Using page 0, turning off looping.
Benchmark info not found - benchmarking... done.
Your maximum speed is 511909 keys/s per CPU core.
Nothing happens
But ...
Ask for work... got blocks [1466349440-1466357631] (8589 Mkeys)
went to
Ask for work... got blocks [1466443152-1466452879] (10200 Mkeys)
I think it did update somehow.
[/quote]
If there was an output after LBC -u (new generator found) yes. Else: no
LBC -u must fetch a new generator. If it doesn't (for whatever reasons), you must fetch it manually (unpack, rename, chmod a+x, yadda yadda)
then LBC -x, then profit
Rico
ubuntu@instance-5:~/collider$ ./LBC -u
Finished update run - system up to date.
ubuntu@instance-5:~/collider$ ./LBC -x
Testing mode. Using page 0, turning off looping.
Benchmark info not found - benchmarking... done.
Your maximum speed is 511909 keys/s per CPU core.
Nothing happens
But ...
Ask for work... got blocks [1466349440-1466357631] (8589 Mkeys)
went to
Ask for work... got blocks [1466443152-1466452879] (10200 Mkeys)
I think it did update somehow.
tried with sudo ./LBC -u nothing happens.
Finished update run - system up to date.
ubuntu@instance-5:~/collider$ ./LBC -x
Testing mode. Using page 0, turning off looping.
Benchmark info not found - benchmarking... done.
Your maximum speed is 511909 keys/s per CPU core.
Nothing happens
But ...
Ask for work... got blocks [1466349440-1466357631] (8589 Mkeys)
went to
Ask for work... got blocks [1466443152-1466452879] (10200 Mkeys)
I think it did update somehow.
tried with sudo ./LBC -u nothing happens.
Hi colliders!
New versions of the SSE42, AVX2 and Skylake hrd-core generators are now available. These use the arulbero-ECC library instead of the libsecp256k1 and give a nice keyrate bump. It's not as big as for the GPU-accelerated versions (soon to come), but still pretty decent. It seems the sse42 version profits most.
SSE42: +79%
AVX2: +29%
Skylake: +6%
There will be no SSE41 or 32bit-generic versions available. If you run one of these, put a "no_update": 1, line in your lbc.json.
edit: but there will be a 64bit generic binary available!
How to update?
1) End your LBC by pressing "e" and wait until you see the shell prompt.
2) type in ./LBC -u, it will update the generator (and may even the BLF)
3) type in ./LBC -x to test and benchmark the new generator. You should see a better maximum keyrate.
4) Ready to go!
Rico
New versions of the SSE42, AVX2 and Skylake hrd-core generators are now available. These use the arulbero-ECC library instead of the libsecp256k1 and give a nice keyrate bump. It's not as big as for the GPU-accelerated versions (soon to come), but still pretty decent. It seems the sse42 version profits most.
SSE42: +79%
AVX2: +29%
Skylake: +6%
There will be no SSE41 or 32bit-generic versions available. If you run one of these, put a "no_update": 1, line in your lbc.json.
edit: but there will be a 64bit generic binary available!
How to update?
1) End your LBC by pressing "e" and wait until you see the shell prompt.
2) type in ./LBC -u, it will update the generator (and may even the BLF)
Code:
# LBC -u
New generator found. (DL-size: 0.54MB)
BLF patch found. (DL-size: 206.96MB)
Finished update run - system up to date.
3) type in ./LBC -x to test and benchmark the new generator. You should see a better maximum keyrate.
4) Ready to go!
Rico
So your prohability with 14 million addresses for each key you are generating at the moment is about:
1-e^((-2^23.7389*(2^23.7389-1))/(2*2^160)) = 6.70521 × 10^-35
or simpler as we have big numbers:
(2^23.7389)^2/(2*2^160) = 6.70521 × 10^-35
1-e^((-2^23.7389*(2^23.7389-1))/(2*2^160)) = 6.70521 × 10^-35
or simpler as we have big numbers:
(2^23.7389)^2/(2*2^160) = 6.70521 × 10^-35
Interesting. I thought the probability is 1/2(160-23.7389) = ~ 9.52 x 10-42
Now I am at the moment in the middle of the rollout of the new client, so I cannot dig deeper into this, but I will certainly have a look for the explanation of this 7 orders of magnitude discrepancy.
Thanks!
Rico
This revelation is of equal, if not greater, significance to Alan Guth's discovery in 1979 of the exponential expansion of the universe, doubling in size 100 times for 10-35 seconds right after the Big Bang.
An instant 7-magnitude increase in the Bitcoin Big Banger, in one post! This should win the Nobel Cryptocurrency Prize.
I see the client is created in perl, will there be support for windows? strawberry perl or such.
An ISO Live Image may be good for people who have a GPU and are not running linux.
Cuda and OCL?
An ISO Live Image may be good for people who have a GPU and are not running linux.
Cuda and OCL?
At the moment an easy-to-use ISO for windows users would be the best option (in any case for the GPU client, but also welcome for the CPU client).
The generator is written in C/OpenCL. As for a native windows client: The Perl client (LBC) is not the problem (it does run on Strawberry and we had a windows version already - the source still has rudimentary support for it in it's DNA), the problem is the generator using some libc memory management intrinsics.
=> I would happily give GPUauth=1 to anybody who would do a decent easy-to-use robust GPU (Nvidia and AMD) and CPU support LBC image ISO.
Of course it would still have to go through a rigorous review process for trojans/viruses etc. but I believe it would be a very welcome addition for the Win-users with nifty GPUs.
Ideally, it would be a minimal Linux which would boot from USB stick. As for "live-image" - LBC does auto-update itself (BLF, generator and the LBC itself), so there should be a writable partition on the USB stick else people would have to load a new version of the ISO over and over again. (and I wouldn't have the time to review the images that often - thus not authorize them)
Rico
I can have a look at this as I have a fair bit of experience with linux and arch seems a good use for this. dont worry about the gpuauth=1 yet, il see if I can get an iso up and running with the client and right dependencies installed first and then I can get back to you on that. Also being arch and minimal it will be easier to go over for the review process.
I also have a fair bit of time on my hands at the moment, so it may do me some good
I see the client is created in perl, will there be support for windows? strawberry perl or such.
An ISO Live Image may be good for people who have a GPU and are not running linux.
Cuda and OCL?
An ISO Live Image may be good for people who have a GPU and are not running linux.
Cuda and OCL?
At the moment an easy-to-use ISO for windows users would be the best option (in any case for the GPU client, but also welcome for the CPU client).
The generator is written in C/OpenCL. As for a native windows client: The Perl client (LBC) is not the problem (it does run on Strawberry and we had a windows version already - the source still has rudimentary support for it in it's DNA), the problem is the generator using some libc memory management intrinsics.
=> I would happily give GPUauth=1 to anybody who would do a decent easy-to-use robust GPU (Nvidia and AMD) and CPU support LBC image ISO.
Of course it would still have to go through a rigorous review process for trojans/viruses etc. but I believe it would be a very welcome addition for the Win-users with nifty GPUs.
Ideally, it would be a minimal Linux which would boot from USB stick. As for "live-image" - LBC does auto-update itself (BLF, generator and the LBC itself), so there should be a writable partition on the USB stick else people would have to load a new version of the ISO over and over again. (and I wouldn't have the time to review the images that often - thus not authorize them)
Rico
edit: I see you all left the #666 post for me - how nice of you.
I see the client is created in perl, will there be support for windows? strawberry perl or such.
An ISO Live Image may be good for people who have a GPU and are not running linux.
Cuda and OCL?
An ISO Live Image may be good for people who have a GPU and are not running linux.
Cuda and OCL?
So your prohability with 14 million addresses for each key you are generating at the moment is about:
1-e^((-2^23.7389*(2^23.7389-1))/(2*2^160)) = 6.70521 × 10^-35
or simpler as we have big numbers:
(2^23.7389)^2/(2*2^160) = 6.70521 × 10^-35
1-e^((-2^23.7389*(2^23.7389-1))/(2*2^160)) = 6.70521 × 10^-35
or simpler as we have big numbers:
(2^23.7389)^2/(2*2^160) = 6.70521 × 10^-35
Interesting. I thought the probability is 1/2(160-23.7389) = ~ 9.52 x 10-42
Now I am at the moment in the middle of the rollout of the new client, so I cannot dig deeper into this, but I will certainly have a look for the explanation of this 7 orders of magnitude discrepancy.
Thanks!
Rico
@rico666
"....Because we assume the private keys to addresses with funds on them are also uniformly distributed in the search space, this means that the effective search space until any collision for these is found is 160bit - log2(9000000)bit = ~137bit."
I might be wrong, but because of this (wolfram alpha expression):
1-(((2^21)! * (2^160 über 2^21))/((2^160)^(2^21)))
"....Because we assume the private keys to addresses with funds on them are also uniformly distributed in the search space, this means that the effective search space until any collision for these is found is 160bit - log2(9000000)bit = ~137bit."
I might be wrong, but because of this (wolfram alpha expression):
1-(((2^21)! * (2^160 über 2^21))/((2^160)^(2^21)))
I'm not sure I follow this. (mathematically I do, but semantically I don't)
Where do you get the 221 from? Thats 2M.
Quote
I assume the possibilty of hitting one of your approximately 2,5 mil target addresses is:
1.5*10^-36 or 1/(2^119)
1.5*10^-36 or 1/(2^119)
We have 14 mil target addresses at the moment.
Quote
If you agree with the things written above, I would recommend to expand the target address space to 2^23 (8.x mil pub keys)
and gain a 4 times higher possibility to hit one of your targets:
2.4*10^-35 or or 1/(2^115)
and gain a 4 times higher possibility to hit one of your targets:
2.4*10^-35 or or 1/(2^115)
It's still 160 - log2(14000000) = ~ 136.26bit search space
Rico
Hello Rico,
what I wanted to submit was that in terms of prohability you should consider to expand your public address list for comparision from about 2.x million to 8.y million pub keys. I have read somewhere, maybe in some older postings, that your address list covers only 2.5 million addresses. As you know the birthday paradox (https://en.wikipedia.org/wiki/Birthday_problem) says with an increasing number of persons (in our example: public addresses) the prohability to find 2 with the same birthdays (in our example: RIPEMD-160 hash number) increases exponentially.
2 million addesses give us a prohability of:
1-(((2^21)! * (2^160 über 2^21))/((2^160)^(2^21))) = 1.504 × 10^-36
or differently:
1-e^((-2^21*(2^21-1))/(2*2^160)) = 1.504 × 10^-36
8 million addesses give us a prohability of:
1-e^((-2^23*(2^23-1))/(2*2^160)) =2.407 × 10^-35
4 times more public addresses give a 16 times higher prohability to hit one target. That´s why I suggested to expand the public key address space. Want I did not know at the time of writing that you already expand your list to 14 million addresses.
So your prohability with 14 million addresses for each key you are generating at the moment is about:
1-e^((-2^23.7389*(2^23.7389-1))/(2*2^160)) = 6.70521 × 10^-35
or simpler as we have big numbers:
(2^23.7389)^2/(2*2^160) = 6.70521 × 10^-35
Besides that, I came across an interessting article about increasing the prohability of collision through interating hash functions:
https://security.stackexchange.com/questions/61756/wont-all-hashes-collide-after-enough-iterations-with-a-static-salt
and
http://crypto.stackexchange.com/questions/15068/entropy-when-iterating-cryptographic-hash-functions
N(hash1)>N(hash2)>N(hash3)>N(hash4)....>N(hashx)
due to collisions and the lacking to use the full 256 rsp. 160 bit space.
4 hashes have to be done to create a bitcoin address. This is surely reducing the number of different pub addresses further.
The collisions and mapping errors of each hash level causing an exponential effect in reducing the output.
That is giving you a better prohability than 6.70521 × 10^-35.
Reagrds,
Janu$$
Hi rico,
good work! The new beta-client does a really nice job here:
A big jump compared to the "old" client and his ~6.6 Mkeys/s
good work! The new beta-client does a really nice job here:
Quote
real-duke@C1-Ubuntu:~/gcollider$ ./LBC
GPU authorized: yes
Will use 4 CPUs.
Ask for work... got blocks [1439310448-1439333807] (24494 Mkeys)
oooooooooooo -snip- ooooooooooooooooooooooo (20.32 Mkeys/s)
with "time": 20 (my default setting)GPU authorized: yes
Will use 4 CPUs.
Ask for work... got blocks [1439310448-1439333807] (24494 Mkeys)
oooooooooooo -snip- ooooooooooooooooooooooo (20.32 Mkeys/s)
A big jump compared to the "old" client and his ~6.6 Mkeys/s
I'm going to consolidate the DB a little bit. There are Ids that evidently made it so far as of installing the LBC client and even fetching work, setting the password, but never delivered a single block, so I'm going to clean up that cruft (except unlimitedwings) as no work is lost.
"Earlier access"
Anyone with GPUauth=1, a Skylake or a AVX2 capable CPU and a GPU of course, please PM me if you'd like to try the newdouble-speed (that was for me, for you it's) triple-speed client before official release.
Rico
Code:
201 d569c66deb88b4547fd880d1f7a6d7e5 0 n/a
202 Broutinjoel 0 n/a
203 panglaoIsland 0 n/a
204 2d3c673afd7a8ed0c3ce1d839c6787af 0 n/a
205 _1_2_3_4_ 0 n/a
206 cc575cfb261730ae3d27950af8514acd 0 41d 12h 5m 6s
207 Hoolakawoola 0 n/a
208 A_N_T_R_a_t 0 47d 44m 55s
209 Pelumpu_ 0 28d 13h 35m 54s
210 f21c9d1b1b8a74a7c988b0a1f890c151 0 n/a
211 unlimitedwings 0 15h 8m 55s
212 fddf2ed38e2046733ea7ac2ab72439ec 0 95d 18h 50m 25s
213 9fa2f89ede80442a0fcc7ecd17667770 0 n/a
214 d99429a21f453dd575644a291daa517a 0 n/a
215 _TOM______ 0 n/a
216 DarkMann 0 n/a
217 ratel007 0 n/a
218 ExploitAgency 0 19d 14h 55m 4s
219 tg43te4t534 0 n/a
"Earlier access"
Anyone with GPUauth=1, a Skylake or a AVX2 capable CPU and a GPU of course, please PM me if you'd like to try the new
Rico
Stop the excessive quoting or else. I have also nuclear options.
Rico
Rico
24 hours with no quote, no reply??!! I saw my life colliding before my eyes!
I apologize for myself, becoin, and everybody else, going forward for the next 2160 posts.
@rico666
"....Because we assume the private keys to addresses with funds on them are also uniformly distributed in the search space, this means that the effective search space until any collision for these is found is 160bit - log2(9000000)bit = ~137bit."
I might be wrong, but because of this (wolfram alpha expression):
1-(((2^21)! * (2^160 über 2^21))/((2^160)^(2^21)))
"....Because we assume the private keys to addresses with funds on them are also uniformly distributed in the search space, this means that the effective search space until any collision for these is found is 160bit - log2(9000000)bit = ~137bit."
I might be wrong, but because of this (wolfram alpha expression):
1-(((2^21)! * (2^160 über 2^21))/((2^160)^(2^21)))
I'm not sure I follow this. (mathematically I do, but semantically I don't)
Where do you get the 221 from? Thats 2M.
Quote
I assume the possibilty of hitting one of your approximately 2,5 mil target addresses is:
1.5*10^-36 or 1/(2^119)
1.5*10^-36 or 1/(2^119)
We have 14 mil target addresses at the moment.
Quote
If you agree with the things written above, I would recommend to expand the target address space to 2^23 (8.x mil pub keys)
and gain a 4 times higher possibility to hit one of your targets:
2.4*10^-35 or or 1/(2^115)
and gain a 4 times higher possibility to hit one of your targets:
2.4*10^-35 or or 1/(2^115)
It's still 160 - log2(14000000) = ~ 136.26bit search space
Rico
A thief that tried but didn't succeed is still a thief, right?
No, Nothing has been stolen, A thieve who steals nothing is a nothing.
HERE WE ASK AAAAAGAIN:
1. Prove you have the privkey.
2. Prove it is not yours.
1. Prove you have the privkey.
2. Prove it is not yours.
Proving one has the privkey to a 80k BTC address discards the need for any further proof down the line.
The laws of physics don't apply anymore etc. https://www.youtube.com/watch?v=G_Cvtr86H5g
And your notorious requirement of negative proof... almost looks like you beg for upgrading your title from mascot to jester.
Rico
Speaking of physics, finding a collision is similar to quantum particles, only the observer really knows, lol.
Speaking of privkeys, finding a privkey is different to finding a collision, lol.
Yes, but finding a privkey to an existing address with funds on it are proof of a collision.
NO!
It just proves you've "found" the privkey for that bitcoin address. To find a collision you have to find TWO privkeys to the same bitcoin address!
Ugh, my God, you want a collision in EC.
Ugh, yeah! I told ya from the very beginning.The name of this "project" is misleading!
"Large Bitcoin Private Key Finder"
It can serve both purposes!
And that doesn't sound nearly as cool.
Of course. I can understand this. A pool of "thieves" doesn't sound nearly as cool as pool of "researchers".
Or you just pretend to be the Bitcoin Black Swan? lol
Now now, you can't call anyone a thief if they haven't stolen anything, right?
A thief that tried but didn't succeed is still a thief, right?
As some of you remember, arulbero proposed some changes to the ECC generation process.
Meanwhile, he also implemented a drop-in replacement of EC computations and I integrated it.
After a frenzy search for a bug which I introduced by premature optimization ... the new LBC client works!
It ran now through for 7 hours straight, delivering over 530 Gkeys. (~ 21 Mkeys/s)
To give you a comparison: The current GPU client for download does 7.5 Mkeys/s on my notebook.
My development version (bloom@GPU, some optimizations) did 9 Mkeys/s.
Now, that libsecp256k1 got replaced by the arulbero-ECC: 19-20 Mkeys/s
GPU usage up from 34% to roughly 80% - there is still a capacity margin we have ;-)
PS: So of course I will make this version available after some more tests and compiling the packages.
Rico
Meanwhile, he also implemented a drop-in replacement of EC computations and I integrated it.
After a frenzy search for a bug which I introduced by premature optimization ... the new LBC client works!
It ran now through for 7 hours straight, delivering over 530 Gkeys. (~ 21 Mkeys/s)
Code:
Ask for work... got blocks [1406666464-1406672415] (6241 Mkeys)
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo (19.91 Mkeys/s)
Code:
$ nvidia-smi
Sat Mar 25 07:37:19 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 378.13 Driver Version: 378.13 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Quadro M2000M Off | 0000:01:00.0 Off | N/A |
| N/A 63C P0 N/A / N/A | 2201MiB / 4042MiB | 81% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 21092 C ./gen-hrdcore-skylake+gpu-linux64 540MiB |
| 0 21094 C ./gen-hrdcore-skylake+gpu-linux64 540MiB |
| 0 21095 C ./gen-hrdcore-skylake+gpu-linux64 540MiB |
| 0 21096 C ./gen-hrdcore-skylake+gpu-linux64 540MiB |
+-----------------------------------------------------------------------------+
To give you a comparison: The current GPU client for download does 7.5 Mkeys/s on my notebook.
My development version (bloom@GPU, some optimizations) did 9 Mkeys/s.
Now, that libsecp256k1 got replaced by the arulbero-ECC: 19-20 Mkeys/s
GPU usage up from 34% to roughly 80% - there is still a capacity margin we have ;-)
PS: So of course I will make this version available after some more tests and compiling the packages.
Rico
Stop the excessive quoting or else. I have also nuclear options.
Rico
Rico
Take it easy, Putin.
Stop the excessive quoting or else. I have also nuclear options.
Rico
Rico
Jump to: