Large Bitcoin Collider Thread 2.0 - page 19.

Real-Duke

legendary

Activity: 3486

Merit: 2287

Wheel of Whales 🐳

Quote from: rico666 on May 01, 2017, 09:37:42 AM

edit: Version 1.140 on Server which does not need this workaround. Problem was only in "LBC -x" - not in normal operation.

Hmm but ./LBC -x still gives an error here!
Starting with ./LBC seems to work...but correct way?

Code:

real-duke@C1-Ubuntu:~/gcollider$ ./LBC -u
New client '1.140-LBC.bz2' found.
Finished update run - system up to date.

real-duke@C1-Ubuntu:~/gcollider$ ./LBC -x
GPU authorized: yes
Will use 4 CPUs.
Testing mode. Using page 0, turning off looping.
Benchmark info not found - benchmarking... done.
Your maximum speed is 5246127 keys/s per CPU core.
Generator validity check failed.
real-duke@C1-Ubuntu:~/gcollider$ ./LBC
GPU authorized: yes
Will use 4 CPUs.
Ask for work... got blocks [4941330847-4941353694] (23957 Mkeys)
oo
real-duke@C1-Ubuntu:~/gcollider$ ls -all
insgesamt 526140
drwxrwxr-x  2 real-duke real-duke      4096 Mai  1 21:38 .
drwxr-xr-x 28 real-duke real-duke      4096 Mai  1 21:31 ..
-rwxrwxrwx  1 real-duke real-duke     55018 Feb 19 13:41 1.021-LBC
-rwxrwxrwx  1 real-duke real-duke     55108 Mär  8 22:47 1.031-LBC
-r--r--r--  1 real-duke real-duke     63819 Mai  1 00:20 1.067-LBC
-r--r--r--  1 real-duke real-duke     63819 Mai  1 00:20 1.138-LBC
-rw-rw-r--  1 real-duke real-duke        28 Mai  1 21:34 bench.pst
-rwxrwxrwx  1 real-duke real-duke      4488 Mär  9 18:16 diagnostics-OpenCL.txt
-rw-rw-r--  1 real-duke real-duke 536870912 Apr 24 23:41 funds_h160.blf
-rwxrwxrwx  1 real-duke real-duke    395888 Apr 16 20:59 gen-hrdcore-avx2+gpu-linux64
-rwxrwxrwx  1 real-duke real-duke   1131720 Apr 16 20:31 gen-hrdcore-avx2-linux64
-rwxrwxrwx  1 real-duke real-duke     20814 Mär 26 13:45 hash160_deparsed.cl
-r-xr-xr--  1 real-duke real-duke     63826 Mai  1 21:33 LBC
-rwxrwxrwx  1 real-duke real-duke       124 Apr 16 22:56 lbc.json
real-duke@C1-Ubuntu:~/gcollider$

Real-Duke

legendary

Activity: 3486

Merit: 2287

Wheel of Whales 🐳

LOL nice rumours about my hardware Grin

C1 is my Intel Standalone PC and C2 is my little AMD Notebook...both are no ARM Systems as far as I know Wink

"C" means nothing more than "Computer" in this case.

rico666

legendary

Activity: 1120

Merit: 1037

฿ → ∞

Quote from: SopaXT on May 01, 2017, 11:26:00 AM

Rico, he's running it on a machine named "Ubuntu-C2", which suggests of it being an ODROID-C2. That's an ARM single board computer and LBC is not going to run on it.

I see "C1-Ubuntu", which I would now intuitively translate to "Collider1-Ubuntu".
Given the fact that he is running a GTX1060, I seriously doubt he has an ARM board below that.
Given another fact, that the LBC client has recognized AVX2, I would be very interested to see an ARM with avx2 features.

As mentioned, the problem was that "LBC -x" did not start the generator explicitely with the "./" current path prefix. It's fixed in 1.140

SopaXT

full member

Activity: 157

Merit: 113

Rico, he's running it on a machine named "Ubuntu-C2", which suggests of it being an ODROID-C2. That's an ARM single board computer and LBC is not going to run on it.

rico666

legendary

Activity: 1120

Merit: 1037

฿ → ∞

Quote from: shortcircuit on May 01, 2017, 09:06:33 AM

Can't exec "gen-hrdcore-avx2+gpu-linux64": File not found yadda

Temporary fix:

Code:

export PATH=$PATH:.

edit: Version 1.140 on Server which does not need this workaround. Problem was only in "LBC -x" - not in normal operation.

shortcircuit

member

Activity: 169

Merit: 14

Quote from: rico666 on May 01, 2017, 06:05:52 AM

Quote from: Real-Duke on May 01, 2017, 05:37:01 AM

have done the update to the new generator yesterday evening and all went fine over night.
Just now I have shutdown the LBC and make the following run:

Code:

real-duke@C1-Ubuntu:~/gcollider$ ./LBC -u
Finished update run - system up to date.

real-duke@C1-Ubuntu:~/gcollider$ ./LBC -x
GPU authorized: yes
Will use 4 CPUs.
Testing mode. Using page 0, turning off looping.
Benchmark info not found - benchmarking... Can't exec "gen-hrdcore-avx2+gpu-linux64": Datei oder Verzeichnis nicht gefunden at ./LBC line 2030.
Can't benchmark generator: No such file or directory at ./LBC line 2030.

The file "gen-hrdcore-avx2+gpu-linux64" is of course in the LBC directory with rights 777 but now you can see Im unable to start the collider again ( Have a working backup somewhere Wink

)

Hm. Nevertheless:

Code:

> ls -al

so I can see what is there and what not.

Already posted in the german section. Here's the ouput of ls -al

Quote

drwxr-xr-x 2 bert bert 4096 May 1 14:40 .
drwxr-xr-x 4 bert bert 4096 May 1 12:25 ..
-rw-r--r-- 1 root root 48076 May 1 14:40 diagnostics-LBC.txt
-rw-r--r-- 1 root root 536870912 May 1 14:23 funds_h160.blf
-r-xr-xr-- 1 root root 199200 May 1 14:19 gen-hrdcore-generic-linux64
-rwxr-xr-x 1 root root 63819 May 1 13:50 LBC

mrgiacalone

newbie

Activity: 2

Merit: 0

Quote

As I understand it, the use of GPU processing capacity may be unused depending on the number of cores the CPU has. Does this mean that running LBC on a multi-GPU rig with only an Intel Celeron Dual Core would not necessarily use all the GPUs capacity?

As of now, this is exactly what it means. I am thrilled to say, that this is actually something I will be working on next and GPU usage/saturation will raise at least by a factor of 2 again in the next couple of Huh

(days/few weeks).

Thanks for the reply.
At the moment I have a 6xGPU rig with a dead mobo/cpu that I will be splitting into two 3xGPU rigs. I intend on using one of them for new mining projects (of the like of LBC), so I'll be checking your updates (for the next days/few weeks) to do some testing on multi-GPU rigs. (Hopefully I'll get my GPU auth flag ready as well with my CPU). My main interest is on the performance with this low CPU high GPU combination, which are very common in GPU rigs at the moment.

Quote

As you can see in the stats The pool is slowing down, which is because our top-contributor https://lbc.cryptoguru.org/stats/Unknownhostname is shutting down capacities. Without him, the pool has around 200-250 Mkeys/s. (Maybe 400 if _KULME_ starts his monster machine Wink)

Therefore my next focus and priority will be again on the generator to squeeze some more speed out of it and also to make it available on more machines.

Perhaps this will also be a good time to increase incentives to a broader range of contributors to reduce dependence on fewer of them.

MRG

rico666

legendary

Activity: 1120

Merit: 1037

฿ → ∞

Quote from: Real-Duke on May 01, 2017, 05:37:01 AM

have done the update to the new generator yesterday evening and all went fine over night.
Just now I have shutdown the LBC and make the following run:

Code:

real-duke@C1-Ubuntu:~/gcollider$ ./LBC -u
Finished update run - system up to date.

real-duke@C1-Ubuntu:~/gcollider$ ./LBC -x
GPU authorized: yes
Will use 4 CPUs.
Testing mode. Using page 0, turning off looping.
Benchmark info not found - benchmarking... Can't exec "gen-hrdcore-avx2+gpu-linux64": Datei oder Verzeichnis nicht gefunden at ./LBC line 2030.
Can't benchmark generator: No such file or directory at ./LBC line 2030.

The file "gen-hrdcore-avx2+gpu-linux64" is of course in the LBC directory with rights 777 but now you can see Im unable to start the collider again ( Have a working backup somewhere Wink

)

Hm. Nevertheless:

Code:

> ls -al

so I can see what is there and what not.

Real-Duke

legendary

Activity: 3486

Merit: 2287

Wheel of Whales 🐳

Hi rico,

have done the update to the new generator yesterday evening and all went fine over night.
Just now I have shutdown the LBC and make the following run:

Code:

real-duke@C1-Ubuntu:~/gcollider$ ./LBC -u
Finished update run - system up to date.

real-duke@C1-Ubuntu:~/gcollider$ ./LBC -x
GPU authorized: yes
Will use 4 CPUs.
Testing mode. Using page 0, turning off looping.
Benchmark info not found - benchmarking... Can't exec "gen-hrdcore-avx2+gpu-linux64": Datei oder Verzeichnis nicht gefunden at ./LBC line 2030.
Can't benchmark generator: No such file or directory at ./LBC line 2030.
real-duke@C1-Ubuntu:~/gcollider$ ./LBC
GPU authorized: yes
Will use 4 CPUs.
Benchmark info not found - benchmarking... Can't exec "gen-hrdcore-avx2+gpu-linux64": Datei oder Verzeichnis nicht gefunden at ./LBC line 2030.
Can't benchmark generator: No such file or directory at ./LBC line 2030.
real-duke@C1-Ubuntu:~/gcollider$

The file "gen-hrdcore-avx2+gpu-linux64" is of course in the LBC directory with rights 777 but now you can see Im unable to start the collider again ( Have a working backup somewhere Wink

)

rico666

legendary

Activity: 1120

Merit: 1037

฿ → ∞

Quote from: SopaXT on May 01, 2017, 03:52:40 AM

I *do* have projects of my own - SopaXorzTaker on GitHub.

Hm. I see.

This is really my final take on it. All subsequent writeups of you in this thread will be deleted unseen. You may as well copy them to /dev/null
I'm answering under the theoretical premises your proposed PoW would actually be an unfakeable PoW - which it isn't. Which renders all of this discussion moot.
But hey - why not:

Quote

you'd only need 200*16 = 3200 keys per second to be generated, while my machine does ~800 kkey/sec.

You are comparing peas to apples. 800 Kkeys/s incremental key generation compared to 3200 starting keys.
For an incremental key, you simply have to do 1/2 of a G addition. For a starting key, you have to do n G multiplications (or additions if you have a precomputed table) where n is the number of 1-bits in your privkey. "your 800k-machine" can therefore do about 6666 starting keys/s in our current entropy range. Nice number, but too low.

Also, the LBC server is a VM and there is no chance to get a GPU in there. As of now, the server would be able to cope with around 50 Tkeys/s key generation capacity with no code changes whatsoever.

Please build YOUR OWN "LBC Server" project. SHOW me, how it's done by doing it, not by babbling about it.

Quote

I feel it necessary to argue, as I think some of your claims are incorrect, such as this one. I'd love to hear your feedback and arguments on that.

Again, your feeling is based on your ignorance (in this case about the key generation process). I did way more than I should to nurture your attention needs. Now begone.

SopaXT

full member

Activity: 157

Merit: 113

Quote from: rico666 on May 01, 2017, 02:57:55 AM

Quote from: SopaXT on May 01, 2017, 12:02:28 AM

Rico, did you see my suggestion above? Please reply, I really want to know if it's applicable.

No, it's not. It's a variation of what arulbero has suggested further above. To all the problems of your previous suggestion, this adds the problem of scalability to the server (We have - in principle - a 1:many relation between server and clients, and we cannot afford to let the server perform key generation computations).

Moreover:

Quote

For the time being, I consider all security-related issues resolved and am as of now not willing to participate in any discussion regarding "LBC security", novel proof-of-work suggestions and the like. If you must stretch this topic, please do so in v1 thread or show me some code. Else: 404

I have addressed every valid security concern in the LBC client. So far, you have brought nothing to the table of value. No working code, no proof of concept. You have no projects of your own, no track of record , no nothing. You only brought stir to the LBC project. But you consider it somehow (I honestly do not know where you take that self-confidence) legit to demand my attention and even answers. You are in no position for that. Am I being clear?

You are this close ----> <------ to my ignore list. Please read and understand in the 1st post of this thread what that means (key phrase is: retro-active). The only thing keeping you from there is, that it could be perceived as martyrdom if I simply kicked you there. But the time nears where I do not care. Your constant gnat buzzing is like a developer DoS and I will not let you swamp me with that.

More than anyone else, your contribution to the LBC meritocracy is negative so far. Before I even consider looking at any of your output ever again, you will have to provide some Gods own code or concept of value for the LBC. Including a prototype implementation. Until then: Try to learn as much as you can and should your fingers tickle and urge you to do a writeup: DON'T.

I *do* have projects of my own - SopaXorzTaker on GitHub.
If you find it appropriate to take this post to your attention, I should argue with you on the above.
Key generation for the challenge is not that expensive. Assuming that your server has ~200 regular clients, and you get a work requests from all of them (which is exaggerated) every second and there's 16 challenge keys per work request, you'd only need 200*16 = 3200 keys per second to be generated, while my machine does ~800 kkey/sec. I feel it necessary to argue, as I think some of your claims are incorrect, such as this one. I'd love to hear your feedback and arguments on that.

~~Additionally, you could potentially reuse the challenge keys for less security and more performance, and then you'd need only 16 keys per second to keep up with the clients.~~
EDIT: that's not possible as every client has a different work from the server, so the challenge has to be different too.
Am I wrong?

rico666

legendary

Activity: 1120

Merit: 1037

฿ → ∞

Quote from: mrgiacalone on April 30, 2017, 06:27:26 PM

alternate connection and follow your instructions for the "Clubbed to Death" method.

Yep - that "clubbed to death" method works always - because actually a human is doing what the computer should do.

Now that I will have more time to spend on usability and infrastructure, I certainly hope to get that automated install more robust and user-friendly.
(e.g. biggest problem of FTP updates was, that the LBC client on some installations just sees an empty directory and I have no idea why.)
With moving the updates of client and generator to HTTPS, I also am using JSON output from the Nginx server for directory listings
(see e.g. https://lbc.cryptoguru.org/static/generators/) which should also help to get the update/install more robust as no brittle output parsing is necessary anymore.

This is actually the authoritative source. FTP will be phased out as soon as the last sub-1.138 client is in use.

Quote

The CPU computes 4096 uncompressed public keys and moves them to GPU
The GPU computes 4096 hash160 of this and 4096 hash160 of the compressed equivalents
The 8192 hashes are moved back to the CPU which performs a bloom filter search on them.

This process is repeated 4096 times before you see a 'o' on your screen, which takes around 8 seconds on modern CPU/GPU combinations. Depending on the speed of the GPU, its load may be low - e.g. just 10% per CPU core.

This is not entirely true anymore. The 8192 hashes moving back is not done anymore, the GPU does the bloom filter search now. Also, using a faster ECC (done by arulbero) than what libsecp256k1 provided, allowed the CPU to feed the GPU faster. All in all, the load one CPU core does on GPU is about twice what it was when the quoted text above was written.

Quote

As I understand it, the use of GPU processing capacity may be unused depending on the number of cores the CPU has. Does this mean that running LBC on a multi-GPU rig with only an Intel Celeron Dual Core would not necessarily use all the GPUs capacity?

As of now, this is exactly what it means. I am thrilled to say, that this is actually something I will be working on next and GPU usage/saturation will raise at least by a factor of 2 again in the next couple of Huh

(days/few weeks).

rico666

legendary

Activity: 1120

Merit: 1037

฿ → ∞

Quote from: SopaXT on May 01, 2017, 12:02:28 AM

Rico, did you see my suggestion above? Please reply, I really want to know if it's applicable.

No, it's not. It's a variation of what arulbero has suggested further above. To all the problems of your previous suggestion, this adds the problem of scalability to the server (We have - in principle - a 1:many relation between server and clients, and we cannot afford to let the server perform key generation computations).

Moreover:

Quote

For the time being, I consider all security-related issues resolved and am as of now not willing to participate in any discussion regarding "LBC security", novel proof-of-work suggestions and the like. If you must stretch this topic, please do so in v1 thread or show me some code. Else: 404

I have addressed every valid security concern in the LBC client. So far, you have brought nothing to the table of value. No working code, no proof of concept. You have no projects of your own, no track of record , no nothing. You only brought stir to the LBC project. But you consider it somehow (I honestly do not know where you take that self-confidence) legit to demand my attention and even answers. You are in no position for that. Am I being clear?

You are this close ----> <------ to my ignore list. Please read and understand in the 1st post of this thread what that means (key phrase is: retro-active). The only thing keeping you from there is, that it could be perceived as martyrdom if I simply kicked you there. But the time nears where I do not care. Your constant gnat buzzing is like a developer DoS and I will not let you swamp me with that.

More than anyone else, your contribution to the LBC meritocracy is negative so far. Before I even consider looking at any of your output ever again, you will have to provide some Gods own code or concept of value for the LBC. Including a prototype implementation. Until then: Try to learn as much as you can and should your fingers tickle and urge you to do a writeup: DON'T.

SopaXT

full member

Activity: 157

Merit: 113

Rico, did you see my suggestion above? Please reply, I really want to know if it's applicable.

BurtW

legendary

Activity: 2646

Merit: 1137

All paid signature campaigns should be banned.

I am currently writing a thread related to the the Bitcoin address space and need to know something about your project:

What is the maximum key generation rate you have every recorded, even for a short period of time?

Thanks!

Please see my related thread here: https://bitcointalksearch.org/topic/the-case-for-moving-from-a-160-bit-to-a-256-bit-bitcoin-address-1895455

mrgiacalone

newbie

Activity: 2

Merit: 0

Hello rico,

While trying to set up LBC on a Ubuntu VM I ran into the "Benchmark info not found - benchmarking... 'gen-hrdcore-sse42-linux64' not found/executable." issue mentioned in the first thread (https://bitcointalksearch.org/topic/m.16539529). In my case this was probably because of a poor connection on the desktop hosting the VM. The links in the first thread don't seem to work as they are. However, I was able to download the files from the FTP folders using an alternate connection and follow your instructions for the "Clubbed to Death" method.

Generators: (ftp://ftp.cryptoguru.org/LBC/generators/)

(i.e. 170327-5f48315d6c68d76825b578327dcbf5c9.gen-hrdcore-sse42-linux64.bz2. Extract and rename to gen-hrdcore-sse42-linux64).

BLF: (ftp://ftp.cryptoguru.org/LBC/blf/)

Rename BLF file to "funds_h160.blf"
(i.e. 1170422-433ed5196898900563a5d6dcf7f6a87d.blf.bz2. Extract and rename to funds_h160.blf)

After that it was simple enough to get it working. While I wait for the GPU threshold to be reached, I have a question about the GPU process (http://lbc.cryptoguru.org:5000/man/user#gpu).

Quote

The CPU computes 4096 uncompressed public keys and moves them to GPU
The GPU computes 4096 hash160 of this and 4096 hash160 of the compressed equivalents
The 8192 hashes are moved back to the CPU which performs a bloom filter search on them.

This process is repeated 4096 times before you see a 'o' on your screen, which takes around 8 seconds on modern CPU/GPU combinations. Depending on the speed of the GPU, its load may be low - e.g. just 10% per CPU core.

As I understand it, the use of GPU processing capacity may be unused depending on the number of cores the CPU has. Does this mean that running LBC on a multi-GPU rig with only an Intel Celeron Dual Core would not necessarily use all the GPUs capacity?

Cheers,

MRG

rico666

legendary

Activity: 1120

Merit: 1037

฿ → ∞

There is a new LBC client version (1.138) available

You can fetch it either manually from https://lbc.cryptoguru.org/static/client/LBC
or your 1.067 client will get it - for the last time - from ftp://ftp.cryptoguru.org/LBC/client/1.138-LBC.bz2

All subsequent LBC client and generator updates will be from https://lbc.cryptoguru.org/static/
Moving the source of updates for executables from FTP to HTTPS is also the main change 1.067->1.138

Other changes:

ssl_opts => { verify_hostname => 1 },, host name verification on by default
changed generator benchmark from qx -> open pipe
limiting loop to 5 if time given < 5
-f will read another .json config file than the default lbc.json

LBC in the News:

Another VICE Motherboard 2nd guessing copy:

http://forklog.com/bolshoj-bitkoin-kollajder-ugrozhaet-polzovatelyam-vzlomom-koshelkov/

I found only out after the server log started to mention this URL as origin of some accesses. Of course Google Translate suggests, that the usual "never in a billion years" talk is going on in Cyrillic script too. Wink

As you can see in the stats The pool is slowing down, which is because our top-contributor https://lbc.cryptoguru.org/stats/Unknownhostname is shutting down capacities. Without him, the pool has around 200-250 Mkeys/s. (Maybe 400 if _KULME_ starts his monster machine Wink

)

Therefore my next focus and priority will be again on the generator to squeeze some more speed out of it and also to make it available on more machines. For the time being, I consider all security-related issues resolved and am as of now not willing to participate in any discussion regarding "LBC security", novel proof-of-work suggestions and the like.

If you must stretch this topic, please do so in v1 thread or show me some code. Else: 404

SopaXT

full member

Activity: 157

Merit: 113

I don't remember if I suggested the below challenge-based proof-of-work system already:

Server sends work to the client, with a range of private keys to test against a bloom filter of hashes of the public keys.
Server also generates a random challenge private key in the range and sends the public key hash to the client (which then recalculates the bloom filter, including it)
The challenge for the client is to send back the challenge private key to the server. If the challenge is failed, the server bans the client and ignores connections from it.

The challenge could be faked if the client stops the calculations after finding the challenge key, but the current client can also be tweaked to send invalid work too.
There is a possibility of using multiple challenge keys to make the forging of the work as computationally hard as doing the actual computations.
Again, please feel free to correct me and criticize this idea appropriately!

rico666

legendary

Activity: 1120

Merit: 1037

฿ → ∞

Quote from: Razzoel on April 28, 2017, 02:40:37 AM

It would be interesting if you guys could try to recover "burned" coins somehow. That way, the lost coins in the ecosystem won't go to waste.

That is the plan. Acually these lost coins are those that will remain unclaimed and these will go sooner or later to the LBC Pot for client disbursement.

When Satoshi said

Quote

Lost coins only make everyone else’s coins worth slightly more. Think of it as a donation to everyone.

he forgot to mention the donation being involuntary. So might as well go to Pool members instead of "everyone".

onemanatatime

full member

Activity: 294

Merit: 101

Aluna.Social

Quote from: rico666 on April 28, 2017, 01:22:13 AM

Quote from: onemanatatime on April 28, 2017, 01:10:43 AM

Are you planning any kind of crowdfunding or seed funding in the near future to expand your project?

No.

I must say, that a crowdfunding project for an ASIC did cross my mind, but down that path is probably more pain than fun, so ... no.

At least not within the next 2-3 years? Wink

Cool.. will keep an eye on the progress of this project. All the best rico666 !

Topic: Large Bitcoin Collider Thread 2.0 - page 19. (Read 57468 times)