Pages:
Author

Topic: VanitySearch (Yet another address prefix finder) - page 6. (Read 32966 times)

full member
Activity: 1232
Merit: 242
Shooters Shoot...
Guys I updated my VanitySearch clone to work on GPU as well as CPU, so you guys shouldn't have any problems with that. I'm still debugging some issues with ".*" and ".+" wildcards that citb0in spotted, and the GPU build has a nasty and unexpected 5x slowdown. Which is pretty weird since almost all of the GPU codebase is the same as the original.

What exactly is different about your clone versus the original? Thanks.

My clone has regex support instead of just the two wildcards ? and * (had to rip out all the legacy wildcard code out in the process because it was overcomplicating things) and it also support searching for multiple prefixes simultaneously.

It is available at https://github.com/ZenulAbidin/VanitySearch (warning: still experimental software).
Can you give me an example of how regex works/is different than current program?

current program can search strings like:

1NotATeth
1NotATeth*

How is regex different? I read your github page

Does your version still support full address search?

also, current program can search up to 32 million prefixes simultaneously (I haven't tried more than 32 million).

legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
Guys I updated my VanitySearch clone to work on GPU as well as CPU, so you guys shouldn't have any problems with that. I'm still debugging some issues with ".*" and ".+" wildcards that citb0in spotted, and the GPU build has a nasty and unexpected 5x slowdown. Which is pretty weird since almost all of the GPU codebase is the same as the original.

What exactly is different about your clone versus the original? Thanks.

My clone has regex support instead of just the two wildcards ? and * (had to rip out all the legacy wildcard code out in the process because it was overcomplicating things) and it also support searching for multiple prefixes simultaneously.

It is available at https://github.com/ZenulAbidin/VanitySearch (warning: still experimental software).
hero member
Activity: 1443
Merit: 513
Guys I updated my VanitySearch clone to work on GPU as well as CPU, so you guys shouldn't have any problems with that. I'm still debugging some issues with ".*" and ".+" wildcards that citb0in spotted, and the GPU build has a nasty and unexpected 5x slowdown. Which is pretty weird since almost all of the GPU codebase is the same as the original.
Ive always noticed a slowdown on verbose wildcard vanities. specially ones like 1test*test.
usually, its indexes much slower with split key generation + wildcards toss in a split key +1test*test even with -c it seems to take a while longer than it should.

full member
Activity: 1232
Merit: 242
Shooters Shoot...
Guys I updated my VanitySearch clone to work on GPU as well as CPU, so you guys shouldn't have any problems with that. I'm still debugging some issues with ".*" and ".+" wildcards that citb0in spotted, and the GPU build has a nasty and unexpected 5x slowdown. Which is pretty weird since almost all of the GPU codebase is the same as the original.

What exactly is different about your clone versus the original? Thanks.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
Guys I updated my VanitySearch clone to work on GPU as well as CPU, so you guys shouldn't have any problems with that. I'm still debugging some issues with ".*" and ".+" wildcards that citb0in spotted, and the GPU build has a nasty and unexpected 5x slowdown. Which is pretty weird since almost all of the GPU codebase is the same as the original.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
I pushed my fixes to the gpu branch, but apparently someone is using my Vast GPU, so I can't test the GPU build until it becomes free again.

Hey guys in regards to recent events with coldkey sweeping funded wallets,
 I proposed this app's (VanitySearch) split key feature for makers. (and of course https://1splitkey.com service that's built off of it. because that's why I made it to simplify it.)
This notion seems to be rejected when I propose it, and these guys are proposing less than ideal solutions, ignoring this likely due to it being "Me" that proposed it.
This kind of makes me feel some type of way about the community in general.
https://bitcointalksearch.org/topic/m.61631653


However I see this as a potential opportunity to shutdown my free service and release the sourcecode and walk away from the project.
In all likelihood this is the only way 1splitkey will be utilized.
I have restraints about the implications of repurposing splitkeys backend. which contains tesla agents and things of that nature for client control and server communications.
They are used in a manner that's obvious the same way client side gpu miners are set up.

Should I take one for the team and release the SC for the good of the physical's community?

If I do this, I'll be shutting 1SK down as it will even be harder to fight bad actors.
 
I'll let you guys decide. I dont want to see all the sweat equity from this just get brushed off, it's a beautiful system that's currently just toiling in obscurity.

Is this generating any revenue for you right now?

If it's just sitting there collecting hosting fees, then yeah, you can release the code, and maybe one of us will find a way to utilize it.
hero member
Activity: 1443
Merit: 513
Hey guys in regards to recent events with coldkey sweeping funded wallets,
 I proposed this app's (VanitySearch) split key feature for makers. (and of course https://1splitkey.com service that's built off of it. because that's why I made it to simplify it.)
This notion seems to be rejected when I propose it, and these guys are proposing less than ideal solutions, ignoring this likely due to it being "Me" that proposed it.
This kind of makes me feel some type of way about the community in general.
https://bitcointalksearch.org/topic/m.61631653


However I see this as a potential opportunity to shutdown my free service and release the sourcecode and walk away from the project.
In all likelihood this is the only way 1splitkey will be utilized.
I have restraints about the implications of repurposing splitkeys backend. which contains tesla agents and things of that nature for client control and server communications.
They are used in a manner that's obvious the same way client side gpu miners are set up.

Should I take one for the team and release the SC for the good of the physical's community?

If I do this, I'll be shutting 1SK down as it will even be harder to fight bad actors.
 
I'll let you guys decide. I dont want to see all the sweat equity from this just get brushed off, it's a beautiful system that's currently just toiling in obscurity.









full member
Activity: 1232
Merit: 242
Shooters Shoot...
So is it just when the -check is used? I'm confused lol.

How does it impact searching for keys...

No it literally breaks the address finding process too.

As in, it finds the correct address/private key, but then when it goes to verify the address is correctb(all this happens in Vanity.cpp) that fails and you get a warning because of botched base58 checksums.

This means that VanitySearch will never exit even if it finds the right keys.
OK, a few followup questions.

Does this only impact Linux, or does it impact Windows as well?

Is it hit or miss (the bug) because obviously I find addresses and their keys when running tests.

Is there a test to run that will duplicate these botched checksums?
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
So is it just when the -check is used? I'm confused lol.

How does it impact searching for keys...

No it literally breaks the address finding process too.

As in, it finds the correct address/private key, but then when it goes to verify the address is correctb(all this happens in Vanity.cpp) that fails and you get a warning because of botched base58 checksums.

This means that VanitySearch will never exit even if it finds the right keys.
full member
Activity: 1232
Merit: 242
Shooters Shoot...
Can you enlighten me to what exactly the bug is/is doing? Is it only the -check unit test?

It's not just in the unit test, it's also in the main loop when it's generating Base58 addresses (the checksums made after an equality comparison or regex match, specifically).

In the above code piece for creating the checksum, it runs fine without any optimizations at all.

But, when -fstrict-aliasing is switched on as a result of using -O2, the compiler assumes that variables of different types being assigned to each other have the same size (as in sizeof()) and optimizes out some internal ASM that would've otherwise been generated.

The ASM being removed here is parts of the assignment statements in the snippet above (and in the non-SSE version). Those areas still have junk values from when those automatic variables were declared.

So the SHA256 checksum contains some garbage bytes. When this is encoded into Base58, it can only generate wrong addresses. And this is despite all private keys being 100% correct.

That's why it works in debug mode and not in production.
So is it just when the -check is used? I'm confused lol.

How does it impact searching for keys...
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
Can you enlighten me to what exactly the bug is/is doing? Is it only the -check unit test?

It's not just in the unit test, it's also in the main loop when it's generating Base58 addresses (the checksums made after an equality comparison or regex match, specifically).

In the above code piece for creating the checksum, it runs fine without any optimizations at all.

But, when -fstrict-aliasing is switched on as a result of using -O2, the compiler assumes that variables of different types being assigned to each other have the same size (as in sizeof()) and optimizes out some internal ASM that would've otherwise been generated.

The ASM being removed here is parts of the assignment statements in the snippet above (and in the non-SSE version). Those areas still have junk values from when those automatic variables were declared.

So the SHA256 checksum contains some garbage bytes. When this is encoded into Base58, it can only generate wrong addresses. And this is despite all private keys being 100% correct.

That's why it works in debug mode and not in production.
full member
Activity: 1232
Merit: 242
Shooters Shoot...
An update about the chronic address bug on VanitySearch:

I have found the source of the error. In case you didn't know, everything released after VanitySearch 1.13 fails to math any base58 address, on newer GCC (11+, and maybe 10 as well)

This has to do with the sha256.cpp and sha256_sse.cpp files inside the hash/ folder. Specifically, sha256_checksum and sha256sse_checksum functions.

These functions fail to make the correct checksum when you are running with optimization flags on, but I managed to get even more specific than that.

It appears that a single O2 optimization flag, -fstrict-aliasing is causing all the chaos. Taking the SSE version of this function for example:

Code:
void sha256sse_checksum(uint32_t *i0, uint32_t *i1, uint32_t *i2, uint32_t *i3,
  uint8_t *d0, uint8_t *d1, uint8_t *d2, uint8_t *d3) {

  __m128i s[8];

  _sha256sse::Initialize(s);
  _sha256sse::Transform2(s, i0, i1, i2, i3);

#ifndef WIN64
  uint32_t *s32 = (uint32_t *)(&s[0]);
  *((uint32_t *)d0) = __builtin_bswap32(s32[3]);
  *((uint32_t *)d1) = __builtin_bswap32(s32[2]);
  *((uint32_t *)d2) = __builtin_bswap32(s32[1]);
  *((uint32_t *)d3) = __builtin_bswap32(s32[0]);
#else
  *((uint32_t *)d0) = _byteswap_ulong(s[0].m128i_u32[3]);
  *((uint32_t *)d1) = _byteswap_ulong(s[0].m128i_u32[2]);
  *((uint32_t *)d2) = _byteswap_ulong(s[0].m128i_u32[1]);
  *((uint32_t *)d3) = _byteswap_ulong(s[0].m128i_u32[0]);
#endif

}

Aliasing is an optimization technique where the compiler assumes that you're never going to cast variables into crazy types of different sizes and makes some fast but unsafe memory writes as a result.

It's unsafe because if the type indeed is cast into a larger size type, then some garbage will be written into the higher parts of the variable. That's where the gibberish in the checksum characters comes from. The parameters d0-3 are expanded into uint32 and then assigned, and that's where strict aliasing messes things up.

Something similar happens with sha256_checksum but with memcpy instead.

These functions were introduced in the following commit: https://github.com/JeanLucPons/VanitySearch/commit/ea177b7b36c0db66f110d4358fd4fd4704a6603d right before v1.14 was released.

To fix this mess, all you have to do is pass -fno-strict-aliasing (please write this carefully) in the CXXFLAGS in the makefile.

I am working on a patched version of this where only the offending functions get this flag. So my codebase is spaghetti right now. Please stand by.
Can you enlighten me to what exactly the bug is/is doing? Is it only the -check unit test?
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
An update about the chronic address bug on VanitySearch:

I have found the source of the error. In case you didn't know, everything released after VanitySearch 1.13 fails to math any base58 address, on newer GCC (11+, and maybe 10 as well)

This has to do with the sha256.cpp and sha256_sse.cpp files inside the hash/ folder. Specifically, sha256_checksum and sha256sse_checksum functions.

These functions fail to make the correct checksum when you are running with optimization flags on, but I managed to get even more specific than that.

It appears that a single O2 optimization flag, -fstrict-aliasing is causing all the chaos. Taking the SSE version of this function for example:

Code:
void sha256sse_checksum(uint32_t *i0, uint32_t *i1, uint32_t *i2, uint32_t *i3,
  uint8_t *d0, uint8_t *d1, uint8_t *d2, uint8_t *d3) {

  __m128i s[8];

  _sha256sse::Initialize(s);
  _sha256sse::Transform2(s, i0, i1, i2, i3);

#ifndef WIN64
  uint32_t *s32 = (uint32_t *)(&s[0]);
  *((uint32_t *)d0) = __builtin_bswap32(s32[3]);
  *((uint32_t *)d1) = __builtin_bswap32(s32[2]);
  *((uint32_t *)d2) = __builtin_bswap32(s32[1]);
  *((uint32_t *)d3) = __builtin_bswap32(s32[0]);
#else
  *((uint32_t *)d0) = _byteswap_ulong(s[0].m128i_u32[3]);
  *((uint32_t *)d1) = _byteswap_ulong(s[0].m128i_u32[2]);
  *((uint32_t *)d2) = _byteswap_ulong(s[0].m128i_u32[1]);
  *((uint32_t *)d3) = _byteswap_ulong(s[0].m128i_u32[0]);
#endif

}

Aliasing is an optimization technique where the compiler assumes that you're never going to cast variables into crazy types of different sizes and makes some fast but unsafe memory writes as a result.

It's unsafe because if the type indeed is cast into a larger size type, then some garbage will be written into the higher parts of the variable. That's where the gibberish in the checksum characters comes from. The parameters d0-3 are expanded into uint32 and then assigned, and that's where strict aliasing messes things up.

Something similar happens with sha256_checksum but with memcpy instead.

These functions were introduced in the following commit: https://github.com/JeanLucPons/VanitySearch/commit/ea177b7b36c0db66f110d4358fd4fd4704a6603d right before v1.14 was released.

To fix this mess, all you have to do is pass -fno-strict-aliasing (please write this carefully) in the CXXFLAGS in the makefile.

I am working on a patched version of this where only the offending functions get this flag. So my codebase is spaghetti right now. Please stand by.
hero member
Activity: 630
Merit: 731
Bitcoin g33k
Anyone want to lend me their GPU for a few days so I can fix my broken VanitySearch build (and maybe Kangaroo as well as people are starting to complain about the hash table not working and stuff)?

I have $10 in Vast.ai from a while ago but the card used to find it has since expired and I have no way to fund it now.

@NotATether
I've prepared a machine for you. Check your mailbox (DM)
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
Anyone want to lend me their GPU for a few days so I can fix my broken VanitySearch build (and maybe Kangaroo as well as people are starting to complain about the hash table not working and stuff)?

I have $10 in Vast.ai from a while ago but the card used to find it has since expired and I have no way to fund it now.
You want me to ship you one? Smiley

Well that won't really make any sense to me, as I only got (quite old) laptops handy at the moment, and the power utility is a b*tch.

I think just put it in a DIY box and slap some temporary SSH access on it for me to get in and debug VanitySearch should do the trick.
full member
Activity: 1232
Merit: 242
Shooters Shoot...
Anyone want to lend me their GPU for a few days so I can fix my broken VanitySearch build (and maybe Kangaroo as well as people are starting to complain about the hash table not working and stuff)?

I have $10 in Vast.ai from a while ago but the card used to find it has since expired and I have no way to fund it now.
You want me to ship you one? Smiley
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
Anyone want to lend me their GPU for a few days so I can fix my broken VanitySearch build (and maybe Kangaroo as well as people are starting to complain about the hash table not working and stuff)?

I have $10 in Vast.ai from a while ago but the card used to find it has since expired and I have no way to fund it now.
hero member
Activity: 630
Merit: 731
Bitcoin g33k
I'll be surprised if this bug is in JLP's VanitySearch build as well.

original VanitySearch of JLP works fine, no issues. We certainly would heard about existing bugs
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
what's the output of
Code:
./VanitySearch -check

read HERE and HERE

try to compile with gcc 9 instead of the latest gcc 11

in Linux try to set optimization flag -O2 to -O1 in Makefile
In Windows / cygwin64 the same

try IntMod.cpp change
Code:
static Int _P; // Field characteristic in IntMod.cpp

to
Code:
static Int Field_P;
and all other occurences in file

VanitySearch -check unit test is failing because of "Invalid private key checksum", and indeed, all parts of the address are equal except for the checksum part. So I will investigate this.

I'll be surprised if this bug is in JLP's VanitySearch build as well.
hero member
Activity: 630
Merit: 731
Bitcoin g33k
what's the output of
Code:
./VanitySearch -check

read HERE and HERE

try to compile with gcc 9 instead of the latest gcc 11

in Linux try to set optimization flag -O2 to -O1 in Makefile
In Windows / cygwin64 the same

try IntMod.cpp change
Code:
static Int _P; // Field characteristic in IntMod.cpp

to
Code:
static Int Field_P;
and all other occurences in file
Pages:
Jump to: