List of all Bitcoin addresses ever used - currently UNavailable on temp location - page 3.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: icopress on December 10, 2021, 02:31:20 PM

LoyceV, can you tell me where I can find a list of all companies represented on the coinmarketcap with verified accounts in third-party networks?

No.

Quote

For example, Twitter filter or Etherscan filter, etc.

I'm so glad I barely know what "Twitter" or "Etherscan" means Cheesy

I stay away from shitcoins and spammy websites.

Quote

I apologize in advance for moving away from the topic of discussion, but I need advice ... and I figured that since you are a data guru, you can tell me where to look for what I need, (I would appreciate any feedback). Tongue

Sorry, I have no useful feedback on this Sad

It seems more like a manual thing to collect custom data on custom networks.

icopress

legendary

Activity: 1456

Merit: 5874

light_warrior ... 🕯️

LoyceV, can you tell me where I can find a list of all companies represented on the coinmarketcap with verified accounts in third-party networks? For example, Twitter filter or Etherscan filter, etc. I apologize in advance for moving away from the topic of discussion, but I need advice ... and I figured that since you are a data guru, you can tell me where to look for what I need, (I would appreciate any feedback). Tongue

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Finally, this service is back online! I got myself a new VPS:

Quote from: LoyceV on August 01, 2020, 04:05:46 AM

The data
See alladdresses.loyce.club (new location)

I'm still working on an update, I expect to upload it tomorrow.

This server allows 8 TB of bandwidth per month (and I'm still hoping to have it doubled because of the Black Friday deal), but doesn't have the disk space to run updates. I'm not sure yet how often I'll update it (after tomorrow's update).
For now: enjoy!

An update on my statistics from August last year:

Some interesting (?) statistics (updated until blockchair_bitcoin_outputs_20211202)
Total address count: 1,967,537,866
1... address count: 1,227,646,688
3... address count: 543,236,056
bc1q... address count: 145,894,845
...-... (with a "dash") address count: 50,758,929

Unique address count: 927,366,160
1... address count: 547,749,478
3... address count: 272,205,566
bc1q... address count: 89,702,297
...-... (with a "dash") weird address count: 17,707,628

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: mrxtraf on September 19, 2021, 03:00:21 AM

And what is the difference, s, m, d?

As I said these are undocumented. Basically this particular explorer decided to use their own convention for naming these types of scripts and never told anyone how they are doing it or what they mean.
If I had to guess, they are probably computing the hash of the script (possibly SHA1 or MD5) then enter that smaller hash in their database to search based on a hash. The starting letter (s, m, ...) may be a quick indicator on where in the database to look them up.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: mrxtraf on September 19, 2021, 03:30:42 AM

But not found Sad

My mistake, sorry. Read my updated post

mrxtraf

member

Activity: 259

Merit: 47

Quote from: LoyceV on September 19, 2021, 03:02:35 AM

Quote from: mrxtraf on September 19, 2021, 03:00:21 AM

But where to look at the principle of formation. And what is the difference, s, m, d?

Nobody knows Cheesy

But if you enter one of them into Blockchair.com's Search field, it shows the transaction.

I try. But not found Sad

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: mrxtraf on September 19, 2021, 03:00:21 AM

But where to look at the principle of formation. And what is the difference, s, m, d?

Nobody knows Cheesy

~~But if you enter one of them into Blockchair.com's Search field, it shows the transaction.~~
Sorry, I forgot it works differently: if you edit them into the URL of a Bitcoin address on Blockchair.com, it shows the transaction.
Example: blockchair.com/bitcoin/address/s-ffd80dee5966fb23c1a483b28f6bfcbc.

mrxtraf

member

Activity: 259

Merit: 47

Quote from: pooya87 on September 19, 2021, 02:56:40 AM

Quote from: mrxtraf on September 19, 2021, 02:49:56 AM

What the addres with prefix s- and prefix m- ?

They are not addresses, they are output scripts that don't have any corresponding address and blockchair.com explorer uses an undocumented method to convert them into these strange looking strings.

I understand that these are scripts. But where to look at the principle of formation. And what is the difference, s, m, d?

pooya87

legendary

Activity: 3472

Merit: 10611

Quote from: mrxtraf on September 19, 2021, 02:49:56 AM

What the addres with prefix s- and prefix m- ?

They are not addresses, they are output scripts that don't have any corresponding address and blockchair.com explorer uses an undocumented method to convert them into these strange looking strings.

mrxtraf

member

Activity: 259

Merit: 47

What the addres with prefix s- and prefix m- and prefix d- ?

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: BoraxTributary on August 12, 2021, 05:48:20 PM

This is up to date as of 2021-08-12.

If it's the same file notatether.com published, it was last updated in May.

I still haven't found a decent affordable replacement host.

BoraxTributary

newbie

Activity: 5

Merit: 0

Another way to generate the file yourself, from a local copy of the blockchain: https://github.com/graymauser/btcposbal2csv

BoraxTributary

newbie

Activity: 5

Merit: 0

For anyone stumbling open this in the future, I have created a torrent for this file, which accesses the web link.

Web Link: https://files.notatether.com/public/loycev/addresses_sorted.txt.gz

Torrent Magnet URL

.torrent file, good until 2022-09

This is up to date as of 2021-08-12. It will not be updated in the future, but is being left here to help, at least for now.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: PrimeNumber7 on June 12, 2021, 09:28:25 PM

Have you considered creating a script to update your existing list once you have a list of addresses? Your script could start at block_n and add an address if it is not already in your list. This should reduce read/write operations pretty significantly.

That's what I tried to do for the addresses in chronological order, but it gave inconsistent results (and due to the time it takes to test it, I'm still not sure what caused it). I could probably do the same for the sorted addresses, by using comm instead of sort. That means only reading the compressed big file from disk, reading and sorting the weekly addition, and writing the new compressed file to disk. This does make more sense and reduces disk writes

PrimeNumber7

copper member

Activity: 1666

Merit: 1901

Amazon Prime Member #7

Quote from: LoyceV on June 06, 2021, 03:01:35 PM

Quote from: NotATether on June 06, 2021, 01:51:54 PM

I'm trying to figure out how much RAM it uses in the worst case.

I can limit sort's memory usage. I'm more worried about the grinding this causes on the hard drive. I have no idea how much data gets read and written to sort 30 GB, but I assume every bit gets pickup up at least several times.

Quote from: NotATether on June 12, 2021, 02:59:14 AM

Quote from: LoyceV on June 06, 2021, 03:01:35 PM

Quote from: NotATether on June 06, 2021, 01:51:54 PM

I'm trying to figure out how much RAM it uses in the worst case.

I can limit sort's memory usage. I'm more worried about the grinding this causes on the hard drive. I have no idea how much data gets read and written to sort 30 GB, but I assume every bit gets pickup up at least several times.

On second thought, I just had one of my servers' disks fail a couple days ago (all data was lost), so I'm not comfortable running these updating scripts on the rest of my hardware with all that grinding until I can set up a proper backup plan for my TBs of data.

Have you considered creating a script to update your existing list once you have a list of addresses? Your script could start at block_n and add an address if it is not already in your list. This should reduce read/write operations pretty significantly.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: NotATether on June 12, 2021, 02:59:14 AM

I'm not comfortable running these updating scripts on the rest of my hardware

No worries, I'll just wait for the right hosting offer again.
This is the reason I don't want to do a lot of testing on my laptop SSD too.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

Quote from: LoyceV on June 06, 2021, 03:01:35 PM

Quote from: NotATether on June 06, 2021, 01:51:54 PM

I'm trying to figure out how much RAM it uses in the worst case.

I can limit sort's memory usage. I'm more worried about the grinding this causes on the hard drive. I have no idea how much data gets read and written to sort 30 GB, but I assume every bit gets pickup up at least several times.

On second thought, I just had one of my servers' disks fail a couple days ago (all data was lost), so I'm not comfortable running these updating scripts on the rest of my hardware with all that grinding until I can set up a proper backup plan for my TBs of data.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: NotATether on June 06, 2021, 01:51:54 PM

I'm trying to figure out how much RAM it uses in the worst case.

I can limit sort's memory usage. I'm more worried about the grinding this causes on the hard drive. I have no idea how much data gets read and written to sort 30 GB, but I assume every bit gets pickup up at least several times.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

Quote from: LoyceV on June 06, 2021, 09:29:51 AM

Memory requirements can be low (then sort uses more tmp files instead).

This is what I'm worried about. I remember you writing somewhere along the lines of the sort process takes an obscene amount of memory on this file, or maybe that was in the addresses-with-a-balance project. That's why I'm trying to figure out how much RAM it uses in the worst case.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: NotATether on June 06, 2021, 01:06:14 AM

Leaving it single threaded (as most shell commands already are) will probably be alright as long as it doesn't take more than a few hours.

But if it has obscene memory requirements then it'll be too much for my box. I only got 9 out of 16GB ram free and I hate force-rebooting the notatether.com webserver Undecided

It uses a bunch of pipes, so the load it causes is more like 2-3 than 1. Memory requirements can be low (then sort uses more tmp files instead). It should be done within a few hours, and doesn't need frequent updates (once every 2 weeks will be fine).

Topic: List of all Bitcoin addresses ever used - currently UNavailable on temp location - page 3. (Read 4161 times)