Pages:
Author

Topic: List of all Bitcoin addresses with a balance (Read 7785 times)

legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
February 03, 2024, 08:27:42 AM
What I mean by that is that it is a real address available onchain
It's easy to generate billions of "real" addresses, but they're meaningless until they receive an on-chain Bitcoin payment.

Quote
And that would explain why my address doesn't show in the list of all address and balances above zero
Bitcoin's blockchain is a ledger for anything that ever happened. Things made up by Faucetpay internally have nothing to do with Bitcoin.
newbie
Activity: 13
Merit: 0
An example being a deposit address on Faucetpay.io is not in the list
That's because the address was never used on-chain. I don't use Faucetpay, but I guess they use the Bitcoin address as "account number". Anyone can do anything they want off-chain, there's no way to index that.

Yeah I had a quick look at one of the addresses and blockchair has some history data, so I think they use it like a clearing account.

What I mean by that is that it is a real address available onchain but also offchain (some of my terminology might be a bit wrong but feel free to correct me if there's better ways to elaborate on how it works).

Offchain usage (or is that considered sidechain) is like the network of systems that can talk direct to the (in this case the one i looked at, faucetpay) network/system, allowing transfers to be fee free if the sending system chooses to do so (transactions don't show in blockchair in this case).

Onchain usage is normal real transfers via the block chain, registers in the historical data (eg on blockchair) and then at some stage (algo? time? amount? don't know) is moved out of that address to another address (hence the clearing) - but in the interface that I see the amount is still allocated to me.  So my balance to my (faucetpay) address is buried in another address, hopefully readily available to me when i need to spend it!

And that would explain why my address doesn't show in the list of all address and balances above zero, as when they clear it they move the whole balance out.

Cheers
legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
An example being a deposit address on Faucetpay.io is not in the list
That's because the address was never used on-chain. I don't use Faucetpay, but I guess they use the Bitcoin address as "account number". Anyone can do anything they want off-chain, there's no way to index that.
newbie
Activity: 13
Merit: 0
Quote
Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets.
What do you mean by "off-chain"? Bitcoin addresses can only be on-chain (unless they're unused, in that case they're nowhere).

An example being a deposit address on Faucetpay.io is not in the list of 50+ million addresses (source data http://addresses.loyce.club-blockchair/bitcoin_addresses_and_balance_LATEST.tsv.gz from a few days ago at least).

In this particular case the address starts with 1EZASQnt79J and can be sent to from many locations (do they all have their own faucetpay integration feature outside of the BTC blockchain - I don't know).

Cheers
legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
Many thanks for making this available, liked it enough to login to BCT and post for the first time in years!!
You're welcome Smiley

Quote
Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets.
What do you mean by "off-chain"? Bitcoin addresses can only be on-chain (unless they're unused, in that case they're nowhere).
newbie
Activity: 13
Merit: 0

I don't know which file you have downloaded but I am confused about your procedure because: the list, which also contains the balances, is sorted by default by balance with the highest at the top and the lowest at the bottom. So you don't have to sort anything Wink YMMV

Fair comment.  guess it future proofs any changes to the sorting of the source data as long as the balance data is in the same column Smiley

Cheers

hero member
Activity: 630
Merit: 649
Bitcoin g33k
zcat datafile.tar.gz | grep -v address | sort -k2 -nr | nl -w8 - > outputfilename.txt

This will rank from largest as #1 down to smallest (over 50 million addresses currently).

I don't know which file you have downloaded but I am confused about your procedure because: the list, which also contains the balances, is sorted by default by balance with the highest at the top and the lowest at the bottom. So you don't have to sort anything Wink YMMV
newbie
Activity: 13
Merit: 0
Many thanks for making this available, liked it enough to login to BCT and post for the first time in years!!


I started playing with the all addresses with balance data and made a poor version of a full ranked list.  Simple one line command in bash or git bash (and others) if people are interested.

For those that want to, download the data file locally and use something like this (probably more efficient ways to do this but I was in a hurry Smiley ) :

zcat datafile.tar.gz | grep -v address | sort -k2 -nr | nl -w8 - > outputfilename.txt

This will rank from largest as #1 down to smallest (over 50 million addresses currently).


Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets.  Is this an area that interests you to expand the dataset?  (I don't know much about how these sort of wallets/systems work under the covers at this stage so not sure what is in the art of the possible)

Assuming the places that run these wallets provide access to the data in some way of course, at this stage I haven't gone down the path of opening any discussion with blockchair directly.

Enjoy!
legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
I just extracted the data using 7-zip and those text lines are in the result file. Maybe some bytes were mixed (shifted)
If that's the case, write to 7-zip. I just use tar, and it works fine.
newbie
Activity: 3
Merit: 0
Guys, thank you for your discovery and answers.
It looks a bit strange because I always use copy/paste and so on.
But the lines I see in the file I downloaded this morning.
I am ready to provide screenshots to all of mentioned addresses.
But, I already have an idea why it can be so. Bad if it affected not only me.
I just extracted the data using 7-zip and those text lines are in the result file. Maybe some bytes were mixed (shifted)
But I like more the idea of different witness versions.
Screenshot for some lines: https://ibb.co/tJJhGRL
Blockchain on the source site: https://blockchair.com/bitcoin/address/bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
Why do I see only the address bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t on blockchair.com but nowhere else?
This address is empty and not included in my list of funded addresses.

The LoyceVs list contains the line.
No, it does not.

Quote
So, the question was "Why does the list contain the line if the address doesn't exist in the real blockchain?".
Search the differences:
https://blockchair.com/bitcoin/address/bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
https://blockchair.com/bitcoin/address/bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t

Lesson learned: use copy paste and carefully check each character. If I type an address on an offline computer, I create a checksum on both just to be sure I typed it correctly.
hero member
Activity: 630
Merit: 649
Bitcoin g33k
Sorry, I didn't mention that my question is related to the thread, but indeed.
The LoyceVs list contains the line. So, the question was "Why does the list contain the line if the address doesn't exist in the real blockchain?".
The same for these addresses (for convenience, the first one is also included):
bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
bc1sw50qgdz25j
--snip--

I see. But I can't find the Bitcoin address bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t that you originally mentioned in the current LoyceV funded addresses list:

even up to this prefix there is no single hit
Code:
$ grep 'bc1qpcmm7k' blockchair_bitcoin_addresses_and_balance_LATEST.tsv

The same for these addresses (for convenience, the first one is also included):
bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
bc1sw50qgdz25j
--snip--

The first line has a typo, see the address you posted at the beginning and see this address you mentioned:
bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t

Apparently you have misspelled the address in your initial question. You have a typo Smiley Being that said, the address with the prefix 'bc1p' is in the LoyceV list:
Code:
$ grep 'bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t' blockchair_bitcoin_addresses_and_balance_LATEST.tsv 
Quote
bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t   41354

Now we are one step further. These are special other address types. You can find more information HERE.

Hope this answers your question.
citb0in
newbie
Activity: 3
Merit: 0
Sorry, I didn't mention that my question is related to the thread, but indeed.
The LoyceVs list contains the line. So, the question was "Why does the list contain the line if the address doesn't exist in the real blockchain?".
The same for these addresses (for convenience, the first one is also included):
bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
bc1sw50qgdz25j
bc1pewum2h6sap5ch5lf9qns4t0ufe9f6wf3z5s225
bc1zw508d6qejxtdg4y5r3zarvaryvaxxpcs
bc1pvw7t698cn379acgstnzwtnn8r6qxernnsrul3z
bc1pgptwf4gcrwy3qvq26p3cwapagaxyvsd8syuwhq
bc1putjagjerw0qw9qmy0wxu2pfk0v5t8gu7ranaxg
bc1pj7gr7lpxdnf6fm82jyy602t7xs9jjf3cp30yc7
bc1prkqmm25t4fn84k3j0dvlhkut4yeyuwurekel3c
bc1pxcsyvgrxyp8jqmeqtqs8sgpkyq7zq0sxpf58k
bc1p042rxsz3865566ykcd956pymkn0fxkn8lq96jx
bc1px5sy2gr9yp8zqm3q2us8wgp4yq4jq0gf5cpy9
bc1pxusywgr8ypgzqupqtys8jgphyq4zqgcmufa07
bc1p23jk6urvv96x2gp3yqszqgpqyqszqgqgxs8es
bc1p8ysyjgrfypfzqu3q9usrqgpeyqnzqfgv63qmh
bc1pxvsyxgrrypxzqmpq25s82gpnypajqlg7ux9e3
bc1pxqsrzgpjyqejqdpqx5srvgphyquzqwgc3wgd8
bc1pdv6mhmhayxp7xjxut0zhdpexskf7te5wfa53pv
bc1pjw7qg604staq70s86sfdq80zy5df2ljdl9n370
bc1p8qsysgrgypgjqufqtgs85gpcyqjzqsqul7r68
bc1pxysyzgrpyp9zq63q2vs8xgp3ypdjqhgfsx3d3
bc1pchjs0jhpga0lrqwwn26k6a9v5hj9pr8e522vq8
bc1pxgsyygrzyp9jq6eq2ss8ggpjyq5zq2g4sz4gk
bc1pxssyggryypxjqmfq2cs8vgp5yqsjq0ctxl0l2
bc1zqyqs3juw9m
bc1pm9jzmujvdqjj6y28hptk859zs3yyv78hlz84pm
hero member
Activity: 630
Merit: 649
Bitcoin g33k
Hi,
Why do I see only the address bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t on blockchair.com but nowhere else?
BR

simply said, its an invalid bitcoin address. Just because blockchair displays it, doesn't mean it exists. Blockchair seems to accept any address via URL parameter, even invalid ones like the one you mentioned.

Valid characters in in bech32 addresses are: 023456789acdefghjklmnpqrstuvwxyz
So invalid characters are: 1bio

Try enter this URL and you will get a similar result of what you asked for:
https://blockchair.com/bitcoin/address/bc1q123testbio123testbio123

However, your question is off-topic. Please make sure that in future you select a suitable subforum in which your question fits. If in doubt, create a new thread.
newbie
Activity: 3
Merit: 0
Hi,
Why do I see only the address bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t on blockchair.com but nowhere else?
BR
legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
September 29, 2023, 04:28:04 AM
I've moved this project to another (donated) server. Everything should be back online again, let me know if there are any issues.
legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
September 11, 2023, 05:01:30 AM
Since HASH160 keeps coming up, I'll link World's fastest and simplest block parser for those who (only) need all HASH160 here. This is for all addresses ever used though.
legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
For all of you, a script which downloads and converts LoyceV addresses to brainflayer bloom filter.
I've added it to the OP:
If you want to create a brainflayer bloom filter: pbies wrote a guide.

Note that I haven't tested it, so as always, do your own research. You shouldn't be messing with bloom filters if you don't know what you're doing anyway Wink
full member
Activity: 222
Merit: 124
For all of you, a script which downloads and converts LoyceV addresses to brainflayer bloom filter.
Taken are all >1 mBTC addresses.

Code:
#!/usr/bin/env bash
set -e
echo "### Public addresses downloader and converter to bloom filter ###"

echo [01/12] Download the addresses
wget http://addresses.loyce.club/blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz

echo [02/12] Unpack the archive
pv blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz | gunzip > blockchair_bitcoin_addresses_and_balance_LATEST.tsv

echo [03/12] Check lowest balance line
line=$(grep -m 1 -bne $'\t99999$' blockchair_bitcoin_addresses_and_balance_LATEST.tsv | cut -d ":" -f 2)

echo [04/12] Get only with proper balance
head -c $line blockchair_bitcoin_addresses_and_balance_LATEST.tsv > int1.txt

echo [05/12] Get rid of first line
tail -n +2 int1.txt > int2.txt

echo [06/12] Get only addresses, sort and write to file
pv int2.txt | cut -f 1 | sort > gte1mBTC.txt

echo [07/12] Search for incompatible addresses
line=$(grep -m 1 -bne $'^bc1p' "gte1mBTC.txt" | cut -d ":" -f 2)

echo [08/12] Get only compatible addresses
head -c $line "gte1mBTC.txt" > int3.txt

echo "[09/12] Convert addresses to hash160 ($(wc -l < int3.txt) lines)"
pv int3.txt | ~/brainflayer/brainflayer -o int4.txt

echo [10/12] Sort and uniq hashes
pv -c int4.txt | sort -u | pv -c > hash160.txt

echo [11/12] Convert hex to bloom filter
~/brainflayer/hex2blf "hash160.txt" bloom.blf

echo [12/12] Finished!
>&2 echo -ne "\x7"
legendary
Activity: 3262
Merit: 16303
Thick-Skinned Gang Leader and Golden Feather 2021
OK, can you please publish the way you generate the TSV file?
I did publish that, read the OP.

Quote
The latest one on the website is already a week old.
Thanks for this, I kinda forgot about this data project when I started another (big) download from Blockchair. Since I'm only allowed to download one connection at a time, it blocked other updates.
I'm running an update now, it should be available in about 3 hours.

Quote
Maybe someone needs more frequent updates, or to have a backup plan should your website go down or stop being updated (these things happen...)
If that happens, they'll have to figure it out on their own. If you're interested in discussing different parsers, it's probably best to open another topic about it.
Pages:
Jump to: