Topic: List of all Bitcoin addresses with a balance - page 2. (Read 9169 times)
The list is currently a few days behind again
Quote
comm -12 Bitcoin_addresses_LATEST.txt <(cat wallet.txt | fromdos | sort | uniq)
Quote
If you need fromdos, install tofrodos, as it tells you to do.
Now it worked. Thank you very much
Compare two files, one with addresses and second your one with addresses you are searching for:
Sorry, i do not know what i am doing wrong,
I tested with several addresses and only found 1
https://i.ibb.co/R3v2fJk/1.jpg
https://i.ibb.co/M9NXhgL/2.jpg
Code:
#!/usr/bin/env bash
comm -12 <(pv -cN 1st "$1") <(pv -cN 2nd "$2")
>&2 echo -ne "\x7"
comm -12 <(pv -cN 1st "$1") <(pv -cN 2nd "$2")
>&2 echo -ne "\x7"
Sorry, i do not know what i am doing wrong,
I tested with several addresses and only found 1
https://i.ibb.co/R3v2fJk/1.jpg
https://i.ibb.co/M9NXhgL/2.jpg
I use these 3 scripts:
Get addresses and balances and unpack them:
Take only addresses, sort and uniq them:
Compare two files, one with addresses and second your one with addresses you are searching for:
Get addresses and balances and unpack them:
Code:
#!/usr/bin/env bash
wget http://addresses.loyce.club/blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz
pv blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz | gunzip > blockchair_bitcoin_addresses_and_balance_LATEST.tsv
>&2 echo -ne "\x7"
wget http://addresses.loyce.club/blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz
pv blockchair_bitcoin_addresses_and_balance_LATEST.tsv.gz | gunzip > blockchair_bitcoin_addresses_and_balance_LATEST.tsv
>&2 echo -ne "\x7"
Take only addresses, sort and uniq them:
Code:
#!/usr/bin/env bash
pv -B 1M blockchair_bitcoin_addresses_and_balance_LATEST.tsv | cut -f 1 | sort -u > addrs-with-bal.txt
>&2 echo -ne "\x7"
pv -B 1M blockchair_bitcoin_addresses_and_balance_LATEST.tsv | cut -f 1 | sort -u > addrs-with-bal.txt
>&2 echo -ne "\x7"
Compare two files, one with addresses and second your one with addresses you are searching for:
Code:
#!/usr/bin/env bash
comm -12 <(pv -cN 1st "$1") <(pv -cN 2nd "$2")
>&2 echo -ne "\x7"
comm -12 <(pv -cN 1st "$1") <(pv -cN 2nd "$2")
>&2 echo -ne "\x7"
I tried a lot and could not put to work these commands on linux mint
I've deleted your post because of the unnecessary long quote.Your screenshot already shows the problem:
[/quote]Quote
comm -12 Bitcoin_addresses_LATEST.txt <(cat wallet.txt | sort | uniq)
The file with balances is indeed not sorted. And you can't use it, because comm matches whole lines. Remove the balances and sort it, or download the file without balances.Quote
comm -12 Bitcoin_addresses_LATEST.txt <(cat wallet.txt | fromdos | sort | uniq)
If you need fromdos, install tofrodos, as it tells you to do.Running this as root is bad practice.
I get this data from Blockchair.com, and their last update is from April 19. Their other data is still being updated, so it looks like something is stuck with updating the Bitcoin addresses.
The daily updates are back to normal 
The files haven't been updated in over a week
I get this data from Blockchair.com, and their last update is from April 19. Their other data is still being updated, so it looks like something is stuck with updating the Bitcoin addresses.Files haven't been updated since 19 April 2024.
I posted both posts in the wrong topic. It's correct now, the other post is here. Both updates got stuck, one was waiting for the other, which was waiting for Blockchair. In short: the list of all addresses with a balance is not updated at the moment. The files haven't been updated in over a week
It looks like the daily updates didn't happen, which means the weekly update had no fresh data. I'm not sure why, I'll check it tomorrow.Update: it's fixed. It was waiting for another stuck update. The weekly update is running now, this takes a few hours to complete. Thanks for reporting this.
Files haven't been updated since 19 April 2024.
The files haven't been updated in over a week, though the source files on blockchair are still updated. Don't see any maintenance status mentioned on the download site or this topic.
What I mean by that is that it is a real address available onchain
It's easy to generate billions of "real" addresses, but they're meaningless until they receive an on-chain Bitcoin payment.Quote
And that would explain why my address doesn't show in the list of all address and balances above zero
Bitcoin's blockchain is a ledger for anything that ever happened. Things made up by Faucetpay internally have nothing to do with Bitcoin. An example being a deposit address on Faucetpay.io is not in the list
That's because the address was never used on-chain. I don't use Faucetpay, but I guess they use the Bitcoin address as "account number". Anyone can do anything they want off-chain, there's no way to index that.Yeah I had a quick look at one of the addresses and blockchair has some history data, so I think they use it like a clearing account.
What I mean by that is that it is a real address available onchain but also offchain (some of my terminology might be a bit wrong but feel free to correct me if there's better ways to elaborate on how it works).
Offchain usage (or is that considered sidechain) is like the network of systems that can talk direct to the (in this case the one i looked at, faucetpay) network/system, allowing transfers to be fee free if the sending system chooses to do so (transactions don't show in blockchair in this case).
Onchain usage is normal real transfers via the block chain, registers in the historical data (eg on blockchair) and then at some stage (algo? time? amount? don't know) is moved out of that address to another address (hence the clearing) - but in the interface that I see the amount is still allocated to me. So my balance to my (faucetpay) address is buried in another address, hopefully readily available to me when i need to spend it!
And that would explain why my address doesn't show in the list of all address and balances above zero, as when they clear it they move the whole balance out.
Cheers
An example being a deposit address on Faucetpay.io is not in the list
That's because the address was never used on-chain. I don't use Faucetpay, but I guess they use the Bitcoin address as "account number". Anyone can do anything they want off-chain, there's no way to index that. Quote
Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets.
What do you mean by "off-chain"? Bitcoin addresses can only be on-chain (unless they're unused, in that case they're nowhere).An example being a deposit address on Faucetpay.io is not in the list of 50+ million addresses (source data http://addresses.loyce.club-blockchair/bitcoin_addresses_and_balance_LATEST.tsv.gz from a few days ago at least).
In this particular case the address starts with 1EZASQnt79J and can be sent to from many locations (do they all have their own faucetpay integration feature outside of the BTC blockchain - I don't know).
Cheers
Many thanks for making this available, liked it enough to login to BCT and post for the first time in years!!
You're welcome Quote
Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets.
What do you mean by "off-chain"? Bitcoin addresses can only be on-chain (unless they're unused, in that case they're nowhere). I don't know which file you have downloaded but I am confused about your procedure because: the list, which also contains the balances, is sorted by default by balance with the highest at the top and the lowest at the bottom. So you don't have to sort anything
YMMVFair comment. guess it future proofs any changes to the sorting of the source data as long as the balance data is in the same column
Cheers
zcat datafile.tar.gz | grep -v address | sort -k2 -nr | nl -w8 - > outputfilename.txt
This will rank from largest as #1 down to smallest (over 50 million addresses currently).
This will rank from largest as #1 down to smallest (over 50 million addresses currently).
I don't know which file you have downloaded but I am confused about your procedure because: the list, which also contains the balances, is sorted by default by balance with the highest at the top and the lowest at the bottom. So you don't have to sort anything
YMMV
Many thanks for making this available, liked it enough to login to BCT and post for the first time in years!!
I started playing with the all addresses with balance data and made a poor version of a full ranked list. Simple one line command in bash or git bash (and others) if people are interested.
For those that want to, download the data file locally and use something like this (probably more efficient ways to do this but I was in a hurry ) :
zcat datafile.tar.gz | grep -v address | sort -k2 -nr | nl -w8 - > outputfilename.txt
This will rank from largest as #1 down to smallest (over 50 million addresses currently).
Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets. Is this an area that interests you to expand the dataset? (I don't know much about how these sort of wallets/systems work under the covers at this stage so not sure what is in the art of the possible)
Assuming the places that run these wallets provide access to the data in some way of course, at this stage I haven't gone down the path of opening any discussion with blockchair directly.
Enjoy!
I started playing with the all addresses with balance data and made a poor version of a full ranked list. Simple one line command in bash or git bash (and others) if people are interested.
For those that want to, download the data file locally and use something like this (probably more efficient ways to do this but I was in a hurry
) :zcat datafile.tar.gz | grep -v address | sort -k2 -nr | nl -w8 - > outputfilename.txt
This will rank from largest as #1 down to smallest (over 50 million addresses currently).
Unfortunately it seems the BTC addresses that I have are off-chain / micro wallets. Is this an area that interests you to expand the dataset? (I don't know much about how these sort of wallets/systems work under the covers at this stage so not sure what is in the art of the possible)
Assuming the places that run these wallets provide access to the data in some way of course, at this stage I haven't gone down the path of opening any discussion with blockchair directly.
Enjoy!
I just extracted the data using 7-zip and those text lines are in the result file. Maybe some bytes were mixed (shifted)
If that's the case, write to 7-zip. I just use tar, and it works fine.
Guys, thank you for your discovery and answers.
It looks a bit strange because I always use copy/paste and so on.
But the lines I see in the file I downloaded this morning.
I am ready to provide screenshots to all of mentioned addresses.
But, I already have an idea why it can be so. Bad if it affected not only me.
I just extracted the data using 7-zip and those text lines are in the result file. Maybe some bytes were mixed (shifted)
But I like more the idea of different witness versions.
Screenshot for some lines: https://ibb.co/tJJhGRL
Blockchain on the source site: https://blockchair.com/bitcoin/address/bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
It looks a bit strange because I always use copy/paste and so on.
But the lines I see in the file I downloaded this morning.
I am ready to provide screenshots to all of mentioned addresses.
But, I already have an idea why it can be so. Bad if it affected not only me.
I just extracted the data using 7-zip and those text lines are in the result file. Maybe some bytes were mixed (shifted)
But I like more the idea of different witness versions.
Screenshot for some lines: https://ibb.co/tJJhGRL
Blockchain on the source site: https://blockchair.com/bitcoin/address/bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
Why do I see only the address bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t on blockchair.com but nowhere else?
This address is empty and not included in my list of funded addresses.The LoyceVs list contains the line.
No, it does not.Quote
So, the question was "Why does the list contain the line if the address doesn't exist in the real blockchain?".
Search the differences:https://blockchair.com/bitcoin/address/bc1qpcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
https://blockchair.com/bitcoin/address/bc1pqcmm7krqrmsjr4k84r85v6s9rhnl9warprrp0t
Lesson learned: use copy paste and carefully check each character. If I type an address on an offline computer, I create a checksum on both just to be sure I typed it correctly.
Jump to: