[Question] Fast way to check bulk address bitcoin balance

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

Quote from: albert0bsd on November 14, 2023, 08:28:30 AM

Quote from: DanielHoffman on November 13, 2023, 11:25:58 PM

Hi i want to check a bulk addresses balance, actually i use request.get in python way with blockstream but it's too slow (around 100 addresses in 46 seconds)

Why people always want to do this? It is always slow.

The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

Then load that file into a database/bloom filter or some other custom method.

Check your list against that method.

WIth this way you can be able to check million of address in less than a second

In fact you can even make your own list if you download Blockchair's transaction dumps (I think they are daily IIRC) and then process the transactions using your own filter and scripts to add up all the UTXOs on a particular address and subtracting the ones you sent.

You need quite a few terabytes to make this possible, but it lets you do a lot of cool stuff.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

apogio

hero member

Activity: 560

Merit: 1060

Quote from: LoyceV on November 16, 2023, 08:50:10 AM

3. The data is updated daily. I assume OP is trying to brute-force Bitcoin addresses, in which case that doesn't matter.

Are there still people trying to brute-force keys? I am amazed...

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: albert0bsd on November 14, 2023, 08:28:30 AM

The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

Then load that file into a database/bloom filter or some other custom method.

This is what I would use to quickly check millions of addresses:

Quote from: LoyceV on June 11, 2020, 10:30:34 AM

How to use
The most likely use is to check a long list of Bitcoin addresses for a remaining balance.
On Linux, use this to find matching addresses (after extrating the compressed .gz file of course):

Code:

comm -12 Bitcoin_addresses_LATEST.txt <(cat mylist.txt | sort | uniq)

Bitcoin_addresses_LATEST.txt: the extracted latest version downloaded from addresses.loyce.club.
mylist.txt: your own list of addresses, one address per line.

This takes only seconds to check millions of addresses.

Quote from: bitmover on November 14, 2023, 01:06:07 PM

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

Three things:
1. The server can spit out that file within 46 seconds Tongue

2. OP mentioned "bulk", and I don't consider his 100 address test to be bulk.
3. The data is updated daily. I assume OP is trying to brute-force Bitcoin addresses, in which case that doesn't matter.

Even if it takes a few minutes to download, after that you can check billions of addresses per hour.

@OP: what are you trying to accomplish?

apogio

hero member

Activity: 560

Merit: 1060

Quote from: albert0bsd on November 16, 2023, 07:45:27 AM

Do you read all the posts before reply?

I edited my post above and explained you have already mentioned it. I didn't carefully read your post, sorry.

Quote from: albert0bsd on November 14, 2023, 08:28:30 AM

OP is a lazy newbie, he want the code already done, just to run in one-click, he is not developer, maybe he doesn't know what a hashmap or a bloom filter is... if you see he only reply to the person who actually offer some code snippet

Ok, but OP can also import the file from LoyceV's website to an excel or something else and do the process manually. Excel would crash, now that I think about it.

In fact, OP wants something that can't be done with just one click.

albert0bsd

hero member

Activity: 862

Merit: 662

Quote from: apogio on November 16, 2023, 04:43:00 AM

Another solution to the problem:
...

Do you read all the posts before reply?

Quote from: albert0bsd on November 14, 2023, 08:28:30 AM

...
The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance
...

OP is a lazy newbie, he want the code already done, just to run in one-click, he is not developer, maybe he doesn't know what a hashmap or a bloom filter is... if you see he only reply to the person who actually offer some code snippet

apogio

hero member

Activity: 560

Merit: 1060

Another solution to the problem:

Look into LoyceV's datasets: https://bitcointalksearch.org/topic/list-of-all-bitcoin-addresses-with-a-balance-5254914

1. Download the daily data from there (it includes every address that has a balance).
2. Import them in a DB
3. Set your code to ask the DB if it includes the address you want. This will be much faster.

Optional tip: If you import batch addresses from the DB into a hashmap then you can check if it contains the address ULTRA FAST.

EDIT: It has already been mentioned by Yamane_Keto & albert0bsd above.

btc_enigma

hero member

Activity: 691

Merit: 569

Feel free to see the following github repo that uses several blockexplorer APIs to check address balances

https://github.com/xylevy/BTCSteroids

PS: Blockonomics API actually allows to lookup multiple addresses in one call and should be quite useful for this task

albert0bsd

hero member

Activity: 862

Merit: 662

Quote from: DanielHoffman on November 15, 2023, 02:36:57 PM

Thanks but already do check balance in same 50 seconds Embarrassed

, honestly i want to do this very fast but it's okay

You are very lazy, you omit a lot of good answers.

DanielHoffman

newbie

Activity: 26

Merit: 0

Quote from: seek3r on November 14, 2023, 02:58:41 AM

Using requests.get is indeed very slow.

To check the balance of Bitcoin addresses in bulk, you would want an API that supports batch requests.
This would speed up the whole process since you can query multiple addresses within a single API call. The Blockchain.com API supports batch requests for address balances.

Code:

import requests

def fetch_balances(addresses):
address_list = '|'.join(addresses)

api_endpoint = f'https://blockchain.info/balance?active={address_list}'

response = requests.get(api_endpoint)
data = response.json()

for address, info in data.items():
balance = info['final_balance']
print(f"Address: {address}, Balance: {balance} ")

addresses = ['address1', 'address2', 'address3', ...]

fetch_balances(addresses)

A simple python script that would help you to fetch the balances of multiple addresses. The output would be in satoshis due to API of Blockchain.com

Thanks but already do check balance in same 50 seconds Embarrassed

, honestly i want to do this very fast but it's okay

seek3r

legendary

Activity: 1260

Merit: 2014

Quote from: Yamane_Keto on November 15, 2023, 09:08:14 AM

A simple python script that would help you to fetch the balances of multiple addresses. The output would be in satoshis due to API of Blockchain.com
The blockchain.info API has a long history of sudden shutdowns, in which you will not receive a notification or alert, and the support team responds slowly and may continue for several days without knowing the cause of the problem.
blockchair.com/API is better.

But the OP asked for a solution with batch requests and afaik blockchair's API does not support this in a single API call. Wink

Blockchairs API typically works with individual queries for each address or transaction. This means you would need to send a separate request for each address you want to check, which can be less efficient than a batch request, especially for a large number of addresses.

An alternative to blockchain.com could be the Blockcypher API, but I have no experience with it.

Yamane_Keto

hero member

Activity: 406

Merit: 443

Quote from: seek3r on November 14, 2023, 02:58:41 AM

Code:

import requests

def fetch_balances(addresses):
address_list = '|'.join(addresses)

api_endpoint = f'https://blockchain.info/balance?active={address_list}'

A simple python script that would help you to fetch the balances of multiple addresses. The output would be in satoshis due to API of Blockchain.com

The blockchain.info API has a long history of sudden shutdowns, in which you will not receive a notification or alert, and the support team responds slowly and may continue for several days without knowing the cause of the problem.
blockchair.com/API is better.

Quote from: albert0bsd on November 14, 2023, 08:28:30 AM

The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

It seems that addresses.loyce.club depends on the data that comes from blockchair_bitcoin_addresses_and_balance_DATE.tsv.gz. Therefore, even if you download this file and try to manage the database and then create the API, you are accessing the indirect blockchair/API, which I think would be slower, if If you have the resources, buy a paid service.

albert0bsd

hero member

Activity: 862

Merit: 662

Quote from: bitmover on November 14, 2023, 02:32:17 PM

And looks like op is looking for a performance boost

In that case a custom host like mempool.space can be installed to handle all the block chain data already parsed and processed in this way you can change your API end point to this localhost server without any restriction..

Quote from: CryptoDeivid on November 14, 2023, 07:50:11 PM

This time may vary depending on the specific data being used and the processing involved, I thing, but 46 seconds is a relatively short amount of time.

Once that the data is downloaded an proccessed the waiting time will be just small.

yhiaali3

legendary

Activity: 1848

Merit: 1982

Fully Regulated Crypto Casino

There are many sites that provide this service for free or paid. You can try this free site:

https://cointool.app/batchCheckBalance/btc

It gives good and fast results, and the other good thing about this site is that it gives aggregate address balance results for coins other than Bitcoin on many networks such as:
BTC, ETH, Arbitrum, BSC, Optimism, TRX, HECO, Solana, Polygon,.......etc

CryptoDeivid

jr. member

Activity: 40

Merit: 24

Quote from: albert0bsd on November 14, 2023, 01:29:01 PM

Quote from: bitmover on November 14, 2023, 01:06:07 PM

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

Well that is true the data may bot be updated, but it is a good point to start no? If someone do this he can keep updated their database with other sources scripts and this may avoid all external web APIs limits.

About the 46 Seconds... it is not bvious?? The time that i mention is after download and process the data.

This time may vary depending on the specific data being used and the processing involved, I thing, but 46 seconds is a relatively short amount of time.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

Quote from: albert0bsd on November 14, 2023, 01:29:01 PM

Quote from: bitmover on November 14, 2023, 01:06:07 PM

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

Well that is true the data may bot be updated, but it is a good point to start no? If someone do this he can keep updated their database with other sources scripts and this may avoid all external web APIs limits.

About the 46 Seconds... it is not bvious?? The time that i mention is after download and process the data.

It is amazing to use this file, especially for privacy purposes. As he is looking all addresses, nobody knows which one is his.

But data probably don't get updates in every block (like a block explorer). And looks like op is looking for a performance boost

albert0bsd

hero member

Activity: 862

Merit: 662

Quote from: bitmover on November 14, 2023, 01:06:07 PM

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

Well that is true the data may bot be updated, but it is a good point to start no? If someone do this he can keep updated their database with other sources scripts and this may avoid all external web APIs limits.

About the 46 Seconds... it is not bvious?? The time that i mention is after download and process the data.

bitmover

legendary

Activity: 2352

Merit: 6089

bitcoindata.science

I think the problem with those free api is that you might even get blocked doing 100 requests in less than a minute.

I made this javascript tool which allows you to do search for address balances, but it can't handle 100 addresses (I even used a try/catch method with a seocnd api when you get to api request limits)

Take a look

https://bitcoindata.science/bitcoin-balance-check

Quote from: albert0bsd on November 14, 2023, 08:28:30 AM

Quote from: DanielHoffman on November 13, 2023, 11:25:58 PM

Hi i want to check a bulk addresses balance, actually i use request.get in python way with blockstream but it's too slow (around 100 addresses in 46 seconds)

Why people always want to do this? It is always slow.

The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

Then load that file into a database/bloom filter or some other custom method.

Check your list against that method.

WIth this way you can be able to check million of address in less than a second

This is not faster than 46 seconds (as you have to download a huge file)... and the data might not be updated.

albert0bsd

hero member

Activity: 862

Merit: 662

Quote from: DanielHoffman on November 13, 2023, 11:25:58 PM

Hi i want to check a bulk addresses balance, actually i use request.get in python way with blockstream but it's too slow (around 100 addresses in 46 seconds)

Why people always want to do this? It is always slow.

The correct way to do this faster is download an updated list of address with balance List of all Bitcoin addresses with a balance

Then load that file into a database/bloom filter or some other custom method.

Check your list against that method.

WIth this way you can be able to check million of address in less than a second

seek3r

legendary

Activity: 1260

Merit: 2014

Using requests.get is indeed very slow.

To check the balance of Bitcoin addresses in bulk, you would want an API that supports batch requests.
This would speed up the whole process since you can query multiple addresses within a single API call. The Blockchain.com API supports batch requests for address balances.

Code:

import requests

def fetch_balances(addresses):
address_list = '|'.join(addresses)

api_endpoint = f'https://blockchain.info/balance?active={address_list}'

response = requests.get(api_endpoint)
data = response.json()

for address, info in data.items():
balance = info['final_balance']
print(f"Address: {address}, Balance: {balance} ")

addresses = ['address1', 'address2', 'address3', ...]

fetch_balances(addresses)

A simple python script that would help you to fetch the balances of multiple addresses. The output would be in satoshis due to API of Blockchain.com

Topic: [Question] Fast way to check bulk address bitcoin balance (Read 383 times)