Author

Topic: Blockchain data download (Read 178 times)

newbie
Activity: 3
Merit: 7
November 30, 2023, 04:09:52 AM
#14
Bonus: LoyceV's small Linux commands for handling big data. If you have anything to add, please post there!
Aha... good stuff!  Thanks
 Smiley
newbie
Activity: 3
Merit: 7
November 30, 2023, 03:58:49 AM
#13
As an alternative to having to download stuff, I took a brief look to see if it was feasible to do using dune.com. There is a table there with bitcoin balance (balances_bitcoin.satoshi_day), and there are tables with inputs, outputs and TXs, so theoretically, using these tables, and filtering the joins with date intervals and minimum balance values, one should be able to derive the information on the platform.
Thanks for mentioning dune.  I have a login there but thought it was just ETH.  I will have another look. 

newbie
Activity: 3
Merit: 7
November 30, 2023, 04:21:58 AM
#11
I would like to analyze zombie addresses with a balance, sometimes called dormant addresses.
I don't think this is the data you're looking for. It doesn't have any information on when addresses were funded. You'll need the full blockdata.
Correct.  I should have said something like address, input, output, net and final TX date.
I'm actually trying to find a way to get similar output to this defunct google code project:
https://docs.google.com/spreadsheets/d/1xTROekDerP1TPOB3SOD_1bbQr580BPqbhF3YHdO96pw/edit?pli=1#gid=189298223
But that method looks useless without 2TB of memory.
 Smiley
So I was thinking to manipulate previously dumped data.

As an alternative to having to download stuff, I took a brief look to see if it was feasible to do using dune.com. There is a table there with bitcoin balance (balances_bitcoin.satoshi_day), and there are tables with inputs, outputs and TXs, so theoretically, using these tables, and filtering the joins with date intervals and minimum balance values, one should be able to derive the information on the platform.
Thanks for mentioning dune.  I have a login there but thought it was just ETH.  I will have another look.  

Bonus: LoyceV's small Linux commands for handling big data. If you have anything to add, please post there!
Aha... good stuff!  Thanks
 Smiley
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
November 29, 2023, 06:14:09 AM
#10
Shouldn't OP get what he wants by performing union between data on that link and http://alladdresses.loyce.club/ ?
Interesting idea, but I don't think so. That would put all funded addresses in chronological order of the first received transaction. That doesn't mean they're "dormant", they could have received more recent transactions too.

I can think of a combination of "comm" and "sort" to combine those 2 lists. If there's any interest, I'll do it, but only once. With limited RAM and HDD, I expect it to take a few hours to sort this long list.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
November 28, 2023, 11:07:26 AM
#9
Ascii.  So I can use Awk, grep and sed. ;^)
I like those Smiley
Bonus: LoyceV's small Linux commands for handling big data. If you have anything to add, please post there!

Not practical option when they limit download speed at only 10 kB/s.
Actually, it's 100 kB/s nowadays, so it "only" takes you 3 months to download Tongue
That's why I've created a mirror:
This (donated) server burns through rougly 12 TB of bandwidth per month.
legendary
Activity: 2338
Merit: 10802
There are lies, damned lies and statistics. MTwain
November 28, 2023, 05:33:43 AM
#8
<…>  Any suggestions? <…>
I figure you want to be able to list addresses such as those that figure on this site, that already list the top balance dormant addresses (in out terms, disregarding ins) for periods of time ranging from 1 to 9 years.

As an alternative to having to download stuff, I took a brief look to see if it was feasible to do using dune.com. There is a table there with bitcoin balance (balances_bitcoin.satoshi_day), and there are tables with inputs, outputs and TXs, so theoretically, using these tables, and filtering the joins with date intervals and minimum balance values, one should be able to derive the information on the platform.

I gave it a quick go, but despite using up some credits to use the middle sized engine, I kept on getting timeouts for a relatively simple query that tried to obtain all TXs with a current balance of over 2K BTCs and no outputs to date. It could be an alternative to explore further though, in case you don’t eventually create a custom solution on your own environment.
hero member
Activity: 1722
Merit: 801
November 27, 2023, 10:16:29 PM
#7
Yes, I do plan to run my own node, but it seems like parsing my own blockchain data may be not the best way to go.
Running a full node will help you a lot in privacy because you can use Tor and broadcast your transaction with your own full node.

Did you read this guide to run a full node with cheap cost?
[Guide] How to run a Bitcoin Core full node for under 50 bucks!

Quote
I've got lots to work with now.
Running a full node can help the Bitcoin network too because full nodes validate Bitcoin blocks. When you run a full node, you are contributing to Bitcoin network.
Live nodes: https://bitnodes.io/

To use blockchain data, you will have to do more tasks by yourself and sure it will take more time and efforts from you.
newbie
Activity: 3
Merit: 7
November 27, 2023, 07:00:46 AM
#6
Wow! Thanks to everyone for responding so quickly.
Yes, I do plan to run my own node, but it seems like parsing my own blockchain data may be not the best way to go.
I've got lots to work with now.
 Smiley
sr. member
Activity: 406
Merit: 443
November 27, 2023, 03:29:11 AM
#5
Any suggestions?
The site above collects data from Blockchair Database Dumps. You can do this by purchasing API packages form any block explorer and then using Python to arrange them or do whatever you want.

Bitcoin block data (728 GB): inputs, outputs and transactions
You can trust the data that comes from this site, but if you do not trust it or Blockchair, it is better to run a full node.
legendary
Activity: 2394
Merit: 5531
Self-proclaimed Genius
November 27, 2023, 02:20:30 AM
#4
I found http://addresses.loyce.club.
Is this the best/a really good site or are there other repositories to draw from?
That is the list maintained by LoyceV, link to profile: https://bitcointalksearch.org/user/loycev-459836
The user is one of the most prominent members here in the forum with lots of good contributions.

As for the list's safety and data integrity, there's no way to verify, but refer to LoyceV's trust rating which is in very high positive standing.
full member
Activity: 420
Merit: 120
November 27, 2023, 01:52:46 AM
#3
Here are Some popular sites where you can download blockchain data from
That is a page to Download Bitcoin Core wallet software. It's not a link to download Bitcoin blockchain data.

After you download the software, you will have two options to run your wallet, prune node or full node.
Both will download all blockchain data but prune node will delete data from past blocks that exceed the storage you set. Full node will store all blockchain data on your device.


Quote
At blockchair.com you can download blockchain data at https://blockchair.com/dumps#database
sr. member
Activity: 336
Merit: 495
Math + Code = Blockchain 😁
November 27, 2023, 01:42:47 AM
#2
Here are Some popular sites where you can download blockchain data from
You can make use of an external software to parse it
Do you intend on running a node?
 
newbie
Activity: 3
Merit: 7
November 27, 2023, 12:36:29 AM
#1
Hi All,

I'm looking for pre-parsed blockchain data downloads.  Ascii.  So I can use Awk, grep and sed. ;^)
I found http://addresses.loyce.club.
Is this the best/a really good site or are there other repositories to draw from?

I would like to analyze zombie addresses with a balance, sometimes called dormant addresses.
I did see some python code that might be helpful.

Any suggestions?
Thanks.
Jump to: