Topic: Bitcoin block data (728 GB): inputs, outputs and transactions (Read 3356 times)

I also suspect to not get 8 full cores, but 8 half or maybe even 8 quarter cores. They're quite a lot slower than my (older) Xeon server, while these are AMD EPYC 7551P cores.
The CPU steal was kinda expected, at least they're not suspending me. I used to process all my Trust updates a lot faster, now it takes over a day. But that's okay, as there are 6 more days left in the week and I'm not in a rush (for this).
My Merit update also takes longer, it used to finish on Sunday, but it's still running. It only uses one core, and the CPU steal ended. So if hammering all 8 cores for many hours in a row leads to 60% steal, I can live with that.
I've been screwed by disappearing servers in the past. In this case, I could have paid per month to reduce the risk, but took the risk out of convenience. As for the lack of online discussion: servarica_hani joined Lowendtalk in 2013.
For what it's worth: I've improved my backup strategy. I now make backups of everything.

It's really cheap. Although 60% CPU steal (means you get 40% of CPU computing power ?) and very few online discussion about serverica is somewhat concerning. As for about "Dedicated Core", lowendtalk website claim it only applies to VDS (not VPS) while serverica website claim it also applies to their VPS.

I used this Black Friday offer, or more specifically: 8 Slices of this one + 2 TB extra SAN storage.
It says "Dedicated Core", but I had my doubts about that, considering the price. Support told me I can use the CPUs to the max, but after many hours of Trust data processing, I saw 60% CPU steal this morning. That's unfortunate, but doesn't really matter for what I need. I'll adjust my usage a bit so it takes a bit longer at lower CPU load. I don't mind if the oldest pages have to wait a bit longer to update their navigation bar, but I'd like other tasks to not be delayed too much.
According to Lowendtalk, the host has a good reputation. All in all I think it's a good deal.

As far as I know, they get you on bandwidth charges. On my old (dedicated) server, I used about 10-20 TB per month. On this server, I'm at 4 TB in 5 days. That's why I choose 250 Mbit unlimited instead of 10 Gbit with a 32 TB bandwidth limit.

Use storage from a cloud provider - you'll get better results than a shared disk, unless your VPS is doing heavy computations on the data.    For a VPS, you should buy the best processor/RAM you can afford.

Out of curiosity, what VPS do you use? While i can easily find VPS which offer 8 core/32GB RAM at slightly higher price, usually it comes with much smaller storage.

Update:
It took 2 days to get this data from Bitcoin Core, but it's ready. Daily updates should be okay again.

Update!

After my (sponsored) dedicated server disappeared last week, I had to rush to get a new server. I'm still in the process of restoring data. From all my large data projects, this is the first one to be back online. This data reached 1 TB now. I've updated the sizes in this quote:

VPS server specs:
8 cores AMD EPYC 7551P
32 GB RAM
1 TB NVMe
2 TB SAN (Storage Area Network)
I paid $220 for a year.

Let me know if anything fails.

---

---

block_hash_version_versionHex_merkleroot_time_mediantime_nonce_bits_difficulty_ chainwork_nTx_strippedsize_size_weight.tsv.gz is still updating.

This project wasn't updated for over a week. I think my other download got so big, it was still running at the time this cronjob started. And Blockchair only allows one connection at a time, so it was rejected every day.
I'm now doing a manual update, and from now on updates will happen 4 hours later. That should give me many years before this same problem will occur.

If anyone wants this data, speak up NOW:
I want my disk space back, so I'll wipe this 855 GB soon. It took a lot of time to download, so if anyone wants a copy for some reason, better speak up now

Bro, you have infinite patience.   

This data from input transactions data, file name  

blockchair_bitcoin_inputs_20100304.tsv.gz

This file with r,s,z ?


block_id   transaction_hash   index   time   value   value_usd   recipient   type   script_hex   is_from_coinbase   is_spendable   spending_block_id   spending_transaction_hash   spending_index   spending_time   spending_value_usd   spending_sequence   spending_signature_hex   spending_witness   lifespan   cdd      
42495   44f339ccc78c6395cca0163a5a3da7ff4b76bff329ec8e1f45996859717385ac   0   2010-02-25 9:22:30   5000000000   0.5   1K5ULJgsJR1xQsSDmPsFCFsFgEmKkMtNQP   pubkey   4104127c91de8d72c7e1cb21cbf784b074e3a4d556c8ca8f17b046a25dbbbb4e7b05a3881cefcfe 52475877ea654fcebe7901be53a0bef042050905c7aa21293b137ac   1   1   43736   0aaad829c52080b76292ebb168f684f264c5d4a092643063afaddacfb83e6553   0   2010-03-04 14:25:07   0.5   4294967295   493046022100a738f0a95a5ffdcfd688e33a05d2d118c0cba2bb994a5cad94ab0c59027c5542022 100d81a0d56454abd9ea240ce4c02c1fcda29c68216d4558ab96149fa65379a26e901      622957   360.5075231      
42365   687f509be527da767f8d44332111221a92e951c1b2d6f64e5d9539e2af4efa29   0   2010-02-24 12:26:00   5000000000   0.5   13izg2rAiGaFFpUCqBKeUm8BSohqfJpoS4   pubkey   41041e99b07917e497b0e98215fa05ffb926aa1b3ec507b4d621a584c4bf093828433858d9e63b7 19ec7e838f539ac5a9fecad99bdf099393d1b670270cc46733350ac   1   1   43759   15293ee1b14f9a2c5041b8952e89aa6eca455c7a08a9f9048fbadbd5bae1c60b   0   2010-03-04 17:36:55   0.5   4294967295   473044022015e67872d04797f7ac31dd59750c670954522bea767ab37bd3c1a710dcfcbbab02205 cd8bc98e6070cf0afed8968f8ef5f3ada1f51b9a4d0bb817b3a70e035d4d4c201      709855   410.7957176      
42017   3304b670a5e5a864e2ad416b52300cf355038c67789c199775cb4bfa4ad8ddc2   0   2010-02-22 23:32:50   5000000000   0.5   1PiuHkfNomTJRnMbywnNjhrgbHDJNPWkEc   pubkey   4104b205906f0304ae29c994125279f4be7b2de3ab2bddbf8607ad24f4d627bbbeb9fc076159d2d 5a87f4bbf418bc2bcfe8ba06dda1512061903bcb00842a2a627fcac   1   1   43759   15293ee1b14f9a2c5041b8952e89aa6eca455c7a08a9f9048fbadbd5bae1c60b   1   2010-03-04 17:36:55   0.5   4294967295   49304602210083ffc2fd4853517c9a51c3744b0aa8bf3700e2822d74813ae6b75053d6372bfd022 1009466c70a0180688e459da015357bbc2ab9aeb0de3beac0cb1efaba623eca30fe01      842645   487.6417824      
43034   3c34970dba0eeb2960001e77d667d3e9df00da0ba0c0351bf4af10ef9b67e4ce   0   2010-02-28 14:51:06   5000000000   0.5   14hJYeFAjckhuzpJkUj6mont2yZheizUXt   pubkey   4104bc6235e6ad560ab0f1f09120454948755a055ba591db3019302b11732637a53155315057a15 b7de9afddd5ce4e722f59c6fc46ac87bffd827a84e7340985ac68ac   1   1   43759   15293ee1b14f9a2c5041b8952e89aa6eca455c7a08a9f9048fbadbd5bae1c60b   2   2010-03-04 17:36:55   0.5   4294967295   483045022100c683817cc7010389e185144a7fe411af7b8564ca8a13dbf9c27dcd1986685afc022 01fae568aa4be674f993957f4030119569cd6e06ae87894a67a5b9bf44d17833101      355549   205.7575231      
43567   eafb39f3340676d900190badabb898756e2862594c280927061d8ded5d3c1343   0   2010-03-03 14:45:17   5000000000   0.5   17FdnuFaT8wuuBSyaJDBPZ3cLLgr8bpVzn   pubkey   4104780f11944bbabef7753de80994abd0fb01f739cd461c317c805ccaaac82908955a49bc5e1b9 c1c2a10c0713c972b531870f6e2d20bb9565316b9c0dbbba1af9aac   1   1   43759   15293ee1b14f9a2c5041b8952e89aa6eca455c7a08a9f9048fbadbd5bae1c60b   3   2010-03-04 17:36:55   0.5   4294967295   47304402204b19320ece79b7a3b4a70eacffbd22828468fb682822aa5693fa5084e05990aa02207 02899a2ea9781f38a89ac5d56ce3824bc19cd15ca9a3b4e5611600cb10d165701      96698   55.95949074      
43256   529e358c5a5bab4f8aedad3e1c304bc49dfa0b69b439b49f15589567e3dbb0c3   0   2010-03-01 20:01:46   5000000000   0.5   1MKDNdVs2zRY75LJh5DYaF954AgwtYAwy4   pubkey   4104615a801f9da7e4365768009b1606fd13ae8df9187ce79759d7e23a021a3e73e950555df6213 9d4f252e9d01a60990f07c78cb1ecab1ae241c7308b79e6948f26ac   1   1   43759   15293ee1b14f9a2c5041b8952e89aa6eca455c7a08a9f9048fbadbd5bae1c60b   4   2010-03-04 17:36:55   0.5   4294967295   483045022100d57a8cc989301c0c5c3b3c47dc38bd3c11d5eef69d8f83d72c540a659760c210022 017a4b4b6e4ec0ee37a6655186dce4bb564b2ed285c317d7b9a87666469b5f8cd01      250509   144.9704861      
43569   43ccf58532ba0d51c2b3ad71066417c7473c6b6f4e70e4f8ed0c665b758eb2d8   0   2010-03-03 15:06:12   5000000000   0.5   18ePkq5kbYnHMBTr6VMEcsPm6DzxXAnfuR   pubkey   410443bd83a3c18343736b26511d7225bd63b5ba3a5ef032c097bddaa1426d31318ed0c1c5b6168 236a967acf94d3621011bf09e638af913e6dab00098b719425d04ac   1   1   43759   15293ee1b14f9a2c5041b8952e89aa6eca455c7a08a9f9048fbadbd5bae1c60b   5   2010-03-04 17:36:55   0.5   4294967295   483045022100ae71a3f8516280088d441ac20ac8d9c1f33bd6a05385c96d809599965b8e66ba022 07ae75074af73903209a7c1d9cc9cefcccb4f374b7cb6fc6020bd818d0adf3ff201      95443   55.23321759      
38596   cb3d1c126178ab5d560f12a966f3b592e705773e08c3c1177005f16a17e4d875   0   2010-02-06 6:02:51   5000000000   0.5   1PmVA4Hnf3a8h8NzPtkXpWS4xvxa2o4Kiz   pubkey   4104c78455c3267666e8e7a9e2c6ac4f5d1225c2556a50d9b79d6f6d97bd4413f7c463ce6b31863 2a40f4004cc36cc0f319c2e0c05de3bf54d4c1c8365699bcc65f6ac   1   1   43789   4681ae540cf953bf69ac890af6eba3647dcd3469b5f20052723a3ec3414ec2a6   0   2010-03-04 20:34:45   0.5   4294967295   493046022100ee23a1d3cad9ac03b5aa6845ebe68615b52d4a5830d40dd838015f67690291f6022 100aa702b9d4490ed255b6a641f00e08a612ff242e379fc20b6112d32c7305a0e8b01      2298714   1330.274306      
31271   e3077f5229f41ec6db52ec5c3308666ec1b64e4e5e3f303a8873ac84122fe275   0   2009-12-24 1:05:46   5000000000   0.5   16ECYkGoqCw4SeXvGbbPmQzjqCggjobige   pubkey   41048205e75a1fb24902b71c0c1f49b993c1e0ff31dcd8bd636873eecd1a9837287f70a2fe13ece c492ab8541340f8a19558bb1dbb6752d984ffc6f29533ea83ba5dac   1   1   43789   4681ae540cf953bf69ac890af6eba3647dcd3469b5f20052723a3ec3414ec2a6   1   2010-03-04 20:34:45   0.5   4294967295   48304502205853639a66e15d0dd805de9841a5a974892b9869fd2fcf894f89c3721efdc8e602210 0ab2d66be1b4c183587ae481e81e6143a88e66b2ef0453c65fc0e282329fec93701      6118139   3540.589699   

---

I find, r,s,z data in input tx files, not in output.


Thank you   

See the samples in the OP. Or download one file from each directory. If you find it, post it here.

Bro, in what files on your server  r,s,z data of BTC ?  In files with transactions, outupts or

I looking for bulk data , not json etc....


Thank.You.

Bump!

Zensi never made me an offer. Blockchair also has BCH, Dash, Doge, LTC or Zcash data dumps available (for a very slow download).

You can find the data at gz.blockchair.com/bitcoin-cash/. I don't really want to add hundreds of GB of Forkcoin data to my server. If you need help downloading or processing the data, feel free to make me an offer.

pls add all bch addresses

I only added this centralized shitcoin to utilize the 2 TB storage disk, but adding several gigabytes per week filled it. This gives enough space for a long time again.

I'll create a TSV with this:
I left out the previous and next block hash to reduce the size.

It should be done in about 11 hours, which means I'll share the output tomorrow. I expect the file to be around 150 MB, so I didn't bother making smaller daily files.
Once done, this will get daily updates, up to the same block as Blockchair's data dumps (so not the latest blocks from my own Bitcoin Core).

Update
block_hash_version_versionHex_merkleroot_time_mediantime_nonce_bits_difficulty_ chainwork_nTx_strippedsize_size_weight.tsv.gz (80 MB).

I've updated the OP

Hi Loyce and thanks for adding "merkleroot" to your data. Would you mind to add also the rest ?

From how to dump or parse information from all available blocks on a full-node, I realized there's more block data missing from these data dumps.

Example:

If anyone wants a dump of this data, just ask. I don't mind adding it.