Bitcoin block data (728 GB): inputs, outputs and transactions - page 4.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

I've finalized my update, and after 10 months, I'm happy to announce This project is back online!

Quote from: LoyceV on January 07, 2021, 12:33:04 PM

Back online! The data
See this new location instead of blockdata.loyce.club
le6-1-81flash.pulsedmedia.com/public-loyceclu/inputs/ (469 GB)
le6-1-81flash.pulsedmedia.com/public-loyceclu/outputs/ (145 GB)
le6-1-81flash.pulsedmedia.com/public-loyceclu/transactions/ (52 GB)

New! Torrents!
inputs.torrent
outputs.torrent
transactions.torrent
For privacy, you may want to consider using a VPN so other users can't see your IP address.

Feel free to test (and/or seed) the torrents if you have some bandwidth and disk space to waste

I haven't tested the daily updates yet. Let me know if anything needs my attention.

ABCbits

legendary

Activity: 2856

Merit: 7410

Crypto Swap Exchange

Quote from: LoyceV on March 09, 2022, 01:37:31 PM

Why?

Because i wanted to test whether DHT works.

Quote from: LoyceV on March 09, 2022, 01:37:31 PM

And how? When I open a torrent in Transmission, it just starts and doesn't ask to remove trackers.

On qbittorrent-nox, you can choose not to start torrent after adding it. It has tracker tab where you can remove each tracker manually.

Quote from: LoyceV on March 09, 2022, 01:37:31 PM

I'm used to only downloading Torrents with many seeders, and those usually start within seconds. I have no idea why this takes so long. The Seedbox doesn't seem to lack capacity.

I have no idea either, but the weird thing is "transaction" torrent start within few seconds.

Quote from: LoyceV on March 09, 2022, 01:37:31 PM

CLI? Even though I use CLI for most things, I've never used it for Torrent.

I only use CLI to start the program, qbittorrent-nox offer web UI.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ABCbits on March 09, 2022, 09:06:16 AM

1. I remove tracker on both torrent before i start download process.

Why? And how? When I open a torrent in Transmission, it just starts and doesn't ask to remove trackers.

Quote

2. "output" downloaded started some time (about 15 - 30 minutes) after i add it.

I'm used to only downloading Torrents with many seeders, and those usually start within seconds. I have no idea why this takes so long. The Seedbox doesn't seem to lack capacity.

Quote

3. Average speed was 24-30MB/s. Although "transaction" have 2x speed after it connected to 2 seeder.

I think "outputs" shifted seeding to the other Seedbox at some point. I expected both to seed simultaneously.

Quote

4. I use qbittorent-nox with default configuration.

CLI? Even though I use CLI for most things, I've never used it for Torrent.

Quote

I think i can conclude that you made the torrent files correctly and there's no problem with your seedbox.

Thanks! I have about 100 GB upload to go, I expect it to be done tomorrow. After that, I'll create 3 brand new Torrent files with the most complete data (including correct file dates). I'll also schedule daily downloads to update the latest files again, update OP, and add the new download link.
I won't update the Torrent file that often, which means it'll miss the latest days (or weeks, months....).

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ABCbits on March 09, 2022, 07:33:40 AM

But it's not practical option.

Indeed. I don't want users to manually add IPs.

Quote

Testing it now.

Only one out of 2 Seedboxes is currently uploading (at a very decent speed).

I'm uploading the rest of "inputs", after that I'll fix the file dates and see if I can get a more permanent solution for hosting those files.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ABCbits on March 09, 2022, 05:53:42 AM

Since it include option "Private Torrent", usually it means DHT is used by default. And when i tried it on my VPS (where i remove all tracker before start the torrent), the DHT is working and i can start the download.

To test this, I tried to create a new one with "Private torrent" enabled, and 4MB piece size. It didn't show up, so I deleted the existing one first. It still didn't show up. I'm now testing without "Private" again: this works.

Quote

Yeah, usually i just rely on DHT and share my IP:Port (if needed) so other user can it directly to list of peers on their torrent software.

How would that IP/port-sharing work? Usually I just click the Torrent or Magnet link, and don't need to manually add IPs.

Quote

My VPS have spare bandwidth, do you want me to try to see whether i can complete the download?

Feel free to test it

ABCbits

legendary

Activity: 2856

Merit: 7410

Crypto Swap Exchange

Quote from: LoyceV on March 06, 2022, 03:26:12 PM

I'd appreciate some pointers from someone with experience creating Torrents.

More information would be appreciated. For example, how do you create the torrent? In past i used mktorrent (to create torrent) and qbittorrent (on both server and client to upload/download) to transfer data due to unstable connection on my end. I don't use any tracker, but i disable private flag so i could use DHT. However, my data size was smaller.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: seoincorporation on March 08, 2022, 01:51:22 PM

You need a VPS with a huge disk and unlimited band bandwidth, and those aren't cheap at all

The Seedbox actually isn't that bad: It has 6TB monthly bandwidth, and when it reaches the limit it continues at 100 Mbit/s.

Quote

You can buy a laptop (workstation) with a 1TB SSD and configure it as a server with NO-IP. This way your bandwidth will be limited only by upload and download speed, but not for the amount of data. And you can run the server in your house.

There are several problems with that: First, I don't want to expose my home IP address for this. And I don't want my upload bandwidth to be maxed out for several days in a row when someone downloads 600 GB.
Third problem is that I don't think my internet provider would appreciate it if I upload a few TB each month, so they'll probably start throttling my bandwidth. And on top of that, given current electricity prices, running a system 24/7 is more expensive than the Seedbox.

If I can't get the Torrents to work, the files can still be downloaded directly through http, but I haven't even finished uploading everything yet. And if barely anyone is going to use this, I might just have to give up on this project entirely at some point.

seoincorporation

legendary

Activity: 2982

Merit: 2681

Top Crypto Casino

Hello LoyceV.

After reading the thread I see you are having problems with the amount of data and the way to share it, and I was thinking about a solution or how I would do it.

You need a VPS with a huge disk and unlimited band bandwidth, and those aren't cheap at all, but I know a cheap way to handle this.

You can buy a laptop (workstation) with a 1TB SSD and configure it as a server with NO-IP. This way your bandwidth will be limited only by upload and download speed, but not for the amount of data. And you can run the server in your house.

If you like the idea and want some help with the server configuration just let me know. Wink

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ABCbits on March 08, 2022, 07:07:04 AM

Really? It's weird since rTorrent (the backend) support DHT.

These are the only options it gives:
Image loading...

Quote

Can you use different tool (which support DHT) on your Seedbox?

I can start the following:

Code:

Deluge
Rclone
qBittorrent

I haven't tried those yet.

Quote

2. You might want to check this thread about torrent piece size, https://torrentinvites.org/f29/piece-size-guide-167985/.

I read about that already, it's a compromise. I could select a smaller piece size, but that means it has many pieces to deal with. I'll try 4MB next time.

Quote

3. Take note not all tracker accept any kind of torrent.

That's the tricky part: I couldn't find much about it, only a list of recommended trackers. That's what makes me think the slow download start might be related to the trackers.

ABCbits

legendary

Activity: 2856

Merit: 7410

Crypto Swap Exchange

Quote from: LoyceV on March 07, 2022, 07:52:57 AM

Quote from: ABCbits on March 07, 2022, 07:18:35 AM

how do you create the torrent?

I have a Seedbox at PulsedMedia.com, which by default uses ruTorrent. I clicked "Create Torrent", selected the data, and added some trackers that I found in other torrents. Then, I set the Piece size as large as possible (16 MB, because it's 50+ GB of data). After a couple of minutes, it was ready and I could save the .torrent file.

1. ruTorrent (which use rTorrent as backend) is reliable (according to discussion on different forum), although it's development has slowed down in past 3 years. However it should able to create torrent without problem.
2. You might want to check this thread about torrent piece size, https://torrentinvites.org/f29/piece-size-guide-167985/.
3. Take note not all tracker accept any kind of torrent.

Quote from: LoyceV on March 07, 2022, 07:52:57 AM

Quote

In past i used mktorrent (to create torrent) and qbittorrent (on both server and client to upload/download) to transfer data due to unstable connection on my end. I don't use any tracker, but i disable private flag so i could use DHT. However, my data size was smaller.

When creating the torrent, ruTorrent doesn't have the option to use DHT.

Really? It's weird since rTorrent (the backend) support DHT. Can you use different tool (which support DHT) on your Seedbox?

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ABCbits on March 07, 2022, 07:18:35 AM

how do you create the torrent?

I have a Seedbox at PulsedMedia.com, which by default uses ruTorrent. I clicked "Create Torrent", selected the data, and added some trackers that I found in other torrents. Then, I set the Piece size as large as possible (16 MB, because it's 50+ GB of data). After a couple of minutes, it was ready and I could save the .torrent file.

Quote

In past i used mktorrent (to create torrent) and qbittorrent (on both server and client to upload/download) to transfer data due to unstable connection on my end. I don't use any tracker, but i disable private flag so i could use DHT. However, my data size was smaller.

When creating the torrent, ruTorrent doesn't have the option to use DHT.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: PawGo on March 06, 2022, 03:10:11 PM

I have just started download, but I have only one seed, for Transactions.
I will leave it and see if it finish.

Thanks for testing. I see it's indeed uploading now. My own download (from the new server) hasn't started yet.
My guess is there might be a problem with the Trackers I choose.

Update:
My own download started and completed overnight. Judging by the amount of sent data, your download stopped after about a third.
I'd appreciate some pointers from someone with experience creating Torrents.

ABCbits

legendary

Activity: 2856

Merit: 7410

Crypto Swap Exchange

Quote from: LoyceV on January 17, 2022, 12:07:31 PM

Quote from: ABCbits on January 17, 2022, 08:25:14 AM

For $10, that VPS is very cheap. Do you mind share the name of the VPS provider?

RamNode.com. This is the "MASSIVE KVM", which means it's HDD and not meant for heavy computing tasks.

Looks like it could meet my needs. I have bit concern about payment processor they use though (Bitpay and Payssion), let's hope one of them won't ask for KYC verification or strict blacklist.

PawGo

legendary

Activity: 952

Merit: 1367

Quote from: LoyceV on March 06, 2022, 02:53:37 PM

Quote from: LoyceV on February 24, 2022, 04:45:53 AM

Torrents!
outputs.torrent
transactions.torrent

Has anyone tried these torrents? No data was uploaded, so if you tried and it's not working, please let me know.

I got a new server today: 1 TB, so enough to add the "inputs" data too. I'm currently trying to Torrent the files from the old to the new server, but so far it's not doing anything. Hence my question if anyone else tried

I have just started download, but I have only one seed, for Transactions.
I will leave it and see if it finish.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: LoyceV on February 24, 2022, 04:45:53 AM

Torrents!
outputs.torrent
transactions.torrent

Has anyone tried these torrents? No data was uploaded, so if you tried and it's not working, please let me know.

I got a new server today: 1 TB, so enough to add the "inputs" data too. I'm currently trying to Torrent the files from the old to the new server, but so far it's not doing anything. Hence my question if anyone else tried

ABCbits

legendary

Activity: 2856

Merit: 7410

Crypto Swap Exchange

Quote from: LoyceV on January 17, 2022, 07:32:10 AM

The best (temporary) option for hosting I found so far is this:

Quote

$10/mo ($0.015/hr)
2 CPU, 2GB RAM, 650GB HDD

This offers 7.5TB/month bandwidth, and charges $4/TB for additional bandwidth.
Would anyone be interested in this data at this price?

As someone who run full node and aware of GitHub existance (which host script to extract Bitcoin Core data), i'm not interested.

P.S. For $10, that VPS is very cheap. Do you mind share the name of the VPS provider?

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

New hosting
On my quest for new hosting, I ended up on a seedbox. It's not big enough for all data, but to start, I've uploaded outputs and transactions. See: blockdata. Unfortunately, there's no nicer URL for now.

Torrents!
outputs.torrent
transactions.torrent
~~I think the problem is with torrent trackers, but I'm not sure how to fix it.~~ It works now

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ABCbits on January 18, 2022, 08:08:16 AM

I have bit concern about payment processor they use though (Bitpay and Payssion), let's hope one of them won't ask for KYC verification or strict blacklist.

Last time I funded my account was last May 2021, using a low-fee Forkcoin through Bitpay. I funded it with $10 at a time, just in case they'd hit me with KYC. They didn't. But as far as I know that's no longer possible.
I've never used Payssion.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: ABCbits on January 17, 2022, 08:25:14 AM

For $10, that VPS is very cheap. Do you mind share the name of the VPS provider?

RamNode.com. This is the "MASSIVE KVM", which means it's HDD and not meant for heavy computing tasks.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

The best (temporary) option for hosting I found so far is this:

Quote

$10/mo ($0.015/hr)
2 CPU, 2GB RAM, 650GB HDD

This offers 7.5TB/month bandwidth, and charges $4/TB for additional bandwidth.
Would anyone be interested in this data at this price?

Topic: Bitcoin block data (728 GB): inputs, outputs and transactions - page 4. (Read 2869 times)