Topic: Bitcoin block data (728 GB): inputs, outputs and transactions - page 3. (Read 3356 times)

It is unlikely that Loyce would be dealing with Cogent directly, but rather would be dealing with one of Cogent's customers.

Even if someone's bandwidth is "unmetered", I can assure you that usage is still "monitored". As you note, Cogent is going to oversell their capacity, and most likely, Cogent's customer who sells VPS services will also oversell their capacity. If Loyce is constantly sending hundreds of GB's worth of data to the internet, there will be less capacity for other customers to send their own data to the internet, which will degrade service for others.

The files that Loyce is hosting total over 660 GB. It would not take much for Loyce to run into hitting the multi TB range with files of that size, especially considering that it is trivial for someone to request those files multiple times.

What's the problem? Doesn't both scp and rsync use ssh when copying data from different device?


You could run Bitcoin Core and processing script on local device, then upload the result to your VPS/seedbox.


I think he meant average terminal user, although i would argue nano is more popular option.

Not unless he is working with a really small provider. More and more 1GB un-metered for co-location is the standard.^* Or if it is metered it's in the multi TB range.
Bandwidth has gotten so cheap at data centers that it's pointless to rate it anymore. Case in point, Cogent just made us an offer for a 1GB fiber loop to our rack for under $400 a month all in. 3 years ago that same circuit was well over $1500. Hurricane is under $2500 for a 10GB circuit. And we are a very small buyer of bandwidth. "Real" companies that are buying multi 100GB circuits are paying very very little.

-Dave

^* Most places are going for a 10 to 1 over subscription so you probably may not get the 1GB all the time but the point is the same.

Ahh, yes, there it is. I thought I remembered giving you this advice, but I couldn't find the discussion in this thread.

AWS and GCS have the same pricing structure, generally speaking. If you hosted the files on either of those platforms, it would cost approximately US$20 per month. This is not the same as hosting your files on a VPS. Accessing and transferring the files would be much quicker compared to using a VPS.

I believe you have previously stated that you don't want to use AWS because you don't want to have to use a credit card to pay, nor associate your IRL identity with the service. My argument would be that this is really your only option, as using a VPS with a smaller provider (as you have been doing) is eventually going to result in your account being shut down, or your bandwidth being exhausted.

There might be other cloud storage providers that can offer storage buckets at a lower price that might not be as reliable or have as high of throughput that offers their services for less. Your project is not one that is critical to always have access to all your data at a moment's notice, so this may be okay. It is possible you can find one that will be willing to accept crypto for their services.

If you use a storage bucket on AWS or GCS, someone will need to pay $0.09/gb to transfer the file to the internet, or $0.01 to transfer your file to another data center run by the same cloud provider located on the same continent where your data is housed (or $0.00/gb -- free - within the same datacenter location). You can configure the storage bucket so that the person downloading the file will need to pay the transfer charges.

We discussed this already. But I just got a very nice Xeon-powered dedicated server (no more VPS!) from an anonymous donation, so I'm covered for now.

I'm curious: what would it cost to store a TB in a storage bucket?

It is ~never appropriate to store that much data on a VPS. You are much better off storing these files in a storage bucket. If you try any other solution, you will either quickly hit your transfer limitations, or your files will eventually get taken down because you are taking up too many resources.

Hosing your files in a storage bucket will mean that others can ~instantly download your files (limited only by their own bandwidth and computer equipment). Allowing people to download your files from the internet will be expensive, however, this can be addressed by configuring your bucket such that the requestor (the person downloading your file) will pay for the egress bandwidth.

All major cloud providers offer storage buckets. Many smaller cloud providers do as well.

Big update

First: the Torrents won't last. I got an anonymous sponsor for a dedicated server. The Seedbox expires on April 6, so the Torrens won't work much longer:

The data is back at it's original location:
This server is has a 50 TB/month bandwidth limit. So far, at most a few people per month downloaded this (crazy amount of) data, so it should be sufficient.

Note: Some files are missing, those are still being updated now. By tomorrow automated daily updates should be on track again.

For projects like this topic I don't really need AWS-level uptime, so I gladly go for the budget hosts. I run this project at Racknerd (good deals via Lowendtalk.com), and (so far) I'm quite happy with it. If I look at prices on their own site, it's much higher.
You could also go for RamNode, pay by the hour and less cheap, but from what I've seen RamNode has very solid performance too.
I don't think any of those are run from their garage I run this project at Gullo's Hosting (again: see Lowendtalk.com for deals), which has the unique feature that it's run by one guy. Servers are international, so not from his garage, but you'll always deal with the same guy. He'll try his best, which for this project is everything I need.

I also found out that several of the higher range VPS providers don't accept Bitcoin, or demand a copy of my passport (yeah, right!) first.

That's not a VPS, so won't help me much.

;-) Your point.
The same applies to the privacy policy and the washing machine manual.


Ops. I did not use it, only transactions dump. It is up to you if you want to use it, even partially.

Yesterday I was looking for some hosting services for BTC and honestly speaking I haven't found anything I would trust. Once I tried one hosting just to compare with my main one, but service and reliability was terrible. I had impression all that companies which offer servers for BTC are running their "datacenters" somewhere in the garage.
Have you seen https://www.sync.com/pricing-individual/ ?

See:
(nobody ever reads the OP)

Memory usage to get balances: ~18GB. That would take a strong VPS.

Wait, I am lost. How do you currently extract the data you publish?
For operations on local blocks I have used that parser: https://github.com/gcarq/rusty-blockparser
Configuration was really easy, processing of course takes some time.

People still use FTP these days? I remember using FTP with FileZilla long time ago.


I would demand partial/full refund.

I've updated all links to the new host. Torrents should work again too:

---

---

Apart from the fact that I wouldn't know how to do this, I don't really want to add more load to my local PC.

I used wget instead, this should preserve file dates.

It can't be: the average user doesn't use vi.

Is that true that if you tell average user to leave vi, you will get a natural RNG or at least a good example of password (compliant with any strong password policy)?

My VPS doesn't have enough storage for "inputs" torrent, so i didn't try until download is finished. But it starts almost immediately with similar speed on previous speed.

Ugh ... that leaves me with no other choice than use touch.



(Actually I'd prefer to do that .. can't do rsync straight to that appliance and I would rather avoid setting up a buffer VM just for rsync)

---