Topic: Whole Bitcoin Core 0.15 blockchain database on Google Drive - page 2. (Read 2342 times)

There was someone else earlier (see my test-link) having the same problem on Windows, and he eventually moved his pruned blockchain to a RAM-drive. That solved his problem.
If you haven't checked the number of read/write-actions to your hdd, you can't rule it out from being the bottleneck.

I'm no export on Windows, but as far as I know most flash drives don't use write cache (to be safe when you suddenly unplug it).

That's not what I experienced. A very simple way to test it: when you start your download, and blocks are still small, there's much less checking per block. At that point I download at the maximum speed my internet connection can deliver. It only slows down later, when blocks require more checking.

All of the time all of my own main components (CPU, RAM, download bandwidth) were busy at 2-5% (peak loads 10-12%). My ONLY possible bottleneck could have been the hdd which was a pretty fast 7200 rpm, but still a usual hard drive (not SSD).
But! Trying to figure out why the download was so slow at a moment I copied all Bitcoin Core 0.14.2 (already downloaded) data directories into a fast USB 3.0 128 Gb flash drive.  The syncing process did become a little bit faster, but not significantly!
So, I am still sure the bottleneck can be anywhere but never inside my PC or my internet channel.

I assume it was insufficient number of other WIDE UPLOAD bandwidth, accepting incoming connections full Bitcoin Core nodes.

Of course that's the key, but my point is still valid: it all depends on your PC.
Downloading to an SSD is much faster than downloading to a hdd.

That being said, my test also showed downloading a pruned chain on ramdrive did less blocks per minute than downloading blocks to my full chain on hdd. It took 70 seconds for 11 hours of blocks, that's about 1 block per second (see my test).
I don't really want to spend the time to do a full test, but from this small test, I extrapolate I can download a full blockchain to my hdd in 2-3 days.

I have no experience on Windows though, especially a system low on RAM can take much longer.

"PRUNED TO a RAM drive" - that's the key to your success,as I understand it.

Try to download the full 160Gb (not pruned) blockchain from the start, then tell us how much time it has taken you

As far as I understand, you can just install Bitcoin Core and -rescan the database:
Assuming you download a legit Bitcoin Core, it will verify all blocks on your disk and you can be absolutely sure you have a legit blockchain.

That being said, I wouldn't use this myself, I'm totally fine letting Bitcoin Core download it's own blocks.

As a test, I recently downloaded a full blockchain (pruned to a RAM drive) in less than 24 hours, on a several years old PC with an average internet connection. Given the fact that I can do this, your bottleneck must be related to your PC or internet connection.

First of all, I created a special thread on security risks :
https://bitcointalksearch.org/topic/security-risks-in-downloading-the-whole-bitcoin-blockchain-from-external-srces-2323576

Secondly, I wrote a great number of posts in  https://bitcointalksearch.org/topic/do-run-a-bitcoin-core-full-node-now-2303869 explaining :
how long I had to be waiting (weeks) till my Bitcoin Core 0.14.2 fully synced itself at last. And the bottleneck was NOT in any way related to my PC or internet bandwidth.

I know that in 0.15 version the great Core developers team did implement some changes and improvements to accelerate the process. But I think it'll still take weeks, not hours or days.

(I should admit yet,  that in June when I was syncing my node, I couldn't be syncing 24/7.  If  (as now) I could, then the process could have actually been finished much faster)

And due to the dangerous, absolutely unnecessary after the Segwit activation, 2Mb (2x) threat to the greatest values in my own life - the True Bitcoin (most of my savings in Bitcoin) and its great Developers Team, I just wanted to help some smart and responsible people to start their own Bitcoin Core full nodes as soon as possible!

As for trust, it would be a million times safer of course if the complete archive could still be downloaded from the developer's site. I understand that the Developers team just couldn't have imagined such an 2x 'emergency situation', so for that and some other reasons they stopped offering such full blockchain downloads.
But for this November I'd suggest they themselves offered full archives similar to mine. But of course I am too much a newbie to give advices here

Just playing devil's advocate here..

That's great that you've "tested the archive"... But how can we "test" you? A "member" with 70 odd posts...

What guarantee do we have that this Blockchain is "legit"... Other than you word?

IMO, by downloading this and using it, you are potentially weakening your node, not strengthening the network.

Install Core, let it sync "naturally".

Friend, what kind of threat can you see here?
I assure you, Google Drive would never replace my files, or do other similar meanness)
So, download safely (if you haven't started your full Bitcoin Core node yet) and don't worry about anything you mentioned

I'm not sure why you use the word "inadequate" - We know what Google is doing, and the way they are demonetising videos and distorting search results. Why would anyone trust them with something that poses a threat to their future ambitions.  I agree that Google offers many useful free services, and I use them for such things as listing domain names for sale, and traffic analysis. Using them for something that is at the heart of my savings - no way.

Dear friend, one should never fall into such inadequate extremes.
Google Drive is an excellent inexpensive service, and if we take a little help from it, then it'll be extremely beneficial for everyone! So don't worry about that

I'm a convinced anti-globlist, and I believe that the dominance and control exercised by Google, Amazon, Ebay, Microsoft and their associates and sunsidiaries are counter to the interests of most surfers. Bitcoin was started to escape this, and the use of Google to distribute any Bitcoin related products seems to be extremely unwise.

OK, it seems like all parts are available now for a total of 131.62GB. Google seems to be throttling download to below 100 megabits per second, so I get the ETA of over 3 hours. I'll post again when its done, but otherwise it looks good: JDownloader recognized all 9+1+2 parts, correctly recognized their sizes and availability.

Maybe not as fast as that "axel" mentioned by the other poster, but otherwise seems flawless.

Edit: OK, I forgot about this download. It finished properly, uncompresses properly without password, there wasn't any strange files in it and the Bitcoin client started off this directory properly and resumed syncing without problem.

Also a reminder: decompression utilities restore the files in a fairly fragmented state. Defragment the uncompressed directory to improve overall performance.

Try downloading right now, seems part 11 is available again from now on

The problem is due to my experiments with opening direct links

I'm comparing all properties, everything, part 11 is all the same as part 12,but for an unknown reason it is not shown inside the folder.  I am trying to find solution, wait a little bit more

Wait for a moment, I'll  check it and fix immediately

Part 11 seems to be missing from the 3rd link.

With your Google ID, you also get Google Sites space. I haven't done it, but why not a sftp server?  Just a thought, I think you get 15gb.  check it out!

I Googled a little, found many sites (including Google itself) that claimed to produce direct download links.
It seemed to work with small files only.

On my 10Gb files those direct links did still lead to a web page that asked for confirmation to download the files despite Google could not check it for viruses.

I do not have enough free time to deal with all that stuff now, if someone knows how to get direct download links for  Large files in Google Drive, then let us know here, please. Later I would create such links for all 12 parts of the archive

OK, no problem at all, of course! If Google drive can provide those direct links, I will create them!
Just wait a little bit

> I'm not quite sure about multi thread parallel downloading from Google Drives,
> maybe they do not provide the option, maybe they do. Personally, I use a Firefox
> download Extension that allows reliable downloads. When I tried it, the
> downloading was on the max available speed all time.

There is a reason for the direct link.
The direct link may be easily emailed to anyone.
Multi-stream download utilities may be used (for example, `axel`).
The download procedure may be implemented with simple script (using
Python, Go or bash).
With direct link people do not need to use GUI at all - which is the
case with server-based approach.
You decide.
But if you want to make this backup really useful for many peoples,
think of storing the files on the server with direct links.