Pages:
Author

Topic: [ANN] Bitcoin blockchain data torrent - page 11. (Read 211194 times)

rme
hero member
Activity: 756
Merit: 504
July 31, 2014, 05:16:49 PM
Seeding new torrent 24/7
newbie
Activity: 3
Merit: 0
July 31, 2014, 04:22:37 PM
Since I was curious about the new torrent, here's some info:

new bootstrap.dat contains up to block height 313000 (July 8 )

The new block data (this is using the hexidecimal form) matches my bitcoind's:

$ ./blockheight -l 295001 < bootstrap-313000.dat | sha256sum
6cccd54b215799d91b6781b190903fb5673cf6413d2a93c5ebe26223e3b2de21  -
$ for((i = 295001; i < 313001; i++)); do  ./bitcoin-cli getblock `./bitcoin-cli getblockhash $i` false ; done | sha256sum
6cccd54b215799d91b6781b190903fb5673cf6413d2a93c5ebe26223e3b2de21  -

Previous data matches:

$ dd if=bootstrap-313000.dat bs=335633 count=52206 2>/dev/null | sha1sum
78a4b41ba7ed823c78cd9ca81a6de393c0ccc226  -
$ sha1sum bootstrap-295000.dat
78a4b41ba7ed823c78cd9ca81a6de393c0ccc226  bootstrap-295000.dat


Can you re-add a http:// based tracker?  The UDP trackers aren't supported by older transmission (2.13 from EPEL).  The exodus.desync.com tracker in the previous torrent seems to be working well.

Edit: UDP trackers actually do support IPv6, but support varies depending on your torrent software.
legendary
Activity: 1596
Merit: 1100
July 31, 2014, 02:53:49 PM
What about other trackers? Open, less politicized,  trackers, that aren't associated with thepiratebay.se? Or maybe differently politicized trackers associated with other organizations? While those 4 trackers are reliable and well defended against DDoS, they are also the ones that are most commonly blocked/censored/filtered by various software nannies.

I don't pretend to be a tracker expert, and actively solicit suggestions.

I would certainly bias towards less politicized + more reliable trackers.

Please post your tracker suggestions.

legendary
Activity: 2128
Merit: 1073
July 31, 2014, 12:30:08 PM
Does tracker.ccc.de work for anyone anywhere in the world besides .at, .ch & .de?

Maybe it should be changed to "udp://open.demonii.com:1337" like all the newer torrents? This new tracker is accessible worldwide.

Edit: After reviewing the recent statistics of reach-ability of demonii.com I no longer recommend it.

What about other trackers? Open, less politicized,  trackers, that aren't associated with thepiratebay.se? Or maybe differently politicized trackers associated with other organizations? While those 4 trackers are reliable and well defended against DDoS, they are also the ones that are most commonly blocked/censored/filtered by various software nannies.
legendary
Activity: 1596
Merit: 1100
July 31, 2014, 09:39:36 AM
Beta-testing and pre-seeding a new torrent at http://gtf.org/garzik/bitcoin/bootstrap.dat.torrent

Please only join if you already have an old bootstrap.dat file, or you built the new one yourself using linearize.py and bitcoind.

Code:
magnet link:
magnet:?xt=urn:btih:682759b001fac5d3b0d8517d2a179af2e65ebed6&dn=bootstrap.dat&tr=udp%3A%2F%2Ftracker.openbittorrent.com%3A80&tr=udp%3A%2F%2Ftracker.publicbt.com%3A80&tr=udp%3A%2F%2Ftracker.ccc.de%3A80&tr=udp%3A%2F%2Ftracker.istole.it%3A80
hero member
Activity: 560
Merit: 509
I prefer Zakir over Muhammed when mentioning me!
July 16, 2014, 05:46:46 AM
Alright. It took 26 minutes to download.
Now starting the wallet..

It says it's importing data.... fine...

10 minutes in... it has imported about 170'000 blocks....

1 hour later ... 230k blocks.

1.5 hours later .... importing block 250k.... importing sure takes a lot of time...  Sad but we are getting there...

2 hours, 270k (that's end of november 2013)... at this rate it should take another hour to finish...

I was wrong... 3 hours, it's at 283k... I give up... I have to get some sleep... lol...

PS progress bar is at about 40%... but don't think its an accurate estimation of time needed.

But if it is, then it might take another 3-4 hours to stop.... will see tomorrow...

You can use this command for increasing syncing speed from your computer not from internet. Smiley
Code:
-dbcache=           Set database cache size in megabytes (4 to 4096, default: 100)

You can use this command to decrease the connection to number of peers, if it is affecting your internet browsing.
Code:
-maxconnections=    Maintain at most connections to peers (default: 125)
IMO it won't affect in a great way but might affect in a very low speed networks.

Kindly,
        MZ
legendary
Activity: 1484
Merit: 1007
spreadcoin.info
July 16, 2014, 01:21:26 AM
7 weeks to go.
it took 7 hours to get here.
It'll probably finish now in a 1 hour or so...

Now it switched to synchronizing with network for the remaining blocks that weren't in the torrent.
legendary
Activity: 1484
Merit: 1007
spreadcoin.info
July 15, 2014, 06:26:43 PM
Alright. It took 26 minutes to download.
Now starting the wallet..

It says it's importing data.... fine...

10 minutes in... it has imported about 170'000 blocks....

1 hour later ... 230k blocks.

1.5 hours later .... importing block 250k.... importing sure takes a lot of time...  Sad but we are getting there...

2 hours, 270k (that's end of november 2013)... at this rate it should take another hour to finish...

I was wrong... 3 hours, it's at 283k... I give up... I have to get some sleep... lol...

PS progress bar is at about 40%... but don't think its an accurate estimation of time needed.

But if it is, then it might take another 3-4 hours to stop.... will see tomorrow...
legendary
Activity: 1484
Merit: 1007
spreadcoin.info
July 15, 2014, 06:15:28 PM
I am checking this out now.

A month ago it took me about a day to download the whole blockchain with the wallet.
I will see how fast the torrent and validating is.

At the moment it's very promising, like 25 MB/s download speed.
sd
hero member
Activity: 730
Merit: 500
July 15, 2014, 04:53:54 PM
Would be too much to ask if they added like, not daily but maybe weekly updated blockchain file? Everytime someone new comes to Bitcoin, the first thing they say is how much of a pain in the ass it is to sync, unless they are using 3rd party wallets that is, but im too used to the simple niceness of the classic wallet to change  Smiley

There isn't a 'they', there is only an 'us'. If you think it should be done then do it. I was going to but my tests showed that using a bootstrap file doesn't load data into bitcoind any faster than pulling the data from the live network for blocks beyond 295000. The recent blocks seem more complicated than earlier ones. Maybe a bigger bootstrap file would still be useful to some people.

I really think that most users should be using a light client like electrum anyway. Interested companies, organizations, and technical users should be running the full nodes, not every guy who wants to make small purchases.
legendary
Activity: 1361
Merit: 1003
Don`t panic! Organize!
July 15, 2014, 01:20:27 PM
Looks like update needed Smiley
I made a linear block chain up to 309000, 7zip it and put into torrent.
It is like 12,3GB.
My home upload is ~500kb/s so it will take some time to move it to better seeds... hopefully one day tops Smiley
magnet link:
Code:
magnet:?xt=urn:btih:8347fcc3df300e5a31e2304442c128e308256dfc&dn=bitcoin309000&tr=udp%3a%2f%2ftracker.openbittorrent.com%3a80&tr=udp%3a%2f%2ftracker.publicbt.com%3a80&tr=udp%3a%2f%2ftracker.istole.it%3a80%2fannounce&tr=udp%3a%2f%2fopen.demonii.com%3a1337
Have fun Smiley

Looks like at least 10 ppl downloaded it but noone seed? :/
sr. member
Activity: 322
Merit: 250
July 15, 2014, 09:17:01 AM
Would be too much to ask if they added like, not daily but maybe weekly updated blockchain file? Everytime someone new comes to Bitcoin, the first thing they say is how much of a pain in the ass it is to sync, unless they are using 3rd party wallets that is, but im too used to the simple niceness of the classic wallet to change  Smiley
sd
hero member
Activity: 730
Merit: 500
July 15, 2014, 08:40:00 AM
One flaw in your test...

I didn't go into exact details about how I tested because discussions like this tend to never end. If there was a significant advantage or disadvantage to using the torrent I think I would have seen it.

Please do treat my results with scepticism and test yourself.
newbie
Activity: 52
Merit: 0
July 10, 2014, 04:18:50 AM
The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.

Experimentation being the basis of science I tested your theory. I used a AWS m3.xlarge with a 60GB SSD block device for the bitcoin client. I created a bootstrap.dat for block 305000 and compared the time it takes to update from block 295000 to 305000 via my bootstrap.dat and by pulling blocks from the live network. It turns out you are right.

Time to update from 295000 to 305000 with a custom bootstrap.dat: 3845 seconds
Time to update from 295000 to 305000 by connecting to the live network: 3848 seconds

As a 3 second difference on a 1 hour load is meaningless I can only conclude there is no reason to update this torrent. I'll keep throwing bandwidth at it.


One flaw in your test:  I'm pretty sure when you give the Bitcoin client a bootstrap.dat file it doesn't just "update" but instead rescans the entire blockchain database on your hard drive.  That's why the bootstrap.dat is meant to be used on a fresh install because if you already have a good chunk of the blockchain downloaded you're usually better off just letting the client update from the network.

Also not everyone has the bandwidth of an extra large AWS instance.  With my slow 3mbps/384kbps DSL line, I may be the exception in that I could possibly rebuild my local database faster than I could download the latest blocks.
sd
hero member
Activity: 730
Merit: 500
July 09, 2014, 03:36:04 AM
The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.

Experimentation being the basis of science I tested your theory. I used a AWS m3.xlarge with a 60GB SSD block device for the bitcoin client. I created a bootstrap.dat for block 305000 and compared the time it takes to update from block 295000 to 305000 via my bootstrap.dat and by pulling blocks from the live network. It turns out you are right.

Time to update from 295000 to 305000 with a custom bootstrap.dat: 3845 seconds
Time to update from 295000 to 305000 by connecting to the live network: 3848 seconds

As a 3 second difference on a 1 hour load is meaningless I can only conclude there is no reason to update this torrent. I'll keep throwing bandwidth at it.
legendary
Activity: 2618
Merit: 1007
July 07, 2014, 08:23:02 AM
The verification speed when verifying blocks past the latest checkpoint is far slower than getting the blocks from the network usually.
sd
hero member
Activity: 730
Merit: 500
July 07, 2014, 07:31:12 AM
After checkpoints there is little need to get data from a torrent file instead of the network itself, since it is likely a CPU or disk IO bound operation, not network speed.

Also with this other torrent file, there is no way for me (other than downloading the whole file and decompressing it) to recreate + verify it locally.

Network speed is an issue, that's why we are using a torrent. There are currently 14620 blocks between the end of this torrent and the end of the blockchain. Bitcoind should download about as fast as torrent does, but sadly it doesn't.

Bitcoind verifies this data when it imports it, just like when it verifies data it pulls off random network nodes. The only possible attack is a 50%+1, and if an attacker could pull that off they would do it on the main network, or more likely they would just mine half the coins instead.

EDIT: For clarity
newbie
Activity: 8
Merit: 0
July 07, 2014, 06:40:22 AM
Hey, thanks for the torrent. I downloaded the file and placed it in my btc folder. I open the client and it says its importing the block from the disk, but nothing hapens, it still says 0 out of 309616 blocks processed. It´s been like that for 1 hour. Is this normal? Thank you Smiley

EDIT: OK, its now syncronizing and the bootstrap.dat changed the name to bootstrap.dat.old, still at 0 blocks...

NEW EDIT: It doesnt sync, what should I do? Thanks.
legendary
Activity: 2618
Merit: 1007
July 06, 2014, 12:41:02 PM
After checkpoints there is little need to get data from a torrent file instead of the network itself, since it is likely a CPU or disk IO bound operation, not network speed.

Also with this other torrent file, there is no way for me (other than downloading the whole file and decompressing it) to recreate + verify it locally.
sd
hero member
Activity: 730
Merit: 500
July 06, 2014, 12:12:25 PM
I'm not going to seed some random stranger's torrent, sorry.

If you connect a full node to the bitcoin network you also download data from random strangers. As I understand it the data in this file is subjected to exactly the same checks as data downloaded by the bitcoin client itself. Should the data be misformatted it will be rejected. If some of the end blocks got reorganized the client will sort it out once it connects to the network proper.

Of course you are right to be paranoid but I don't see the need for this torrent to align with checkpoints in the code itself.
Pages:
Jump to: