Pages:
Author

Topic: Experimental pre-0.8 builds for testing - page 6. (Read 19841 times)

hero member
Activity: 686
Merit: 500
Bitbuy
December 11, 2012, 10:47:37 AM
#13
Do you happen to know when the way the blockchain gets downloaded gets changed? Perhaps even downloading in parallel from multiple peers?

When someone implements an improvement. I plan to have a look at this myself soon, but no promises.

Quote
Edit: And is there an ETA for the 0.8 release? I'm so looking forward to all these great changes to hit to main release Smiley

It still needs a lot of testing, and several things are missing before we can have a new stable release.

Sounds great, keep up the good work Smiley And no need to rush Smiley
legendary
Activity: 1072
Merit: 1181
December 10, 2012, 07:01:48 PM
#12
Do you happen to know when the way the blockchain gets downloaded gets changed? Perhaps even downloading in parallel from multiple peers?

When someone implements an improvement. I plan to have a look at this myself soon, but no promises.

Quote
Edit: And is there an ETA for the 0.8 release? I'm so looking forward to all these great changes to hit to main release Smiley

It still needs a lot of testing, and several things are missing before we can have a new stable release.
hero member
Activity: 686
Merit: 500
Bitbuy
December 10, 2012, 04:25:35 PM
#11
I have to admit I expected a way shorter time than that.
If you're syncing from the network even if your own connection is fast you're subject to luck of the draw on which peers you pull from. This is one reason Pieter was suggesting loadblock runs above.


Do you happen to know when the way the blockchain gets downloaded gets changed? Perhaps even downloading in parallel from multiple peers?

Edit: And is there an ETA for the 0.8 release? I'm so looking forward to all these great changes to hit to main release Smiley
staff
Activity: 4242
Merit: 8672
December 10, 2012, 02:44:44 AM
#10
I have to admit I expected a way shorter time than that.
If you're syncing from the network even if your own connection is fast you're subject to luck of the draw on which peers you pull from. This is one reason Pieter was suggesting loadblock runs above.
member
Activity: 105
Merit: 11
December 09, 2012, 11:51:41 PM
#9
I downloaded the bitcoin-0.7.1-244-g3ccb06f-win32-setup.exe, installed on my test laptop (Dual core 2GHz Intel, 2GB DDR, with XP) and, without any other extra parameters, it took ~6:40 hours to process 193395 blocks of transaction history. The cpu use was so far ~ 25%.

After this I had to restart the client (the download apparently stopped) and the cpu usage increased to ~75% (sometimes 97-99%). Later it dropped to ~65% while download speed was ~ 250 KB/s.

A new restart was needed after 9:43 hours. The download bar at the bottom of the window disappeared, hopefully this won't scare a regular user.

Finally, the block chain fully synchronized after ~13:00 hours...


I have to admit I expected a way shorter time than that.

Disclaimer: your mileage may vary.
hero member
Activity: 991
Merit: 1011
December 09, 2012, 08:12:11 PM
#8
turbo build just crashed on win 7 64. ran for quite a few hours and then came weirdo "this program requested visual studio runtime library to terminate it in an unusual way"-or something-message, which i successfully closed by reflex  Undecided

was doing normal blockchain download stuck around 93% with 30 connections.
hero member
Activity: 714
Merit: 500
December 09, 2012, 07:46:53 PM
#7
"C:\Program Files\Bitcoin\bitcoin-qt.exe" -reindex -par=4 -dbcache=500 -logtimestamps -benchmark

3h22m
@ Windows7

Were you already synced with 0.8 before attempting that reindex?  If so that was horribly slow.


Yes, before reindex, i'm running pre0.8 1025build version, synced.

Laptop:  Intel Core i3, 4GB ram
staff
Activity: 4242
Merit: 8672
December 09, 2012, 12:46:08 PM
#6
"C:\Program Files\Bitcoin\bitcoin-qt.exe" -reindex -par=4 -dbcache=500 -logtimestamps -benchmark

3h22m
@ Windows7

Were you already synced with 0.8 before attempting that reindex?  If so that was horribly slow.
legendary
Activity: 1974
Merit: 1029
December 09, 2012, 09:22:27 AM
#5
Starting from an empty ~/.bitcoin directory (having only bootstrap.dat and a bitcoin.conf with an rpc password) and running this:

Code:
nice bitcoind -logtimestamps -loadblock=dot-bitcoin/blk0001.dat -loadblock=dot-bitcoin/blk0002.dat -loadblock=dot-bitcoin/blk0003.dat -dbcache=50 -benchmark &

Took 1 hour 57 to reach height 211494 when at that moment the latest block was 211518. As soon as 211519 was found some minutes later the remaining blocks were processed/downloaded/whatever.

4 CPU cores. RAM usage reached 420 Mb RSS, after synchronization it ended up at 375 Mb. Restarting the daemon leaves it at 130 Mb. I've never used -dbcache until now, if that matters. Usual RAM usage until now was 80-90 Mb just after starting, then going up slowly as days passed (like in, 130 Mb after ~10 days).
hero member
Activity: 714
Merit: 500
December 09, 2012, 01:02:31 AM
#4
"C:\Program Files\Bitcoin\bitcoin-qt.exe" -reindex -par=4 -dbcache=500 -logtimestamps -benchmark

3h22m
@ Windows7
newbie
Activity: 56
Merit: 0
December 08, 2012, 11:43:17 PM
#3
This is v0.7.1-244-g3ccb06f-beta starting from a completely bare directory, downloading the blockchain from a peer on the same LAN to remove internet connection speed as a factor.

Quote
time bitcoin-qt -par=8 -connect=192.168.XXX.XXX

real    48m58.429s
user    114m35.786s
sys     2m33.817s

Again with a larger cache.

Quote
time bitcoin-qt -par=8 -connect=192.168.XXX.XXX -dbcache=4096

real    42m10.028s
user    109m33.141s
sys     1m9.843s

Multithreading seems to be working well!
legendary
Activity: 1400
Merit: 1013
December 08, 2012, 09:47:10 PM
#2
This is v0.7.1-244-g3ccb06f-beta starting from a completely bare directory, downloading the blockchain from a peer on the same LAN to remove internet connection speed as a factor.

Quote
time bitcoin-qt -par=8 -connect=192.168.XXX.XXX

real    48m58.429s
user    114m35.786s
sys     2m33.817s

Again with a larger cache.

Quote
time bitcoin-qt -par=8 -connect=192.168.XXX.XXX -dbcache=4096

real    42m10.028s
user    109m33.141s
sys     1m9.843s
legendary
Activity: 1072
Merit: 1181
December 08, 2012, 07:18:32 PM
#1
Hello all,

I've uploaded a new set of builds of bitcoind/Bitcoin-Qt to my builds page. The new "turbo" build is based on current master on git, plus some more experimental patches (Hal Finney's optimized secp256k1 code, and parallel script verification). The source code for these can be found here (.tgz archive here).

Disclaimer: don't use these for any serious stuff - I am not responsible for destroyed wallets or exploded CPU's.

Testing of these is very welcome.

As the most invasive change is in the block/transaction validation system, this is the most interesting part to test. I am very interested in reports about speed, CPU usage, memory usage, ... during initial block synchronization, in particular on Windows systems. Obviously, crashes or other unexpected behaviour is even more important.

As these builds use the new 0.8 database code ("ultraprune", as I called it before), they will not use the existing block files/database from 0.7.2 or earlier. There are several ways to import the data:
  • Put bootstrap.dat in your data directory.
  • Start the program with options -loadblock=path/to/old/blk0001.dat -loadblock=path/to/old/blk0002.dat.
  • If you already have a full 0.8 database (perhaps from a next-test build), you can use the new -reindex option to rebuild your index from scratch, redoing the block validation process, reusing the block data you already have on disk.
  • Obviously, you can also just let it synchronize from network, but as the current block-fetch logic is somewhat of a hack, you may get unlucky, and hit a slow peer, or have downloads that stop for long periods of time. It is advised to use -connect=IP to connect to a single fast peer (you can use my server, 178.18.90.41, for this purpose if needed) for that.

As these builds use parallel script verification (only enabled after block 193000, the last checkpoint), you may wish to control how many threads are used (don't use more than the number of CPU cores you have). By default, it will try to autodetect, but you can use the -par=N option to select the number of threads. One interesting thing you can test is comparing -par=1 with higher numbers, and check the speedup (in particular of the blocks after 193000).

Other options which may be useful to play with:
  • -dbcache=N tweaks the amount of cache used. N is a number in megabytes (default is 25), but higher numbers may result in significant speedups.
  • -logtimestamps adds timestamps to the output in the debug.log file, which may be very useful to compare speed.
  • -benchmark produces detailed timing information during block validation (not very accurate on Windows, unfortunately).
Pages:
Jump to: