Stupidly slow bitcoin core syncing | Bitcointalksearch.org

shorena

copper member

Activity: 1498

Merit: 1528

No I dont escrow anymore.

Quote from: grue on March 04, 2016, 09:28:14 PM

Are you using windows? If so, while you're syncing, can you open resource monitor and post a screenshot of the overview tab?

I just remembered that it might also be interesting to see whether you are still downloading (can be seen via Help -> Debug Window -> Network Traffic).

grue

legendary

Activity: 2058

Merit: 1452

Are you using windows? If so, while you're syncing, can you open resource monitor and post a screenshot of the overview tab?

shorena

copper member

Activity: 1498

Merit: 1528

No I dont escrow anymore.

Quote from: ethought on March 03, 2016, 07:56:15 AM

Running 0.12 on an Amazon instance and it is taking days to sync.

I noticed my daemon is only showing 8 connections.

The last few thousand blocks are going at a snails pace.

8 connections is pretty normal during sync. If 0.12 is slow its very likely the CPU. I managed to fully sync 0.12 in ~8.5 hours on 2 virtual i5 cores.

ethought

legendary

Activity: 1316

Merit: 1000

Running 0.12 on an Amazon instance and it is taking days to sync.

I noticed my daemon is only showing 8 connections.

The last few thousand blocks are going at a snails pace.

cr1776

legendary

Activity: 4228

Merit: 1313

You could have a block that is corrupted on disk which might mean re-downloading the block chain.

Quote from: zvs on January 05, 2016, 07:26:30 AM

Mine always crashes around the same block.

Then starts over from block 1, 'reading the blockchain', with 1 core. Which takes forever, and then gets an error and aborts again after 10 hours or so.

like this

Code:

2016-01-05 11:53:27.625338 LoadBlockIndexDB: last block file = 410
2016-01-05 11:53:27.629531 LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=94, size=60445069, heights=391569...391856, time=2016-01-03...2016-01-05)
2016-01-05 11:53:27.629589 Checking all blk files are present...
2016-01-05 11:53:27.700450 LoadBlockIndexDB: transaction index enabled
2016-01-05 11:53:27.700552 LoadBlockIndexDB: hashBestChain=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f height=0 date=2009-01-03 18:15:05 progress=0.000000
2016-01-05 11:53:27.701030 init message: Verifying blocks...
2016-01-05 11:53:27.701201 block index 16563ms
2016-01-05 11:53:27.701415 No wallet support compiled in!
2016-01-05 11:53:27.701433 init message: Activating best chain...
2016-01-05 11:53:27.764422 - Load block from disk: 0.36ms [0.00s]
2016-01-05 11:53:27.764506 - Sanity checks: 0.03ms [0.00s]
2016-01-05 11:53:27.764534 - Fork checks: 0.03ms [0.00s]
2016-01-05 11:53:27.764586 - Connect 1 transactions: 0.05ms (0.050ms/tx, 0.000ms/txin) [0.00s]
2016-01-05 11:53:27.764602 - Verify 0 txins: 0.07ms (0.000ms/txin) [0.00s]
2016-01-05 11:53:27.764653 - Index writing: 0.05ms [0.00s]
2016-01-05 11:53:27.764678 - Callbacks: 0.03ms [0.00s]
2016-01-05 11:53:27.764695 - Connect total: 0.30ms [0.00s]
2016-01-05 11:53:27.764710 - Flush: 0.01ms [0.00s]
2016-01-05 11:53:27.764725 - Writing chainstate: 0.01ms [0.00s]
2016-01-05 11:53:27.764782 UpdateTip: new best=00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048 height=1 log2_work=33.000022 tx=2 date=2009-01-09 02:54:25 progress=0.000000 cache=
0.0MiB(1tx)

It verifies the blocks starting from 1, with a single core. Attempting a 'bitcoin-cli stop' says that it's currently reading block index, lol.

I had more success w/ the XT client, but I don't think they're really much different at all?

zvs

legendary

Activity: 1680

Merit: 1000

https://web.archive.org/web/*/nogleg.com

Mine always crashes around the same block.

Then starts over from block 1, 'reading the blockchain', with 1 core. Which takes forever, and then gets an error and aborts again after 10 hours or so.

like this

Code:

2016-01-05 11:53:27.625338 LoadBlockIndexDB: last block file = 410
2016-01-05 11:53:27.629531 LoadBlockIndexDB: last block file info: CBlockFileInfo(blocks=94, size=60445069, heights=391569...391856, time=2016-01-03...2016-01-05)
2016-01-05 11:53:27.629589 Checking all blk files are present...
2016-01-05 11:53:27.700450 LoadBlockIndexDB: transaction index enabled
2016-01-05 11:53:27.700552 LoadBlockIndexDB: hashBestChain=000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f height=0 date=2009-01-03 18:15:05 progress=0.000000
2016-01-05 11:53:27.701030 init message: Verifying blocks...
2016-01-05 11:53:27.701201 block index 16563ms
2016-01-05 11:53:27.701415 No wallet support compiled in!
2016-01-05 11:53:27.701433 init message: Activating best chain...
2016-01-05 11:53:27.764422 - Load block from disk: 0.36ms [0.00s]
2016-01-05 11:53:27.764506 - Sanity checks: 0.03ms [0.00s]
2016-01-05 11:53:27.764534 - Fork checks: 0.03ms [0.00s]
2016-01-05 11:53:27.764586 - Connect 1 transactions: 0.05ms (0.050ms/tx, 0.000ms/txin) [0.00s]
2016-01-05 11:53:27.764602 - Verify 0 txins: 0.07ms (0.000ms/txin) [0.00s]
2016-01-05 11:53:27.764653 - Index writing: 0.05ms [0.00s]
2016-01-05 11:53:27.764678 - Callbacks: 0.03ms [0.00s]
2016-01-05 11:53:27.764695 - Connect total: 0.30ms [0.00s]
2016-01-05 11:53:27.764710 - Flush: 0.01ms [0.00s]
2016-01-05 11:53:27.764725 - Writing chainstate: 0.01ms [0.00s]
2016-01-05 11:53:27.764782 UpdateTip: new best=00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048 height=1 log2_work=33.000022 tx=2 date=2009-01-09 02:54:25 progress=0.000000 cache=
0.0MiB(1tx)

It verifies the blocks starting from 1, with a single core. Attempting a 'bitcoin-cli stop' says that it's currently reading block index, lol.

I had more success w/ the XT client, but I don't think they're really much different at all?

LFC_Bitcoin

legendary

Activity: 3556

Merit: 9709

#1 VIP Crypto Casino

Quote from: Prelude on January 01, 2016, 12:03:36 PM

The issue is not with my computer. I started noticing this with the release of 0.10 or so. 0.11.2 still has the same issue for me. Being 24 hours behind can take half an hour to sync! Completely unacceptable. The wallet is installed on a Samsung SSD in a 5960X rig with a gigabit fiber connection, so don't tell me my computer is at fault. Seems I'm not the only one with this issue, but haven't been able to find a solution. Will I have to stop using the core wallet in favor of something else?

I agree that the current speed to sync is unacceptable but what can we do other than change to a crappy web wallet or lightweight client?

Not much.

It's just the way it is, as others have said we should see several improvements in the next release.

One of the older releases used to take me about 5 minutes to even open the wallet, I much prefer being able to see what % I'm on before the wallet & addresses load.

You still have the ridiculous sync wait then though Sad

shorena

copper member

Activity: 1498

Merit: 1528

No I dont escrow anymore.

I was away for a few days so perfect time to clock sync.

Was behind ~40 hours when I started. It was clearly working out of order, clearing several hours at once after some time. CPU is i5 4. Gen and stayed at 80% on all 4 cores. Disk is a cheap Crucial SSD.

Code:

2 min. - 105 MB - 3 nodes - 26h
3 min. - 150 MB - 4 nodes - 19h
4 min. - 153 MB - 4 nodes - done
now* - 206 MB - 23 nodes
* few hours later

Net monitor:

Jet Cash

legendary

Activity: 2814

Merit: 2472

https://JetCash.com

We need more full nodes, and you won't get them unless they get some reward for helping other nodes to sync.

unamis76

legendary

Activity: 1512

Merit: 1012

Quote from: shorena on January 01, 2016, 12:35:21 PM

What does the core build in bandwith monitor show? Does it show a large spike of downloading data at the beginning? This is usually the case for me, once the data is downloaded it takes a bit (not 30 minutes though) until its shown as fully synced. I have at max 32MBit/s downstream, but I never saturate that with core anyway.

If that is indeed the case, the reason would be disk or CPU.

If the data trickles in instead, its probably your connection to other nodes.

This is also the case for me, bandwidth monitor looks the same. ~ 24 hours worth of blockchain data usually takes from 30 to 45 minutes (old Core 2 Duo, 2GB Ram, 1TB WD Caviar Blue), but I've had it take an hour.

Core just takes time, it's just how it is

Not syncing at all is indeed strange, though...

CIYAM

legendary

Activity: 1890

Merit: 1086

Ian Knowles - CIYAM Lead Developer

Quote from: Parazyd on January 01, 2016, 01:04:56 PM

You can find extra-fast nodes to connect to here: https://blockchain.info/hub-nodes
Might help.

Yup - used that when I was in Aus - didn't help.

Parazyd

hero member

Activity: 812

Merit: 587

Space Lord

You can find extra-fast nodes to connect to here: https://blockchain.info/hub-nodes
Might help.

CIYAM

legendary

Activity: 1890

Merit: 1086

Ian Knowles - CIYAM Lead Developer

Quote from: shorena on January 01, 2016, 12:51:20 PM

For your case I wonder if you cant connect to one of the miner nodes. They probably dont have their IPs public, but maybe you can get one if you ask. Connecting directly to them with -connect=IP could help you catch up.

I did actually try adding nodes manually (via the console window) to help but unfortunately that didn't work either.

In general when it comes to bandwidth issues I tend to assume that because I live in China I will have troubles but because I could not sync the Bitcoin blockchain in Australia (when I had a month to do so) I do think that bandwidth can't be the only reason.

shorena

copper member

Activity: 1498

Merit: 1528

No I dont escrow anymore.

Quote from: CIYAM on January 01, 2016, 12:39:37 PM

Quote from: shorena on January 01, 2016, 12:35:21 PM

What does the core build in bandwith monitor show?

Unfortunately when I was in Australia I didn't bother to look at that - from here I typically see problems that are due to the GCF (very slow ping times, etc.).

I wish I had paid more attention to it in Australia (I was busy with some "real life" stuff at that time) but I would be very surprised if connectivity and bandwidth where the issue when I was there (am pretty sure I ran Bitcoin with and without a SOCKS proxy whilst I was there and my proxy is not in China).

For your case I wonder if you cant connect to one of the miner nodes. They probably dont have their IPs public, but maybe you can get one if you ask. Connecting directly to them with -connect=IP could help you catch up.

CIYAM

legendary

Activity: 1890

Merit: 1086

Ian Knowles - CIYAM Lead Developer

Quote from: shorena on January 01, 2016, 12:35:21 PM

What does the core build in bandwith monitor show?

Unfortunately when I was in Australia I didn't bother to look at that - from here I typically see problems that are due to the GCF (very slow ping times, etc.).

I wish I had paid more attention to it in Australia (I was busy with some "real life" stuff at that time) but I would be very surprised if connectivity and bandwidth where the issue when I was there (am pretty sure I ran Bitcoin with and without a SOCKS proxy whilst I was there and my proxy is not in China).

shorena

copper member

Activity: 1498

Merit: 1528

No I dont escrow anymore.

What does the core build in bandwith monitor show? Does it show a large spike of downloading data at the beginning? This is usually the case for me, once the data is downloaded it takes a bit (not 30 minutes though) until its shown as fully synced. I have at max 32MBit/s downstream, but I never saturate that with core anyway.

If that is indeed the case, the reason would be disk or CPU.

If the data trickles in instead, its probably your connection to other nodes.

CIYAM

legendary

Activity: 1890

Merit: 1086

Ian Knowles - CIYAM Lead Developer

I have been unable to sync for the last six months at least (if I run Bitcoin it will never catch up).

At first I thought it was just because I live in China but when I visited Australia for a month it still never caught up (despite having a reasonable home bandwidth connection there).

It might simply be that the number of txs and the fact that I am using an average laptop (that is now at least 5 years old) which is preventing it from catching up. I understand in the next release sigs will be processed much faster so hopefully I can still run a full node after that (otherwise I am going to have to use a SPV wallet).

Parazyd

hero member

Activity: 812

Merit: 587

Space Lord

Well, there are really a few factors that affect this...
Your CPU is one of them, notice how usage goes up when you are syncing. It's also a huge amount of data.
Other thing, the nodes you're connected to probably don't have a Gbit fiber connection, so that might be a reason for slowness too.

unamis76

legendary

Activity: 1512

Merit: 1012

Bitcoin Core being "slow" isn't a defect, it's just how it is. Maybe the nodes you're connected to aren't really that fast? I don't know. Either way that's something that one cannot really improve...

Prelude

legendary

Activity: 1596

Merit: 1000

The issue is not with my computer. I started noticing this with the release of 0.10 or so. 0.11.2 still has the same issue for me. Being 24 hours behind can take half an hour to sync! Completely unacceptable. The wallet is installed on a Samsung SSD in a 5960X rig with a gigabit fiber connection, so don't tell me my computer is at fault. Seems I'm not the only one with this issue, but haven't been able to find a solution. Will I have to stop using the core wallet in favor of something else?

Topic: Stupidly slow bitcoin core syncing (Read 4150 times)