[CHALLENGE] Run A Bitcoin Node: 14 Days To 14 Merits - page 139.

Ultegra134

hero member

Activity: 1750

Merit: 904

Day 6:

Quote

{
"chain": "main",
"blocks": 616620,
"headers": 825030,
"bestblockhash": "00000000000000000001d432b156f481a94336c1a6cbd41a47b583369e1306e8",
"difficulty": 15466098935554.65,
"time": 1581235581,
"mediantime": 1581233155,
"verificationprogress": 0.5511284619745094,
"initialblockdownload": true,
"chainwork": "00000000000000000000000000000000000000000c84fe30b14d2ef7720e174f",
"size_on_disk": 34091967163,
"pruned": true,
"pruneheight": 587181,
"automatic_pruning": true,
"prune_target_size": 34999369728,
"warnings": ""
}

Turned off the computer yesterday, minor progress to 55%.

spiker777

sr. member

Activity: 1007

Merit: 279

Payment Gateway Allows Recurring Payments

Day 2 Update

Code:

{
"chain": "main",
"blocks": 269537,
"headers": 825029,
"bestblockhash": "00000000000000033fa77127650d4853a0af8af8b4b4051373a5c076304556aa",
"difficulty": 510929738.0161518,
"time": 1384414352,
"mediantime": 1384410333,
"verificationprogress": 0.02845592873068672,
"initialblockdownload": true,
"chainwork": "0000000000000000000000000000000000000000000003a27a14cdea5884ebb6",
"size_on_disk": 14311329974,
"pruned": true,
"pruneheight": 0,
"automatic_pruning": true,
"prune_target_size": 299999690752,
"warnings": ""
}

SilverCryptoBullet

member

Activity: 97

Merit: 43

My prune node today. 44.7%.

Code:

{
  "chain": "main",
  "blocks": 580794,
  "headers": 824987,
  "bestblockhash": "000000000000000000181726006a4ff8bf4db7cfad46aa5e0b8daa3b46759b86",
  "difficulty": 7409399249090.253,
  "time": 1560570497,
  "mediantime": 1560567947,
  "verificationprogress": 0.4469682672667246,
  "initialblockdownload": true,
  "chainwork": "000000000000000000000000000000000000000006afcf56ae8a47c3cbb86143",
  "size_on_disk": 4850407692,
  "pruned": true,
  "pruneheight": 577447,
  "automatic_pruning": true,
  "prune_target_size": 4999610368,
  "warnings": ""
}

Quote from: NotATether on January 09, 2024, 03:20:36 AM

I was busy yesterday and forgot to post the progress. Apologies!

Can I join, please

I actually run the node 7 days.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: LoyceV on January 06, 2024, 01:09:33 PM

My setup:
Bitcoin Core 26.0 on Ubuntu Linux 20 LTS, with settings:

Code:

./bitcoind -dbcache=4096 -prune=550

Every hour I store the output of:

Code:

./bitcoin-cli getblockchaininfo
free # for memory usage
ifconfig # for bandwidth usage

My system:
Intel i3 laptop with 8 GB RAM, no swap, SSD, VPN and about 70 Mbit/s fibre (shared with the rest of the house).

Results!
Hourly snapshots:
Memory usage (in kB): the column under "used" is actual RAM in use by running programs, the amount under "buff/cache" is file cache. So my 8 GB ram has 1.4 GB in use, and 6 GB as cache. I'm not sure how Bitcoin Core's dbcache is counted here, it looks like it's part of "buff/cache":

Code:

total used free shared buff/cache available
0: Mem: 7896464 886744 3540780 71976 3468940 6631040
1: Mem: 7896464 1330572 1431872 67472 5134020 6191636
2: Mem: 7896464 1430260 370860 56728 6095344 6102660
3: Mem: 7896464 1415924 212176 55604 6268364 6118096
4: Mem: 7896464 1406264 265548 55612 6224652 6127556
5: Mem: 7896464 1464192 604184 56784 5828088 6068652
6: Mem: 7896464 1438192 466904 56788 5991368 6094572
7: Mem: 7896464 1453828 536400 56812 5906236 6078956
8: Mem: 7896464 1471348 475660 57364 5949456 6061140
9: Mem: 7896464 1450140 496528 57364 5949796 6082164
10: Mem: 7896464 1462976 282460 56852 6151028 6070016
11: Mem: 7896464 1433148 383072 56812 6080244 6099500
12: Mem: 7896464 1345788 679108 56768 5871568 6186660
13: Mem: 7896464 1362892 570220 58164 5963352 6168708
14: Mem: 7896464 1470504 689756 59260 5736204 6060004
15: Mem: 7896464 1466812 736848 58212 5692804 6064640
16: Mem: 7896464 1468596 667628 59388 5760240 6061412
17: Mem: 7896464 1459996 600740 59456 5835728 6070040
18: Mem: 7896464 1471352 564384 59272 5860728 6058980
19: Mem: 7896464 1453480 635228 59480 5807756 6076660
20: Mem: 7896464 1462944 319308 58824 6114212 6067856
21: Mem: 7896464 1465848 379816 59596 6050800 6063936
22: Mem: 7896464 1476788 228816 60656 6190860 6051860
23: Mem: 7896464 1379152 326068 60616 6191244 6149720
24: Mem: 7896464 1435832 263568 59068 6197064 6094576
25: Mem: 7896464 1477692 544540 58100 5874232 6053500
26: Mem: 7896464 1449552 512540 58512 5934372 6081168
27: Mem: 7896464 1465168 710916 59616 5720380 6064528
28: Mem: 7896464 1500760 139072 57912 6256632 6030720
29: Mem: 7896464 1458464 299044 59600 6138956 6071308
30: Mem: 7896464 1516848 491032 57772 5888584 6014708
31: Mem: 7896464 1515076 189824 59604 6191564 6015588
32: Mem: 7896464 1510132 137576 57704 6248756 6023336
33: Mem: 7896464 1514920 203540 57264 6178004 6020008
34: Mem: 7896464 1498772 125608 59504 6272084 6037380
35: Mem: 7896464 1419124 135508 59432 6341832 6118808
36: Mem: 7896464 1489180 159092 59456 6248192 6049372
37: Mem: 7896464 1495980 188368 59560 6212116 6042884
38: Mem: 7896464 1487280 487328 58468 5921856 6049808
39: Mem: 7896464 1468228 149468 57704 6278768 6066864
40: Mem: 7896464 1473212 131732 58660 6291520 6061800

Code:

0: "verificationprogress": 1.054359111951477e-09,
1: "verificationprogress": 0.04891353970646389,
2: "verificationprogress": 0.09407763328821177,
3: "verificationprogress": 0.1452094064674858,
4: "verificationprogress": 0.1999485637578325,
5: "verificationprogress": 0.2502986361236066,
6: "verificationprogress": 0.2948632578813903,
7: "verificationprogress": 0.3253893081185931,
8: "verificationprogress": 0.3667325462341976,
9: "verificationprogress": 0.4181769295434056,
10: "verificationprogress": 0.4684429325547408,
11: "verificationprogress": 0.5167469289823825,
12: "verificationprogress": 0.5560739987615844,
13: "verificationprogress": 0.5856895281560425,
14: "verificationprogress": 0.6149000806640802,
15: "verificationprogress": 0.6362808879763396,
16: "verificationprogress": 0.6572300384174818,
17: "verificationprogress": 0.6743030443020936,
18: "verificationprogress": 0.6898836424965333,
19: "verificationprogress": 0.707189535621089,
20: "verificationprogress": 0.7257414464822198,
21: "verificationprogress": 0.7432379893745559,
22: "verificationprogress": 0.7591546335013464,
23: "verificationprogress": 0.7755112799090741,
24: "verificationprogress": 0.7921896358067945,
25: "verificationprogress": 0.8087255859251153,
26: "verificationprogress": 0.8238380369120022,
27: "verificationprogress": 0.8390669815384693,
28: "verificationprogress": 0.8552901640231105,
29: "verificationprogress": 0.8702372187018386,
30: "verificationprogress": 0.8939735382422083,
31: "verificationprogress": 0.9077083890464666,
32: "verificationprogress": 0.9246245153988459,
33: "verificationprogress": 0.9385434003500938,
34: "verificationprogress": 0.9489972191119592,
35: "verificationprogress": 0.9542581918963842,
36: "verificationprogress": 0.9589770006924087,
37: "verificationprogress": 0.9668372904078799,
38: "verificationprogress": 0.9784829067702089,
39: "verificationprogress": 0.9874556287852982,
40: "verificationprogress": 0.997143865345627,

Converted to progress per hour:
Image loading...

The dip after 7 hours may have been caused by internet congestion.
During the first hours, the sync speed was limited by my internet speed. After 13-15 hours, it's clear that bandwidth is no longer the limiting factor. I'm not sure what caused the dip in progress at 35-36 hours: if I have to guess, blocks were mostly filled with Ordinals at that time.
During the last hours, sync speed was limited by my SSD speed (and the lack of RAM).

It would be interesting to compare sync times between Windows and Linux, on the exact same hardware. Unfortunately, I don't have Windows to test this.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

I was busy yesterday and forgot to post the progress. Apologies!

Progress:

Code:

Day 1:
spiker777

Day 2:
apogio
oktana

Day 5:
tech30338
Ultegra134
DYING_S0UL

tech30338

full member

Activity: 728

Merit: 151

Defend Bitcoin and its PoW: bitcoincleanup.com

Day 5

Code:

{
  "chain": "main",
  "blocks": 824969,
  "headers": 824969,
  "bestblockhash": "000000000000000000037c59e3d40eddc1d97d53494f5fb735abf3cacab4a165",
  "difficulty": 73197634206448.34,
  "time": 1704785372,
  "mediantime": 1704783139,
  "verificationprogress": 0.9999988824061389,
  "initialblockdownload": false,
  "chainwork": "000000000000000000000000000000000000000064be9d7a17228c188f70ff1a",
  "size_on_disk": 4902415622,
  "pruned": true,
  "pruneheight": 822506,
  "automatic_pruning": true,
  "prune_target_size": 4999610368,
  "warnings": ""
}

apogio

hero member

Activity: 560

Merit: 1060

DAY 2:

Quote

{
"chain": "main",
"blocks": 824966,
"headers": 824966,
"bestblockhash": "000000000000000000004f3a85a15a4a66d8d25c719e2eb6fcbc282365b38f6e",
"difficulty": 73197634206448.34,
"time": 1704783575,
"mediantime": 1704782509,
"verificationprogress": 0.9999969936535228,
"initialblockdownload": false,
"chainwork": "000000000000000000000000000000000000000064bdd5c15a751e9a19088eef",
"size_on_disk": 612971397864,
"pruned": false,
"warnings": ""
}

DYING_S0UL

sr. member

Activity: 322

Merit: 318

The Alliance Of Bitcointalk Translators - ENG>BAN

DAY 5 progress:

Ultegra134

hero member

Activity: 1750

Merit: 904

Day 5:

Quote

{
"chain": "main",
"blocks": 592868,
"headers": 824904,
"bestblockhash": "0000000000000000000ec5096aadf8f69d16109bc0bf7df8d388fb79ee3715a3",
"difficulty": 10771996663680.4,
"time": 1567407779,
"mediantime": 1567404528,
"verificationprogress": 0.4960159278174696,
"initialblockdownload": true,
"chainwork": "000000000000000000000000000000000000000008310a5325954201c052c0c7",
"size_on_disk": 32080535585,
"pruned": true,
"pruneheight": 568134,
"automatic_pruning": true,
"prune_target_size": 34999369728,
"warnings": ""
}

Half way through, 4 years and 18 weeks behind. We're getting closer.

spiker777

sr. member

Activity: 1007

Merit: 279

Payment Gateway Allows Recurring Payments

Great Initiative @NotATether I always wanted to try Bitcoin Core but heard that it'll consume a lot of Data so I was kinda of confused. but now to take part in your challenge I am going to have a Experience Bitcoin Core Too And hopefully learn something new as well.

Day 1 For Me:

It is my first time with Bitcoin Core in fact with any nodes. so Please guide me if I am doing it wrong.

o_e_l_e_o

legendary

Activity: 2268

Merit: 18748

Quote from: LoyceV on January 08, 2024, 06:33:52 AM

Quote from: NotATether on January 08, 2024, 06:21:19 AM

The workload is equivalent to downloading a 600GB file in your browser and scanning it for viruses.

It's a lot more than that. My (laptop) sync is almost done, and my disk %util is now consistently around 96%. It has written 4.58 TB to disk in 42 hours.
When it's done, I'll create a progress graph.

Loyce is correct. The blocks folder will be ~550 GB. The chainstate, however, is what is constantly being read and written to.

The total size of the chainstate will only be ~9 GB once all is said and done, but it is updated after every block to remove all spent UTXOs and update all newly created UTXOs. Ideally this would be stored entirely within your RAM and only flushed to disk when you shutdown your node, but few people are running Core with a big enough dbcache, and so the chainstate file on your SSD is constantly being updated, especially during the IBD. Once you are fully synced and running, then Core will cache as much of the chainstate as possible in your RAM (up to your dbcache limit), and will optimize what it can cache to RAM (i.e. will preferentially cache newer UTXOs which are more likely to be spent rather than ones which have been dormant for years, specifically to minimize how much needs to be written to the chainstate file).

The more RAM you allocate, the less your chainstate will be written to. The less often you shutdown your node, the less your chainstate will be written to.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

Quote from: apogio on January 08, 2024, 06:53:24 AM

Quote from: NotATether on January 08, 2024, 06:31:16 AM

It's specified in megabytes and it can be a maximum of 16GB.

Actually it's specified in mebibytes, according to this from bitcoin's github:

Quote

-dbcache= - the UTXO database cache size, this defaults to 450. The unit is MiB (1024).
The minimum value for -dbcache is 4.

It doesn't really make a big difference, I just mention it for the sake of accuracy.

Well, yeah. Megabytes is technically 1000 Kilobytes, but the only people who use that measurement are disk & memory manufacturers, everyone else considers them a factor of 1024 although technically it is not.

apogio

hero member

Activity: 560

Merit: 1060

Quote from: NotATether on January 08, 2024, 06:31:16 AM

It's specified in megabytes and it can be a maximum of 16GB.

Actually it's specified in mebibytes, according to this from bitcoin's github:

Quote

-dbcache= - the UTXO database cache size, this defaults to 450. The unit is MiB (1024).
The minimum value for -dbcache is 4.

It doesn't really make a big difference, I just mention it for the sake of accuracy.

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: NotATether on January 08, 2024, 06:21:19 AM

The workload is equivalent to downloading a 600GB file in your browser and scanning it for viruses.

It's a lot more than that. My (laptop) sync is almost done, and my disk %util is now consistently around 96%. It has written 4.58 TB to disk in 42 hours.
When it's done, I'll create a progress graph.

Quote

the process of syncing your Core Node one time doesn't affect your disk life much if at all.

For a modern disk, that's correct. My 2 TB Samsung has 1200 TBW.
One of my old SSDs can only write 36 TB. It won't instantly kill it, but it takes a significant chunk out of it's life.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

Quote from: SilverCryptoBullet on January 08, 2024, 06:22:44 AM

With this speed, it is quick enough but I want to learn

How to set dbcache?
How to know how the sync progress kills % of my disk life?

Use the -dbcache setting in the command line or add dbcache=the value to your bitcoin.conf file. It's specified in megabytes and it can be a maximum of 16GB. Although most people should use 2 or 4GB of dbcache. There is little benefit to make it higher.

Are you using an HDD or SSD?

HDDs are designed to handle petabytes or at the very least, many, many terabytes of read/write IO total bytes. SSDs a bit less than that, but regardless, the process of syncing your Core Node one time doesn't affect your disk life much if at all. Having said that, if your disk is failing SMART tests, do not use any disk-intensive apps like running a Node, and get it replaced.

SilverCryptoBullet

member

Activity: 97

Merit: 43

Quote from: NotATether on January 08, 2024, 06:21:19 AM

Yes, everyone can join. No, it will not kill your disk. The workload is equivalent to downloading a 600GB file in your browser and scanning it for viruses. Will that break your disk? No. No matter whether it is an HDD or SSD.

With this speed, it is quick enough but I want to learn

How to set dbcache?
How to know how the sync progress kills % of my disk life?

I have Core i7, 8GB RAM and think I won't to do anything with dbcache, want to know only.

NotATether

legendary

Activity: 1568

Merit: 6660

bitcoincleanup.com / bitmixlist.org

Yes, everyone can join. No, it will not kill your disk. The workload is equivalent to downloading a 600GB file in your browser and scanning it for viruses. Will that break your disk? No. No matter whether it is an HDD or SSD.

SilverCryptoBullet

member

Activity: 97

Merit: 43

Can I join it?

I actually run a prune node since 3 Jan but on my desktop because I have my laptop to repair something for a few days. The syncing on desktop is very low.

Today, I copied the prune node data to my laptop, and it syncs very quickly.

Code:

{
"chain": "main",
"blocks": 420024,
"headers": 824850,
"bestblockhash": "00000000000000000165e59d1ba94b2b386a4d80aa318077941ac020c979e432",
"difficulty": 213398925331.3239,
"time": 1468095911,
"mediantime": 1468095039,
"verificationprogress": 0.1489870150749924,
"initialblockdownload": true,
"chainwork": "0000000000000000000000000000000000000000001f2d93b0bc10f3ee0455d7",
"size_on_disk": 3332447503,
"pruned": true,
"pruneheight": 416588,
"automatic_pruning": true,
"prune_target_size": 4999610368,
"warnings": ""
}

Reading LoyceV's posts, I am worrying that it will kill my laptop and its disk. Should I continue?

apogio

hero member

Activity: 560

Merit: 1060

Great initiative NotATether.

I am obviously running nodes since I started my bitcoin journey, but since I have less than 1000 merits, I will participate with one of my nodes.

DAY 1:

Code:

{
  "chain": "main",
  "blocks": 824839,
  "headers": 824839,
  "bestblockhash": "00000000000000000000e205f4449a5ddbbe5aa29964d00b05c6b88e4ad3757c",
  "difficulty": 73197634206448.34,
  "time": 1704704580,
  "mediantime": 1704699308,
  "verificationprogress": 0.9999952006027929,
  "initialblockdownload": false,
  "chainwork": "0000000000000000000000000000000000000000649ccedf7c72035b2f18ad28",
  "size_on_disk": 612733635152,
  "pruned": false,
  "warnings": ""
}

LoyceV

legendary

Activity: 3290

Merit: 16489

Thick-Skinned Gang Leader and Golden Feather 2021

Quote from: LoyceV on January 07, 2024, 01:41:35 PM

Another experiment:

Code:

./bitcoind -dbcache=4096 -prune=550 -datadir=/dev/shm/bitcoin

I'll keep hourly status-snapshots again.

I'm running this on a 6-core Xeon (E-2236) with 32 GB RAM and 1 GBit internet (and HDD, but I won't be using that).

This took 11 hours and 10 minutes to sync

Unfortunately, I didn't store data from getblockchaininfo. Somehow bitcoin-cli couldn't connect to the daemon now. So I can't tell how the progress estimate changed over time.

Topic: [CHALLENGE] Run A Bitcoin Node: 14 Days To 14 Merits - page 139. (Read 32901 times)