Pages:
Author

Topic: [CHALLENGE] Run A Bitcoin Node: 14 Days To 14 Merits - page 90. (Read 19213 times)

sr. member
Activity: 1582
Merit: 281
Eloncoin.org - Mars, here we come!
I'm in! I started about 4 hours ago and this is my progress. The time left is showing just 3 days but it’s not always so, sometimes it reset and goes back to an average of 12 days or so.


These are my settings. I set pruning to 20GB (I think it should be more because my external ssd drive which I am using has over 500GB of space).

legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
Which SSD do you use and what is its TBW?
It's an older 128 GB disk, with 70 TBW. It's not something I worry about now, since that disk has served me for many years and still has enough left, but I was surprised that just 1 blockchain download will now take 10-20% of it's "life". It has written another 0.3 TB since my previous post.



Another experiment:
Code:
./bitcoind -dbcache=4096 -prune=550 -datadir=/dev/shm/bitcoin
I'll keep hourly status-snapshots again.

I'm running this on a 6-core Xeon (E-2236) with 32 GB RAM and 1 GBit internet (and HDD, but I won't be using that).
I expect this one not to slow down when chainstate grows, because it won't be low on memory (and even though dbcache isn't that high, that doesn't matter when using a RAM-drive for storage).
hero member
Activity: 714
Merit: 1010
Crypto Swap Exchange
I checked something else: writing 20-50 MB/s for a few days is very destructive for an SSD! Until now, I've written 2.6 TB in less than a day, which is already more than 10% of all writes that SSD has seen in it's entire life! My current estimate for this old disk is that this is going to cost 10-20% of the SSD's TBW (TeraBytes Written, a design-life span for SSDs). This is destructive for old SSDs with little RAM!

Which SSD do you use and what is its TBW?

Now that you made me curious about the wear of my own SSD in my Umbrel node (mine is a Samsung SSD 860 Evo 1TB), I checked S.M.A.R.T. attributes:
Code:
=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 1
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
  9 Power_On_Hours          0x0032   094   094   000    Old_age   Always       -       25807
 12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       787
177 Wear_Leveling_Count     0x0013   087   087   000    Pre-fail  Always       -       184
179 Used_Rsvd_Blk_Cnt_Tot   0x0013   100   100   010    Pre-fail  Always       -       0
181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
190 Airflow_Temperature_Cel 0x0032   059   047   000    Old_age   Always       -       41
195 Hardware_ECC_Recovered  0x001a   200   200   000    Old_age   Always       -       0
199 UDMA_CRC_Error_Count    0x003e   100   100   000    Old_age   Always       -       0
235 Unknown_Attribute       0x0012   099   099   000    Old_age   Always       -       41
241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       202678239705

This SSD has endured two full intial blockchain downloads and 24/7 operation as Umbrel node for about two years now. Total Power_On_Hours indicates almost three years of use but the first year of life was not as storage device for a Bitcoin node.

Attribute ID# 177 Wear_Leveling_Count is to be read as remaining percentage of endurance. So after something over a total of 94TiBW (1TiB is 240 bytes) my SSD is at 87% of remaining endurance. I'm not worried at all now. I was a bit worried when I read LoyceV's post.

I hope this is not off-topic here.
sr. member
Activity: 322
Merit: 318
The Alliance Of Bitcointalk Translators - ENG>BAN
DAY 4 progress:

legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
I'm currently at 2020-12-31 (63.38% progress). CPU-usage is around 40%, memory-usage at 23%, chainstate is 5.0 GB and bandwidth continuously varies between 0 and 100%.
System load is 2.6. iostat -dx /dev/sda 5 shows disk %util around 60%, and iotop shows reading 1-3 MB/s, and writing 20-50 MB/s. Memory usage of bitcoind is 12.0%.
Update: with 5.6 GB chainstate, progress (and bandwidth consumption) is going down. Disk reads are increasing while CPU-consumption is decreasing.
I checked something else: writing 20-50 MB/s for a few days is very destructive for an SSD! Until now, I've written 2.6 TB in less than a day, which is already more than 10% of all writes that SSD has seen in it's entire life! My current estimate for this old disk is that this is going to cost 10-20% of the SSD's TBW (TeraBytes Written, a design-life span for SSDs). This is destructive for old SSDs with little RAM!
hero member
Activity: 714
Merit: 1010
Crypto Swap Exchange
Some months ago I had some issues with my Raspi Umbrel node which I run for testing purposes only, not a big fan of Umbrel, (1TB SATA SSD, Raspi 4B with 8GB RAM) and I decided to throw away any Umbrel app (i.e. delete Bitcoin, Electrs, Lightning node containers on it). I didn't want to setup Umbrel completely from scratch though, because I also wanted to test how fast a (and in particular my) Raspi can perform the initial blockchain download (IBD) under slightly optimized conditions.

After deleting all Umbrel containers I made sure the blockchain files were truely gone. Then I re-installed the Umbrel Bitcoin Core container, opted for ~4.5GB of dbcache size, according to debug.log Umbrel passed dbcache=4883 to bitcoind.

Quote from: debug.log
2023-06-22T18:00:42Z Bitcoin Core version v25.0.0 (release build)
...
... Cache configuration:
... * Using 2.0 MiB for block index database
... * Using 610.1 MiB for transaction index database
... * Using 533.9 MiB for basic block filter index database
... * Using 8.0 MiB for chain state database
... * Using 3729.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)

My Umbrel is configured to only communicate via Tor outside of my local network. My internet connection has stable 100MBit/s for download and 40MBit/s upload bandwidth. My Raspi isn't overclocked but it is actively cooled with a fan which keeps it under 40°C even at higher workloads.

I will list the timestamps of UpdateTip log entries at intervals of 50k blocks as processed by my Raspi Umbrel below:
Quote from: debug.log

2023-06-22T18:01:53Z UpdateTip: new best=00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048 height=1 version=0x00000001 log2_work=33.000022 tx=2 date='2009-01-09T02:54:25Z' progress=0.000000 cache=0.0MiB(1txo)
...
2023-06-22T18:12:36Z UpdateTip: new best=000000001aeae195809d120b5d66a39c83eb48792e068f8ea1fea19d84a4278a height=50000 version=0x00000001 log2_work=48.367456 tx=50780 date='2010-04-10T16:22:18Z' progress=0.000060 cache=6.7MiB(33635txo)
...
2023-06-22T18:19:45Z UpdateTip: new best=000000000003ba27aa200b1cecaad478d2b00432346c3f1f3986da1afd33e506 height=100000 version=0x00000001 log2_work=58.648173 tx=216577 date='2010-12-29T11:57:43Z' progress=0.000256 cache=17.5MiB(104639txo)
...
2023-06-22T18:29:47Z UpdateTip: new best=0000000000000a3290f20e75860d505ce0e948a1d1d846bec7e39015d242884b height=150000 version=0x00000001 log2_work=67.003783 tx=1718407 date='2011-10-20T13:44:51Z' progress=0.002032 cache=165.1MiB(1240022txo)
...
2023-06-22T18:50:39Z UpdateTip: new best=000000000000034a7dedef4a161fa058a2d67a173a90155f3a2fe6fc132e0ebf height=200000 version=0x00000002 log2_work=68.741562 tx=7316696 date='2012-09-22T10:45:59Z' progress=0.008650 cache=345.2MiB(2621907txo)
...
2023-06-22T19:35:19Z UpdateTip: new best=000000000000003887df1f29024b06fc2200b55f8af8f35453d7be294df2d214 height=250000 version=0x00000002 log2_work=71.012098 tx=21491097 date='2013-08-03T12:36:23Z' progress=0.025408 cache=991.4MiB(7315303txo)
...
2023-06-22T20:33:26Z UpdateTip: new best=000000000000000082ccf8f1557c5d40b21edabb18d2d691cfbf87118bac7254 height=300000 version=0x00000002 log2_work=78.499549 tx=38463789 date='2014-05-10T06:32:34Z' progress=0.045474 cache=1541.7MiB(11807717txo)
...
2023-06-22T21:58:11Z UpdateTip: new best=0000000000000000053cf64f0400bb38e0c4b3872c38795ddde27acb40a112bb height=350000 version=0x00000002 log2_work=82.531372 tx=63960713 date='2015-03-30T22:17:14Z' progress=0.075615 cache=2593.3MiB(19434736txo)
...
2023-06-23T05:55:40Z UpdateTip: new best=000000000000000004ec466ce4732fe6f1ed1cddc2ed4b328fff5224276e3f6f height=400000 version=0x00000004 log2_work=84.183059 tx=112477766 date='2016-02-25T16:24:44Z' progress=0.132956 cache=2921.5MiB(20675352txo)
...
2023-06-23T13:17:17Z UpdateTip: new best=0000000000000000014083723ed311a461c648068af8cef8a19dcd620c07a20b height=450000 version=0x20000000 log2_work=85.875657 tx=190757161 date='2017-01-25T22:11:29Z' progress=0.225462 cache=3542.6MiB(25890028txo)
...
2023-06-23T20:47:40Z UpdateTip: new best=00000000000000000024fb37364cbf81fd49cc2d51c09c75c35433c3a1945d04 height=500000 version=0x20000000 log2_work=87.684014 tx=283595039 date='2017-12-18T18:35:25Z' progress=0.335151 cache=1905.8MiB(12482102txo)
...
2023-06-24T03:21:22Z UpdateTip: new best=000000000000000000223b7a2298fb1c6c75fb0efc28a4c56853ff4112ec6bc9 height=550000 version=0x20000000 log2_work=90.011241 tx=356588225 date='2018-11-14T02:35:41Z' progress=0.421371 cache=3917.7MiB(28934852txo)
...
2023-06-24T12:21:01Z UpdateTip: new best=00000000000000000007316856900e76b4f7a9139cfbfba89842c8d196cd5f91 height=600000 version=0x20000000 log2_work=91.230115 tx=466297405 date='2019-10-19T00:04:21Z' progress=0.550934 cache=1862.6MiB(12118385txo)
...
2023-06-24T23:39:25Z UpdateTip: new best=0000000000000000000060e32d547b6ae2ded52aadbc6310808e4ae42b08cc6a height=650000 version=0x20000000 log2_work=92.317149 tx=571876797 date='2020-09-25T23:39:25Z' progress=0.675557 cache=2766.8MiB(19500715txo)
...
2023-06-25T12:39:54Z UpdateTip: new best=0000000000000000000590fc0f3eba193a278534220b2b37e9849e1a770ca959 height=700000 version=0x3fffe004 log2_work=93.063032 tx=669566382 date='2021-09-11T04:14:32Z' progress=0.790797 cache=2384.8MiB(16335145txo)
...
2023-06-25T23:48:00Z UpdateTip: new best=0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e height=750000 version=0x3119a000 log2_work=93.683772 tx=757720472 date='2022-08-18T18:02:52Z' progress=0.894757 cache=1966.1MiB(12875717txo)
...
2023-06-26T16:46:31Z UpdateTip: new best=00000000000000000000a2c9c72ff100692ba39a89d5b84417a8b9d0e947db39 height=796033 version=0x2fffe000 log2_work=94.265140 tx=856792482 date='2023-06-26T16:46:05Z' progress=1.000000 cache=3589.5MiB(22517925txo)

If there's interest, I can also post UpdateTip logs at some constant time elapsed intervals, maybe every 6h would be convenient?

As you can see the IBD to current tip at block height 796033 on 2023-06-26T16:46:31Z (received time, not blocktime) took little less than 95 hours in total on such "weak" hardware like a Raspi 4B with 8GB RAM and 1TB SATA SSD, UASP active for USB3-SATA-adapter.
full member
Activity: 532
Merit: 125
Defend Bitcoin and its PoW: bitcoincleanup.com
Day 3

Code:
{
  "chain": "main",
  "blocks": 824738,
  "headers": 824738,
  "bestblockhash": "0000000000000000000015b6e3b72063bc3dcefb37b60fee1ba3ab59c988b931",
  "difficulty": 73197634206448.34,
  "time": 1704629572,
  "mediantime": 1704626490,
  "verificationprogress": 0.9999967297732044,
  "initialblockdownload": false,
  "chainwork": "000000000000000000000000000000000000000064828ae956f89d0ef207422b",
  "size_on_disk": 4913725693,
  "pruned": true,
  "pruneheight": 822275,
  "automatic_pruning": true,
  "prune_target_size": 4999610368,
  "warnings": ""
}
sr. member
Activity: 1582
Merit: 281
Eloncoin.org - Mars, here we come!
My system:
Intel i3 laptop with 8 GB RAM, no swap, SSD, VPN and about 70 Mbit/s fibre (shared with the rest of the house).

I haven't tried this with 8 GB RAM since the Ordinal spam largely increased chainstate. If this doesn't lead to surprises, I expect it to be done on Monday. If it does lead to surprises, I'll try the same with 16 GB RAM.

Oh wow, I felt it would require a higher specification system. My system is actually higher than yours, it’s a gaming system from barely a few years back so I believe if you can do it with this specifications then I can too. The difference is just the internet speed. I’ll just start today anyways and just see how it goes.
hero member
Activity: 1540
Merit: 744
Day 4:
Quote

{
  "chain": "main",
  "blocks": 535612,
  "headers": 824729,
  "bestblockhash": "00000000000000000018c60c528db50835e40501abd09149e03458f47c8f89a0",
  "difficulty": 5949437371609.53,
  "time": 1533641189,
  "mediantime": 1533637263,
  "verificationprogress": 0.3665898299353761,
  "initialblockdownload": true,
  "chainwork": "000000000000000000000000000000000000000002a40999cde10e58dab526f5",
  "size_on_disk": 34202607878,
  "pruned": true,
  "pruneheight": 501138,
  "automatic_pruning": true,
  "prune_target_size": 34999369728,
  "warnings": ""
}
Almost 37%, 5 years and 21 weeks behind.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
You can join, but you won't get any merit for it obviously  Smiley
That's okay, I'm doing it because I was curious Smiley
I'm currently at 2020-12-31 (63.38% progress). CPU-usage is around 40%, memory-usage at 23%, chainstate is 5.0 GB and bandwidth continuously varies between 0 and 100%.
System load is 2.6. iostat -dx /dev/sda 5 shows disk %util around 60%, and iotop shows reading 1-3 MB/s, and writing 20-50 MB/s. Memory usage of bitcoind is 12.0%.
I'm not sure what's the current bottleneck, nothing seems to be maxed out. I think most of the idsk writes are updating chainstate. That makes it tempting to do the same on a ram drive.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
Hey guys, can I still join? Would install it on a PI or NAS.

Happy to be part of the community ☺️

Of course, this contest is open perpetually so you can join at any time.
newbie
Activity: 1
Merit: 0
Hey guys, can I still join? Would install it on a PI or NAS.

Happy to be part of the community ☺️
legendary
Activity: 2870
Merit: 7490
Crypto Swap Exchange
Quote
Contacted my isp
Try speedtest.net Smiley

FWIW i've seen few ISP has relative slow speed, but remains fast when you perform test with speedtest.net or similar website Roll Eyes. Sometimes i'd just download stuff (such as video from youtube using yt-dlp) for less biased result.
legendary
Activity: 1568
Merit: 6660
bitcoincleanup.com / bitmixlist.org
Progress:

Code:
Day 3:
tech30338
Ultegra134
DYING_S0UL

Screw it, I'm doing it Cheesy

You can join, but you won't get any merit for it obviously  Smiley
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
I'm now tempted to do a full blockchain download with hourly snapshots, to see how the remaining time estimate changes.
Screw it, I'm doing it Cheesy

My setup:
Bitcoin Core 26.0 on Ubuntu Linux 20 LTS, with settings:
Code:
./bitcoind -dbcache=4096 -prune=550

Every hour I store the output of:
Code:
./bitcoin-cli getblockchaininfo
free # for memory usage
ifconfig # for bandwidth usage

My system:
Intel i3 laptop with 8 GB RAM, no swap, SSD, VPN and about 70 Mbit/s fibre (shared with the rest of the house).

I haven't tried this with 8 GB RAM since the Ordinal spam largely increased chainstate. If this doesn't lead to surprises, I expect it to be done on Monday. If it does lead to surprises, I'll try the same with 16 GB RAM.
hero member
Activity: 1540
Merit: 744
Day 3:
Quote

{
  "chain": "main",
  "blocks": 470743,
  "headers": 824629,
  "bestblockhash": "000000000000000001866dbc6430f507c267cda701eb0aac8eaf385e83daad5a",
  "difficulty": 678760110082.9902,
  "time": 1497153878,
  "mediantime": 1497146450,
  "verificationprogress": 0.254143698617925,
  "initialblockdownload": true,
  "chainwork": "0000000000000000000000000000000000000000005f96f0b4b0e4eb705a6640",
  "size_on_disk": 33094651308,
  "pruned": true,
  "pruneheight": 439692,
  "automatic_pruning": true,
  "prune_target_size": 34999369728,
  "warnings": ""
}
Forgot to keep my computer turned on overnight, still making progress though.
sr. member
Activity: 322
Merit: 318
The Alliance Of Bitcointalk Translators - ENG>BAN
DAY 4 progress:

Note: Ignore the per hour. It's usually slow at this time (evening).




Or your disk Wink

Try speedtest.net Smiley
Disk is fine. It's normally below 50%. On rare cases, sometimes it's going go 60-70% but never 100%  Smiley

Last I checked it was something like 5.6+ Mbps  Smiley
full member
Activity: 532
Merit: 125
Defend Bitcoin and its PoW: bitcoincleanup.com
i'd say choose depends on partial amount of free/unused RAM. Although if you use Windows, i don't expect it'll have relative small amount of free/unused RAM.

My RAM usages is 45-50% on average. And the CPU would get up to 30-50% sometimes. So we can assume my internet speed sucks.  Tongue

Usually no, since it's cause by either:
1. CPU or disk bottleneck.
2. Bitcoin Core happen connect to node with slow connection speed or physically very far from where you live (which cause slower download speed).
There's a third option: your ISP may not give you the full bandwidth.

I'm now tempted to do a full blockchain download with hourly snapshots, to see how the remaining time estimate changes.

No 1 is not the case for me, like I said above. I actually did tried that turning on & off. And it works but for a short amount of time.

A moment ago it was 0.07% per hour. Restarted the bitcoin core and now it's 3.00% per hour (eventually it will go down).

Miracle miracle miracle... Wink

P.S: Contacted my isp, said they'll increase my speed but will have to wait a couple of days, as my current internet package still has some days left.
So you have upgraded your internet speed? if there are too much connections in your area it will probably slow during peak hours, the speed upgrade will be useless since your still in the same facility, specially if your box is full or loaded, but hoping that your upgrade will solve your problem.
try also to close other applications and some who runs in your background, chrome also eats alot of memory this days.
legendary
Activity: 3290
Merit: 16489
Thick-Skinned Gang Leader and Golden Feather 2021
My RAM usages is 45-50% on average. And the CPU would get up to 30-50% sometimes. So we can assume my internet speed sucks.  Tongue
Or your disk Wink

Quote
Contacted my isp
Try speedtest.net Smiley
sr. member
Activity: 322
Merit: 318
The Alliance Of Bitcointalk Translators - ENG>BAN
i'd say choose depends on partial amount of free/unused RAM. Although if you use Windows, i don't expect it'll have relative small amount of free/unused RAM.

My RAM usages is 45-50% on average. And the CPU would get up to 30-50% sometimes. So we can assume my internet speed sucks.  Tongue

Usually no, since it's cause by either:
1. CPU or disk bottleneck.
2. Bitcoin Core happen connect to node with slow connection speed or physically very far from where you live (which cause slower download speed).
There's a third option: your ISP may not give you the full bandwidth.

I'm now tempted to do a full blockchain download with hourly snapshots, to see how the remaining time estimate changes.

No 1 is not the case for me, like I said above. I actually did tried that turning on & off. And it works but for a short amount of time.

A moment ago it was 0.07% per hour. Restarted the bitcoin core and now it's 3.00% per hour (eventually it will go down).

Miracle miracle miracle... Wink

P.S: Contacted my isp, said they'll increase my speed but will have to wait a couple of days, as my current internet package still has some days left.
Pages:
Jump to: