Do you need to manually reset the nodes every week?

theymos

administrator

Activity: 5222

Merit: 13032

Quote from: BlindMayorBitcorn on February 23, 2016, 09:43:52 PM

I've personally never run anything more complicated than an SPV wallet. When MIT came aboard I had this I guess naive idea that soon enough every research University in the world would be running full nodes and that that would be more than enough. How wrong was I? Am I a part of the problem? Are SPV nodes a drag on the system, in that they make it that much more convenient to not run a full one?

If not enough of the economy is backed by full nodes, then Bitcoin is totally insecure. See here and here.

Lightweight nodes are also a slight drain on the network's capacity, but that's a much less pressing and serious problem than the economic issue.

BlindMayorBitcorn

legendary

Activity: 1260

Merit: 1116

I've personally never run anything more complicated than an SPV wallet. When MIT came aboard I had this I guess naive idea that soon enough every research University in the world would be running full nodes and that that would be more than enough. How wrong was I? Am I a part of the problem? Are SPV nodes a drag on the system, in that they make it that much more convenient to not run a full one?

theymos

administrator

Activity: 5222

Merit: 13032

I was occasionally having problems with this on one of my nodes, but 0.12.0's mempool limiter totally fixes it.

GamerSg

newbie

Activity: 23

Merit: 0

Just set minrelaytxfee to 0.0001 and you will have a much smaller and stable mempool.

The default setting allows spam to be accepted into the mempool and sticks around forever because those transactions will never be mined.

shorena

copper member

Activity: 1498

Merit: 1528

No I dont escrow anymore.

Quote from: tspacepilot on February 21, 2016, 05:08:38 PM

Quote from: shorena on February 21, 2016, 03:43:18 AM

Not once a week, but I see similar things on my node. The TX that keep pilling up mostly pay a very low fee, so it could be avoided by setting limitfreerelay minrelaytxfee lower and high respectivly.

Sho, is that drop on the 7th of feb where you're restarting your node to clear your mempool? If that's the case then it seems like at least one long-time node runner (you) *is* resetting your node regularly to avoid the pileup.

Its actually a crash with a quick reboot afterwards. 2 GB RAM is no longer enough to let it run just like that. Its about time again, maybe I should just reboot this time. The new server will have 6GB so its a matter for finding time to set up the new one.

tspacepilot

legendary

Activity: 1456

Merit: 1081

I may write code in exchange for bitcoins.

Quote from: shorena on February 21, 2016, 03:43:18 AM

Not once a week, but I see similar things on my node. The TX that keep pilling up mostly pay a very low fee, so it could be avoided by setting limitfreerelay minrelaytxfee lower and high respectivly.

Sho, is that drop on the 7th of feb where you're restarting your node to clear your mempool? If that's the case then it seems like at least one long-time node runner (you) *is* resetting your node regularly to avoid the pileup.

shorena

copper member

Activity: 1498

Merit: 1528

No I dont escrow anymore.

Not once a week, but I see similar things on my node. The TX that keep pilling up mostly pay a very low fee, so it could be avoided by setting limitfreerelay minrelaytxfee lower and high respectivly.

Foxpup

legendary

Activity: 4535

Merit: 3188

Vile Vixen and Miss Bitcointalk 2021-2023

Quote from: johnyj on February 19, 2016, 11:25:17 PM

If a block contains 1MB data, and typically 2000 transactions, then each transaction is estimated at 0.5KB. In fact I have seen some blocks containing almost 3000 transactions, and cointape estimate average transaction to be 258 bytes right now

I always heard it was around 600 bytes. Tradeblock says it's currently 540 bytes. Blocks containing 2,000 transactions are not typical.

Quote from: johnyj on February 19, 2016, 11:25:17 PM

each of them should be 25K

No, they should average 25kB. Most of the space is taken up by a very small number of exceptionally large transactions (possibly broadcast as DOS attacks), so most transactions will be far below average.

johnyj

legendary

Activity: 1988

Merit: 1012

Beyond Imagination

Quote from: Foxpup on February 19, 2016, 10:32:55 PM

Quote from: johnyj on February 19, 2016, 08:58:19 PM

each of them is 0.5KB maximum

What are you talking about? Even average transactions are larger than that, and mempool transactions generally aren't average anyway: most of these transactions are unconfirmed for a reason, and usually that reason is their size.

If a block contains 1MB data, and typically 2000 transactions, then each transaction is estimated at 0.5KB. In fact I have seen some blocks containing almost 3000 transactions, and cointape estimate average transaction to be 258 bytes right now

I have checked the rawmempool and indeed found some very large transactions, but to match your number (40k transactions for 1GB), each of them should be 25K, which is almost 100 times larger than average transaction, all 100+ sigop transactions?

Foxpup

legendary

Activity: 4535

Merit: 3188

Vile Vixen and Miss Bitcointalk 2021-2023

Quote from: johnyj on February 19, 2016, 08:58:19 PM

each of them is 0.5KB maximum

What are you talking about? Even average transactions are larger than that, and mempool transactions generally aren't average anyway: most of these transactions are unconfirmed for a reason, and usually that reason is their size.

johnyj

legendary

Activity: 1988

Merit: 1012

Beyond Imagination

Quote from: Foxpup on February 19, 2016, 05:48:08 AM

What exactly is the problem? Linear mempool growth is normal and expected for a constant number of transactions per second, at least until old transactions stop being broadcast and an equilibrium is reached. Many have complained that said equilibrium ends up being too large, so starting from version 0.12 Bitcoin Core has a configurable limit on how large the mempool can grow, after which it will start dropping the transactions with the lowest fees.

For reference, my node's mempool usually hovers around 40k transactions in 1GB, which I think is typical.

Thanks, good to know this is normal, because I have observed this phenomenon for years. I never cared too much since it is still small comparing to the total free memory. However I'm seeing tradeblock using a different way to calculate the mempool size, it is always related to the unconfirmed transactions in mempool, not growing linearly

In your case, 40K transactions would result in 20M maximum mempool, not 1GB. Since you have 40K transactions, and each of them is 0.5KB maximum, what are all those other memories used for?

ScripterRon

full member

Activity: 136

Merit: 120

Quote from: Foxpup on February 19, 2016, 05:48:08 AM

What exactly is the problem? Linear mempool growth is normal and expected for a constant number of transactions per second, at least until old transactions stop being broadcast and an equilibrium is reached. Many have complained that said equilibrium ends up being too large, so starting from version 0.12 Bitcoin Core has a configurable limit on how large the mempool can grow, after which it will start dropping the transactions with the lowest fees.

For reference, my node's mempool usually hovers around 40k transactions in 1GB, which I think is typical.

My node has been hovering around 11,000 mempool transactions lately, although right now it is at 7100.

Foxpup

legendary

Activity: 4535

Merit: 3188

Vile Vixen and Miss Bitcointalk 2021-2023

What exactly is the problem? Linear mempool growth is normal and expected for a constant number of transactions per second, at least until old transactions stop being broadcast and an equilibrium is reached. Many have complained that said equilibrium ends up being too large, so starting from version 0.12 Bitcoin Core has a configurable limit on how large the mempool can grow, after which it will start dropping the transactions with the lowest fees.

For reference, my node's mempool usually hovers around 40k transactions in 1GB, which I think is typical.

tspacepilot

legendary

Activity: 1456

Merit: 1081

I may write code in exchange for bitcoins.

Quote from: johnyj on February 19, 2016, 03:01:02 AM

Quote from: tspacepilot on February 19, 2016, 01:12:26 AM

There may be something going on here if you say that your stats are different from what you're seeing on other people's nodes. In any case, it's definitely not regular procedure to be restarting your nodes every week. Bitcoin is reasonably stable software and you should be able to get significant uptime if everything is configured correctly. On the other hand, 80MB doesn't really seem all that large (but you're right that that linear growth is troubling), I'm guessing something's configured incorrectly here. I think it's hard to diagnose further without more details.

This is not my machine, but my machine is having similar problem and now it is over 225MB after 10 days of running. Maybe I'm just too lazy to build the bitcoind from source

I'm rebuilding bitcoin and bitcoin-qt because of another question in this board. I'll kick it off and run it for a while and tell you if I see something similar. I imagine that long before then someone else who's on here and has a node that's been up for a while can chime-in about whether 225MB is a lot. Is your memory usage still climbing on that linear pattern after 10 days?

johnyj

legendary

Activity: 1988

Merit: 1012

Beyond Imagination

Quote from: tspacepilot on February 19, 2016, 01:12:26 AM

There may be something going on here if you say that your stats are different from what you're seeing on other people's nodes. In any case, it's definitely not regular procedure to be restarting your nodes every week. Bitcoin is reasonably stable software and you should be able to get significant uptime if everything is configured correctly. On the other hand, 80MB doesn't really seem all that large (but you're right that that linear growth is troubling), I'm guessing something's configured incorrectly here. I think it's hard to diagnose further without more details.

This is not my machine, but my machine is having similar problem and now it is over 225MB after 10 days of running. Maybe I'm just too lazy to build the bitcoind from source

tspacepilot

legendary

Activity: 1456

Merit: 1081

I may write code in exchange for bitcoins.

There may be something going on here if you say that your stats are different from what you're seeing on other people's nodes. In any case, it's definitely not regular procedure to be restarting your nodes every week. Bitcoin is reasonably stable software and you should be able to get significant uptime if everything is configured correctly. On the other hand, 80MB doesn't really seem all that large (but you're right that that linear growth is troubling), I'm guessing something's configured incorrectly here. I think it's hard to diagnose further without more details.

johnyj

legendary

Activity: 1988

Merit: 1012

Beyond Imagination

http://bitcointicker.co/networkstats/
You can see from this site, for more than 10 days, the mempool size steadily grew beyond 80MB, and the operator did a restart of the process, then it start to build from 0 again, but it always goes up no matter what is the queue size of transactions

And this is not the same as I see on tradeblock, they seems to have it under 10MB most of the time

Topic: Do you need to manually reset the nodes every week? (Read 1394 times)