Topic: The Lightning Network node experience - page 9. (Read 3480 times)

Yesterday, I performed a triangle swap with two other bitcointalk members. Each of us opened a 0.05 BTC channel to the other person. This way, we gained additional 0.10 BTC capacity, which means that this swap has doubled my node's capacity. I paid 222 sat for the opening transaction. We dropped the fees to 0 and TheJuice's script took care of the rebalancing. On the same day, I routed 6 transactions from the channel I opened and I got paid 222.76 satoshi. The channel paid itself off in just a few hours.

Note that on some days (ex. today) I routed a few small payments (< 1 sat). It's impossible to tell it from the graph at this point.

I have 2 -- one dynamically adjust fees based on the proportion of the sats on each side (and some other parameters). The rebalance script looks to get atleast 2M sats outgoing on each channel -- however it will only rebalance if the expected fees from future transactions are higher (by 3x) than the fee to rebalance.

What does your rebalance script do? Does it adjust fees once one side of a channel reaches a certain point? Or does it send a invoice to yourself that is paid to a channel with a high balance to a channel with a low balance?

There are 4 major reason:

1) It's part of the game to get higher rankings. A lot of the rankings take into consideration average channel size. When I first started out I had the limit at 1M sats and have been slowly increasing. Higher rankings help with #4 below.
2) I've found that smaller channels almost never generate significant transactions -- I suspect this is both b/c smaller nodes open smaller channels and it's harder to keep sats on both sides of a small channel.
3) My rebalance script targets 2M sats on each side; just makes the scripting easy with no small channels.
4) Once you get a rather large node you start having "gravity", meaning people open channels to you randomly. This is great b/c your node grows without having to invest more capital. I found most times people open the smallest possible channel -- so by now using a 5M sat make people who want to connect open a larger channel than they might have otherwise. The other reason is closing channels can be expensive (esp force closed ones), so I don't want small nodes who may shut down in a month to open to me. I want long term node partners.

If BTC crosses 100k, I'll probably drop my min channel size to 4M sats.

I have considered if I should let people connect for less who want to use my node as a connection for their lightning node wallet (as opposed to another routing node) -- maybe one of these days i'll do that.

I am actually thinking about opening a channel to your node. Is there any particular reason why you reject channels smaller than 5 million satoshi? Do you want to avoid frequent rebalancing this way?

my node also runs for about 4 months and has 17 channels. it forwarded 616 payments which collected 978 sats. i've set the basefee to 1 sat and no percentage fee for all channels. i played around with the fees in the beginning (thats why i earned more sats than the number of forwards)

i only added the summary plugin to the node for now

some weeks ago i wanted to rebalance a channel with a trusted peer (all of the balance is on my side, pushing over 50% and the peer sending the amount onchain back to me), but the keysend to the lnd node didn't work. i've sent christian decker an email with the logs and he said he will look into it to debug it, but i never heard back... it still isn't working

apart from that i didn't have any bigger problems, but i am also not using my lightning node for a lot of my own economic activity at the moment (only some small donations)

I've been meaning to do a larger write up of my experience once I hit 1M sats in fees.

I run the node Prince, https://1ml.com/node/02644f80b5d32ed9a9888690571159692a17d7ead7db2df5124a8e2a72a8447d30

I am currently ranked 8th on the BOS rankings (https://fulmo.org/bos-score.html) and 12th on lightning terminal rankings. I have 8-9 BTC on my node with about 110 channels (I have more incoming that outcoming liquidity, so that 8BTC is not my investment).

Top line numbers: Over the past 5 months have routed over 6700 payments worth over 1.7 billion sats earning me about 900k sats in routing fees; I've spent about 100k sats in on-chain fees and rebalancing fees. For those doing the math, that comes out to about 0.5% APR on my locked-up capital. Although my volume is steadily increasing and I may be able to get closer to 1% in the next year.

I have everything automated including fees and re-balancing.
-- I use lnd-charge for fees: https://github.com/accumulator/charge-lnd
-- I use c-otto's script for automatic re-balancing: https://c-otto.de/

Happy to answer more questions -- oh, and feel free to open a channel to me. The best way to get started is to open liquidity triangles with people (ie A->B; B->C; C->A). Happy to be a part of a couple here if there is interest. I do require a minimum of 5M sats for any incoming channels.

Yes, I know about the listforwards command outputting the exact reasons, but not for all failed payments. I also know about the description of each error. However, at least me, cannot understand by only reading the descriptions in that link. So, it's almost useless to me to read them. :p

I'm not sure we can check if a payment was split or not. What it seems to me is that, at least for some payments, there are several attempts and as the values changes by small amounts, I assume it tried different routes, hence, different fees.

You need to use lightning-cli listforwards command to see the exact error. Here you can find a list of errors with explanations.


Maybe the payment was split into multiple chunks and some of them failed? It would make sense to route one part through you again if it worked in some previous attempt.

No problem @_Rath.

I still have tons of failed transactions with no apparent reason. I have 5 pages of failed transactions at November 9th. It's 48 failed attempts, of which only 6 were LOCAL_FAILED. All the others are simply FAILED.
I have enough liquidity in the outgoing channels of these attempts, so I can't understand why, even the LOCAL_FAILED ones, failed.



There is no outgoing channel for these LOCAL_FAILED. Have no idea why.

Then, the other attempts all failed further down in the route. What I question in my mind is if when a payment fails, for a given amount and route, the next attempts will try to use exactly the same route or will it try new routes? If it tries the same route, what could be the reasons for the failures? And if it tries a different route for each attempt, is it normal to fail like 10 or 12 times before it finally gives up on trying to route this payment?

Thanks @darkv0rt3x for sharing your data. As I have already told you, I spent some time manually adjusting the fees for each channel every few days and it looks like I found the right values.

November looks really promising for my node. I am also about to open two new channels beside the one with n0nce.

Hi there...

Here is my contribution. My noded URI is in my signature.
My node has 7 channels open, with a total of 4,500,222 sats of outbound capacity and 6,286,444 sats of inbound capacity. I'm running a bare metal Debian BullsEye, Bitcoin Core v0.21, C-Lightning v0.10.2 and RTL v0.10.1-beta. I have a few other non-related services running in this device (RockPro64, Samsung NVMe drive and 1TB external USB drive) but I don't think they are relevant for this matter.

My node is not very active as apparently I'm using some high fees that are probably putting me kind of isolated in the network. I'll lower the fees as soon as I can to see if there is some more traffic through it.
My node has been running for over a year but it only started forwarding payments successfully since January this year. Here is my data.


January, 2021


February, 2021


March, 2021


April, 2021


May, 2021


June, 2021


July, 2021


August, 2021


September, 2021


October, 2021


So, as we can see, I'm using high fees for sure. Not much activity these 2 last months and there's nothing in November so far!
I remember I used a plugin to auto-adjust fees and I made the mistake of making it auto-start whenever the node was restarted which made the plugin to adjust fees too often and make fees too low too.

I don’t see any reason why adjusting fees couldn’t be automated depending on the specific circumstances.

If a lot of BTC is required to earn from running a LN node, perhaps a reasonable interest amount should be charged against earnings to account for what could have been earned by lending your bitcoin to whatever reputable DeFi company.

Interesting, thanks for you insight! So it might actually be worth to connect 'already connected' peers after all, cool cool. More experimentation possibilities

There are some writeups / articles online and also YouTube videos where people show their dashboard etc., but from what I've seen so far, to be profitable you need to put in a lot of manual work. We all know time is money & time is limited; so all that work for a few satoshi is usually not worth it. The 'maximum profits' I've seen so far, usually required a lot of BTC locked up + work under normal hourly wage.

Thats interesting. I wonder if some people are closing channels (or not closing channels) based on fees on various routes. If there is competition offering free routes, it might not be worth it to tie up bitcoin, but once a channel is closed for likely low future fees, they might not open a new channel due to the associated initial costs of doing so (tx fees).

It would be interesting to see how much someone might be able to earn running a LN node if they were trying to maximize their profit.

Sure, that's what I have been doing as well. Unfortunately, those channels remained silent as I periodically dropped the fees so I decided to make their use free of charge. One channel suddenly started routing transactions again in both directions even after I raised the fees. I will try rebalancing the other channel through a circular payment for a reasonable fee. It became unbalanced just a few hours after it was opened so I might earn back the cost of rebalancing and make some profit easily.

When your channel becomes unbalanced, you can reduce the fees on the unbalanced side so that you will receive transactions on the side that has a low balance.

You can also use dynamic pricing so that as one side gets low, the fee rate will change to encourage transactions to be routed to that side of the channel.

Whoops, I missed n0nce's reply.


That's exactly what I had been avoiding until I made a mistake. I opened a channel to both Bitfinex and Nicehash. I had been routing fairly large payments between those two channels until Bitfinex bumped their fee rate from 1 ppm to 100 ppm. While I was trying to figure it out, I learnt that Nicehash and Bitfinex have had a channel for quite some time. It looks like my node was routing payments between them whenever their channel was unbalanced, which happened quite often. I still keep those two channels open because I keep routing payments through them from/to other channels.


It might sound easy, but it's a huge pain in practice. I mostly use ZigZag.io and Boltz for non-dual-funded channels. When I was running LND node, I tried Loop but it failed 90% of the time. Since I have a couple of active channels, I might try circular rebalance soon. Also, it makes sense to wait a couple of days and see if the channel will balance itself.

The easiest and the fastest way for you would be to use the lncli fwdinghistory command. If you have some spare time, you could install either ThunderHub or RTL. I haven't used the former one since it's LND exclusive but it should be as good as RTL or even better for channel management. It will be much easier for you to interpret all the data if you use either of them. The graphs I posted were created by RTL.

Yeah, it depends a ton about who you're connecting to and also the size of those channels. To maximize routing and profits, some people actually manually monitor their channels. Some recommendations I picked up:

* Don't create channel between 2 already connected nodes. (e.g. check via LN explorers)
* If a channel isn't routing anything for a long time, close it.
* If a channel starts to get used much, but only in one direction, rebalance it so it continues being used.
* Look for nodes with much usage, but no 1-hop connection and connect them through you.
* Adapt fees up / down in the channels that are used much, like sometimes your fees are just too high; sometimes users keep paying if you increase them since you're the only route to some location.

This is just off the top of my mind, not sure if I can find the original 'list' anymore. We may just create a new thread for that. I don't feel qualified for it myself though, since I was never a big routing node myself.

With regards to 'LN node experience', I will post in here in the future when I find the time to write down all my experiences and make a concise but informative post of it.