Tips to Keep Your Node Afloat on the High Seas

[Olaoluwa Osuntokun]

Hi y'all,

I thought it'd be useful to cross-post an X-thread of mine from earlier
this year on this mailing list:
https://twitter.com/roasbeef/status/1737566027574333736?t=u2GJP4QZNAzQvlKXCKAbWg&s=19.
It describes various configuration and behavioral changes node operators can
make to better navigate the current on-chain fee climate. We're in a bit of
a fallow period right now (~50 sat/byte) to get into the next block, but
forecasts point toward even choppier waters due to both exogenous and
endogenous events, so it's prudent to update prior mental models and
automation in light of the new climate.

The biggest take away from that post is that node operators should increase
the `max-commit-fee-rate-anchors` to something around 100 sat/byte or so.
This controls that max fee rate of the commitment transaction for anchor
channels (if you aren't already defaulting to anchor channels, then start
there!). Anchor channels let us avoid having to guess the fee rate that will
confirm a force close up front, but we still need to be able to get into the
mempool in the first place. Increasing this value gives lnd more breathing
room to fee bump to get above the min-purging-fee rate (around ~23 sat/vbyte
rn). If your anchor commitment fee rate isn't above that value, then it may
not propagate well, or at all.

If all node operators properly set the max anchor fee rate config option,
then we should be able to reduce the amt of cascading force closes we're
seeing lately. This happens when no one in the entire HTLC route can get
their force close confirmed before the timeout. Usually this is rare, but
when fee rates spike up quickly to 200 sat/vbyte, it can be more common.
Node operators can also further increase their CLTV delta from the default
80 blocks (~13 hours) to something in the range of hundreds of blocks
(days).

Have no fear, we're working on re-prioritizing some improvements to: fee
estimation, go-to-chain neuritics, configuration options, and more for the
upcoming 0.18 release. Some of the items we're re-examining for
prioritization include:

  * An optional mode that will cease to go to chain for payments where
    you're the originator. As a mobile node, you aren't really routing at
    all, so there's no actual risk if you don't go and timeout an outgoing
    HTLC when it gets close to the timeout.

  * Automatically setting the max anchor fee rate value mentioned above.
    Node operators shouldn't have to set this at all. Initially it was a
    config option as we weren't 100% sure what the right value should be,
    but IMO now it's clear: the value should be set to the max of your
    mempool's high water mark and a smoothed sample of the `feefilter` (BIP
    133) values your peers send.

    It isn't enough just to worry about your own node's mempool, as if all
    your peers have high `feefilter` values, then you'll never see a large
    class of transactions (lower fee rate so it isn't sent), which means
    your mempool may never actually get "full", so it never purges anything,
    causing you to think that 10 sat/byte is enough to propagate thru the
    network, when it's actually 20 sat/byte.

    This will also serve as another layer of fee sanity check for neutrino
    nodes.

  * Have an optional config option to stop going to chain for dust HTLCs.
    You can compute what an economic fee rate + HTLC amt combo should be,
    and just not go to chain for anything below that. Ultimately, the
    protocol needs a new extension that uses a different accounting tab for
    dust HTLCs (as rn they eat away at the max allotted HTLC amt). In the
    ideal scenario, these just get cancelled back eventually. In the worst
    case, it eats into your available forwarding bandwidth, as the HTLCs
    take up otherwise available slots.

    Node operators can also increase the `min_htlc` amt to achieve a similar
    effect. It should be set to the size of an HTLC you'd be willing to go
    to chain for. This is one area where multiple channels are nice: you can
    have a few beefy channels for large HTLC amts with a higher `min_htlc`
    setting, and some smaller ones with more lax limits.

  * Dynamic fee bumping for force closes. Today users can manually bump
    using `BumpFee`, but otherwise, today we'll periodically get a fresh
    estimation for the next block. This serves to increase our fee rate over
    time if fees are also increasing, but means we won't react quickly to
    changes.

    We have a few candidate designs here, which are right now mostly blocked
    on some predatory refactoring in the sweeper.

  * An optional mode for optimistic incoming HTLC cancellation. With this
    mode, you'd cancel the incoming HTLC potentially before your outgoing
    force close even confirms. This can also serve to reduce unnecessary
    force close cascades, but has some risk as: if you cancel early, then
    you risk that the outgoing HTLC is eventually redeemed on or off chain.


We're also working on some exciting projects that will enable even more
efficient utilization of the chain for channel opens. Stay tuned for more
news on that front!


-- Laolu