{"type":"rich","version":"1.0","author_name":"npub1zlxd3xlzjhq2ue03e5m5p2w6mp8v3dkhq5r39flsftjjsje04wvsdd2k4w","author_url":"https://nostr.ae/npub1zlxd3xlzjhq2ue03e5m5p2w6mp8v3dkhq5r39flsftjjsje04wvsdd2k4w","provider_name":"njump","provider_url":"https://nostr.ae","html":"📅 Original date posted:2022-09-23\n📝 Original message:\nDear Matt and Lightning developers,\n\nthose are excellent and important questions that I should probably have\naddressed more explicitly in the article / ml-post! Let me add what I\ncurrently know and begin with your second question as I think I think the\nanswer will be more objective / verifiable while my response to the gossip\nquestion is a bit more speculative at this point:\n\nOn Fri, Sep 23, 2022 at 10:43 AM Matt Corallo \u003clf-lists at mattcorallo.com\u003e\nwrote:\n\nb) What are the privacy implications of the naive \"update on drained\n\u003e channel\", and have you done any\n\u003e analysis of the value of this type of gossip update at different levels of\n\u003e privacy?\n\u003e\n\nWe know from information theory that the distribution with the highest\nentropy is the uniform distribution [1] which in the context of a channel\nwith capacity of `c` means `log(c+1)` bits as shown in [2].\n\nNow if a given drain leads to a pair of `htlc_maximum_msat` one could in\ndeed also go the other way around and look at the pair of\n`htlc_maximum_msat` and estimate the *past* drain. And of course also the\n(the non uniform) liquidity distribution from the past. Note that now with\nthe better `htlc_maximum_msat` pair the distribution should be closer to\nuniform again which is actually best from an information theoretic point of\nview as this maximizes the entropy in the channel.\n\nI have just created and uploaded a small python script / notebook [3]  from\nwhich we can see for example how much information about the shape of the\npast liquidity distribution we would learn if we could induce from the\n`htlc_maximum_msat` pair that the drain was 0.75. So from the notebook I\nquote:\n\nAssume we learnt a drain of 0.75\n=================================\nEntropy of uniform: 20.00 bits.\nEntropy of the channel with drain: 19.17 bits\nInformation gain: 0.83 bits\n\n\nYes we learnt less than 1 bit of information by knowing the shape of the\ndistribution from the last payments. This much is learnt by a single probe.\nGiven that we expect our payment failure rates to drop by an order of\nmagnitude and the fact that I could probably learn the drain from the on\nchain signal or gossip anyway I personally would consider this information\nleakage to be acceptable. But of course people may not agree with me and\njust not set their `htlc_maximum_msat` flag in the proposed manner. Also it\nis obvious that higher drain values produce a more heavily depleted channel\nand we might learn more bits (in case of a 0.95 drain we learnt about 3\nbits.)\n\na) How much gossip overhead do you expect this type of protocol to\n\u003e generate/is there a useful\n\u003e outcome for this type of update even if you limit gossip updates to\n\u003e once/twice/thrice per day?\n\u003e\n\nAs written in the article I expect the setting of the valves as throtteling\ndevices to be rather stable as I would assume the drain on channels should\nnot be impacted too heavily by installing valves. This assumption is\ncertainly more reasonable if nodes already used min cost flow solvers to\nsend out payments and less reasonable if nodes still use ad-hoc splitting\nheuristics and restrict their path finding on channels with sufficiently\nlarge `htlc_maximum_msat` values for the amount of the partial payment.\n\nIn any case I expect that once the valve is in a good setting for a\nparticular channel / drain the setting can stay as long as the drain won't\nchange significantly. In general I would not expect drain on channels to\nchange heavily over time (though I certainly would not expect it to be\nstatic / stable either). Certainly fee changes in the network (and other\nfactors) may eventually produce changes in drain on channels which would\nalso have to eventually be reflected in new settings for the valve.\n\nThe experimental algorithm idea that I presented in cell 12 of the original\nnotebook (which I also used in the privacy consideration code from above)\ncomputes the histogram of liquidity of the last N routing requests (failed\nand successful ones) that a node saw on the channel and then makes a\ndecision weather to open or close the valve a bit more. Using this greedy\nstrategy we saw in the notebook that a node could find decent values in\nlogarithmic many steps . So while the setup of the valve may trigger a few\ngossip messages (but each one only after a sufficient amount of routing\nrequests have been processed) and while I expect dynamics in drain on the\nchannel I don't expect such adoptions should have to happen in real time or\neven several times per day and put unnecessary load / spam to gossip. But I\nguess we can only see this by operating nodes and probably there are some\nnodes where updates occure more frequently than with other nodes.\n\nThat being said of course if the peers of a channel work together and both\nexchange information / collaborate in finding a good `htlc_maximum_msat`\npair (similalry as our protocol for finding fees in a mutual close) they\nwill probably need less gossip messages than in the case where both nodes\nindependently try to find decent settings.\n\nI am happy if people have more insights into this and challenge my\nexpectation (because it actually really is only an expectation / intuition\nat this point). That being said: Yes even with only a few gossip messages\nper day I expect for the given reasons the setup of valves to be possible\nand useful!\n\nwith kind regards Rene\n\n[1]:\nhttps://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution#Uniform_and_piecewise_uniform_distributions\n[2]: https://arxiv.org/abs/2103.08576\n[3]:\nhttps://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/Privacy%20Considerations%20of%20signaling%20past%20drain%20via%20%60htlc_maximum_msat%60%20pairs.ipynb\n\n\u003e\n\u003e Thanks,\n\u003e Matt\n\u003e\n\u003e On 9/22/22 2:40 AM, René Pickhardt via Lightning-dev wrote:\n\u003e \u003e Good morning fellow Lightning Developers,\n\u003e \u003e\n\u003e \u003e I am pleased to share my most recent research results [1] with you. They\n\u003e may (if at all) only have a\n\u003e \u003e small impact on protocol development / specification but are actually\n\u003e mainly of concern to node\n\u003e \u003e operators and LSPs. I still thought they may be relevant for the list.\n\u003e \u003e\n\u003e \u003e While trying to estimate the expected liquidity distribution in depleted\n\u003e channels due to drain via\n\u003e \u003e Markov Models I realized that we can exploit the `htlc_maxium_msat`\n\u003e setting to act as a control\n\u003e \u003e valve and regulate the \"pressure\" coming from the drain and mitigate the\n\u003e depletion of channels. Such\n\u003e \u003e ideas are btw not novel at all and heavily used in fluid networks [2].\n\u003e Thus it seems very natural\n\u003e \u003e that we do the same on the Lightning Network.\n\u003e \u003e\n\u003e \u003e In the article we show within a theoretic model how expected payment\n\u003e failure rates per channel may\n\u003e \u003e drop significantly by up to an order of magnitude if channels set up\n\u003e proper asymmetric\n\u003e \u003e `htlc_maximum_msat` pairs.\n\u003e \u003e\n\u003e \u003e We furthermore provide in our iPython notebook [3] two experimental\n\u003e algorithmic ideas with which\n\u003e \u003e node operators can find decent `htlc_maximum_msat` values in a greedy\n\u003e fashion. One of the algorithms\n\u003e \u003e does not even require to know the drain or payment size distribution or\n\u003e build the Markov model but\n\u003e \u003e just looks at the liquidity distribution in the channel at the last x\n\u003e routing attempts and adjusts\n\u003e \u003e the `htlc_maximum_msat` value if the distribution is to far away from a\n\u003e uniform distribution.\n\u003e \u003e\n\u003e \u003e Looking forwards for your thoughts and feedback.\n\u003e \u003e\n\u003e \u003e with kind regards Rene\n\u003e \u003e\n\u003e \u003e\n\u003e \u003e [1]:\n\u003e \u003e\n\u003e https://blog.bitmex.com/the-power-of-htlc_maximum_msat-as-a-control-valve-for-better-flow-control-improved-reliability-and-lower-expected-payment-failure-rates-on-the-lightning-network/\n\u003e \u003c\n\u003e https://blog.bitmex.com/the-power-of-htlc_maximum_msat-as-a-control-valve-for-better-flow-control-improved-reliability-and-lower-expected-payment-failure-rates-on-the-lightning-network/\n\u003e \u003e\n\u003e \u003e [2]: https://en.wikipedia.org/wiki/Control_valve \u003c\n\u003e https://en.wikipedia.org/wiki/Control_valve\u003e\n\u003e \u003e [3]:\n\u003e \u003e\n\u003e https://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/htlc_maximum_msat%20as%20a%20valve%20for%20flow%20control%20on%20the%20Lightnig%20network.ipynb\n\u003e \u003c\n\u003e https://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/htlc_maximum_msat%20as%20a%20valve%20for%20flow%20control%20on%20the%20Lightnig%20network.ipynb\n\u003e \u003e\n\u003e \u003e\n\u003e \u003e --\n\u003e \u003e https://ln.rene-pickhardt.de \u003chttps://ln.rene-pickhardt.de\u003e\n\u003e \u003e\n\u003e \u003e _______________________________________________\n\u003e \u003e Lightning-dev mailing list\n\u003e \u003e Lightning-dev at lists.linuxfoundation.org\n\u003e \u003e https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev\n\u003e\n\n\n-- \nhttps://www.rene-pickhardt.de\n-------------- next part --------------\nAn HTML attachment was scrubbed...\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220923/b0791f3f/attachment-0001.html\u003e"}