{"type":"rich","version":"1.0","author_name":"npub1aslmpzentw224n3s6yccru4dq2qdlx7rfudfnqevfck637cjt6esswfqmx","author_url":"https://nostr.ae/npub1aslmpzentw224n3s6yccru4dq2qdlx7rfudfnqevfck637cjt6esswfqmx","provider_name":"njump","provider_url":"https://nostr.ae","html":"📅 Original date posted:2021-11-15\n📝 Original message:\nHi Rene,\n\n\n\u003e First I am happy that you also agree that reliability can and should be\n\u003e expressed as a probability as discussed in [0].\n\u003e\n\nProbability based routing is not new to me. I've implemented a form of that\nin lnd in march 2019: https://github.com/lightningnetwork/lnd/pull/2802,\nfollowed by several rounds of refinement.\n\n\n\u003e The problem that you address is that of feature engineering[1]. Which\n\u003e consists of two (or even more) steps:\n\u003e\n\u003e 1.) Feature selection: That means in payment delivery we will compute a\n\u003e min cost flow [2] with a chosen cost function (historically people used\n\u003e dijkstra seach for single paths with the cost function representing the\n\u003e weights on the edges of the graph -which is what most folks currently still\n\u003e do). While [2] and I personally agree with you that the cost function\n\u003e should be a combination the two features fees and reliability (as in\n\u003e successprobability) Matt Corallo righfully pointed out [3] that other\n\u003e features might be chosen in the future to deliver more optimal results. For\n\u003e example implementations currently often use CLTV as a feature (which I\n\u003e honestly find horrible) and I am currently investigating if one could add\n\u003e latency of channels or - for known IP addresses - either the geo distance\n\u003e or IP distance.\n\u003e\n\nI am aware that there are more candidate features, but my question is\nspecifically about the ones that I mentioned.\n\n2.) Combining features: This is the question that you are asking. Often\n\u003e people use a linear weighted sum to combine features. This is what often\n\u003e happens implicitly in neural networks. While this is often good enough and\n\u003e while it is often practical to either learn the weights or give users a\n\u003e choice there are many situation where the weighted linear sum does not work\n\u003e well with the selected features. An example for the weighted sum is the\n\u003e risk-factor in c-lightning that could have been used to decide if one\n\u003e wanted the dijkstra seach to either optimize for CLTV delta or for paid\n\u003e routing fees. Also in our paper [2] in which we discuss the same two\n\u003e features that you mentioned we explain how a linear sum of two features can\n\u003e be optimal due to the lagrangian bounding principle. However in practice\n\u003e (of machine learning) it has been shown that using the harmonic mean [4]\n\u003e between features often works very well without the necessity to learn a\n\u003e weight / parameter. This has for example been done when c-lightnign\n\u003e recently switched to probabilistic path finding [5]. In this thread you\n\u003e find a long discussion and evaluation how the harmonic mean outperformed\n\u003e the linear sum.\n\u003e\n\nObviously features can be combined in a multitude of ways, but I am looking\nfor something that is anchored to some kind of understandable starting\npoint. What I did in lnd is to work with so called 'payment attempt cost'.\nA virtual satoshi amount that represents the cost of a failed attempt. If\nyou put a high price on failed attempts, pathfinding will tend towards more\nreliable routes even if they require a higher fee. To me, the idea of\nputting a (virtual) cost on a payment attempt is tangible and ideally the\nmath should follow from that. I don't want zero parameters, because I think\nthat ultimately the fee/reliability trade-off is up to the user to decide\non.\n\nJoost\n\n\u003e\n-------------- next part --------------\nAn HTML attachment was scrubbed...\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211115/0be67f54/attachment.html\u003e"}