Nostr notes by

📅 Original date posted:2023-05-06 🗒️ Summary of this ...

2023-06-19T17:42:04Z

In reply to nevent1q…tze2
_________________________

📅 Original date posted:2023-05-06
🗒️ Summary of this message: A member of the lightning community expressed concern over the lack of public communication due to off-topic emails and a slow moderator. They questioned if there is a better place for discussions.
📝 Original message:
Is there a better place to have public communication? Unfortunately since one off topic email was sent here, it's been a ghost town. It appears that there's many emails being held and only one moderator that checks them once a week.

Would hate to see this list die but wondering if there's a better place for discussions?

Tony

-------- Original Message --------
On Apr 29, 2023, 9:57 PM, niftynei wrote:

> Hi all,
>
> When I joined the lightning community a few years ago, I was relatively new to open source software and specification work. Rusty really impressed on me on the importance of holding conversations, as much as possible in public.
>
> Practically speaking, this encompasses IRC, this mailing list, and github issues/PRs.
>
> The reason for this is twofold. It helps document the range of options considered for technical decisions and it provides an interface point for new participants to contribute to the discussion.
>
> Given some recent mails that were posted to this list, now seems like a good time to reiterate the importance and preference of public communication whenever possible, especially for specification or technical discussions.
>
> ~ nifty
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230506/5bcba216/attachment.html>;

📅 Original date posted:2022-06-13 📝 Original message: Rene, ...

2023-06-09T13:06:12Z

In reply to nevent1q…0t5z
_________________________

📅 Original date posted:2022-06-13
📝 Original message:
Rene,

Thanks for your reply.

> Also wallets tend to have poor utxo management. So looking at the on-chain signal one can probably guess for a p2wsh to which two nodes it might belong and try them first.

That was going to be one of my next steps. I thought about parsing through the data from https://github.com/lnresearch/topology in order to get a link between every unpsent p2wsh transaction up to 6 hops forward from the opening transaction between two nodes. Then keep a list of every node and their possible p2wsh transactions and probe with those. Should cut down a lot but I have not run through this dataset problem yet. Curious if you think that would be the best process moving forward with this.

Otherwise, going through LSP's with the assumption set, like I did with ACINQ would probably yield further results.

Regards,
Tony
------- Original Message -------
On Wednesday, June 8th, 2022 at 12:42 AM, René Pickhardt <r.pickhardt at googlemail.com> wrote:

> Dear Tony,
>
> Thank you for putting emphasis on this. I was actually waiting for someone to publicly exploit this.
>
>> The reason this is possible is because [...] currently channel IDs are based on UTXO's. Scid aliases may be the biggest benefit here, but the use of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`, and `amount_below_minimum` have been the biggest helpers in exploiting channel privacy.
>
> Just for reference the exploit with short_channel_ids is known since 2019:
>
> https://github.com/lightning/bolts/issues/675
>
> Though it is nice you point out explicitly the use of error codes of onions.
>
>> By creating a probe guessing the Channel ID based on unspent p2wsh transactions, it's a `m * n` problem to probe the entire network, where `m` is utxos and `n` is nodes.
>
> It is the main reason why I didn't do this. Though similar to you probing ACINQ's node one could probabilistically learn which nodes tend to have unannounced channels and gain some speedup by probing those nodes first.
>
> Also wallets tend to have poor utxo management. So looking at the on-chain signal one can probably guess for a p2wsh to which two nodes it might belong and try them first.
>
> These two strategies should reduce the number of tested nodes for a newly seen p2wsh output significantly and probably make it feasible to probe the network as new blocks come in.
>
> With kind regards Rene Pickhardt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220613/10ec2bcf/attachment.html>;

📅 Original date posted:2022-06-08 📝 Original message: Hi ...

2023-06-09T13:06:12Z

In reply to nevent1q…x0wk
_________________________

📅 Original date posted:2022-06-08
📝 Original message:
Hi /dev/fd0,

> Can this be fixed by making error messages less verbose or reveal less information?

Not completely fixed, but much more time consuming to exploit. When a prober gets _just_ the SCID correct, a certain error is passed back. Typically incorrect_cltv_expiry or amount_below_minimum being the error, proving that the channel ID is correct. Then it's just up to getting the other values correct. So if the error messages were generic and did not reveal the Channel ID was correct unless all of the other values were also correct, it requires many more combinations of parameters to probe for, instead of just the Channel ID, then just the CLTV, then the amount, then the other node. Does not fix the problem completely because eventually if you were to get all values correct, it would route to the other node, where the other node creates the incorrect_or_unknown_payment_details error, which you can tell which node creates the error, proving it got to the guessed node.

> Are alias SCIDs necessary to fix it or error messages alone can fix it?

With the above said, I believe it is necessary to move to SCIDs and that error messages alone can't fix it completely. It's better anyways because when an invoice is made with routing hints for unannounced channels, those consuming the invoice can't associate UTXO's with the node creating the invoice if aliases were used.

Also, in addition to LDK implementing it, I know LND is working on it currently too https://github.com/lightningnetwork/lnd/pull/5955

> I love the research and thanks for sharing all the information. I am assuming analytic firms would be using this already.

Thank you! I assume it as well though it's probably easy to tell whether this is happening to your node or not if someone was watching their HTLC routing failure reasons.

Thanks,
Tony

------- Original Message -------
On Tuesday, June 7th, 2022 at 10:36 PM, alicexbt <alicexbt at protonmail.com> wrote:

> Hi Tony,
>
>> The reason this is possible is because probing is a free operation on the Lightning Network after a channel is opened, the error reasons given are way too verbose, and currently channel IDs are based on UTXO's. Scid aliases may be the biggest benefit here, but the use of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`, and `amount_below_minimum` have been the biggest helpers in exploiting channel privacy.
>
> Can this be fixed by making error messages less verbose or reveal less information?
>
>> We should definitely migrate to alias scid's, and encourage every active unannounced channel holder to close, coinjoin, and reopen with an alias. But care should be given in the future when it comes to error reasons revealing information that is meant to be "private". Until this migration happens, it would be beneficial to stop being so specific about errors, this does not really seem to help end users anyways.
>
> Alias SCID would be better for privacy as they allow a node to request a channel by a random value instead of value derived from the on-chain transaction. Are alias SCIDs necessary to fix it or error messages alone can fix it?
>
> I found these pull requests and assuming alias SCID are already implemented in LDK/rust-lightning:
>
> [https://github.com/lightningdevkit/rust-lightning/pull/1311/](https://github.com/lightningdevkit/rust-lightning/pull/1311/commits)
> https://github.com/lightningdevkit/rust-lightning/pull/1351
>
>> I'll be continuing with this probing project while the problem exists, and work on narrowing down the other channel partner and fixing efficiency bugs. I am publicizing the results as I go, so fair warning that if you have any unannounced channels that you assumed were private and need them to be, close them now on the off chance they get revealed. This could have always been happening already already by analytic firms, so I hope by publicizing this we are all on the same playing field. It is also beneficial to get a better estimate of the unknown size of the Lightning Network.
>
> I love the research and thanks for sharing all the information. I am assuming analytic firms would be using this already.
>
> /dev/fd0
>
> Sent with [Proton Mail](https://proton.me/) secure email.
>
> ------- Original Message -------
> On Wednesday, June 8th, 2022 at 7:35 AM, Tony Giorgio via Lightning-dev <lightning-dev at lists.linuxfoundation.org> wrote:
>
>> Hi,
>>
>> For the past few months I have been working on an LDK probing project that searches for unannounced channels on the Lightning Network. For the past week, I have been probing on mainnet and squashing bugs / making optimizations.
>>
>> So far I have found near 445 unannounced channels totaling 1,076,077,750 satoshi's locked across the 3 nodes I have probed, some with just a minimized set (~30,000) of probable channels based on "round payment amount" and "1 or 2 tx output" heuristics on P2WSH UTXO's. Most of them being on Aincq's node found with the minimized set, I've yet to run the complete set with them. There are about ~860,000 P2WSH UTXO's, about ~60,000 of which are public, so the upward limit of possible private channels is around ~800,000.
>>
>> The exact results are publicized here: https://github.com/BitcoinDevShop/hidden-lightning-network/blob/master/data/results/results.json
>>
>> The reason this is possible is because probing is a free operation on the Lightning Network after a channel is opened, the error reasons given are way too verbose, and currently channel IDs are based on UTXO's. Scid aliases may be the biggest benefit here, but the use of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`, and `amount_below_minimum` have been the biggest helpers in exploiting channel privacy.
>>
>> By creating a probe guessing the Channel ID based on unspent p2wsh transactions, it's a `m * n` problem to probe the entire network, where `m` is utxos and `n` is nodes. Without these errors and instead something like `temporary_channel_failure` or a generic indistinguishable error, guessing a Channel ID would come down to an upwards of `m * n * n-1 * ~2000`, which would be each utxo with each pairing of nodes, each with about ~2000 cltv's to guess (numbers are as low as 7 to as high as ~2000). I threw the extra 2000 into the equation because even with `800,000 * 1 * 2000`, it gets much more time consuming to even probe a single node when we're already spending upwards of a day or so for near 1 million or 2 probes. Concurrent probing is possible, but starts to require more locked up liquidity.
>>
>> We should definitely migrate to alias scid's, and encourage every active unannounced channel holder to close, coinjoin, and reopen with an alias. But care should be given in the future when it comes to error reasons revealing information that is meant to be "private". Until this migration happens, it would be beneficial to stop being so specific about errors, this does not really seem to help end users anyways.
>>
>> I'll be continuing with this probing project while the problem exists, and work on narrowing down the other channel partner and fixing efficiency bugs. I am publicizing the results as I go, so fair warning that if you have any unannounced channels that you assumed were private and need them to be, close them now on the off chance they get revealed. This could have always been happening already already by analytic firms, so I hope by publicizing this we are all on the same playing field. It is also beneficial to get a better estimate of the unknown size of the Lightning Network.
>>
>> For more about this project and viewing the dataset, go to http://hiddenlightningnetwork.com
>>
>> Thanks,
>> Tony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220608/7760d9dd/attachment-0001.html>;

📅 Original date posted:2022-06-07 📝 Original message: Hi, ...

2023-06-09T13:06:11Z

In reply to nevent1q…5xfp
_________________________

📅 Original date posted:2022-06-07
📝 Original message:
Hi,

For the past few months I have been working on an LDK probing project that searches for unannounced channels on the Lightning Network. For the past week, I have been probing on mainnet and squashing bugs / making optimizations.

So far I have found near 445 unannounced channels totaling 1,076,077,750 satoshi's locked across the 3 nodes I have probed, some with just a minimized set (~30,000) of probable channels based on "round payment amount" and "1 or 2 tx output" heuristics on P2WSH UTXO's. Most of them being on Aincq's node found with the minimized set, I've yet to run the complete set with them. There are about ~860,000 P2WSH UTXO's, about ~60,000 of which are public, so the upward limit of possible private channels is around ~800,000.

The exact results are publicized here: https://github.com/BitcoinDevShop/hidden-lightning-network/blob/master/data/results/results.json

The reason this is possible is because probing is a free operation on the Lightning Network after a channel is opened, the error reasons given are way too verbose, and currently channel IDs are based on UTXO's. Scid aliases may be the biggest benefit here, but the use of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`, and `amount_below_minimum` have been the biggest helpers in exploiting channel privacy.

By creating a probe guessing the Channel ID based on unspent p2wsh transactions, it's a `m * n` problem to probe the entire network, where `m` is utxos and `n` is nodes. Without these errors and instead something like `temporary_channel_failure` or a generic indistinguishable error, guessing a Channel ID would come down to an upwards of `m * n * n-1 * ~2000`, which would be each utxo with each pairing of nodes, each with about ~2000 cltv's to guess (numbers are as low as 7 to as high as ~2000). I threw the extra 2000 into the equation because even with `800,000 * 1 * 2000`, it gets much more time consuming to even probe a single node when we're already spending upwards of a day or so for near 1 million or 2 probes. Concurrent probing is possible, but starts to require more locked up liquidity.

We should definitely migrate to alias scid's, and encourage every active unannounced channel holder to close, coinjoin, and reopen with an alias. But care should be given in the future when it comes to error reasons revealing information that is meant to be "private". Until this migration happens, it would be beneficial to stop being so specific about errors, this does not really seem to help end users anyways.

I'll be continuing with this probing project while the problem exists, and work on narrowing down the other channel partner and fixing efficiency bugs. I am publicizing the results as I go, so fair warning that if you have any unannounced channels that you assumed were private and need them to be, close them now on the off chance they get revealed. This could have always been happening already already by analytic firms, so I hope by publicizing this we are all on the same playing field. It is also beneficial to get a better estimate of the unknown size of the Lightning Network.

For more about this project and viewing the dataset, go to http://hiddenlightningnetwork.com

Thanks,
Tony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220608/4e5a66be/attachment.html>;