<oembed><type>rich</type><version>1.0</version><author_name>npub19helcfnqgk2jrwzjex2aflq6jwfc8zd9uzzkwlgwhve7lykv23mq5zkvn4</author_name><author_url>https://nostr.ae/npub19helcfnqgk2jrwzjex2aflq6jwfc8zd9uzzkwlgwhve7lykv23mq5zkvn4</author_url><provider_name>njump</provider_name><provider_url>https://nostr.ae</provider_url><html>📅 Original date posted:2018-05-21&#xA;📝 Original message:&gt; What if instead of trying to decide up front which subset of elements will&#xA;&gt; be most useful to include in the filters, and the size tradeoff, we let&#xA;the&#xA;&gt; full-node decide which subsets of elements it serves filters for?&#xA;&#xA;This is already the case. The current &#34;track&#34; is to add new service bits&#xA;(while we&#39;re in the uncommitted phase) to introduce new fitler types. Light&#xA;clients can then filter out nodes before even connecting to them.&#xA;&#xA;-- Laolu&#xA;&#xA;On Mon, May 21, 2018 at 1:35 AM Johan Torås Halseth &lt;johanth at gmail.com&gt;&#xA;wrote:&#xA;&#xA;&gt; Hi all,&#xA;&gt;&#xA;&gt; Most light wallets will want to download the minimum amount of data&#xA;&gt; required to operate, which means they would ideally download the smallest&#xA;&gt; possible filters containing the subset of elements they need.&#xA;&gt;&#xA;&gt; What if instead of trying to decide up front which subset of elements will&#xA;&gt; be most useful to include in the filters, and the size tradeoff, we let the&#xA;&gt; full-node decide which subsets of elements it serves filters for?&#xA;&gt;&#xA;&gt; For instance, a full node would advertise that it could serve filters for&#xA;&gt; the subsets 110 (txid+script+outpoint), 100 (txid only), 011 (script+outpoint)&#xA;&gt; etc. A light client could then choose to download the minimal filter type&#xA;&gt; covering its needs.&#xA;&gt;&#xA;&gt; The obvious benefit of this would be minimal bandwidth usage for the light&#xA;&gt; client, but there are also some less obvious ones. We wouldn’t have to&#xA;&gt; decide up front what each filter type should contain, only the possible&#xA;&gt; elements a filter can contain (more can be added later without breaking&#xA;&gt; existing clients). This, I think, would let the most served filter types&#xA;&gt; grow organically, with full-node implementations coming with sane defaults&#xA;&gt; for served filter types (maybe even all possible types as long as the&#xA;&gt; number of elements is small), letting their operator add/remove types at&#xA;&gt; will.&#xA;&gt;&#xA;&gt; The main disadvantage of this as I see it, is that there’s an exponential&#xA;&gt; blowup in the number of possible filter types in the number of element&#xA;&gt; types. However, this would let us start out small with only the elements we&#xA;&gt; need, and in the worst case the node operators just choose to serve the&#xA;&gt; subsets corresponding to what now is called “regular” + “extended” filters&#xA;&gt; anyway, requiring no more resources.&#xA;&gt;&#xA;&gt; This would also give us some data on what is the most widely used filter&#xA;&gt; types, which could be useful in making the decision on what should be part&#xA;&gt; of filters to eventually commit to in blocks.&#xA;&gt;&#xA;&gt; - Johan&#xA;&gt; On Sat, May 19, 2018 at 5:12, Olaoluwa Osuntokun via bitcoin-dev &lt;&#xA;&gt; bitcoin-dev at lists.linuxfoundation.org&gt; wrote:&#xA;&gt;&#xA;&gt; On Thu, May 17, 2018 at 2:44 PM Jim Posen via bitcoin-dev &lt;bitcoin-&#xA;&gt;&#xA;&gt;&gt; Monitoring inputs by scriptPubkey vs input-txid also has a massive&#xA;&gt;&gt;&gt; advantage for parallel filtering: You can usually known your pubkeys&#xA;&gt;&gt;&gt; well in advance, but if you have to change what you&#39;re watching block&#xA;&gt;&gt;&gt; N+1 for based on the txids that paid you in N you can&#39;t filter them&#xA;&gt;&gt;&gt; in parallel.&#xA;&gt;&gt;&gt;&#xA;&gt;&gt;&#xA;&gt;&gt; Yes, I&#39;ll grant that this is a benefit of your suggestion.&#xA;&gt;&gt;&#xA;&gt;&#xA;&gt; Yeah parallel filtering would be pretty nice. We&#39;ve implemented a serial&#xA;&gt; filtering for btcwallet [1] for the use-case of rescanning after a seed&#xA;&gt; phrase import. Parallel filtering would help here, but also we don&#39;t yet&#xA;&gt; take advantage of batch querying for the filters themselves. This would&#xA;&gt; speed up the scanning by quite a bit.&#xA;&gt;&#xA;&gt; I really like the filtering model though, it really simplifies the code,&#xA;&gt; and we can leverage identical logic for btcd (which has RPCs to fetch the&#xA;&gt; filters) as well.&#xA;&gt;&#xA;&gt; [1]:&#xA;&gt; https://github.com/Roasbeef/btcwallet/blob/master/chain/neutrino.go#L180&#xA;&gt;&#xA;&gt; _______________________________________________ bitcoin-dev mailing list&#xA;&gt; bitcoin-dev at lists.linuxfoundation.org&#xA;&gt; https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev&#xA;&gt;&#xA;&gt;&#xA;-------------- next part --------------&#xA;An HTML attachment was scrubbed...&#xA;URL: &lt;http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180521/92b184b8/attachment.html&gt;</html></oembed>