Nostr notes by

📅 Original date posted:2019-04-03 📝 Original message:Thanks ...

2023-06-07T18:17:22Z

In reply to nevent1q…k4ha
_________________________

📅 Original date posted:2019-04-03
📝 Original message:Thanks James for the post.

I proposed a similar idea [1] back in 2016 with the difference of signing the UTXO-set hash in a gitian-ish way.

While the idea of UTXO-set-syncs are attractive, there are probably still significant downsides in usability (compared to models with less security), mainly:
* Assume the UTXO set is 6 weeks old (which seems a reasonable age for providing enough security) a peer using that snapshot would still require to download and verify ~6048 blocks (~7.9GB at 1.3MB blocks,… probably CPU-days on a phone)
* Do we semi-trust the peer that servers the UTXO set (compared to a block or tx which we can validate)? What channel to we use to serve the snapshot?

If the goal is to run a full node on a consumer device that is also been used for other CPU intense operations (like a phone, etc.), I’m not sure if this proposal will lead to a satisfactory user experience.

The longer I think around this problem, the more I lean towards accepting the fact that one need to use dedicated hardware in his own environment to perform a painless full validation.

/jonas

[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-February/012478.html

> Am 02.04.2019 um 22:43 schrieb James O'Beirne via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org>:
>
> Hi,
>
> I'd like to discuss assumeutxo, which is an appealing and simple
> optimization in the spirit of assumevalid[0].
>
> # Motivation
>
> To start a fully validating bitcoin client from scratch, that client currently
> needs to perform an initial block download. To the surprise of no one, IBD
> takes a linear amount time based on the length of the chain's history. For
> clients running on modest hardware under limited bandwidth constraints,
> say a mobile device, completing IBD takes a considerable amount of time
> and thus poses serious usability challenges.
>
> As a result, having fully validating clients run on such hardware is rare and
> basically unrealistic. Clients with even moderate resource constraints
> are encouraged to rely on the SPV trust model. Though we have promising
> improvements to existing SPV modes pending deployment[1], it's worth
> thinking about a mechanism that would allow such clients to use trust
> models closer to full validation.
>
> The subject of this mail is a proposal for a complementary alternative to SPV
> modes, and which is in the spirit of an existing default, `assumevalid`. It may
> help modest clients transact under a security model that closely resembles
> full validation within minutes instead of hours or days.
>
> # assumeutxo
>
> The basic idea is to allow nodes to initialize using a serialized version of the
> UTXO set rendered by another node at some predetermined height. The
> initializing node syncs the headers chain from the network, then obtains and
> loads one of these UTXO snapshots (i.e. a serialized version of the UTXO set
> bundled with the block header indicating its "base" and some other metadata).
>
> Based upon the snapshot, the node is able to quickly reconstruct its chainstate,
> and compares a hash of the resulting UTXO set to a preordained hash hard-coded
> in the software a la assumevalid. This all takes ~23 minutes, not accounting for
> download of the 3.2GB snapshot[2].
>
> The node then syncs to the network tip and afterwards begins a simultaneous
> background validation (i.e., a conventional IBD) up to the base height of the
> snapshot in order to achieve full validation. Crucially, even while the
> background validation is happening the node can validate incoming blocks and
> transact with the benefit of the full (assumed-valid) UTXO set.
>
> Snapshots could be obtained from multiple separate peers in the same manner as
> block download, but I haven't put much thought into this. In concept it doesn't
> matter too much where the snapshots come from since their validity is
> determined via content hash.
>
> # Security
>
> Obviously there are some security implications due consideration. While this
> proposal is in the spirit of assumevalid, practical attacks may become easier.
> Under assumevalid, a user can be tricked into transacting under a false history
> if an attacker convinces them to start bitcoind with a malicious `-assumevalid`
> parameter, sybils their node, and then feeds them a bogus chain encompassing
> all of the hard-coded checkpoints[3].
>
> The same attack is made easier in assumeutxo because, unlike in assumevalid,
> the attacker need not construct a valid PoW chain to get the victim's node into
> a false state; they simply need to get the user to accept a bad `-assumeutxo`
> parameter and then supply them an easily made UTXO snapshot containing, say, a
> false coin assignment.
>
> For this reason, I recommend that if we were to implement assumeutxo, we not
> allow its specification via commandline argument[4].
>
> Beyond this risk, I can't think of material differences in security relative to
> assumevalid, though I appeal to the list for help with this.
>
> # More fully validating clients
>
> A particularly exciting use-case for assumeutxo is the possibility of mobile
> devices functioning as fully validating nodes with access to the complete UTXO
> set (as an alternative to SPV models). The total resource burden needed to start a node
> from scratch based on a snapshot is, at time of writing, a ~(3.2GB
> + blocks_to_tip * 4MB) download and a few minutes of processing time, which sounds
> manageable for many mobile devices currently in use.
>
> A mobile user could initialize an assumed-valid bitcoin node within an hour,
> transact immediately, and complete a pruned full validation of their
> assumed-valid chain over the next few days, perhaps only doing the background
> IBD when their device has access to suitable high-bandwidth connections.
>
> If we end up implementing an accumulator-based UTXO scaling design[5][6] down
> the road, it's easy to imagine an analogous process that would allow very fast
> startup using an accumulator of a few kilobytes in lieu of a multi-GB snapshot.
>
> ---
>
> I've created a related issue at our Github repository here:
> https://github.com/bitcoin/bitcoin/issues/15605
>
> and have submitted a draft implementation of snapshot usage via RPC here:
> https://github.com/bitcoin/bitcoin/pull/15606
>
> I'd like to discuss here whether this is a good fit for Bitcoin conceptually. Concrete
> plans for deployment steps should be discussed in the Github issue, and after all
> that my implementation may be reviewed as a sketch of the specific software
> changes necessary.
>
> Regards,
> James
>
>
> [0]: https://bitcoincore.org/en/2017/03/08/release-0.14.0/#assumed-valid-blocks
> [1]: https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki
> [2]: as tested at height 569895, on a 12 core Intel Xeon Silver 4116 CPU @ 2.10GHz
> [3]: https://github.com/bitcoin/bitcoin/blob/84d0fdc/src/chainparams.cpp#L145-L161
> [4]: Marco Falke is due credit for this point
> [5]: utreexo: https://www.youtube.com/watch?v=edRun-6ubCc
> [6]: Boneh, Bunz, Fisch on accumulators: https://eprint.iacr.org/2018/1188
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190403/0005961e/attachment.sig>;

📅 Original date posted:2019-03-22 📝 Original message:Hi The ...

2023-06-07T18:17:11Z

In reply to nevent1q…70rg
_________________________

📅 Original date posted:2019-03-22
📝 Original message:Hi

The overhauled version of the former BIP151 has fundamental differences and deserves (requires?) a new BIP.
Calling it „v2 peer-to-peer message transport protocol“ is more accurate since it is no longer only about encryption.

The formatted draft proposal can be found here: https://gist.github.com/jonasschnelli/c530ea8421b8d0e80c51486325587c52

Significant changes compared to the current available BIP151
* A optimised AEAD construct is now proposed (ChaCha20Poly1305 at Bitcoin), reducing the required ChaCha20 rounds (compared to the openSSH version).
* introduce NODE_P2P_V2
* 32bytes-per-side „pseudorandom" key exchange
* the multi message envelope has been removed
* the length of a packet uses now a 3-byte integer with 23 available bits
* introduction of short-command-ID (ex.: uint8_t 13 == INV, etc.) which result in
some v2 messages require less bandwidth then v1
* the key derivation and what communication direction uses what key is now more
specific

First benchmarks of the used primitives
https://github.com/bitcoin/bitcoin/pull/15519#issuecomment-469705289 <https://github.com/bitcoin/bitcoin/pull/15519#issuecomment-469705289>;

Benchmark of the AEAD compared to the HASH (double SHA256)
(Indicates that v2 messages may be more performant):
https://github.com/bitcoin/bitcoin/pull/15649#issuecomment-475782376 <https://github.com/bitcoin/bitcoin/pull/15649#issuecomment-475782376>;

Proposal:

<pre>
BIP: ???
Layer: Peer Services
Title: Version 2 Peer-to-Peer Message Transport Protocol
Author: Jonas Schnelli <dev at jonasschnelli.ch>
Status: Draft
Type: Standards Track
Created: 2019-03-08
License: PD
</pre>

== Abstract ==

This BIP describes a new Bitcoin peer to peer transport protocol with
opportunistic encryption.

== Motivation ==

The current peer-to-peer protocol is partially inefficient and in plaintext.

With the current unencrypted message transport, BGP hijack, block delay attacks
and message tempering are inexpensive and can be executed in a covert way
(undetectable MITM)<ref>[https://btc-hijack.ethz.ch/files/btc_hijack.pdf
Hijacking Bitcoin: Routing Attacks on Cryptocurrencies - M. Apostolaki, A.
Zohar, L.Vanbever]</ref>.

Adding opportunistic encryption introduces a high risk for attackers of being
detected. Peer operators can compare encryption session IDs or use other form
of authentication schemes <ref
name="bip150">[https://github.com/bitcoin/bips/blob/master/bip-0150.mediawiki
BIP150]</ref> to identify an attack.

Each current version 1 Bitcoin peer-to-peer message uses a double-SHA256
checksum truncated to 4 bytes. Roughly the same amount of computation power
would be required for encrypting and authenticating a peer-to-peer message with
ChaCha20 & Poly1305.

Additionally, this BIP describes a way how data manipulation (blocking or
tempering commands by an intercepting TCP/IP node) would be identifiable by the
communicating peers.

Encrypting traffic between peers is already possible with VPN, tor, stunnel,
curveCP or any other encryption mechanism on a deeper OSI level, however, most
of those solutions require significant knowhow in how to setup such a secure
channel and are therefore not widely deployed.

== Specification ==

<blockquote>
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in RFC 2119<ref>[https://tools.ietf.org/html/rfc2119
RFC 2119]</ref>.
</blockquote>

A peer that supports the message transport protocol as defined in this proposal
MUST accept encryption requests from all peers.

Both communication direction share the same shared-secret but have different
symmetric cipher keys.

The encryption handshake MUST happen before sending any other messages to the
responding peer.

If the responding peer closes the connection after sending the handshake
request, the initiating peer MAY try to connect again with the v1 peer-to-peer
transport protocol. Such reconnects allow an attacker to "downgrade" the
encryption to plaintext communication and thus, accepting v1 connections MUST
not be done when the Bitcoin peer-to-peer network uses almost only v2
communication.

=== NODE_P2P_V2 ===

Peers supporting the transport protocol after this proposal MUST signal
<code>NODE_P2P_V2</code>
<pre>
NODE_P2P_V2 = (1 << 11)
</pre>

A peer usually learns an address along with the expected service flags which
MAY be used to filter possible outbound peers.

A peer signaling <code>NODE_P2P_V2</code> MUST accept encrypted communication
specified in this proposal.

Peers MAY only make outbound connections to peers supporting
<code>NODE_P2P_V2</code>.

=== Handshake ===

<pre>
----------------------------------------------------------------------------------------
| Initiator Responder |
| |
| x, X := SECP256k1_KEYGEN() |
| CLIENT_HDATA := X |
| |
| --- CLIENT_HDATA ---> |
| |
| y, Y := SECP256k1_KEYGEN() |
| ECDH_KEY := SECP256k1_ECDH(X,y) |
| SERVER_HDATA := Y |
| |
| <-- SERVER_HDATA ---- |
| |
| ECDH_KEY := SECP256k1_ECDH(x,Y) |
----------------------------------------------------------------------------------------
</pre>

To request encrypted communication (only possible if yet no other messages have
been sent or received), the initiating peer generates an EC secp256k1 ephemeral
key and sends the corresponding 32-byte public key to the responding peer and
waits for the remote 32-byte public key from the counterparty.

ODD secp256k1 public keys MUST be used (public keys starting with 0x02). If the
public key from the generated ephemeral key is an EVEN public key (starting
with 0x03), negating the key and recalculating its public key SHOULD be done.
Only using ODD public makes it more complex to identify the handshake based on
analyzing the traffic.

The handshake request and response message are raw 32byte payloads containing
no header, length or checksum (the pure 32byte payload) and MUST be sent before
anything else.

Public keys starting with the 4-byte network magic are forbidden and MUST lead
to locally re-generate an ephemeral-key.

Pseudocode for the ephemeral-key generation
<pre>
do {
ecdh_key.MakeNewKey();
if (ecdh_key.GetPubKey()[0] == 3) {
ecdh_key.Negate();
}
} while (m_ecdh_key.GetPubKey()[0..3] == NETWORK_MAGIC);
</pre>

Once a peer has received the public key from its counterparty, the shared
secret MUST be calculated by using secp256k1 ECDH.

Private keys will never be transmitted. The shared secret can only be
calculated if an attacker knows at least one private key and the counterparties
public key. This key-exchange is based on the discrete log problem and thus not
sufficiently strong against known forms of possible quantum computer
algorithms. Adding an additional quantum resistant key exchange like NewHope is
possible but out of scope for this proposal.

After a successful handshake, the messages format MUST use the "v2 messages
structure". Non-encrypted v1 messages from the initiating peer MUST lead to an
immediate connection termination.

After a successful handshake, both peers MUST cleanse the ephemeral-session-key
from memory and/or persistence storage.

A peer not supporting this proposal will not perform the described handshake
and thus send a v1 version message.
Peers supporting this BIP MAY optionally allow unencrypted v1 communication by
detecting a v1 version message by the initial 11-byte sequence of <code>4byte
net magic || "version"</code>.

=== Symmetric Encryption Cipher Keys ===

Once the ECDH secret (<code>ECDH_KEY</code>) is calculated on each side, the
symmetric encryption cipher keys MUST be derived with HKDF
<ref>[https://tools.ietf.org/html/rfc5869 HKDF (RFC 5869)]</ref> after the
following specification:

1. HKDF extraction
<code>PRK = HKDF_EXTRACT(hash=SHA256, salt="BitcoinSharedSecret||INITIATOR_32BYTES_PUBKEY||RESPONDER_32BYTES_PUBKEY", ikm=ECDH_KEY)</code>.

2. Derive Key_1_A (K_1 communication direction A)
<code>K1A = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_1_A", L=32)</code>

2. Derive Key_2_A (K_2 communication direction A)
<code>K1B = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_2_A", L=32)</code>

3. Derive Key_1_B (K_1 communication direction B)
<code>K2 = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_1_B", L=32)</code>

3. Derive Key_2_B (K_2 communication direction B)
<code>K2 = HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinK_2_B", L=32)</code>

=== Session ID ===

Both parties MUST also calculate the 256bit session-id using <code>SID =
HKDF_EXPAND(prk=PRK, hash=SHA256, info="BitcoinSessionID", L=32)</code>. The
session-id can be used for authenticating the encryption-session (identity
check).

The session-id MUST be presented to the user on request.

=== ChaCha20-Poly1305 at Bitcoin Cipher Suite ===

==== Background ====

ChaCha20 is a stream cipher designed by Daniel Bernstein and described in
<ref>[http://cr.yp.to/chacha/chacha-20080128.pdf ChaCha20]</ref>. It operates
by permuting 128 fixed bits, 128 or 256 bits of key, a 64 bit nonce and a 64
bit counter into 64 bytes of output. This output is used as a keystream, with
any unused bytes simply discarded.

Poly1305 <ref>[http://cr.yp.to/mac/poly1305-20050329.pdf Poly1305]</ref>, also
by Daniel Bernstein, is a one-time Carter-Wegman MAC that computes a 128 bit
integrity tag given a message and a single-use 256 bit secret key.

The chacha20-poly1305 at bitcoin combines these two primitives into an
authenticated encryption mode. The construction used is based on that proposed
for TLS by Adam Langley in
<ref>[http://tools.ietf.org/html/draft-agl-tls-chacha20poly1305-03 "ChaCha20
and Poly1305 based Cipher Suites for TLS", Adam Langley]</ref>, but differs in
the layout of data passed to the MAC and in the addition of encryption of the
packet lengths.

==== Detailed Construction ====

The chacha20-poly1305 at bitcoin cipher requires two 256 bits of key material as
output from the key exchange. Each key (K_1 and K_2) are used by two separate
instances of chacha20.

The instance keyed by K_1 is a stream cipher that is used only to encrypt the 3
byte packet length field and has its own sequence number. The second instance,
keyed by K_2, is used in conjunction with poly1305 to build an AEAD
(Authenticated Encryption with Associated Data) that is used to encrypt and
authenticate the entire packet.

Two separate cipher instances are used here so as to keep the packet lengths
confidential but not create an oracle for the packet payload cipher by
decrypting and using the packet length prior to checking the MAC. By using an
independently-keyed cipher instance to encrypt the length, an active attacker
seeking to exploit the packet input handling as a decryption oracle can learn
nothing about the payload contents or its MAC (assuming key derivation,
ChaCha20 and Poly1305 are secure).

The AEAD is constructed as follows: for each packet, generate a Poly1305 key by
taking the first 256 bits of ChaCha20 stream output generated using K_2, an IV
consisting of the packet sequence number encoded as an LE uint64 and a ChaCha20
block counter of zero. The K_2 ChaCha20 block counter is then set to the
little-endian encoding of 1 (i.e. {1, 0, 0, 0, 0, 0, 0, 0}) and this instance
is used for encryption of the packet payload.

==== Packet Handling ====

When receiving a packet, the length must be decrypted first. When 3 bytes of
ciphertext length have been received, they may be decrypted.

A ChaCha20 round always calculates 64bytes which is sufficient to crypt 21
times a 3 bytes length field (21*3 = 63). The length field sequence number can
thus be used 21 times (keystream caching).

The length field must be enc-/decrypted with the ChaCha20 keystream keyed with
K_1 defined by block counter 0, the length field sequence number in little
endian and a keystream position from 0 to 60.

Pseudo code example:
<pre>
// init
sequence_nr_payload = 0; //payload sequence number
sequence_nr_length_field = 0; //length field sequence number (will be reused)
aad_length_field_pos = 0; //position in the length field cipher instance keystream chunk

...

// actual encryption
if cache_length_field_sequence_number != sequence_nr_length_field {
cache_keystream_64_bytes = ChaCha20(key=K_1, iv=little_endian(sequence_nr_length_field), counter=0);
cache_length_field_sequence_number = sequence_nr_length_field
}
packet_length = XOR_TO_LE(cache_length_field_sequence_number[aad_length_field_pos - aad_length_field_pos+3], ciphertext[0-3])

sequence_nr_payload++;
aad_length_field_pos += 3; //skip 3 bytes in keystream
if (aad_length_field_pos + 3 > 64) { //if we are outside of the 64byte keystream...
aad_length_field_pos = 0; // reset at position 0
sequence_nr_length_field++; // increase length field sequence number
}
</pre>

Once the entire packet has been received, the MAC MUST be checked before
decryption. A per-packet Poly1305 key is generated as described above and the
MAC tag calculated using Poly1305 with this key over the ciphertext of the
packet length and the payload together. The calculated MAC is then compared in
constant time with the one appended to the packet and the packet decrypted
using ChaCha20 as described above (with K_2, the packet sequence number as
nonce and a starting block counter of 1).

Detection of an invalid MAC MUST lead to immediate connection termination.

To send a packet, first encode the 3 byte length and encrypt it using K_1 as
described above. Encrypt the packet payload (using K_2) and append it to the
encrypted length. Finally, calculate a MAC tag and append it.

The initiating peer MUST use <code>K_1_A, K_2_A</code> to encrypt messages on
the send channel, <code>K_1_B, K_2_B</code> MUST be used to decrypt messages on
the receive channel.

The responding peer MUST use <code>K_1_A, K_2_A</code> to decrypt messages on
the receive channel, <code>K_1_B, K_2_B</code> MUST be used to encrypt messages
on the send channel.

Optimized implementations of ChaCha20-Poly1305 at bitcoin are relatively fast in
general, therefore it is very likely that encrypted messages require not more
CPU cycles per bytes then the current unencrypted p2p message format
(ChaCha20/Poly1305 versus double SHA256).

The initial packet sequence numbers are 0.

K_2 ChaCha20 cipher instance (payload) must never reuse a {key, nonce} for
encryption nor may it be used to encrypt more than 2^70 bytes under the same
{key, nonce}.

K_1 ChaCha20 cipher instance (length field/AAD) must never reuse a {key, nonce,
position-in-keystream} for encryption nor may it be used to encrypt more than
2^70 bytes under the same {key, nonce}.

We use message sequence numbers for both communication directions.

<pre>
------------------------------------------------------------------------------------------
| Initiator Responder |
| |
| AEAD() = ChaCha20Poly1305Bitcoin() |
| MSG_A_CIPH = AEAD(k=K_1_A, K_2_A, payload_nonce=0, aad_nonce=0, aad_pos=0, msg) |
| |
| --- MSG_CIPH ---> |
| |
| msg := AEAD(k=K_1_A,K_2_A, n=0, ..., MSG_A_CIPH) |
| |
------------------------------------------------------------------------------------------
</pre>

==== Test Vectors ====

<pre>
message 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
k1 (DATA) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
k2 (AAD) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

AAD keystream
76 b8 e0 ad a0 f1 3d 90 40 5d 6a e5 53 86 bd 28 bd d2 19 b8 a0 8d ed 1a a8 36 ef cc 8b 77 0d c7 da 41 59 7c 51 57 48 8d 77 24 e0 3f b8 d8 4a 37 6a 43 b8 f4 15 18 a1 1c c3 87 b6 69 b2 ee 65 86

ciphertext
76 b8 e0 9f 07 e7 be 55 51 38 7a 98 ba 97 7c 73 2d 08 0d cb 0f 29 a0 48 e3 65 69 12 c6 53 3e 32

MAC
d2 fc 11 82 9c 1b 6c 1d f1 f5 51 cd 61 31 ff 08
</pre>

<pre>
message 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
k1 (DATA) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
k2 (AAD) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

AAD keystream
76 b8 e0 ad a0 f1 3d 90 40 5d 6a e5 53 86 bd 28 bd d2 19 b8 a0 8d ed 1a a8 36 ef cc 8b 77 0d c7 da 41 59 7c 51 57 48 8d 77 24 e0 3f b8 d8 4a 37 6a 43 b8 f4 15 18 a1 1c c3 87 b6 69 b2 ee 65 86

ciphertext
77 b8 e0 9f 07 e7 be 55 51 38 7a 98 ba 97 7c 73 2d 08 0d cb 0f 29 a0 48 e3 65 69 12 c6 53 3e 32

MAC
ba f0 c8 5b 6d ff 86 02 b0 6c f5 2a 6a ef c6 2e
</pre>

<pre>
message
ff 00 00 f1 95 e6 69 82 10 5f fb 64 0b b7 75 7f 57 9d a3 16 02 fc 93 ec 01 ac 56 f8 5a c3 c1 34 a4 54 7b 73 3b 46 41 30 42 c9 44 00 49 17 69 05 d3 be 59 ea 1c 53 f1 59 16 15 5c 2b e8 24 1a 38 00 8b 9a 26 bc 35 94 1e 24 44 17 7c 8a de 66 89 de 95 26 49 86 d9 58 89 fb 60 e8 46 29 c9 bd 9a 5a cb 1c c1 18 be 56 3e b9 b3 a4 a4 72 f8 2e 09 a7 e7 78 49 2b 56 2e f7 13 0e 88 df e0 31 c7 9d b9 d4 f7 c7 a8 99 15 1b 9a 47 50 32 b6 3f c3 85 24 5f e0 54 e3 dd 5a 97 a5 f5 76 fe 06 40 25 d3 ce 04 2c 56 6a b2 c5 07 b1 38 db 85 3e 3d 69 59 66 09 96 54 6c c9 c4 a6 ea fd c7 77 c0 40 d7 0e af 46 f7 6d ad 39 79 e5 c5 36 0c 33 17 16 6a 1c 89 4c 94 a3 71 87 6a 94 df 76 28 fe 4e aa f2 cc b2 7d 5a aa e0 ad 7a d0 f9 d4 b6 ad 3b 54 09 87 46 d4 52 4d 38 40 7a 6d eb 3a b7 8f ab 78 c9

k1 (DATA) 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f
k2 (AAD) ff 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f

AAD keystream
c6 40 c1 71 1e 3e e9 04 ac 35 c5 7a b9 79 1c 8a 1c 40 86 03 a9 0b 77 a8 3b 54 f6 c8 44 cb 4b 06 d9 4e 7f c6 c8 00 e1 65 ac d6 61 47 e8 0e c4 5a 56 7f 6c e6 6d 05 ec 0c ae 67 9d ce eb 89 00 17

ciphertext
39 40 c1 e9 2d a4 58 2f f6 f9 2a 77 6a eb 14 d0 14 d3 84 ee b3 0f 66 0d ac f7 0a 14 a2 3f d3 1e 91 21 27 01 33 4e 2c e1 ac f5 19 9d c8 4f 4d 61 dd be 65 71 bc a5 af 87 4b 4c 92 26 c2 6e 65 09 95 d1 57 64 4e 18 48 b9 6e d6 c2 10 2d 54 89 a0 50 e7 1d 29 a5 a6 6e ce 11 de 5f b5 c9 55 8d 54 da 28 fe 45 b0 bc 4d b4 e5 b8 80 30 bf c4 a3 52 b4 b7 06 8e cc f6 56 ba e7 ad 6a 35 61 53 15 fc 7c 49 d4 20 03 88 d5 ec a6 7c 2e 82 2e 06 93 36 c6 9b 40 db 67 e0 f3 c8 12 09 c5 0f 32 16 a4 b8 9f b3 ae 1b 98 4b 78 51 a2 ec 6f 68 ab 12 b1 01 ab 12 0e 1e a7 31 3b b9 3b 5a 0f 71 18 5c 7f ea 01 7d db 92 76 98 61 c2 9d ba 4f bc 43 22 80 d5 df f2 1b 36 d1 c4 c7 90 12 8b 22 69 99 50 bb 18 bf 74 c4 48 cd fe 54 7d 8e d4 f6 57 d8 00 5f dc 0c d7 a0 50 c2 d4 60 50 a4 4c 43 76 35 58 58

MAC
98 1f be 8b 18 42 88 27 6e 7a 93 ea bc 89 9c 4a
</pre>

=== v2 Messages Structure ===

{|class="wikitable"
! Field Size !! Description !! Data type !! Comments
|-
| 3 || length & flag || 23 + 1 bits || Encrypted length of ciphertext payload (not counting the MAC tag) in number of bytes (only 2^23 is usable, most significant bit is the rekey-flag)
|-
| 1-13 || encrypted command || variable || ASCII command (or one byte short command ID)
|-
| ? || encrypted payload || ? || The actual data
|-
| 16 || MAC tag || ? || 128bit MAC-tag
|}

Encrypted messages do not have the 4byte network magic.

The maximum message size is 2^23 (8’388’608) bytes. Future communication MAY
exceed this limit and thus MUST be split into different messages.

Decrypting and processing the message before the authentication succeeds (MAC
verified) MUST not be done.

The 4byte sha256 checksum is no longer required because the AEAD (MAC).

Both peers MUST keep track of the message sequence number (uint32) of sent and
received messages for building a 64-bit symmetric cipher IV.

The command field MUST start with a byte that defines the length of the ASCII
command string up to 12 chars (1 to 12) or a short command ID (see below).

==== Short Command ID ====

To save valuable bandwidth, the v2 message format supports message command
short IDs for message types with high frequency. The ID/string mapping is a
peer to peer arrangement and MAY be negotiated between the initiating and
responding peer. A peer conforming to this proposal MUST support short IDs
based on the table below and SHOULD use short command IDs for outgoing messages.

{|class="wikitable"
! Number !! Command
|-
| 13 || INV
|-
| 14 || HEADERS
|-
| 15 || PING
|-
| 16 || PONG
|-
|}

==== Length comparisons between v1 and v2 messages ====

<pre>
v1 in: 4(Magic)+12(Command)+4(MessageSize)+4(Checksum)+36(Payload) == 60
v2 inv: 3(MessageSize&Flag)+1(Command)+36(Payload)+16(MAC) == 56
(93.33%)
</pre>

<pre>
v1 ping: 4(Magic)+12(Command)+4(MessageSize)+4(Checksum)+8(Payload) == 32
v2 pong: 3(MessageSize&Flag)+1(Command)+8(Payload)+16(MAC) == 28
(87.5%)
</pre>

<pre>
v1 block: 4(Magic)+12(Command)+4(MessageSize)+4(Checksum)+1’048’576(Payload) = 1’048’600
v2 block: 3(MessageSize&Flag)+6(CommandStr)+8(Payload)+16(MAC) == 28 = 1’048’601
(100.000095%)
</pre>

=== Re-Keying ===

Re-keying can be signaled by setting the most significant bit in the length
field before encryption. A peer signaling a rekey MUST use the next key for
encryption messages AFTER the message where the signaling has been done.

A peer identifying a rekey by checking the most significant bit in the envelope
length must use the next key for decrypt messages AFTER the message where the
signaling has been detected.

The next symmetric cipher key MUST be calculated by <code>SHA256(SHA256(session
ID || old_symmetric_cipher_key))</code> and the packet sequence number of the
according encryption direction must be set to 0.

Re-Keying interval is a peer policy with a minimum timespan of 10 seconds.

The Re-Keying must be done after every 1GB of data sent (recommended by RFC4253
SSH Transport) or if the last rekey was more than an hour ago.

Peers calculate the counterparty limits and MUST disconnect immediately if a
violation of the limits has been detected.

=== Risks ===

The encryption does not include an authentication scheme. This BIP does not
cover a proposal to avoid MITM attacks during the encryption initialization.
However, peers MUST show the session-id to the user on request which allows to
identify a MITM by a manual verification on a secure channel.

Optional authentication schemes may be covered by other proposals <ref
name="bip150">[https://github.com/bitcoin/bips/blob/master/bip-0150.mediawiki
BIP150]</ref>.

An attacker could delay or halt v2 protocol enforcement by providing a
reasonable amount of peers not supporting the v2 protocol.

== Compatibility ==

This proposal is backward compatible (as long as not enforced). Non-supporting
peers can still use unencrypted communications.

== Reference implementation ==
* Complete Bitcoin Core implementation: https://github.com/bitcoin/bitcoin/pull/14032
* Reference implementation of the AEAD in C: https://github.com/jonasschnelli/chacha20poly1305

== References ==

<references/>

== Acknowledgements ==
* Pieter Wuille and Gregory Maxwell for most of the ideas in this BIP.
* Tim Ruffing for the review and the hint for the enhancement of the symmetric
key derivation

== Copyright ==
This work is placed in the public domain.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190322/307267fd/attachment-0001.html>;
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20190322/307267fd/attachment-0001.sig>;

📅 Original date posted:2018-09-07 📝 Original message:Hi Tim ...

2023-06-07T18:14:34Z

In reply to nevent1q…k3rx
_________________________

📅 Original date posted:2018-09-07
📝 Original message:Hi Tim

Thanks for the feedback.

I agree with all of Gregs answers.

> key. Together with the encrypted packet lengths, the entire data stream looks like random then,
> which is pretty useful against censorship resistance for example. (The only exception is that the
> stream will never start with the magic bytes.)

All-or-none censorship attacks are out of scope for BIP151.
We won’t achieve DPI robustness in this proposal and I think it should not be part of the p2p protocol.

I think all-or-one censorship situations require an additional layer like TOR with OBFS4 (where AFAIK Eligator is used).
Eventually Core does directly support non-tor routed pluggable transports (it's partially already possible via SOCK proxy, but not on a gossip and plugin-launch level).

This does not exclude that we should obfuscate the key exchange as good as we can without blowing up the implementation too much.

The proposed encryption adds a robustness to the thread model with very little costs and low risks.

> "salt = BitcoinSharedSecret||INITIATOR_PUBKEY||RESPONDER_PUBKEY" should just avoid this issue.

This is a good point and I’d like to see more concrete examples how this (the non dynamic salt) could be exploited.

> Re-keying
> =========
> The problem with signalling re-keying in the length field is that the length field is not covered
> by the MAC. So the attacker can flip the signalling bit. The resulting protocol is probably still
> secure but the malleability is certainly not desirable.

In ChaCha20Poly1305 at openssh, the length field is AAD, encrypted with a different key and part of the MAC.

>
> Deterministic rekeying rules may be better. Otherwise there will be implementations that rekey
> every 10 seconds and implementations that just don't rekey at all (rendering the 10 s rekeying
> interval in the opposite direction useless). Different policies also make it possible to
> fingerprint implementations. Another problem is that people will set their policies arbitrarily.
> What's better: 5 min or 30 min? I don't know, but both are reasonable choices. (Thats's very much
> like discussions about ciphers... What's better AES-GCM or ChaCha20/Poly1305? I don't know, but
> again both are reasonable choices.)

The Rekey cost is two times a double-SHA256,… the costs of a rekey is similar to one or two v1 INV message creations.

>
> Symmetric crypto
> ================
> You call it chacha20-poly1305 at bitcoin but what's the difference to the openssh then? Is the
> idea to save a call to chacha here as you mentioned?
>
> I didn't think about this in detail: maybe there are a few meaningful cases where padding could
> hide the message length without too much overhead. (I'm not convinced, just a random thought.)

I think a new message type that could contain message + pad would be trivial.
Would this again be to obfuscate traffic patterns? Anti DPI is not the scope of BIP151.

>
> Misc
> ====
> "The ID/string mapping is a peer to peer arrangement and MAY be negotiated between the
> requesting and responding peer." I think that's overly complicated. I suggest it should just be
> written in stone, again to avoid complexity and to avoid fingerprinting. New implementations are
> necessary anyway, so maybe just use IDs for anything? ASCII is nice if you want to debug your code
> or some random network failure but that's hard anyway when encryption is used.

I wanted to avoid too much central planing here and only cover the ones where it's most efficient (small messages that are used often).
The ASCII commands are in itself somehow pseude-robust against collision.
For a 1MB block message, using a 1-byte short ID (rather then a 6-byte ASCII command) would reduce the bandwidth requirement insignificant (99.99952%).

If we would always have used short IDs in the past, there could have been a collision between XTIN, compact, sendheaders or so.

>
> In general, the entire thing is a little bit underspecified. (I'm aware it's just a draft.)
> A few examples:
> - What should a peer do if the MAC verification fails?
> - What should a peer do if it receives an even key?
> - "Processing the message before the authentication succeeds (MAC verified) MUST not be done."
> That should also apply to the ciphertext. (Or: What is a "message"?). It may be a good idea to
> to refer to the openssh document or steal from it; it does a pretty good job.
> - "Both peers MUST keep track of the message sequence number (uint32) of sent and received
> messages for building a 64-bit symmetric cipher IV." I think you mean nonce when you say IV?
> - What is the initial value of the sequence number?

Good points. Will make them more clear in the BIP.
I was under the false impression that it is obvious to disconnect in those cases.

> - How is a 64-bit nonce formed from one (two?) uint32?

That’s specified in ChaCha20Poly1305 at openssh ("a nonce consisting of the packet sequence number encoded as a uint64“).
But I’ll specified that more clear.

> - What if the uint32 overflows?

The max data before rekey is 1GB, AFAIK it is impossible to overflow.

> - "Re-Keying interval is a peer policy with a minimum timespan of 10 seconds." What if I receive
> too many re-keying requests? Nothing or should I raise the DoS score?

Current implementation proposal does a disconnect. With the risk of fingerprinting options, I think we can leave this open to the implementation?

> - "The Re-Keying must be done after every 1GB of data sent or received" Hm, every peer updates its
> own sending key, so this should just read "sent" instead of "sent or received“?

Yes. Should probably be „sent“,… and eventually a paragraph that states that a peer should disconnect if the remote peer did not rekey within that limit.

> Pseudocode could probably help here.

Agree. Will try to add.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180907/348e23e8/attachment.sig>;

📅 Original date posted:2018-09-03 📝 Original message:Hi ...

2023-06-07T18:14:32Z

In reply to nevent1q…j4k7
_________________________

📅 Original date posted:2018-09-03
📝 Original message:Hi

During work on the implementation of BIP151 [1] I figured out that the current
published proposal could be further optimized.

I wrote an overhauled BIP151 specification with some – partially radical –
changes.

Now it’s unclear to me if this should be published under a new BIP nr. or if it
is acceptable to change the existing 151 proposal.
If a new BIP number would be required, I think withdrawing BIP151 should be
done (which somehow indicates we should alter 151).

The only BIP151 implementation I’m aware of is the one from Armory [2].
BCoins implementation has been removed [3].

The new proposal draft is available here:
https://gist.github.com/jonasschnelli/c530ea8421b8d0e80c51486325587c52

Major changes
=============
- the encryption handshake no longer requires the v1 protocol, it’s a pure
32bytes-per-side „pseudorandom" key exchange that happens before anything else.
- the multi message envelope has been removed.
- a new NODE_ENCRYPTED service bit
- the key derivation and what communication direction uses what key is now more
specific
- the length of a packet uses now a 3-byte integer with 23 available bits
- introduction of short-command-ID (ex.: uint8_t 13 == INV, etc.) which result in
some v2 messages require less bandwidth then v1
- rekeying doesn’t require a message and can be signaled in the most
significant bit in the packet-size field

Points that are in discussion and may be added to the BIP (or to a new one):

Hybrid NewHope key exchange
===========================
The current ECDH key exchange is vulnerable to Shor’s algorithm and is thus not
considered quantum-safe.
Following TORs approach [4] by adding a NewHope [5] key-exchange the handshake
protocol would very likely make the encryption PQ safe with little costs.
There is also a straight forward implementation [6] from the NewHope team that
has been submitted to NIST PQC project.

Inefficiency of ChaCha20Poly1305 at openssh
========================================
The proposed AEAD could eventually be further optimized.
ChaCha20Poly1305 at openssh uses at least three rounds of ChaCha20 which
eventually can be reduced to two (messages below <=64 bytes [inv, ping,
pong,...] only require one round of ChaCha20, but two for the Poly1305 key and
the message length encryption where the Poly1305 key chacha round „throws away“
32 bytes).

I would suggest that we don’t rehash discussions about the general
concept of encrypting the traffic. This has already been discussed [7][8].

I hope we can limit this thread to discuss further ideas for optimisation as well as
technical details of the published proposal or its implementation.

[1] https://github.com/bitcoin/bitcoin/pull/14032
[2] https://github.com/goatpig/BitcoinArmory/pull/510
[3] https://github.com/bcoin-org/bcoin/commit/41af7acfd68b0492a6442865afd439300708e662
[4] https://gitweb.torproject.org/user/isis/torspec.git/plain/proposals/XXX-newhope-hybrid-handshake.txt?h=draft/newhope
[5] https://eprint.iacr.org/2015/1092
[6] https://github.com/newhopecrypto/newhope

[7] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-February/013565.html
[8] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-June/012826.html

Thanks
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180903/82a75538/attachment-0001.sig>;

📅 Original date posted:2018-06-19 📝 Original message:I ...

2023-06-07T18:13:04Z

In reply to nevent1q…0f98
_________________________

📅 Original date posted:2018-06-19
📝 Original message:I agree with matejcik’s point 1 to 3 and especially with point 4.
The mandatory flag (or optional-flag) makes much sense to me.

> -----------
>
> In general, the standard is trying to be very space-conservative,
> however is that really necessary? We would argue for clarity and ease of
> use over space constraints. We think more straightforward approach is
> desired, although more space demanding. What are the arguments to make
> this as small as possible? If we understand correctly, this format is
> not intended for blockchain nor for persistent storage, so size doesn’t
> matter nearly as much.

I don’t see any reasons why space would be an issue.

HWWs probably can’t handle PBST natively since it is not optimised for
presenting various informations in a signing-verification.

A single stream-in of a PSBT through USB (or similar channel) will not work in
many cases since HWW come often with very restrictive RAM constraints.

Furthermore, I forget to mention in my last mail, that registering (or defining)
a mime-type for PSBT would probably a great usability feature.
(Send PSBT by email/messanger and with dbl-click to open feature, etc.)

/jonas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180619/c263f67b/attachment.sig>;

📅 Original date posted:2018-06-19 📝 Original message:> * ...

2023-06-07T18:13:03Z

In reply to nevent1q…vy4f
_________________________

📅 Original date posted:2018-06-19
📝 Original message:> * Key-value map model or set model.
> * Ability for Combiners to verify two PSBT are for the same transaction
> * Optional signing
> * Derivation from xpub or fingerprint
> * Generic key offset derivation
> * Hex encoding?

I think all of Pieters points are valid and reasonable thought, though I’m unsure if it would be worth changing the existing-implementation-breaking things like the k/v set model.
AFAIK things like non-hex-encoding or generic key offset derivation are extensions and would not break existing implementations.

Further thoughts on BIP174 from my side.

Key derivation in multisig:
From my understanding, the signers and the creator must have agreed – in advance to the PSBT use case – on a key derivation scheme.
BIP32 derivation is assumed, but may not always be the case.
Sharing xpubs (the chaincode) may be a concern in non-trust-relationships between signer(s) and the creator (regarding Pieters xpub/fingerprint concerns).
Providing the type 0x03, the bip32 derivation path is one form of a support to faster (or computational possible) derivation of the required keys for signing a particular input.
From my point of view, it is a support of additional metadata shared between creator and signer and provided from the creator to the signer for faster (or computation possible) key deviation.

I think it could be more flexible (generic) in BIP174.
It could be just a single child key {32-bit int}, or just a keypath ({32-bit int}]{32-bit int}…) which is very likely sufficient for a HWW to derive the relevant key without the creation of a lookup-window or other „maps".
It could even be an enciphered payload which was shared during address/redeem-script generation and „loops“ back during a signing request.

Maybe I’m overcomplicating things, but for practical multisig with HWWs, a simple BIP32-child-key-index or BIP32-keypath derivation support field should be sufficient.
A generic „derivation support field“, provided from the signer to the creator during address-generation that just „loops“ back during the PSBT use-cases is probably a overkill.

Thanks
—
/jonas

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180619/e76fa257/attachment-0001.sig>;

📅 Original date posted:2018-06-03 📝 Original message:> I ...

2023-06-07T18:12:47Z

In reply to nevent1q…tlta
_________________________

📅 Original date posted:2018-06-03
📝 Original message:> I have some concerns about the use of Bech32. It is designed for
> detecting 3 errors up to length 1023 (but is then picked specifically
> to support 4 errors up to length 89). However, for error correction
> this translates to just being able to efficiently correct 1 error
> (3/2, rounded down) up to length 1023. You can of course always try
> all combinations of up to N changes to the input (for any N), testing
> the checksum, and comparing the results against the UTXO set or other
> wallet information that may have been recovered. However, the checksum
> at best gives you a small constant speedup here, not a fundamentally
> improved way for recovery.

Thanks Peter

I removed the part in the proposals that made false claims about the error
correction or cpu-intense key recovery.

I wrote some test code and figured out that my Core i7 machine can
do 31’775 operations per seconds of a addr-derivation-comparison
(bech32 decode, bip32 ckd, hash160, Base58check).
This is non-optimized code running non-parallelized.

Just in case someone wants to do more math here.

Without knowing to much about BCHs, ideally there would be a code that
includes the fact that computational costs for error correction can be very
high during a disaster recovery and that we can probably assume that the
user can provide a derivation element like a used address or pubkey.

Deriving one million child keys and comparing them against an address
table will take less than a minute on consumer systems.

> * correct 7 errors = 26 checksum characters (~ length * 1.25)
>
> So it really boils down to a trade-off between length of the code, and
> recovery properties.

I think 5% error correction (7 errors at 555bits) with a 26 char checksum is
probably an acceptable tradeoff.

Resulting string with 26 checksum chars (mockup):
xp1qqqqqq8z4rsgv54z9a92yla4m2yrsqdlwdl7gn6qldvwkuh3zrg66z8ad2snf832tgaxcuv3kmwugzl5x8wtnkj2q3a03ky0kg8p7dvv4czpjqgvv4zgnvv4zgnvv4zgnvv4zgngn
(140 chars)

Versus the bech32 (6 char checksum):
xp1qqqqqq8z4rsgv54z9a92yla4m2yrsqdlwdl7gn6qldvwkuh3zrg66z8ad2snf832tgaxcuv3kmwugzl5x8wtnkj2q3a03ky0kg8p7dvv4czpjqgvv4zgn
(120 chars)

Versus an xpriv:
xprv9wHokC2KXdTSpEepFcu53hMDUHYfAtTaLEJEMyxBPAMf78hJg17WhL5FyeDUQH5KWmGjGgEb2j74gsZqgupWpPbZgP6uFmP8MYEy5BNbyET
(111 chars)

Not sure if the additional 20 characters make the UX worse.
Typing in +20 chars in a disaster recovery is probably acceptable.

> If there is interest, I can construct a code + implementation for any
> of these in a few days probably, once the requirements are clear.

Yes. Please.
Lets first wait for more feedback about the error robustness though.

Thanks
-
Jonas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180603/2669b658/attachment.sig>;

📅 Original date posted:2018-06-03 📝 Original message:Hi The ...

2023-06-07T18:12:47Z

In reply to nevent1q…2r9u
_________________________

📅 Original date posted:2018-06-03
📝 Original message:Hi

The BIP proposal is now available here:
https://gist.github.com/jonasschnelli/68a2a5a5a5b796dc9992f432e794d719

Reference C code is available here:
https://github.com/jonasschnelli/bech32_keys

Feedback, criticism, etc. welcome!

Thanks
—
Jonas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180603/32ff5291/attachment.sig>;

📅 Original date posted:2018-05-30 📝 Original message:Hi ...

2023-06-07T18:12:32Z

In reply to nevent1q…7jun
_________________________

📅 Original date posted:2018-05-30
📝 Original message:Hi

> - Visually Comparing two keys to find if they are same (Important)
> - Different wallet software could set different birthday/gap limit. creating different xpub/xprv for the same set of mathematically derived individual keys. This removes the decoupling between key and wallet metadata

What would be the downside of encoding the same key with different metadata (resulting in different "visual strings“)?
If you import it into the same software, it would be trivial to detect it. If you import it into another software, it probably doesn’t matter.

Visual comparing is eventually a broken concept (agree with Greg) and I doubt that this property is important, and IMHO basic metadata seems more important then this - very likely irrelevant - visual property.

Also, I think a recovery based on a sole xpriv (or + limited amount of meta-data as described in this proposal) is a disaster recovery (or forensic recovery).

Long term, I would wish, if wallet-metadata including transaction based user metadata would be backed up - after encrypted with a key that can be derived from the seed - in a way, where you need the seed to recover that backup thus it can be stored in cheap, insecure spaces.

>
> In fact, same could be argued to add birthday to WIF private key format to let wallet discover funds faster.
>

The proposal I made can be seen as a replacement for WIF (it can replace WIF and xpriv/xpub) since it can encode a single private key into 275bits (still pretty short Bech32 string).

/jonas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180530/ff430ac7/attachment.sig>;

📅 Original date posted:2018-05-29 📝 Original message:Hi ...

2023-06-07T18:12:30Z

In reply to nevent1q…plcu
_________________________

📅 Original date posted:2018-05-29
📝 Original message:Hi

Extended public and private keys are defined in BIP32 [1].

Encoded extended private keys should not be confused with a wallet „seed“
(proposals like BIP39) while they can also partially serve the purpose to
„seed“ a wallet (there may be an overlap in the use-case).

Recovering a wallet by its extended private master key (xpriv; may or may not
be at depth 0) is a complex task with risks of failing to recover all available
funds.

It may be reasonable to consider that recovering a wallet purely based on the
existence of an extended private master key is a forensic funds recovery
process and should probably be the last resort in case of a backup-recovery
situation. A simple example here is, that it was/is possible to have used an
xpriv (referring to extended private master key) in production that is/was used
to derive BIP45 based P2SH multisig addresses (1of1, used by Bitpays BWS for
while), later used for bare BIP45ish multisig 1of1 as well as for P2PKH after
BIP44 & vanilla BIP32 P2WPKH (m/0’/k’).
I’m not aware of any wallet that would recover 100% of those funds, leading to
the risk that forwarding the unspents and destroying the extended master key
may result in coins forever lost.

The case above may be an edge case, but I’m generally under the assumption that
recovering funds based on the sole existence of an xpriv (or seed) without further
metadata is a fragile concept.

Second, the missing birthday-metadata tend to lead to non-optimal blockchain
scans (eventually increased p2p traffic). Recovering funds can take hours.

Additionally, the BIP44 gap limit seems to be a weak construct. The current gap
limit in BIP44 is set to 20 [2] which basically means, handing out more then 20
incoming payment requests (addresses) results in taking the risks that funds
may be destroyed (or at least not detected) during a recovery.
The Gap limit value may also depend on the use case, but the current proposals
do not allow to set an arbitrary value. High load merchants very likely need a
different gap limit value then individuals create a transaction once a year.

During creation time of an xpriv/xpub, it is impossible to know if the created
xpriv will be used for an unforeseen derivation scheme. Future proposals may
want to limit an extended key to a single derivation scheme.

This is an early draft in order to allow discussion that may lead to a possible
proposal.
This proposals could also make BIP 178 obsolete since it can be replace the
WIF[3] standard.

Thanks for feedback
/jonas

------------------------------------

Titel
######
Bech32 encoded key material including metadata

Abstract
########
An error tolerant encoding format for key material up to 520bits with a minimal
amount of metadata.

Motivation
##########
(See above; intro text)

Specification
#############

## Serialization format

1 bit version bit
15 bits (bit 1 to 16) key-birthday (0-32767)
(12 bit gap limit)
3 or 5 bits script type
256 or 512 or 520 bits key material
= Total 275, 545, 553 bits

The initial version bit allows extending the serialization-format in future.
The encoding format must hint the total length and thus allow to calculate the
length of the key material.

The total length for 256 or 512 bit key material is optimised for Bech32 (power
of 5).

### Key material
If the key material length is 520 bits, it must contain an extended public key
If the key material length is 512 bits, it must contain an extended private key
Key material length other then 256, 512, 520 bits and invalid.

If 520 bits are present, first 256 bits are the BIP32 chain code, to second 264
bits (33 bytes) define the public key (according to BIP32)

If 512 bits are present, first 256 bits are the BIP32 chain code, to second 256
bits define the private key

If 256 bits are present, those bits represent a pure private key (or seed)

### Key birthday
A 15 bit timestamp expressed in days since genesis (valid up to ~2098). The
birthday must be set to the first possible derivation of the according extended
key, if unknown, the used seed birthday must be used. If both unknown, 0
(16x0bit) must be used.

### Gap limit delta
12 bits, results in a possible range from 0 to 4095.

If the total decoded serialization length is 275 bits (decode) or if the key
material is 256 bits (encode), the gap limit must not be present.

The base gap limit value is 20 (to disallow insane gap limits). The final gap
limit is the base value + the gap limit delta stored in those 12 bits.
Key derivation gap limit must not be exceeded when deriving child keys and must
be respected during transaction rescans.
Child key derivation must not be possible if gap limit is hit.

### Script type restriction
3 or 5 bits (range 0-7 / 0-31)
0 no restriction
1 P2PKH compressed
2 P2PKH | P2SH
3 P2WPKH P2WSH nested in P2SH
4 P2WPKH | P2WSH

If the total decoded serialization length is 275 bits (decode) or if the key
material is 256 bits (encode), 3 bits are used for the script type. 5 bits are
used for key material with the size of 512, 520 bits.

If the script type restriction is set, the according extended key must only be
used to derive addresses with the selected script type.
This does not stands in contradiction to derivation path proposals ([4]). It
does allow to derive and encode an extended key at a keypath where users assume
restricted script types in derivation due to other supported proposals.

Encoding
########

Bech32 must be used as encoding format (see the Bech32 rational [5]). Encoding
545 or 553 bits (results in 109 resp. 111 x 5 bits) will exceed the Bech32 property of a
guaranteed detection of 4 errors (only 3 are).
It is possible that there are more efficient BCH codes, especially for encoding
extended private keys. Since a Bech32 implementation needs to be present in
modern Bitcoin software, re-using Bech32 will allow to migrate to this proposal
with a minimal implementation effort.
Forensic, cpu-intense key-recovery (including brute-force techniques) may allow
to recover keys beyond the guaranteed error detection limits.

Bech32 HRPs
Mainnet Private Extended: xp
Mainnet Public Extended: xpu
Testnet Private Extended: tp
Testnet Public Extended: tpu
Mainnet Key: pk-
Testnet Key: tk-

Compatibility
###########
Only new software will be able to use these serialization and encoding format.

References
##########

[1] https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki <https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki>;
[2] https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki
[3] https://github.com/bitcoin/bips/blob/master/bip-0178.mediawiki
[4] https://github.com/bitcoin/bips/blob/master/bip-0049.mediawiki
[5] https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki#rationale

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180529/d2332b98/attachment.html>;
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20180529/d2332b98/attachment.sig>;

📅 Original date posted:2017-04-18 📝 Original message:Hi ...

2023-06-07T18:00:23Z

In reply to nevent1q…qhel
_________________________

📅 Original date posted:2017-04-18
📝 Original message:Hi Dave

> A node that stores the full blockchain (I will use the term archival node) requires over 100GB of disk space, which I believe is one of the most significant barriers to more people running full nodes. And I believe the ecosystem would benefit substantially if more users were running full nodes.

Thanks for your proposal.

I agree that 100GB of data may be cumbersome for some systems, especially if you target end user systems (Laptops/Desktops). Though, in my opinion, for those systems, CPU consumption is the biggest UX blocker.
Bootstrapping a full node on a decent consumer system with default parameters takes days, and, during this period, you probably run at full CPU capacity and you will be disturbed by constant fan noise. Standard tasks may be impossible because your system will be slowed down to a point where even word processing may get difficult.
This is because Core (with its default settings) is made to sync as fast as possible.

Once you have verified the chain and you reach the chain tip, indeed, it will be much better (until you shutdown for a couple of days/hours and have to re-sync/catch-up).

1. I agree that we need to have a way for pruned nodes to partially serve historical blocks.
My personal measurements told me that around ~80% of historical block serving are between tip and -1’000 blocks.
Currently, Core nodes have only two modes of operations, „server all historical blocks“ or „none“.
This makes little sense especially if you prune to a target size of, lets say, 80GB (~80% of the chain).
Ideally, there would be a mode where your full node can signal a third mode „I keep the last 1000 blocks“ (or make this more dynamic).

2. Bootstrapping new peers
I’m not sure if full nodes must be the single point of historical data storage. Full nodes provide a valuable service (verification, relay, filtering, etc.). I’m not sure if serving historical blocks is one of them. Historical blocks could be made available on CDN’s or other file storage networks. You are going to verify them anyways,... the serving part is pure data storage.
I’m also pretty sure that some users have stopping running full nodes because their upstream bandwidth consumption (because of serving historical blocks) was getting intolerable.
Especially „consumer“ peers must have been hit by this (little experience in how to reduce traffic, upstream in general is bad for consumers-connections, little resources in general).

Having a second option built into full nodes (or as an external bootstrap service/app) how to download historical blocks during bootstrapping could probably be a relieve for "small nodes“.
It could be a little daemon that downloads historical blocks from CDN’s, etc. and feeds them into your full node over p2p/8333 and kickstarts your bootstrapping without bothering valuable peers.
Or, the alternative download, could be built into the full nodes main logic.
And, if it wasn’t obvious, this must not bypass the verification!

I’m also aware of the downsides of this. This can eventually reduce decentralisation of the storage of historical bitcoin blockchain data and – eventually – increase the upstream bandwidth of peers willing to serve historical blocks (especially in a transition phase to a second „download“-option).
Maybe it’s a tradeoff between reducing decentralisation by killing low resource nodes because serving historical blocks is getting too resource-intense _or_ reducing decentralisation by moving some percentage of the historical data storage away from the bitcoin p2p network.
The later seems more promising to me.

To your proposal:
- Isn’t there a tiny finger-printing element if peers have to pick an segmentation index?
- SPV bloom filter clients can’t use fragmented blocks to filter txns? Right? How could they avoid connecting to those peers?

</jonas>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170418/9cf44da5/attachment.html>;
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170418/9cf44da5/attachment.sig>;

📅 Original date posted:2017-03-06 📝 Original message:I like ...

2023-06-07T17:56:53Z

In reply to nevent1q…sk7t
_________________________

📅 Original date posted:2017-03-06
📝 Original message:I like the BIP. It can reduce workload during implementation on both sides of the API and it allows to show the user more data without implementing tons of proprietary APIs.
It’s not directly Bitcoin specific (example: BIP32 is also not Bitcoin specific), but I think a BIP is the right way for this.

>
>> Apart from that, my feeling is that it could be simplified. Is
>> longpolling useful?
>
> I would think so, at least for Bitcoin since rates can change significantly in
> a short period of time (or can they anymore? I haven't really watched lately.)

Long polling is a simple push concept that works on most type of network configurations (NAT, proxy, etc.).
The only concern I see here is that an public API will quickly fill up the maximum allowed httpd connections.
But it’s solvable.

>
>> And is the historical rate thing really necessary for typical applications?
>
> When displaying historical transactions, it doesn't really make sense to use
> the current market rate, but rather the market rate at the time the payment
> was made. While wallets might simply cache it with the transaction, it would
> be perhaps nicer if it could be automatically restored for seed-only
> recoveries. In any case, if a service/wallet doesn't want to provide/use
> historical information, it can simply not implement that part.

I’m also not sure how useful historical datapoint are. I don’t think the use case where someone wants to restore from a seed and get all exchange rates during the time of the payment is something users are looking for.
However, It’s optional.

>
>> If yes, the client should be allowed to decide on which time scale the
>> data should be. (tick, min, hour, day, ...) That goes together with
>> clearly defining the type field (something like low, high, open, close,
>> but without flexibility). Think of a candle-stick chart basically.
>
> How is the current draft insufficient for this?
>
>> Also, pushing may be more appropriate for "current" rates than polling.
>> Then no polling interval is necessary. On the other hand, this adds
>> complexity in other places, e.g., state.
>
> Pushing is what longpolling does.

Agree with Luke. A „real“ push (though I’d say long-polling is the real push, AFAIK it’s also the technique behind Apple’s iOS push channel) would require a complex server setup that complicates many things like load-balancer, mem-caching, etc.

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170306/ec223a69/attachment.sig>;

📅 Original date posted:2017-01-04 📝 Original message:Hi ...

2023-06-07T17:55:07Z

In reply to nevent1q…9jjv
_________________________

📅 Original date posted:2017-01-04
📝 Original message:Hi

> Unconfirmed transactions are incredibly important for real world use.
> Merchants for instance are willing to accept credit card payments of
> thousands of dollars and ship the goods despite the fact that the
> transaction can be reversed up to 60 days later. There is a very large
> cost to losing the ability to have instant transactions in many or
> even most situations. This cost is typically well above the fraud risk.
>
> It's important to recognize that bitcoin serves a wide variety of use
> cases with different profiles for time sensitivity and fraud risk.
>
I agree that unconfirmed transactions are incredibly important, but not
over SPV against random peers.

If you offer users/merchants a feature (SPV 0-conf against random
peers), that is fundamentally insecure, it will – sooner or later – lead
to some large scale fiasco, hurting Bitcoins reputation and trust from
merchants.

Merchants using and trusting 0-conf SPV transactions (retrieved from
random peers) is something we should **really eliminate** through
education and by offering different solution.

There are plenty, more sane options. If you can't run your own full-node
as a merchant (trivial), maybe co-use a wallet-service with centralized
verification (maybe use two of them), I guess Copay would be one of
those wallets (as an example). Use them in watch-only mode.

For end-users SPV software, I think it would be recommended to...
... disable unconfirmed transactions during SPV against random peers
... enable unconfirmed transactions when using SPV against a trusted
peer with preshared keys after BIP150
... if unconfirmed transactions are disabled, show how it can be enabled
(how to run a full-node [in a box, etc.])
... educate, inform users that a transaction with no confirmation can be
"stopped" or "redirected" any time, also inform about the risks during
low-conf phase (1-5).

I though see the point that it's nice to make use of the "incoming
funds..." feature in SPV wallets. But – for the sake of stability and
(risk-)scaling – we may want to recommend to scarify this feature and –
in the same turn – to use privacy-preserving BFD's.

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170104/76132d83/attachment.sig>;

📅 Original date posted:2017-01-01 📝 Original message:Hi ...

2023-06-07T17:55:03Z

In reply to nevent1q…yhr5
_________________________

📅 Original date posted:2017-01-01
📝 Original message:Hi
> We introduce several concepts that rework the lightweight Bitcoin
> client model in a manner which is secure, efficient and privacy
> compatible.
>
> The BFD can be used verbatim in replacement of BIP37, where the filter
> can be cached between clients without needing to be recomputed. It can
> also be used by normal pruned nodes to do re-scans locally of their
> wallet without needing to have the block data available to scan, or
> without reading the entire block chain from disk.
I started exploring the potential of BFD after this specification.

What would be the preferred/recommended way to handle 0-conf/mempool
filtering – if & once BDF would have been deployed (any type,
semi-trusted oracles or protocol-level/softfork)?

From the user-experience perspective, this is probably pretty important
(otherwise the experience will be that incoming funds can take serval
minutes to hours until they appear).
Using BIP37 bloom filters just for mempool filtering would obviously
result in the same unwanted privacy-setup.

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20170101/91d3f504/attachment.sig>;

📅 Original date posted:2016-08-25 📝 Original message:Sorry ...

2023-06-07T17:53:04Z

In reply to nevent1q…f50h
_________________________

📅 Original date posted:2016-08-25
📝 Original message:Sorry for hijacking the thread again

> As I understand it, you can scan sequentially starting with the genesis
> block (or with a block at around the time when BIP44 was written). Then
> if you find a new transaction, which requires to generate new addresses,
> you generate them and scan further from that point on. This way you can
> scan in a single pass if the scanning process calls you back when it
> finds a transaction and allows you to change the set of addresses on the
> fly.

(I think this case if not completely unrealistic):

What would happen, if a user gave out 21 addresses, then address0 had
receive funds in +180 days after generation where address21 had receive
funds immediately (all other addresses never received a tx).

In a scan, address0 would be detected at <address-birthday>+180 days
which would trigger the resize+20 of the address-lookup-window, but, we
would require to go back 180day in order to detect received transaction
of address21 (new lookup-window) in that case.

Or do I misunderstand something?

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160825/b7df9cb6/attachment.sig>;

📅 Original date posted:2016-08-24 📝 Original message:Hi ...

2023-06-07T17:53:03Z

In reply to nevent1q…96f5
_________________________

📅 Original date posted:2016-08-24
📝 Original message:Hi

> 6 - Finally, and most importantly, BIP39 seed phrases do not have a
> version number. Without a version number, how are you going to derive
> addresses from a BIP39 seed phrase, when wallets start to use to new
> derivation methods (such as SegWit, or Schnorr signatures)? Does it mean
> that a BIP39 compatible wallet will have to check addresses from all the
> derivation methods that ever existed in the past, in order to ensure
> that all coins are correctly retrieved? Or will there be users that
> cannot access their coins because their BIP39 seed phrase is too old for
> newer software?

I totally agree with Thomas.

Another thing that I think could be a BIP misdesign:

BIP44 Gap Limits
From the BIP:

----------
"Address gap limit is currently set to 20. If the software hits 20
unused addresses in a row, it expects there are no used addresses beyond
this point and stops searching the address chain."
----------

* Does that mean, we do a transaction rescan back to the genesis block
for every 20 addresses?
* Or is it a prerequirement to do a wallet recovery after BIP44's to
have access to a full address-indexed blockchain?

Or maybe I'm missing something.

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160824/c679a7ea/attachment.sig>;

📅 Original date posted:2016-08-24 📝 Original ...

2023-06-07T17:53:01Z

In reply to nevent1q…63tn
_________________________

📅 Original date posted:2016-08-24
📝 Original message:>> Additionally, BIP 111 (NODE_BLOOM service bit) has been implemented in Bitcoin
>> Core and derivatives; it is unclear if used by clients yet. Can developers of
>> such clients please comment and let me know: 1) if their software supports
>> this BIP already; 2) if not, do they intend to support it in the future?
>> If and only if there are any clients using this service bit already, I will
>> update BIP 111 to Final Status in 2 weeks also.
>
> Multibit is adding detection of the NODE_BLOOM bit in the next 2-3 weeks.
>
> SPV is kinda broken if the wallet doesn’t do this detection. If your wallet connects only to nodes that don’t support bloom filtering, the wallet never gets updates. We have had a spike in users reporting that their wallet isn't getting updated. To compound the problem, they rescan the blockchain and lose all of their transaction history. It has caused much panic among less technical users.
>
> We believe that failing to detect the NODE_BLOOM bit is the culprit, although it is non-deterministic, so we aren't certain.
>
> I imagine that other SPV wallets are having similar issues. BIP 111 really isn’t optional at this point, so it should be marked final.

SPV Wallets should definitively update to respect NODE_BLOOM. Bloom
filtering is CPU and disk intense and some node operators have disabled
it (or will disabled it) because there is no direct p2p network-health
benefit.

SPV wallets should probably also make use of the new DNS seeder filter
option.
It is running at least on seed.bitcoin.sipa.be and
seed.bitcoin.jonasschnelli.ch.

The filter option allows SPV Wallets to only get nodes that signal
support for NODE_BLOOM.

The syntax is

x<networkservice_flags_filter_as_int>.seed.bitcoin....

Example for NODE_NETWORK together with NODE_BLOOM
dig x5.seed.bitcoin.jonasschnelli.ch
(NETWORK = (1 << 0), NODE_BLOOM = (1 << 2)) = (bin00000101 = (int)5)

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160824/c7311f2b/attachment.sig>;

📅 Original date posted:2016-08-18 📝 Original message:Hi ...

2023-06-07T17:52:52Z

In reply to nevent1q…a783
_________________________

📅 Original date posted:2016-08-18
📝 Original message:Hi

> I have some experience with hardware wallet development and its
> integration and I know it's a mess. But it is too early to define such
> rigid standards yet. Also, TREZOR concept (device as a server and the
> primary source of workflow management) goes directly against your
> proposal of wallet software as an workflow manager. So it is clear NACK
> for me.

The current question – as already mentioned – is we ACK to work together
on a signing protocol or if we NACK this before we even have started.

I'm not saying that the draft proposal I made is the way to go, I'm
happy to NACK it myself in favor of a better proposal.

I strongly recommend to work together on a standard that will have one
central winner: the end user.

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160818/343d9d28/attachment.sig>;

📅 Original date posted:2016-08-18 📝 Original message:Hi ...

2023-06-07T17:52:51Z

In reply to nevent1q…zydk
_________________________

📅 Original date posted:2016-08-18
📝 Original message:Hi

> The main benefit is that you don't need "standard" to solve problem, but
> use natural tools in given environment and programming stack. Build a
> "standard" on top of URI protocol is a huge limitation, which does not
> give any advantage.

Standards can help an ecosystem to grow, can help to sustain a good user
experience.

The hardware wallet vendors have used "natural tools" and look where we
are. We have *native* plugins in Electrum, Copay, etc. for different
hardware wallets. Mostly the plugins are in the code base of the wallet,
which makes it – in theory – impossible to change from the perspective
of the hardware wallet vendor (there is no control of the deployment if
there are bugs in the plugins code).
The plugins functions overlap significant.

I think this is a bad design for security critical applications.

What I want as hardware wallet user:
* I'd like to have a trusted application (layer) where I'm sure I'm
using software provided through my hardware wallet vendor.

What I want as hardware wallet vendor:
* I'd like to be able to provide and update a software layer (app) to my
customer with the ability to provide code signatures and security
updates anytime. I do want to control the user experience.

> We already see issues with dead simple "bitcoin uri" standard, it barely
> works in most of bitcoin apps. Think of vague definitions of parameters
> or ability to send payment requests over it. HW API would be complicated
> by an order of magnitude and I have serious concerns that it will be
> helpful for anything. So why complicate things.

As far as I know most bitcoin wallets do support the bitcoin:// URI
scheme quite well.
I agree that BIP70 is a mess (including the bitcoin:// additions).

The proposed URI scheme would be completely different. The only
similarity is using the URI scheme as transport layer (which is the
proposed long term inter-app communication layer by Apple and Google).

>> How would the library approach work on mobile platforms? Would USB be
> the only supported hardware communication layer?
>
> Interprocess communication/libraries/dependencies on Android are not
> bound to specific transport anyhow. Such library could be used by any
> android app, and the library would implement proper transports for
> various supported vendors. USB for Trezor, NFC for something different
> etc. If the point is "make life of app developers easier", let's do this
> and do not define artifical "standards".

So you propose having one library that would support multiple vendors?
What if new vendors add a new transport layer (lets assume NFC or
Bluetooth), wouldn't that result in every possible consumer of that
library (all wallets) need to update before the new vendors transport
layer could be used, resulting in a huge deployment process probably
require many month until it can be used?

What if there is a critical security issue in the library? How would the
deployment plan looks like?

I really think we should remove the "hardware communication layer" from
wallets and move it towards the hardware vendor app.

What about iOS? Should we just leave that platform unsupported with
hardware wallets?

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160818/ca1a24d9/attachment.sig>;

📅 Original date posted:2016-08-17 📝 Original message:Hi ...

2023-06-07T17:52:50Z

In reply to nevent1q…rall
_________________________

📅 Original date posted:2016-08-17
📝 Original message:Hi Dana

>> The URI scheme does not require any sorts of wallet app level
>> configuration (where the stdio/pipe approach would require to configure
>> some details about the used hardware wallet).
>
> Hi everybody, just thought I’d throw my opinion in here.
>
> The URI scheme is a nice idea, but this ignores the fact that hardware wallet vendors do most of the work on talking between the computer/mobile and the wallet on a lower level of communication. In the case of BitLox, the base protocol is Google’s ProtoBuf. The commands and transaction data is in a “schema” which is then encoded in different methods accessible via ProtoBuf (depending on the data being sent). The advantages of this protocol is that it can be implemented on a wide variety of platforms. (but that’s a whole 'nother discussion)
>
> The URI would be handled waaaaay up in the specific application (such as the mytrezor wallet software or the various standalone wallets) - nowhere near the actual hardware communications layer.

This is maybe a question of the scope.
The BIP I'm proposing would make a clear interface cut between
wallet-with-unsigned-transaction and a signing-device (and maybe between
wallet-requires-pubkey, signing-device generate some pubkeys [or
non-hardened xpub]).

The detached-signing proposal does not duplicate work. It just moves the
current plugin design into a separate application. Plugins in security
and privacy critical wallet software is something that should probably
be avoided.

It's intentional at a high level to allow maximum flexibility at the
hardware interaction layer.

Your protobuf example is a good use-case. You could implement your
custom processes behind the URI scheme (which is probably way more
efficient then writing a couple of wallet plugins where you – at the end
– mostly don't control the deployment and the source-code).

Defining a standard on the hardware interaction layer is possible, but a
fairly different approach.

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160817/3eea9613/attachment-0001.sig>;

📅 Original date posted:2016-08-18 📝 Original message:Hi ...

2023-06-07T17:52:50Z

In reply to nevent1q…vpsv
_________________________

📅 Original date posted:2016-08-18
📝 Original message:Hi

> I fundamentally disagree with the concept of driving signing workflow by
> the wallet software. Wallet software does not know in advance all data
> necessary for the signer to do the job. As Jochen mentioned above,
> Segwit vs Non-segwit use cases are a good example, but there may be many.

I think this is easily solvable. The required data to verify and sign a
(standard) bitcoin transaction (including P2WSH multi-sig) is manageable.

IMO what a signing devices requires in order to sign a (standard)
transaction:
-> serialized tx
-> serialized tx of the inputs
-> scriptPubKey of the inputs
-> inputs redeem-Scripts
-> input amounts
-> position of the change output any maybe its keypath
-> cosigners pubkeys for inputs and changeaddress

This seems to be manageable for a 1 round communication?
Or do I miss something?

> Currently the TREZOR protocol works like device is a server and wallet
> is a client calling methods on it. It's like: "Sign this for me,
> please", "Ok, give me this information", "Here it is", "Now I need this
> another piece".... "There is the signature". Wallet does not know in
> advance what will go next, and it is for sake of simplicity. I'm quite
> happy with the protocol so far.

I think multiple rounds would still be possible with a clever design.
Although I could imaging that >95% of the users transaction would
require only a single "shot".

Whats the benefits of the multiple rounds communication? Would a single
round result in to many data transported?

Passing a 300kb chunk (assuming a large transaction) over a URI scheme
requires a couple of milliseconds on standard Smartphones or PCs.

> Considering the difference in between current hardware, I really don't
> think it is possible to find any minimal URI-based API good enough for
> communicating with all vendors. What I see more likely is some 3rd party
> libraries (JS, C++, Python, ...) defining high-level API and
> implementing hardware-specific protocols and transports as plugins. That
> way vendors are not limited by strict standard and application
> developers and services can integrate wide range of hardware wallets
> easily. However, this can be done already and we do not need any
> standardization process (yet).

The URI-based API allows transmitting data of multiple megabytes while
there is no need for...
* dependencies of any form (library, etc.)
* library support for a particular language
* platform that supports the dependencies of the library (like USBHID,
not supported by iOS)

Can you elaborate what benefits you would get from the library approach
and how the library API would be different form the proposed URI-scheme?

How would the library approach work on mobile platforms? Would USB be
the only supported hardware communication layer?

Thanks
--
</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160818/5ae29972/attachment.sig>;

📅 Original date posted:2016-08-17 📝 Original message:Hi all ...

2023-06-07T17:52:49Z

In reply to nevent1q…rrll
_________________________

📅 Original date posted:2016-08-17
📝 Original message:Hi all

Thanks for the response.

Jochen's points:
===============
Indeed. There are some missing points and I'd like to work this into the
BIP. Thanks for bringing this up.

Along with a support for wallet-creation with a xpub from the signing
device, we might also want to support loading multiple pubkeys into a
keypool from the device (in case someone likes to use hardened
derivation at all levels). I guess this would not be over-complex to
achieve.

Luke's points:
=============

USB / Plugin/Driver problematic
-------------------------------
I don't think it would be wise to set Trezors USB communication
(hardware interface) as "the standard". A) A USB stack/interaction in
wallets should be avoided IMO. B) This approach won't work for some
platforms (like iOS) due to technical and legal restrictions.

In my opinion, each hardware wallet has to provide custom software in
any case. We don't want to standardize how a hardware wallet has to do
backups, recovers, firmware upgrade, etc. and if we agree on that, then
hardware wallets must provide an application (mostly Chrome extensions
today) to implement theses processes.

Also diversity at the hardware interface will reduce centralized risks
for weak security/vulnerabilities.

The proposed URI scheme approach does not require any sorts of
libraries/dependencies. USB HID can be a problem for cross platform
desktop wallets as well as it won't work of one of the major mobile
platform (iOS). USB HID interaction can be restricted or disabled in non
superuser setups where I'm not aware of any restriction on URI-Scheme level.

URI scheme instead of stdio/pipe
--------------------------------
The URI scheme is not ugly. Its a modern way – implemented in almost all
platforms – how applications can interact with each other while not
directly knowing each other. Registering a URI scheme like "bitcoin://"
has some concrete advantages over just piping through stdio.

Also, the stdio/piping approach does not work for mobile platforms
(where the URI scheme works).

The URI scheme does not require any sorts of wallet app level
configuration (where the stdio/pipe approach would require to configure
some details about the used hardware wallet).

Thomase D.'s points:
===================
Standardizing to many layers of the interaction stack (including the
hardware interaction) will very likely result in vendors not sticking to
the standard.

I agree, the URI scheme has some fragility, but at a level where we can
handle it and with the advantage of abstracting the used brand/device
for privacy and security reasons.

> The existing URI scheme, while allowing disambiguate by manufacturer,
provides no way to to enumerate available manufacturers or enabled
wallets.

Most operating systems allow to check if a certain URL-Scheme is
supported (registered), this would allow at least to check for known
major vendors (like trezor, etc.) which should solve most
multi-hardware-wallet use-cases.

The URI return scheme does work fine and with the correct set timeouts
it should result in a neat user experience.
It's the proposed way of application intercommunication in Apple iOS [1]
and Google Android [2].

Conclusion:
===========
* Non of the points convinced me that there is a better alternative to
the proposed URI scheme interaction (please tell me if I'm stubborn).
* Also, we should move the end users UX in the center of the
problems-to-solve (and not overweight the ideal
code-/API-/hardware-interaction-design while ignoring the end user
experience).
* We should try to not over-standardize the interaction with the device
itself to allow flexibility on the hardware wallet vendor side.

[1]
https://developer.apple.com/library/ios/documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/Inter-AppCommunication/Inter-AppCommunication.html
[2] https://developer.android.com/training/basics/intents/sending.html

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160817/63e4d8a4/attachment-0001.sig>;

📅 Original date posted:2016-08-16 📝 Original message:> I ...

2023-06-07T17:52:47Z

In reply to nevent1q…mx5k
_________________________

📅 Original date posted:2016-08-16
📝 Original message:> I think it does not make sense to try to get this standardized for
> current Bitcoin transactions. They are just too complex.
>
> What might be interesting is to have something similar for Segwit and
> Lightning transactions.
>
> * TREZOR performs extended validation of the inputs, when all of
> prev-txs are streamed into the device and validated. Your standard does
> not tackle this at all and I don't think it's worthy to make this
> standard unnecessarily complicated.

I'm aware of this approach but I don't think this makes sense long term.
We need a better way on the protocol level to validate inputs amounts
(where segwit is a first step towards this).

IMO, not having a standard for hardware wallet interfaces/communication
will long term result in reducing the end user experience.

I think we should collaborate together and work out a standard.

My goal is to add hardware wallet support in Bitcoin-Core where adding
proprietary code (plugin-ish) is something we don't want to do for the
sake of security and compatibility.

As said, the "BIP" is very draft and I'm happy to include the input
streaming as part of it (or you could add it if you want because you
have more experience with it).

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160816/75ee098a/attachment.sig>;

📅 Original date posted:2016-08-16 📝 Original message:Hi ...

2023-06-07T17:52:46Z

In reply to nevent1q…akja
_________________________

📅 Original date posted:2016-08-16
📝 Original message:Hi

Unfortunately, there is no standard in how desktop- or mobile-wallets
can interact with a hardware device resulting in wallet vendors adding
plugins with proprietary code for non-standardized interfaces.

I started a BIP (extreme draft, feel free to improve language, grammar
and content) to address this missing part of the ecosystem.

I think it would be extremely helpful if @ledger, @trezor,
@voisin/@breadwallet, @electrum, @bitpay (and more?!) would help working
on a such standard.

The BIP describes two approaches how to communicate (pipe and
URI-scheme) with the signing-devices app, although, in my opinion, all
major platform do support the URI approach (maybe we could drop the pipe
approach then).
The URI approach means that there is no need to configure the
application location in order to start a inter-process(-app) communication.

Mediawiki:
https://github.com/jonasschnelli/bips/blob/8abb51f0b21b6664388f6e88f6fd642c90d25dca/bip-undef-0.mediawiki

</jonas>

---- BIP (rough early stage draft)

<pre>
BIP: ???
Title: Detached Signing
Author: Jonas Schnelli <dev at jonasschnelli.ch>
Status: Draft (early stage!)
Type: Standards Track
Created: 2016-08-02
</pre>

== Abstract ==

This BIP describes a way how wallet applications can decouple sensitive
privatekeys from the internal keychain and interact with a
signing-devices (hardware wallet, "cold" storage) over a generic
interface in order to get signatures.

== Motivation ==

It seems like that the current approach for allowing signing-devices to
interact with third party wallets is to build a plugin [1][2][3]. Adding
plugins for each hardware wallet type will increase possible security
issues and result in multiple proprietary-third-party code within the
wallet application with very similar structures.

A generic interface how wallets can interact with signing-devices would
result in better user experience, less critical code and simpler
adaption for various signing-devices.

== Specification ==

In order to support desktop- and smartphone-wallet-applications, this
BIP describes two slightly different approaches (process pipe and URI
call) in how to interact with the signing-devices. If possible, the
modern URI approach should be chosen.

=== Signing-Device-Controller-Application ===

To allow a generic interface while still allowing different ways how to
internally communicate with the signing device itself (USB, TCP/IP,
air-gapped Qr-Code scanning, etc.) a controller-application is required.

=== General signing process ===

The wallets signing process must be according the following principal:
* Wallet prepares signing-request-object including bitcoin-transaction
or message together with metadata (scriptPubKey, hd-keypath of the inputs)
* Wallet passes signing-request-object to the
signing-device-controller-application
* Signing-device-controller-application processes
signing-request-object, eventually shows UI, user can sign or cancel
* Signing-device-controller-application sends back
signing-response-object with signatures or an error
* Wallet processes signing-response-object and completes data-object
creating process (example: add signatures to transaction and broadcast)

=== Desktop Process Intercommunication ===

Desktop wallets can interact with a signing device over process
intercommunication (pipe) together with a
signing-device-controller-application.
As specified below, the signing-request-object is a URI string passed
through the pipe. The desktop wallet needs to wait (with a recommended
timeout between 1 and 5 minutes) until the signing-response-object will
be sent back by the signing-device-controller-application.

=== Smartphone/URI App Intercommunication ===

Smartphones and modern operating systems are trying to sandbox
applications and interprocess communication (on pipe level) is mostly
disallowed.
On smartphones, we must use URI-schemes.
The wallet can pass information to the
signing-device-controller-application by using a predefined URI scheme.

<code>detatchedsigning://<command>?<querystring>&returnurischeme=<returnurischeme://></code>

The <code>querystring</code> must be URI encoded.
RFC 2616 does not specify a maximum length of URIs (get request). Most
modern smartphone operating system allow URIs up to serval megabytes.
Signing complex data-structure is therefore possible.

The <code>returnurischeme</code> must contain a URI schema where the
result of the signing process should be returned to.
The returnurischeme must be populated and "opened" once the signing
process has been completed (or cancled).

=== Signing Request ===

The signing request is a flexible URI-Query-String that will be used by
the Signing-device-controller-application for user confirmation as well
as for creating the signature.

The URI-query-string must conform to the following format:

<code>detatchedsigning://sign?type=<bitcoin-p2pkh|bitcoin-p2sh|bitcoin-msg>&data=<raw-transaction|message>&inputscripts=<scriptPubKey-input0>,<scriptPubKey-input1>,...&inputhdkeypath=<hdkeypath0>,<hdkeypath1>,...&returnscheme=<returnurischeme></code>

type = type of the data to sign
data = raw unsigned bitcoin transaction or text-message
(optional)inputscripts = scriptPubKey(s) of the inputs in exact order
(optional)inputhdkeypath = hd-keypath of the inputs in exact order
(optional)returnscheme = a URI scheme where the response must be sent to
(smartphone approach)

* inputhdkeypath or inputscripts must be provided.

=== Signing Response ===

The signing response is a flexible URI-Query-String that will be sent
back to the wallet application and must contain the signatures or an
error code.
The URI-query-string can be opened (smartphone approach) or will be sent
back though the interprocess pipe.

<code><returnurischeme>://signresponse?errorcode=<errorcode>&signatures=<signature-input0>,<signature-input0>,...</code>

In case of ECDSA, the returned signatures must be normalized compact
signatures with the size of 64bytes (128 hex chars).

==== Possible error code ====

0 = no error
1 = user canceled
2 = timeout
10 = missing key identifier (missing HD keypath or input scriptpubkey)
11 = unsupported signing type
12 = could not resolve script
50 = unknown internal error

==== Examples ====

===== Simple p2pkh transaction =====
Unsigned raw transaction:
<code>0100000001fd3cd19d0fb7dbb5bff148e6d3e18bc42cc49a76ed2bfd7d760ad1d7907fd9ce0100000000ffffffff0100e1f505000000001976a9149062e542a78d4fe00dcf7cca89c24a8013c381a388ac00000000</code>
(input ced97f90d7d10a767dfd2bed769ac42cc48be1d3e648f1bfb5dbb70f9dd13cfd
vout:1, output: P2PKH mtgQ54Uf3iRTc9kq18rw9SJznngvF5ryZn 1 BTC)

signing-request URI must be:
<code>detatchedsigning://sign?type=bitcoin-p2pkh&data=0100000001fd3cd19d0fb7dbb5bff148e6d3e18bc42cc49a76ed2bfd7d760ad1d7907fd9ce0100000000ffffffff0100e1f505000000001976a9149062e542a78d4fe00dcf7cca89c24a8013c381a388ac00000000&inputscripts=76a914531148ad17fdbffd4bac72d43deea6c7cf0387d088ac&inputhdkeypath=m/0'/0'/1&returnscheme=myapp</code>
The <code>inputhdkeypath</code> is optional in this case

signing-response URI must be:
<code>detatchedsigning://signresponse?error=0&signatures=<128hex-chars></code>

===== Simple a bitcoin message =====
Message: <code>Lorem ipsum dolor sit amet</code>

signing-request URI must be:
<code>detatchedsigning://sign?type=bitcoinmsg&data=Lorem+ipsum+dolor+sit+amet&inputhdkeypath=m/0'/0'/2</code>

signing-response URI must be:
<code>detatchedsigning://signresponse?error=0&signatures=<128hex-chars></code>

=== Support for multiple signing-devices ===
Must operating systems allow only one registered application per
URI-scheme. To support multiple signing-devices, wallets and
signing-devices can optional add support for brand based URI-schemes.

In addition to the standard URI scheme,
signing-devices-controller-applications can register an additional URI
scheme (with the identical request/response syntax and logic) including
a brand-identifier.

Registering a brand-identifier based URI scheme without registering the
default URI scheme is not allowed.

Wallets can detect if a certain brand based URI scheme is supported and
therefore gives user a selection if multiple signing-devices where
detected [4][5].

<code>detatchedsigning<brandid>://</code>

Supported brand-identifiers are:
* trezor
* ledger
* keepkey
* digitalbitbix

== References ==
[1] https://github.com/spesmilo/electrum/pull/1662
[2] https://github.com/spesmilo/electrum/pull/1391
[3] https://github.com/bitpay/copay/pull/3143
[4]
https://developer.apple.com/library/ios/documentation/UIKit/Reference/UIApplication_Class/
[5]
https://developer.android.com/reference/android/content/pm/PackageManager.html
== Acknowledgements ==

== Copyright ==
This work is placed in the public domain.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160816/fe4471a9/attachment.sig>;

📅 Original date posted:2016-08-08 📝 Original message:Hi As ...

2023-06-07T17:52:20Z

In reply to nevent1q…na2l
_________________________

📅 Original date posted:2016-08-08
📝 Original message:Hi

As already mentioned in the recent BIP151 thread
(https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-June/012826.html),
I propose the following authentication scheme to basically allow MITM
detection and rejection in conjunction with BIP151.

The proposed authentication BIP does require BIP151.

The propose BIP does assume, node operators want to build trusted
connections for various reasons.

BIPs mediawiki github page available here:
https://github.com/bitcoin/bips/compare/master...jonasschnelli:2016/07/auth_bip?expand=1

===================================================

BIP: ???
Title: Peer Authentication
Author: Jonas Schnelli <dev at jonasschnelli.ch>
Status: Draft
Type: Standards Track
Created: 2016-03-23

== Abstract ==

This BIP describes a way how peers can authenticate – without opening
fingerprinting possibilities – to other peers to guarantee ownership
and/or allowing to access additional or limited services.

== Motivation ==

We assume peer operators want to limit the access of different services
or increase datastream priorities to a selective subset of peers. Also
we assume peers want to connect to specific peers to broadcast or filter
transactions (or similar action that reveals sensitive informations) and
therefore they want to authenticate the remote peer and make sure that
they have not connected to a MITM.

Benefits with peer authentication:
* Peers could detect MITM attacks when connecting to known peers
* Peers could allow resource hungry transaction filtering only to
specific peers
* Peers could allow access to sensitive information that can lead to
node fingerprinting (fee estimation)
* Peers could allow custom message types (private extensions) to
authenticated peers

A simple authentication scheme based on elliptic cryptography will allow
peers to identify each other and selective allow access to restricted
services or reject the connection if the identity could not be verified.

== Specification ==

The authentication scheme proposed in this BIP uses ECDSA, ___secrets
will never be transmitted___.

___Authentication initialization must only happen if encrypted channels
have been established (according to BIP-151 [1]).___

The ___encryption-session-ID___ is available once channels are encrypted
(according to BIP-151 [1]).

The identity-public-keys used for the authentication must be pre-shared
over a different channel (Mail/PGP, physical paper exchange, etc.). This
BIP does not cover a "trust on first use" (TOFU) concept.

The authentication state must be kept until the encryption/connection
terminates.

Only one authentication process is allowed per connection.
Re-authenticate require re-establishing the connection.

=== Known-peers and authorized-peers database ===
Each peer that supports p2p authentication must provide two users
editable "databases"

# ___known-peers___ contains known identity-public-keys together with a
network identifier (IP & port), similar to the "known-host" file
supported by openssh.
# ___authorized-peers___ contains authorized identity-public-keys

=== Local identity key management ===
Each peer can configure one identity-key (ECC, 32 bytes) per listening
network interface (IPv4, IPv6, tor).
The according identity-public-key can be shared over a different channel
with other node-operators (or non-validating clients) to grant
authorized access.

=== Authentication procedure ===
Authentication after this BIP will require both sides to authenticate.
Signatures/public-keys will only be revealed if the remote peer could
prove that they already know the remote identity-public-key.

# -> Requesting peer sends `AUTHCHALLENGE` (hash)
# <- Responding peer sends `AUTHREPLY` (signature)
# -> Requesting peer sends `AUTHPROPOSE` (hash)
# <- Responding peer sends `AUTHCHALLENGE` (hash)
# -> Requesting peer sends `AUTHREPLY` (signature)

For privacy reasons, dropping the connection or aborting during the
authentication process must not be possible.

=== `AUTHCHALLENGE` message ===
A peer can send an authentication challenge to see if the responding
peer can produce a valid signature with the expected responding peers
identity-public-key by sending an `AUTHCHALLENGE`-message to the remote
peer.

The responding peer needs to check if the hash matches the hash
calculated with his own local identity-public-key. Fingerprinting the
requesting peer is not possible.

32bytes challenge-hash `hash(encryption-session-ID || challenge_type ||
remote-peers-expected-identity-public-key)`

`challenge_type` is a single character. `i` if the
`AUTHCHALLENGE`-message is the first, requesting challenge or `r` if
it's the second, remote peers challenge message.

=== `AUTHREPLY` message ===
A peer must reply an `AUTHCHALLENGE`-message with an `AUTHREPLY`-message.

| 64bytes || signature || normalized comp.-signature || A signature of
the encryption-session-ID done with the identity-key

If the challenge-hash from the `AUTHCHALLENGE`-message did not match the
local authentication public-key, the signature must contain 64bytes of
zeros.

The requesting peer can check the responding peers identity by checking
the validity of the sent signature against with the pre-shared remote
peers identity-public-key.

If the signature was invalid, the requesting peer must still proceed
with the authentication by sending an `AUTHPROPOSE`-message with 32
random bytes.

=== `AUTHPROPOSE` message ===
A peer can propose authentication of the channel by sending an
`AUTHPROPOSE`-message to the remote peer.

If the signature sent in `AUTHREPLY` was invalid, the peer must still
send an `AUTHPROPOSE`-message containing 32 random bytes.

The `AUTHPROPOSE` message must be answered with an
`AUTHCHALLENGE`-message – even if the proposed requesting-peers
identity-public-key has not been found in the authorized_peers database.
In case of no match, the responding `AUTHCHALLENGE`-message must
contains 32 bytes of zeros.

| 32bytes || auth-propose-hash || hash || `hash(encryption-session-ID

== Post-Authentication Re-Keying ==

After the second `AUTHREPLY` message (requesting peers signature ->
responding peer), both clients must re-key the symmetric encryption
according to BIP151 while using ___a slightly different re-key key
derivation hash___.

They both re-key with `hash(encryption-session-ID ||
old_symmetric_cipher_key || requesting-peer-identity-public-key ||
responding-peer-identity-public-key)`

== Identity-Addresses ==
The peers should display/log the identity-public-key as an
identity-address to the users, which is a base58-check encoded
ripemd160(sha256) hash. The purpose of this is for better visual
comparison (logs, accept-dialogs).
The base58check identity byte is `0x0F` followed by an identity-address
version number (=`0xFF01`).

An identity address would look like
`TfG4ScDgysrSpodWD4Re5UtXmcLbY5CiUHA` and can be interpreted as a remote
peers fingerprint.

== Compatibility ==

This proposal is backward compatible. Non-supporting peers will ignore
the new `AUTH*` messages.

== Example of an auth interaction ==

Before authentication (once during peer setup or upgrade)
# Requesting peer and responding peer create each an identity-keypair
(standard ECC priv/pubkey)
# Requesting and responding peer share the identity-public-key over a
different channel (PGP mail, physical exchange, etc.)
# Responding peer stores requesting peers identity-public-key in its
authorized-peers database (A)
# Requesting peer stores responding peers identity-public-key in its
known-peers database together with its IP and port (B)

Encryption
# Encrypted channels must be established (according to BIP-151 [1])

Authentication
# Requesting peer sends an `AUTHCHALLENGE` message
AUTHCHALLENGE:
[32 bytes, hash(encryption-session-ID || "i" ||
<remote-peers-expected-identity-public-key>)]

# Responding peer does create the same hash `(encryption-session-ID ||
"i" || <remote-peers-expected-identity-public-key>)` with its local
identity-public-key
# If the hash does not match, response with an `AUTHREPLY` message
containing 64bytes of zeros.
# In case of a match, response with an `AUTHREPLY` message
AUTHREPLY:
[64 bytes normalized compact ECDSA signature (H)] (sig of the
encryption-session-ID done with the identity-key)

# Requesting peer does verify the signature with the
`remote-peers-identity-public-key`
# If the signature is invalid, requesting peer answers with an
`AUTHREPLY` message containing 32 random bytes
# In case of a valid signature, requesting peer sends an `AUTHPROPOSE`
message
AUTHPROPOSE:
[32 bytes, hash(encryption-session-ID || "p" ||
<client-identity-public-key>)]

# Responding peer iterates over authorized-peers database (A), hashes
the identical data and looks for a match.
# If the hash does not match, responding peer answer with an
`AUTHCHALLENGE` message containing 32 bytes of zeros.
# In case of a match, responding peer sends an `AUTHCHALLENGE` message
with the hashed client public-key
AUTHCHALLENGE:
[32 bytes, hash(encryption-session-ID || "r" ||
<client-identity-public-key>)]
# Requesting peer sends an `AUTHREPLY` message containing 64 bytes of
zeros if server failed to authenticate
# Otherwise, response with signature in the `AUTHREPLY` message
AUTHREPLY:
[64 bytes normalized compact ECDSA signature (H)] (sig of the
encryption-session-ID done with the identity-key)
# Responding peer must verify the signature and can grant access to
restricted services.
# Both peers re-key the encryption after BIP151 including the
requesting-peer-identity-public-key and responding-peer-identity-public-key

== Disadvantages ==

The protocol may be slow if a peer has a large authorized-peers database
due to the requirement of iterating and hashing over all available
authorized peers identity-public-keys.

== Reference implementation ==

== References ==

* [1] [[bip-0151.mediawiki|BIP 151: Peer-to-Peer Communication Encryption]]

== Acknowledgements ==
* Gregory Maxwell and Pieter Wuille for most of the ideas in this BIP.

== Copyright ==
This work is placed in the public domain.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160808/e83bc378/attachment-0001.sig>;

📅 Original date posted:2016-06-30 📝 Original message:> ...

2023-06-07T17:51:43Z

In reply to nevent1q…8v4x
_________________________

📅 Original date posted:2016-06-30
📝 Original message:> Yes, this is exactly what I meant. The complexity of the proposed construction is comparable to that of Bitcoin itself. This is not itself prohibitive, but it is clearly worthy of consideration.
>
> A question we should ask is whether decentralized anonymous credentials is applicable to the authentication problem posed by BIP151. I propose that it is not.
>
> The core problem posed by BIP151 is a MITM attack. The implied solution (BIP151 + authentication) requires that a peer trusts that another is not an attacker.

BIP151 would increase the risks for MITM attackers.
What are the benefits for Mallory of he can't be sure Alice and Bob may
know that he is intercepting the channel?

MITM is possible today, it would still be possible (though under higher
costs) with BIP151.

With BIP151 we would have the basic tool-set to effectively reduce the
risks of being MITMled.

IMO we should focus on the risks and benefits of BIP151 and not drag
this discussion into the realm of authentication. This can and should be
done once we have proposals for authentication (and I'm sure this will
be a heated debate).

The only valid risk I have on my list from you, Eric, is the false sense
of security.

My countermeasure for that would be...
- deploy BIP151 together with the simplest form of authentication
(know_hosts / authorized_keys file, no TOFU only editable "by hand")
- make it more clear (in the BIP151 MOTIVATION text) that it won't solve
the privacy/MITM problem without additional authentication.

Or could you elaborate again – without stepping into the realm of
authentication/MITM (which is not part of the BIP or possible already
today) – why BIP151 would make things worse?

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160630/ddefb09d/attachment.sig>;

📅 Original date posted:2016-06-30 📝 Original ...

2023-06-07T17:51:43Z

In reply to nevent1q…pwev
_________________________

📅 Original date posted:2016-06-30
📝 Original message:>>> The core problem posed by BIP151 is a MITM attack. The implied solution (BIP151 + authentication) requires that a peer trusts that another is not an attacker.
>>
>> BIP151 would increase the risks for MITM attackers.
>> What are the benefits for Mallory of he can't be sure Alice and Bob may
>> know that he is intercepting the channel?
>
> It is not clear to me why you believe an attack on privacy by an anonymous peer is detectable.

If Mallory has substituted the ephemeral keys in both directions, at the
point where Alice and Bob will do an authentication, they can be sure
Mallory is listening.

Simple dummy example:
1.) Encryption setup with ECDH with ephemeral keys after BIP151
2.) Mallory is MITMling the connection. He is substituting both
direction with its own keys
3.) Connection is successfully MITMled
4.) Alice tells Bob "prove me that you are Bob, please sign the
session-ID with your identity key"
5.) Bob signs the sessionID (ECDH secret) with his identity key which
will be unusable for Mallory who has a substituted sessionID in both
directions.
6.) Alice has successfully detected the Mallory

Disclaimer: 4) and 5) are _not_ authentication proposals :-)

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160630/0920253c/attachment.sig>;

📅 Original date posted:2016-06-28 📝 Original ...

2023-06-07T17:51:40Z

In reply to nevent1q…zlj2
_________________________

📅 Original date posted:2016-06-28
📝 Original message:>> In my opinion, the question should be "why would you _not_ encrypt".
>
> 1) creation of a false sense of security

False sense of security is mostly a communication issue.
BIP151 does focus on encryption (not trust).

Are users aware of the fact that ISP/WiFi-Providers can track their
bitcoin spending (if they use SPV+BF) and link it with other internet
traffic or sell the data to anyone who is interested to do correlation?

Are node operators aware of the possibilities that ISPs/Data-Centers,
etc. can hold back peers, etc.?

If there is a false sense of security/anonymity, then we are already
deep into this territory.
BIP151 was designed as a puzzle-pice towards better security and better
censorship resistance. You shouldn't project all sorts of "false sense
of security" into BIP151. Is a stepping stone towards greater security.

> 2) as a tradeoff against anonymity

Can you point out the tradeoffs?
BIP151 does not introduce fingerprinting possibilities.

> 3) benefit does not justify cost

Can you elaborate the costs?
[Extremely simplified]: we need 300 lines of code from openssh
(ChaCha20-Poly1305 at openssl) and some ECDH magic (already in
Bitcoin-Cores codebase) together with two or three (maybe payed)
cryptoanalysis once the implementation is done.

>> There are plenty of other options to solve this problem. stunnel,
>> Bernsteins CurveCP, VPN, etc. which are available since years.
>> But the reality has shown that most bitcoin traffic is still unencrypted.
>
> The question arises from concern over the security of the network in the case where encryption (and therefore authentication) is pervasive.
>
> As you point out, anyone can set up a private network of nodes today. These nodes must also connect to the permissionless network to maintain the chain. These nodes constitute a trust zone within Bitcoin. This zone of exclusion operates as a single logical node from the perspective of the Bitcoin security model (one entity controls the validation rules for all nodes).
>
> Widespread application of this model is potentially problematic. It is a non-trivial problem to design a distributed system that requires authentication but without identity and without central control. In fact this may be more challenging than Bitcoin itself. Trust on first use (TOFU) does not solve this problem.

Yes. There is no plan to adopt a TUFO scheme. Bip151 does not use TUFO
it does not cover "trust" (It just encrypt all traffic).

Imaging Bip151 together with a simple form of preshared EC key
authentication (nonce signing or similar). You could drastically
increase the security/censor-resistance-properties between nodes where
owners have preshared identity keys (with nodes I also mean SPV/wallet
nodes).

And I guess there are plenty of awesome identity management system ideas
tied or not tied to the Bitcoin blockchain out there.
This is also a reason to not cover trust/authentication/identity in BIP151.
It is possible to have multiple authentication schemes.

> In my opinion this question has not received sufficient consideration to warrant proceeding with a network encryption scheme (which concerns me as well, but as I consider it premature I won't comment).

Yes. I think nobody have started implementing BIP151. It's a draft BIP
and I think it's still okay and great that we have this discussion.

BIP151 hopefully has started some brainwork in how encryption and
authentication could work in Bitcoin and I'm happy to deprecate BIP151
if we have found a better solution or if we come to a point where we
agree that BIP151 does make the network security worse.

>> Example: IIRC non of the available SPV wallets can "speak" on of the
>> possible encryption techniques. Encrypting traffic below the application
>> layer is extremely hard to set up for non-experienced users.
>
> Bloom filters can (and IMO should) be isolated from the P2P protocol. Also, if the proposal creates an insecurity its ease of deployment is moot.

If we assume increasing amount of novice users starting with Bitcoin
every day, how should these users run wallets without increasing
centralization by using webwallets or client/central-server wallets?
(which is OT, but an interesting question)

>> On top of that, encryption allows us to drop the SHA256 checksum per p2p
>> message which should result in a better performance on the network layer
>> once BIP151 is deployed.
>
> I would not consider this a performance enhancing proposal. Simply dropping the checksum seems like a better option. But again, it is moot if it creates an insecurity.
>
>> I agree that BIP151 _must_ be deployed together with an authentication
>> scheme (I'm working on that) to protect again MITM during encryption
>> initialization.
>
> At a minimum I would propose that you modify BIP151 to declare a dependency on a future BIP, making BIP151 incomplete without it. I think we can agree that it would be unadvisable to deploy (and therefore to implement) encryption alone.

I think BIP151 does what it says: encryption and laying groundwork for
authentication.
You wouldn't probably say BIP32 is incomplete because it does not cover
a scheme how to recover funds (or BIP141 [SW consensus] is incomplete
because it does not cover p2p [BIP144]).

The missing MITM protection (solvable over auth) is prominent mentioned
in the BIP [1].

(from your other mail):
>> I don't see reasons why BIP151 could weaken the security of the P2P network. Can you point out some specific concerns?
>
> TOFU cannot prevent MITM attacks (the goal of the encryption). Authentication requires a secure (trusted) side channel by which to distribute public keys. This presents what I consider a significant problem. If widespread, control over this distribution network would constitute control over who can use Bitcoin.
> The effort to prevent censorship could actually enable it. I don't think it would get that far. Someone would point this out in the process of vetting the authentication BIP, and the result would be the scrapping of BIP151.

I agree that the secure trusted 2nd channel key-sharing problem can be
significant for large networks and/or connecting to unknown identities.

But as said, there could be multiple ways of sharing identity keys.
If you want to connect your node to serval other trusted nodes, you can
simply physically preshare keys or do it over GPG / Signal App, etc..

And if I have followed the news correctly, there are some clever guys
working on various internet of trust 2.0 proposals...

>>
>> BIP151 does not rely on identities. BIP151 does not use persisted keys
>> (only ephemeral keys).
>
> BIP 151 is incomplete without authentication.

I would agree if you would say, _trusted encryption_ is incomplete with
authentication. But IMO BIP151 is complete and should be deployed
together with one or multiple authentication schemes.

[1] https://github.com/bitcoin/bips/blob/master/bip-0151.mediawiki#risks

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160628/3748c989/attachment.sig>;

📅 Original date posted:2016-06-28 📝 Original message:Hi ...

2023-06-07T17:51:36Z

In reply to nevent1q…5xq2
_________________________

📅 Original date posted:2016-06-28
📝 Original message:Hi Eric

> I haven't seen much discussion here on the rationale behind BIP 151. Apologies if I missed it. I'm trying to understand why libbitcoin (or any node) would want to support it.
>
> I understand the use, when coupled with a yet-to-be-devised identity system, with Bloom filter features. Yet these features are client-server in nature. Libbitcoin (for example) supports client-server features on an independent port (and implements a variant of CurveCP for encryption and identity). My concern arises with application of identity to the P2P protocol (excluding Bloom filter features).
>
> It seems to me that the desire to secure against the weaknesses of BF is being casually generalized to the P2P network. That generalization may actually weaken the security of the P2P protocol. One might consider the proper resolution is to move the BF features to a client-server protocol.
>
> The BIP does not make a case for other scenarios, or contemplate the significant problems associated with key distribution in any identity system. Given that the BIP relies on identity, these considerations should be fully vetted before heading down another blind alley.

In my opinion, the question should be "why would you _not_ encrypt".

1) Transaction censorship
ISPs, WIFI provider or any other MITM, can holdback/censor unconfirmed
transactions. Regardless if you are a miner or a validation/wallet node.

2) Peer censorship
MITM can remove or add entries from a "addr" message.

3) Fingerprinting
ISPs or any other MITM can intercept/inject fingerprinting relevant
messages like "mempool" to analyze the bitcoin network.

4) SPV
For obvious reasons regarding BF (see BIP or above).

5) Goundwork for a "client-server" model over the P2P channel
Fee estimation, bloom-filters, or any other message type that requires
authentication.

I would not reduce BIP151 to only solve the BF privacy/censorship problem.

If we agree that censorship-resistance is one of the main properties of
Bitcoin, then we should definitively use a form of end-to-end encryption
between nodes. Built into the network layer.

There are plenty of other options to solve this problem. stunnel,
Bernsteins CurveCP, VPN, etc. which are available since years.
But the reality has shown that most bitcoin traffic is still unencrypted.
Example: IIRC non of the available SPV wallets can "speak" on of the
possible encryption techniques. Encrypting traffic below the application
layer is extremely hard to set up for non-experienced users.

On top of that, encryption allows us to drop the SHA256 checksum per p2p
message which should result in a better performance on the network layer
once BIP151 is deployed.

I agree that BIP151 _must_ be deployed together with an authentication
scheme (I'm working on that) to protect again MITM during encryption
initialization.

---
</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160628/4b67008b/attachment.sig>;

📅 Original date posted:2016-06-28 📝 Original message:> ...

2023-06-07T17:51:33Z

In reply to nevent1q…kkyw
_________________________

📅 Original date posted:2016-06-28
📝 Original message:> To quote:
>
>> HMAC_SHA512(key=ecdh_secret|cipher-type,msg="encryption key").
>>
>> K_1 must be the left 32bytes of the HMAC_SHA512 hash.
>> K_2 must be the right 32bytes of the HMAC_SHA512 hash.
>
> This seems a weak reason to introduce SHA512 to the mix. Can we just
> make:
>
> K_1 = HMAC_SHA256(key=ecdh_secret|cipher-type,msg="header encryption key")
> K_2 = HMAC_SHA256(key=ecdh_secret|cipher-type,msg="body encryption key")

SHA512_HMAC is used by BIP32 [1] and I guess most clients will somehow
make use of bip32 features. I though a single SHA512_HMAC operation is
cheaper and simpler then two SHA256_HMAC.

AFAIK, sha256_hmac is also not used by the current p2p & consensus layer.
Bitcoin-Core uses it for HTTP RPC auth and Tor control.

I don't see big pros/cons for SHA512_HMAC over SHA256_HMAC.

</jonas>

[1]
https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki#child-key-derivation-ckd-functions

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160628/bbe12795/attachment.sig>;

📅 Original date posted:2016-05-14 📝 Original message:> ...

2023-06-07T17:50:42Z

In reply to nevent1q…64u0
_________________________

📅 Original date posted:2016-05-14
📝 Original message:> On 14/05/16 10:16, Jonas Schnelli via bitcoin-dev wrote:
>> Importing a bip32 wallet (bip44 or not) is still an expert job IMO.
>
> That's simply not true. All reasonable wallets (reasonable = user
> oriented) now use BIP39 mnemonic for doing exactly this.

AFAIK: Bip39 import (cross-wallet) is not supported by Schildbachs
android wallet [1] and Electrum [2] and Breadwallet [3].

But I think forming a BIP39 mnemonic into a extended master private key
is not the problem here.

The problems I see:
* What if the "old" wallet has used more then 1000 addresses? I guess
some wallets do not even create a lookup window up to 1000 addresses.
There is a high chance of loosing funds when doing sweep (move all funds
to a new wallet) operation.

* I guess most or maybe all wallets will keep all keys (the
"lookup-window" keys) in the wallet database which could affect
performance [4]

* I guess most wallets do not offer "moving the funds to a new seed" [5]
which results in not solving the problem of a "lost" or "compromised"
wallet and implies wrong security to the enduser.

* If I import a bip39 mnemonic into a hardware wallet (assume Trezor or
Keepkey) I have to type in the words into my computer which bypasses
some of the security my hardware wallet provides me (MITM seed attack).
Together with the point above this reduces the security of a wallet (in
particular cold storage significant).

Please correct me if I'm wrong.

I just wanted to point out that importing a wallet is a tricky step
especially cross-wallet imports (I think cross wallet imports is an
experts job without further improvements).

[1] https://github.com/bitcoin-wallet/bitcoin-wallet/issues/245
[2] http://docs.electrum.org/en/latest/seedphrase.html
[3] https://github.com/voisine/breadwallet/issues/360
[4] https://github.com/bitcoin-wallet/bitcoin-wallet/issues/158
[5]
https://github.com/voisine/breadwallet/blob/master/BreadWallet/BRRestoreViewController.m#L225

</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160514/22aca5af/attachment.sig>;

📅 Original date posted:2016-05-14 📝 Original message:Hi ...

2023-06-07T17:50:42Z

In reply to nevent1q…l8xp
_________________________

📅 Original date posted:2016-05-14
📝 Original message:Hi

> That's a valid concern, but I don't see the conflict here. In order to
> recover funds from a wallet conforming to BIPXX, you must have wallet
> software that handles BIPXX. Simply making BIPXX backwards compatible
> with previously created BIP44 or BIP43 purpose 0 wallets doesn't change
> this at all.

Maybe I'm going a bit offtopic. Sorry for that.

Importing a bip32 wallet (bip44 or not) is still an expert job IMO.
Also importing can lead to bad security practice (especially without a
sweep).

Users will send around xpriv or import an seed over a compromised
computer to a cold storage, etc.

I don't think users want to import private keys.
They probably want to import the transaction history and send all funds
covered by that seed to a new wallet.

I often though that task is better covered by a little GUI tool or
cli-app/script:
-> Accept different bip32 schematics (bip32 native, bip44, etc.)
-> Accept different bip39 (like) implementation
-> Create large lookup windows
-> Create a sweep transaction to a new address/wallet and sign/broadcast it.
-> Export transaction history (CSV)

But maybe I'm over-complicating things.

--
</jonas>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20160514/fb88c8d0/attachment.sig>;

📅 Original date posted:2016-02-29 📝 Original ...

2023-06-07T17:49:22Z

In reply to nevent1q…sahq
_________________________

📅 Original date posted:2016-02-29
📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi

I’ve been thinking around a solution to reduce nodes bootstrap time
(IBD) as well as a way to reduce the amount of bandwidth/network usage
per node.
Not sure if this idea was/is already discussed, haven’t found anything
in a quick research.

==Title==
Fast bootstrapping with a pre-generated UTXO-set database.

==Abstract==
This documents describes a way how bitcoin nodes can bootstrap faster
by loading a pre-generated UTXO-set datafile with moderate reduction
of the security model.

==Specification==
Bitcoin-core or any other full node client will need to provide a
feature to "freeze" the UTXO-set at a specified height (will require a
reindex). The frozen UTXO-set – at a specific height – will be
deterministic linearized in a currently not specified
data-serializing-format.
Additionally, a serialized form of the current chain-index (chain
containing all block-headers) up to the specified height will be
appended to the pre-generated UTXO-set-datafile.
The datafile will be hashed with a double SHA256.

The corresponding hash will be produced/reproduced and signed (ECDSA)
by a group of developers, ideally the same group of developers who are
also signing deterministic builds (binary distribution).

Full node client implementations that supports bootstrapping from a
pre-generated UTXO-set, need to include...
1.) a set of pubkeys from trusted developers
2.) the hash (or hashes) of the pre-generated UTXO-set-datafile(s)
3.) n signatures of the hash(es) from 2) from a subset of developers
defined in 1)

To guarantee the integrity of developers pubkeys & signatures, methods
like the current gitian build, used in bitcoin-core, must be used.

New nodes could download a copy of the pre-generated UTXO-set, hash
it, verify the hash against the allowed UTXO-sets, verify the ECDSA
signatures from various developers, and continue bootstrapping from
the specified height if the users accepts the amount of valid signatures
.

Sharing of the pre-generated UTXO-set can be done over CDNs,
bit-torrent or any other file hosting solution. It would also be
possible to extend the bitcoin p2p layer with features to
distribute/share a such pre-generated UTXO-set, in chunks and with the
according hashes to detect invalidity before downloading the whole
content (but would probably end up in something very similar to
bit-torrent).

- ----------------------
</jonas>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJW1B1wAAoJECnUvLZBb1PsqzsP/iSdvyhUzy+BZVSZbKXNjk5P
2vrtirI6NvKQd8hHbrcFeLfyswzYc2JWRnX8sATlauIS0pYdr97JriwUGlvxvNrY
iVTDdf8MIVu8zScLQtJbMatpMvsewtqQEidn/yxWIhiCg4G2T5DZmlBU6O4XIKR6
5aPHElGOKZ15EWGHBG7z4owj3MiOaxhD9q5erBbfLPpcm08o6XAv5miqmGnnn3zh
gocg4Gxs6iDygh3b2dCJFwWIVPxF6UVJhyjv2kLZUmEHT2Y2QvdGcLIIewcWHDze
kgoZYmOEowujCbmeJ+LBwgOI0c1N6L/ciomPBne7ILmK4LyUEzyMLJKNYf/sZ8vI
sVlmwZwZZLfILC7mzMAM0pfj99IOW680WHch9v31lWFlxW/bLvLqAO7n3acQuD6s
xCZN2nAhmWC8FnMFxqB3EUz0lX8giV3qRJZjbQMS+ZrngYkAmVv2bAsoLndqf6MO
l9W8B+ICg1KZLGIOF2pUrInpkB6gUALDFnypV4CeIVdeqtk5l4LnCHK6c4++Hl5n
Bv5HQ/wTgKKNFtHBEJpWyYWvAjfFtgUZUKblR+Bh+D7/Gte1ehiYd02KYD4ds9Y4
3gfO8YbAz/I14Yuh2bIlvVKPWnLQBwL5BBioBfvmhV/r6rEpzWvB7H6Fmi1c759l
VlL0GiUV8ar2LlFhEmWk
=lZSy
-----END PGP SIGNATURE-----

📅 Original date posted:2015-10-23 📝 Original ...

2023-06-07T17:43:52Z

In reply to nevent1q…3umd
_________________________

📅 Original date posted:2015-10-23
📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> Here is the beginnings of an implementation to replace leveldb
> with sqlite: https://github.com/jgarzik/bitcoin/tree/2015_sqlite
>
> It builds, but still needs work before passing tests.

Nice work!

Although not sure if we should focus directly on sqlite4 (could be
optional with a configure flag and a subtree [until stable], sqlite3
supported over depends).

Also i personally would recommend to not implement it as a
replacement, instead, support multiple backends (wrapper header /
different wrapper implementations [leveldb / sqlite3 / sqlite4]). But
– agreed – should not be the focus, but a nice additional flexibility
if it not require much more work.
And – in theory – multiple database back-ends would allow migration.

Before investigating to deep, i think we need a dbwrapper bench tool
that represents our needs and our style how we hit the database.
Gavins recently added bench target / bench environment could be used
for that.

/jonas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJWKdlgAAoJECnUvLZBb1Ps7J8P/2L6215fd0rWGv5uvbLSnQvm
hy1T2AaOfH5HXd2m95icaKYk+ugvQAL80Q/67YwZbPLsT4fErgegw8n75Z6nh/Pc
OJ1EtAvD+Kc/Vm0Wcvt421HXwnm4f+j+8eoEvpG6DdC8gfIG+efhM7DljeNbPFrA
ieNe7XrQ1EZ29lMzpQv0nx3bi6tUHOWuazk82B6vnK49MjH7nrUFipcc18nXbSpM
ZKjQakXmfqIBG8QP9ZUYUlW4aoG0oaOoZnGjQA2LeXBJpIPLpE/WPg0XaXubC+No
232CJNtNyUOmjkb2qbep6vSYgqGJNy1HbCU5y3qooxJhFnKdo63CQsyJKSpL/ssi
0lWraNxjbacxsBn+63In3wEkj02orwm2zTO4I77wCrZmJgpvBFb9bZWTeL8DCYSG
DTkZoKWEK74xvm+dNpEWXK9Lm1ltfyhPdaFeMoRDub4w2uuYlk3KuD8Vdy81HYJb
sak8FbkiWw9xx2OP9+G56Arf9W6mnJ3YHJGrY4SXeRAfuNYwGFHIGt6I1JobuHy4
VWmcHuooz/Q9JLUu3Rr3HsEdXYgCmgmWuzTgWG+Hx92Y6XLWV5pOX+vFRO6J5fSm
aTYPD4GsTM3FROXw5ezlXJ+2y1+ITzmrfm03fEDQvoIyH0TwBqBc5sMBBkma5DDR
0HUthPHWCD+AxbBPbRVa
=CUMX
-----END PGP SIGNATURE-----

📅 Original date posted:2015-10-05 📝 Original ...

2023-06-07T17:42:22Z

In reply to nevent1q…888a
_________________________

📅 Original date posted:2015-10-05
📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> Perhaps Pedro wants to also participate in a 2-of-2 cosigning
> arrangement with a merchant that will deliver a laptop to him, so
> Pedro provides this merchant with the same extended public key
> derived from path m/45', and the merchant provides Pedro with his
> own:

What holds you back from using m/i'/45' where i' is your multisig
"account" number?

Within your BIP45 wallet (lets assume Copay), you would not provide
the xpubkey at m/45', instead you would provide your xpubkey at m/i'/45'
.

It's probably no longer pure BIP45.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJWEm34AAoJECnUvLZBb1Ps+TgP/RmIxp31GOF9JDYVvSpaWfjf
NhB7o5/AiQ5rNv2BL5Wdoebb3EHzPTjzf/gGq3vSHQkm+c5rnDO8kTtwlNog7HN3
djtlbdhftlqcWdgBYNRstMj52NlfQKz4exvaPTsFlxUS02VzafPhssde9H7KdBHw
HoePJziOku6ibHW3u2CdEtGHYSRZ0w7xUmGYUjYc9LRa86Gl5aHFOzTVmboQgNW+
MACb30gSjMGP6i4iHsuwYMpc7DJ1SyooqAL64Z+YN/ZJayHatJIxVx8GGPC2HW5g
nj1LuJacK5VA2VtqtnRhVP0IEm4rKSgSzJz/534lB8/6RFejR0VnjzLZcMsa2TcQ
7yWf6xAN3ijMWi8mxG2RL/2xZiH3txpdMLme3e5YyZ8sgDw04zJI8lKRkg8kN8Wo
G8JQ/6hAzK9OhceoCnazE1ODXRiKa/jXvtfdiuzAb7Q3zFbnXNT2b7dshU0eY6iW
GWBmOInBROfoAqt43gQIMC0gpBsQc9JlVN5s6CgPxhEAMsZI/akKm4qufGbvb767
USgzh6UPE+secLkgjZC/lVKmk7rD+7poJjZB5XzaGdKAxzpFTqw/gLXhfG17TVPA
ZwA2UFhnHVitP839RenrCxWORng2F8gDO/ElWTWtwCq7+c2UxEOxSiKfTOahEoFw
LmOUFxn5iMSbdBEbkm54
=cbgc
-----END PGP SIGNATURE-----

📅 Original date posted:2015-09-18 📝 Original ...

2023-06-07T17:40:50Z

In reply to nevent1q…erv3
_________________________

📅 Original date posted:2015-09-18
📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

> On Friday, September 18, 2015 1:07:10 AM Wladimir J. van der Laan
> via bitcoin-dev wrote:
>> At Monday's code sprint we had a good idea to schedule a regular
>> developer meeting in #bitcoin-dev.
>>
>> Attendance is of course voluntary, but it may be good to have a
>> time that many people are expected to be present and current
>> issues can be discussed.
>
> I think it's important to make a point that these meetings are for
> discussions, and explicitly never decisions, to avoid a repeat of
> the P2SH events when people have to miss it.
>
>> Any preference for days/times?
>>
>> What about e.g. every week 15:00-16:00 UTC on Thursday?

+1 for the weekly IRC meetings.

For time and date maybe a Doodle could help:
http://doodle.com/poll/cihug53sa8u4h2in#table

- ---
jonas
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJV+7eaAAoJECnUvLZBb1PsESgQAIQnHDe7owv3OzMvxwupzGaD
IkTsRtCTSntIb75Wb/KYc0y1L3ilSENRTfZ4nNc4QquqTstkhjU5t+u9T3Mak4D3
2/5AOiJhV6OLYav1SC7uSJh0B4halnZlTwclU7NOvmnkg40DUpNxmEbf+RvUZup3
J0EQFxIuhtjIz0HfZTvw6wmstrP3+UJZTbM5fg0FO3TpgmGybAUoQ3eWgRa7v/gR
OUxnAV//Mus2O80/Z+c5KycZ1Dqc/iN7cQsQFt7kEIK0epkJhkTjoRrW9MyQW04d
1jv7d0mjEJt+2EiC8UuwpaW2eFmeFnGR0pL4UCY1QsDzGENyHKNbrVg26v1AzIbB
SNEYN1+fmsXQYosY5t0Z887Ij+u4/GLHciimh5z7fbI5VB1Ng6Wl84maVmP5Zb3L
MHtkIqQ00RX7GIXUp5+u7eKOO0pH9S08tqo5Ag6ceynJ2lh4Wr8BCNghHzH+ybNJ
NG3BaSkQmjxnWjW3XplaYyxz6E4qJ8id7qH4s0iaNKchAfXiCaBtbcMfljlyBSn/
UbzHJk5jlWZEVpxmiMRctFxusk6GI4P+0eRTJrkffskLEjImUN93A8hOLs5Dy+gI
mm/PZKT2S2qKKa6dlI2kpyPZuRbN7+WSi/FwI0YsUDGl+IoDSqTX7WqRY8cY40ji
rUgzYTw3Won3BcjHTe9y
=JsXj
-----END PGP SIGNATURE-----

📅 Original date posted:2015-06-22 📝 Original ...

2023-06-07T15:39:33Z

In reply to nevent1q…29a3
_________________________

📅 Original date posted:2015-06-22
📝 Original message:-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev The
> move is now complete. The previous archive has been fully
> imported and new posts here will now be saved.

Thanks for the work!

What about updating bitcoin.org (https://bitcoin.org/en/development)?
I guess people start looking there...

Same for this...
https://bitcointalk.org/index.php?topic=15527.0

Bip0001 needs also an update...
https://github.com/bitcoin/bips/blob/master/bip-0001.mediawiki

</jonas>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJVh63PAAoJECnUvLZBb1PsppEQAJTanEPNxRGZk88rlQP2OpYy
1Giw5LwBHLQ/ybaiUxCzEHJqztvK+kHblXWsh/W1QtHV0Mi1HqfYLFpwN9a02bAh
2uBGmHURsmIM1Gr/5ayj+Sz5YtvqXKIUMYlI4vnbvRYWV/JEuoTjsWtavb5prZZc
g0VwQkNZBpBbgQ2apmdZ9ywa8sn4ShUSZiVnmi63pGijqOU3E1w0vt193cvCm85F
h535XF6j5ZFNbdC1EIMo6QZXTRUhLTJ+XYTbJZckF+Ncm0uQOPlqo046jynSdgcK
nUfoodKPq/fyOtOayVAd+Pwqat15oMmUd+Y7qzmxEBSewcmfwLKI+BUnlb332JTz
zgeB4k4JW5627iOwa7wably/XHRbPeByqMyEVEX8nMbrypAcHVGWdjkJCVUdjcrZ
u0PgSAytnIytQrbhA+woUlYhitqx8EYZXmiCs+pBuGwAnwxLXS9fnuKKns7w3glB
MR6b9Ab25bSdVmIJ93W4tEcmfTHzWc9aNBU2td1/c+YuLnH+OFJuu15ef4dWlNeR
Ed+f+cZuGXS8kgEnE/XB+HUVWfktktpUgKvmtZ5kUP/vErBwA+/B+i1drOp38kfv
iGuUThCiep38w64dkV9KVJNp5z1k3PMoVCvgIhZjiDoL8YSBzlXL1iuGDKxtRhRk
6IuBDtLlegeL0/I0d3UV
=ShCq
-----END PGP SIGNATURE-----