{"type":"rich","version":"1.0","author_name":"npub1m230cem2yh3mtdzkg32qhj73uytgkyg5ylxsu083n3tpjnajxx4qqa2np2","author_url":"https://nostr.ae/npub1m230cem2yh3mtdzkg32qhj73uytgkyg5ylxsu083n3tpjnajxx4qqa2np2","provider_name":"njump","provider_url":"https://nostr.ae","html":"📅 Original date posted:2012-11-28\n📝 Original message:On Wed, Nov 28, 2012 at 11:43:19AM +0100, Mike Hearn wrote:\n\u003e Peter is correct that there are a few degrees of freedom in protobuf\n\u003e serialization, though far fewer than with JSON.\n\nFWIW I re-read the specs again and turns out my memory was wrong. (I\nlast looked at this about four months ago) Duplicated fields are handled\nin a defined manner, with the last field seen in the serialization being\nthe one whose value is used. Again, repeated fields are treated as\nelements of a list, preserving order.\n\nIt does raise the interesting question do the implementations that don't\npreserve order of unknown fields, preserve the order of multiple unknown\nfields, either repeated or not?\n\n\u003e I'd like to think upstream would be open to resolving these\n\u003e ambiguities.\n\nI gotta admit, I suspect they won't be that open. Protocol buffers was\ndesigned because Google needed a fast serialization method suitable for\nmany different internal projects. Needing round-trip idempotence seems\nlike a rare requirement to me, especially for internal use.\n\n\u003e Re-serialization of an Invoice message in the Payment message is a\n\u003e potential source of mistakes. There's no need to ever concatenate\n\u003e these messages and alternative implementations that don't order\n\u003e serialized fields by tag number are missing an important optimization,\n\u003e so they could be fixed. The main issue is treatment of unknown fields.\n\u003e If/when the Invoice message is extended with other fields that are\n\u003e round-tripped through an old client, the data may get lost. JSON\n\u003e doesn't help resolve that either, of course. There are a few\n\u003e solutions:\n\nWell, actually you can take advantage of the message concatination\nability of protocol buffers to extend a message by simply appending the\nnew fields to the existing thus either defining new fields, or\noverriding old values as required. If you want to de-duplicate though\nyou run into the problem all over again.\n\nOn the other hand JSON handles this case fine too provided that your\nJSON implementation supports dictionary objects with arbitrary fields.\nJust use the object as is and the unknown fields will be re-serialized\nproperly at the other end. Some implementations will have to be careful\nto handle collisions with existing keys in the namespace. (consider in\nPython what would happen if you mapped your object to a class instance,\nand the serialization included the key \"__init__\")\n\nThat said, JSON is quite problematic with numbers. For instance, you\nhave to be careful to keep integers represented as pure integers below\nwhat Javascript can handle, the maximum integer exactly representable in\na double float, or the JSON won't be parsable in Javascript even if many\nother languages handle it fine. Protocol buffers is at least pretty\nexplicit about what size integers are.\n\n\u003e 1) Change the type of the Invoice field in Payment to be \"bytes\" and\n\u003e set it to be the hash of the originally received binary Invoice\n\u003e message. Downside, requires merchants to track all outstanding\n\u003e invoices.\n\u003e 2) Ask protobufs upstream to modify the spec/implementations so\n\u003e ordering of unknown fields is specified. The Python implementation\n\u003e could be extended to support them so Python implementors don't end up\n\u003e with accidental message downgrades.\n\u003e 3) Language of the spec could be changed to explicitly state that the\n\u003e received Invoice may not be binary-identical to the one that was sent,\n\u003e in the case of a client that incorrectly downgrades the message. Thus\n\u003e you'd be expected to check what the Invoice was using merchant_data\n\u003e which is opaque and could just be, eg, a database key on your own end.\n\u003e 4) Instead of submitting the entire Invoice back to the merchant, just\n\u003e the merchant_data could be in the Payment message.\n\u003e \n\u003e Of the four options I prefer the last. What is the use case for\n\u003e resubmitting the entire invoice anyway? Even if protobufs are improved\n\nNote that I think the SignedInvoice message itself is broken, because\nprotobuf implementations have no reason to guarantee that they can give\nyou the serialized bytes of the Invoice sub-message. It's a quite\nspecific use-case that isn't needed for pretty much anything but crypto.\nFWIW I took a quick look at the official API's, C++, Java and Python,\nand as far as I can tell none of them support accessing the binary\nserialization of a message field other than by re-serializing the\nmessage.\n\nReally the invoice field should be declared as bytes serialized_invoice,\nas inconvenient as that is to work with.\n\n\u003e so handling of round-tripping new messages through old [Python]\n\u003e clients is more rigorous, some implementors will probably convert the\n\u003e protobuf objects into some internal forms for whatever reason (or\n\u003e serialize them to a database, etc) and they're very likely to mess up\n\u003e the handling of unknown fields when they do it.\n\nSince the Payment message includes an *untrusted* Invoice that the\nvendor needs to authenticate the whole invoice no matter what on Payment\nreception. In many cases that implies they have to keep some sort of\ndatabase of \"quotes\" or similar anyway as the client can change anything\nthey want otherwise. Again that leads back to the argument of why not\njust stick with the merchant_dat as you suggest, which will usually be\nsome short invoice number attached to a database? A vendor that wants to\noperation a stateless invoicing system can just stuff a HMAC-protected\nserialized invoice into the merchant_data\n\nI guess you could use a mutable invoice field as a way of achieving some\nsort of negotiation protocol, but I think it's better to stick to the\noriginal concept of just ensuring that the user is really paying the\nright amount to the right address.\n\n-- \n'peter'[:-1]@petertodd.org\n-------------- next part --------------\nA non-text attachment was scrubbed...\nName: signature.asc\nType: application/pgp-signature\nSize: 490 bytes\nDesc: Digital signature\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20121128/46255e3d/attachment.sig\u003e"}