{"type":"rich","version":"1.0","author_name":"npub1m230cem2yh3mtdzkg32qhj73uytgkyg5ylxsu083n3tpjnajxx4qqa2np2","author_url":"https://nostr.ae/npub1m230cem2yh3mtdzkg32qhj73uytgkyg5ylxsu083n3tpjnajxx4qqa2np2","provider_name":"njump","provider_url":"https://nostr.ae","html":"📅 Original date posted:2012-11-28\n📝 Original message:On Mon, Nov 26, 2012 at 05:37:31PM -0500, Gavin Andresen wrote:\n\u003e Why not JSON?\n\u003e -------------\n\u003e \n\u003e Invoice, Payment and Receipt messages could all be JSON-encoded. And\n\u003e the Javascript Object Signing and Encryption (JOSE) working group at\n\u003e the IETF has a draft specification for signing JSON data.\n\u003e \n\u003e But the spec is non-trivial. Signing JSON data is troublesome because\n\u003e JSON can encode the same data in multiple ways (whitespace is\n\u003e insignificant, characters in strings can be represented escaped or\n\u003e un-escaped, etc.), and the standards committee identified at least one\n\u003e security-related issue that will require special JSON parsers for\n\u003e handling JSON-Web-Signed (JWS) data (duplicate keys must be rejected\n\u003e by the parser, which is more strict than the JSON spec requires).\n\u003e \n\u003e A binary message format has none of those complicating issues. Which\n\u003e encoding format to pick is largely a matter of taste, but Protocol\n\u003e Buffers is a simple, robust, multi-programming-language,\n\u003e well-documented, easy-to-work-with, extensible format.\n\nI'm not sure this is actually as much of an advantage as you'd expect. I\nlooked into Google Protocol buffers a while back for a timestamping\nproject and unfortunately there are many ways in which the actual binary\nencoding of a message can differ even if the meaning of the message is\nthe same, just like JSON.\n\nFirst of all while the order in which fields are encoded *should* be\nwritten sequentially, parsers are also required to accept the fields in\nany order. There is also a repeated fields feature where the\nfields can either be serialized as one packed key-list pair, or multiple\nkey-value(s) pairs; in the latter case the payloads are concatenated.\n\nThe general case of how to handle a duplicated field that isn't supposed\nto be repeated seems to be undefined in the standard. Yet at the same\ntime the standard mentions creating messages by concatenating two\nmessages together. Presumably parsers treat that case as an error, but I\nwouldn't be surprised if that isn't always true.\n\nImplementations differ as well. The current Java and C++ implementations\nwrite unknown fields in arbitrary order after the sequentially-ordered\nknown fields, while on the other hand the Python implementation simply\ndrops unknown fields entirely. As far as I know no implementation\npreserves order for unknown fields.\n\nFinally, while not a Protocol Buffers specific problem, UTF8 encoded\ntext isn't guaranteed to survive a UTF8-UTFx-UTF8 round trip.  Multiple\ncode point sequences can be semanticly identical so you can expect some\nsoftware to convert one to the other. Similarly lots of languages\ninternally store unicode strings by converting to something like UTF16.\nOne solution is to use one of the normalization forms such as NFKD - an\nidempotent transformation - although I wouldn't be surprised if\nnormalization itself is complex enough that implementation bugs exist,\nnot to mention the fact that the normalization forms have undergone\ndifferent versions.\n\nI think the best way(1) to handle (most) the above by simply treating the\nbinary message as immutable and never re-serializing a deserialized\nmessage, but if you're willing to do that just using JSON isn't\nunreasonable either.\n\n\n1) Of course I went off an created Yet Another Binary Serialization for\nmy project, but I'm young and foolish...\n\n-- \n'peter'[:-1]@petertodd.org\n-------------- next part --------------\nA non-text attachment was scrubbed...\nName: signature.asc\nType: application/pgp-signature\nSize: 490 bytes\nDesc: Digital signature\nURL: \u003chttp://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20121128/ac1eb23f/attachment.sig\u003e"}