From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1YGWRb-0007fG-Op for bitcoin-development@lists.sourceforge.net; Wed, 28 Jan 2015 17:30:23 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of bitpay.com designates 209.85.214.169 as permitted sender) client-ip=209.85.214.169; envelope-from=jgarzik@bitpay.com; helo=mail-ob0-f169.google.com; Received: from mail-ob0-f169.google.com ([209.85.214.169]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1YGWRW-0006MI-1C for bitcoin-development@lists.sourceforge.net; Wed, 28 Jan 2015 17:30:23 +0000 Received: by mail-ob0-f169.google.com with SMTP id va8so20504149obc.0 for ; Wed, 28 Jan 2015 09:30:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=mNx/VKgV9dchBGf/CJftjxXB1sfqwnTuHjBONdI2GmM=; b=iR0p1t7hNt7fCRUWxdTtjq6Jxbf3hgKgmYBdnGHRLXYNqclgmq9V09l8jodj5s+113 KhDrcJ97JccYydPVvvS7x2qfC0pnk5CE6FmMbXXkUm5woGHS+AW0mOFW6tt28cyOFt9J QjxiRGAescojnSrP4tQvob5VmN9fahQQnd7iOiBKDMjXz1rNURduEEuTJm+pcSsm11EF y8LN86bsZii2AoKQ7aveEqvnVbEdej1pA7pyMMGxScd58GoV5h87sIrK5AB6+3AqZEyI 3mxUAcQrS5B2bXxU2XrlPY0WvthvIs+UWltdO2Vw6O1G6MI1nGXzOUkFfAxyUaI6MesG zbww== X-Gm-Message-State: ALoCoQmQOfA20t5QvPzOizJ9zdIjRoZRKgfGOfJPBPmQWKu6K7Xuq6N7qXfgWaDvUBGn8CrKI2Mn X-Received: by 10.182.27.207 with SMTP id v15mr2883145obg.21.1422466213508; Wed, 28 Jan 2015 09:30:13 -0800 (PST) MIME-Version: 1.0 Received: by 10.202.219.196 with HTTP; Wed, 28 Jan 2015 09:29:53 -0800 (PST) In-Reply-To: References: From: Jeff Garzik Date: Wed, 28 Jan 2015 12:29:53 -0500 Message-ID: To: Nicolas DORIER Content-Type: multipart/alternative; boundary=001a11331f505c8c39050db9b93f X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1YGWRW-0006MI-1C Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for encoding? X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jan 2015 17:30:23 -0000 --001a11331f505c8c39050db9b93f Content-Type: text/plain; charset=UTF-8 It is not "fear", it is field experience. JSON has proven to be a bug generator for the reasons already stated. JSON does not include type marshalling and input validation. Protobufs/msgpack/etc. engineered those to occur automatically, because that is an area shown by field experience to be a constant source of bugs and inconsistent parsing/validation behavior. On Wed, Jan 28, 2015 at 11:52 AM, Nicolas DORIER wrote: > For the number of field there is in the spec, I don't consider having a > JSON to schama really worthwhile. > If you fear it is error prone, then we should provide some testing data > for the BIP70. (Which I already did for protobuf, but was rejected, because > deemed no useful thanks to the code generator... But such code generator > gave me inconsistencies with gavin's implementation for example) > > Why do you think type support is very useful in our case ? we have 3 > types, and dealing only with bytes, int, and string. > It cost me more time to find a suitable cross plateform lib for protobuf > (in c#, that works in ios and winrt) than I would by just coding the json > wrapper classes by hand. (JSON libs are more wildspread and supported than > protobuf) > > 2015-01-28 17:04 GMT+01:00 Jeff Garzik : > >> Not to mention the tiresome and error-prone task of writing your own >> JSON-to-schema marshalling code -- or something equivalent to the protobufs >> compiler and libs for JSON. >> >> protobufs -- and its modern competitors such as msgpack -- natively >> provide type support in a way that must be hacked into JSON or XML. >> >> The protobuf/msgpack design is engineered to avoid bugs routinely found >> in JSON parsing code; due to the amount of code & effort involved in JSON >> input sanity checking, bugs and inconsistencies inevitable arise. We have >> seen this in bitcoind with JSON-RPC. >> >> >> >> On Wed, Jan 28, 2015 at 10:42 AM, Mike Hearn wrote: >> >>> On the other hand, if you charge the developer (and not the plateform) >>>> to check certificate validity, it means that you have to develop a >>>> different codebase for all plateform you are targeting, because each >>>> plateform store trusted root certificate in a different manner with >>>> different APIs, and also have different types representing a X509 >>>> Certificate. >>>> >>> >>> That's what cross-platform abstraction libraries are for. Both Java and >>> Qt provide a key store library that can load from either the OS root store >>> or a custom one. If your chosen app platform doesn't, OK, then you'll have >>> to make or find one yourself. Perhaps contribute it upstream or make it a >>> library. But that's not a limitation of BIP70. >>> >>> Just as a reminder, there is no obligation to use the OS root store. You >>> can (and quite possibly should) take a snapshot of the Mozilla/Apple/MSFT >>> etc stores and load it in your app. We do this in bitcoinj by default to >>> avoid cases where BIP70 requests work on some platforms and not others, >>> although the developer can easily override this and use the OS root store >>> instead. >>> >>> Of all possible solutions, using a third party service to convert things >>> to JSON is one of the least obvious and highest effort. I don't know anyone >>> else who arrived at such a conclusion and respectfully disagree that this >>> is a problem with the design choices in BIP70. It sounds like a bizarre >>> hack around lack of features in whatever runtime you're using. >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> Dive into the World of Parallel Programming. The Go Parallel Website, >>> sponsored by Intel and developed in partnership with Slashdot Media, is >>> your >>> hub for all things parallel software development, from weekly thought >>> leadership blogs to news, videos, case studies, tutorials and more. Take >>> a >>> look and join the conversation now. http://goparallel.sourceforge.net/ >>> _______________________________________________ >>> Bitcoin-development mailing list >>> Bitcoin-development@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >>> >>> >> >> >> -- >> Jeff Garzik >> Bitcoin core developer and open source evangelist >> BitPay, Inc. https://bitpay.com/ >> > > -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ --001a11331f505c8c39050db9b93f Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
It is not "fear", it is field experien= ce.

JSON has proven to be a bug generator for the reasons alre= ady stated.

JSON does not include type marshalling and input v= alidation.=C2=A0 Protobufs/msgpack/etc. engineered those to occur automatic= ally, because that is an area shown by field experience to be a constant so= urce of bugs and inconsistent parsing/validation behavior.



<= /div>

On Wed, Jan = 28, 2015 at 11:52 AM, Nicolas DORIER <nicolas.dorier@gmail.com&= gt; wrote:
F= or the number of field there is in the spec, I don't consider having a = JSON to schama really worthwhile.
If you fear it is error prone, t= hen we should provide some testing data for the BIP70. (Which I already did= for protobuf, but was rejected, because deemed no useful thanks to the cod= e generator... But such code generator gave me inconsistencies with gavin&#= 39;s implementation for example)

Why do yo= u think type support is very useful in our case ? we have 3 types, and deal= ing only with bytes, int, and string.
It cost me more time to= find a suitable cross plateform lib for protobuf (in c#, that works in ios= and winrt) than I would by just coding the json wrapper classes by hand. (= JSON libs are more wildspread and supported than protobuf)

2015-01-28 17:04 GMT+01:00 Jeff Garzik <jgarzik@bitpay.com&g= t;:
Not= to mention the tiresome and error-prone task of writing your own JSON-to-s= chema marshalling code -- or something equivalent to the protobufs compiler= and libs for JSON.

protobufs -- and its modern competitors su= ch as msgpack -- natively provide type support in a way that must be hacked= into JSON or XML.

The protobuf/msgpack design is engineered t= o avoid bugs routinely found in JSON parsing code; due to the amount of cod= e & effort involved in JSON input sanity checking, bugs and inconsisten= cies inevitable arise.=C2=A0 We have seen this in bitcoind with JSON-RPC.


On Wed, Jan 28, 2015 at 10:42 AM, Mike Hearn <mike@plan= 99.net> wrote:
=
On th= e other hand, if you charge the developer (and not the plateform) to check certificate validity, it=20 means that you have to develop a different codebase for all plateform=20 you are targeting, because each plateform store trusted root certificate in a different manner with different APIs, and also have different=20 types representing a X509 Certificate.

That's what cross-platform abstraction librari= es are for. Both Java and Qt provide a key store library that can load from= either the OS root store or a custom one. If your chosen app platform does= n't, OK, then you'll have to make or find one yourself. Perhaps con= tribute it upstream or make it a library. But that's not a limitation o= f BIP70.

Just as a reminder, there is no obligatio= n to use the OS root store. You can (and quite possibly should) take a snap= shot of the Mozilla/Apple/MSFT etc stores and load it in your app. We do th= is in bitcoinj by default to avoid cases where BIP70 requests work on some = platforms and not others, although the developer can easily override this a= nd use the OS root store instead.
=C2=A0
Of all possibl= e solutions, using a third party service to convert things to JSON is one o= f the least obvious and highest effort. I don't know anyone else who ar= rived at such a conclusion and respectfully disagree that this is a problem= with the design choices in BIP70. It sounds like a bizarre hack around lac= k of features in whatever runtime you're using.


-----------------------------------------------------------= -------------------
Dive into the World of Parallel Programming. The Go Parallel Website,
sponsored by Intel and developed in partnership with Slashdot Media, is you= r
hub for all things parallel software development, from weekly thought
leadership blogs to news, videos, case studies, tutorials and more. Take a<= br> look and join the conversation now. http://goparallel.sourceforge.net/
_______= ________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment



=
--
Jeff Garzik
Bitcoin core developer and open source evang= elist
BitPay, Inc. =C2=A0 =C2=A0 =C2=A0https://bitpay.com/




--
Jeff Garzik
Bitcoin core developer and open source evangelistBitPay, Inc. =C2=A0 =C2=A0 =C2=A0https://bitpay.com/
--001a11331f505c8c39050db9b93f--