From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8B6EEC0051 for ; Tue, 18 Aug 2020 17:16:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id 7FF19864B2 for ; Tue, 18 Aug 2020 17:16:15 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gPrEtiv0ZcQm for ; Tue, 18 Aug 2020 17:16:14 +0000 (UTC) X-Greylist: delayed 00:15:39 by SQLgrey-1.7.6 Received: from mail-vs1-f46.google.com (mail-vs1-f46.google.com [209.85.217.46]) by whitealder.osuosl.org (Postfix) with ESMTPS id E282486174 for ; Tue, 18 Aug 2020 17:16:13 +0000 (UTC) Received: by mail-vs1-f46.google.com with SMTP id a1so10513618vsp.4 for ; Tue, 18 Aug 2020 10:16:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=voskuil-org.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=Jc53nrAqFJzWpeihvWmPt4pPa8IxoJsUWg6qJW7UN5M=; b=DolzfglfOiWu9znjkfmx0i9K7jHIMlUbIZxHMKdc1vFGc0DpuM2v8vvR+DJr5jvgzo +0f44HohXAKxhDO48sD/n1dsCS7w1nQGz6j3OT/H7edjjwFLXAWsvstrJgPYzkSGJfIX cw7WK1FOSgtpu+1O/2FlZ4MqVCDXjNZzTEMgBBciLLhpLFI8ZxTWc6fvkA4RIjdkRQOa GTllFVoLmYlO6Np9gJpOUtXQKXYVLpAxkzRsVc+U97XIlVHyK4Wd6rsPZmPn6bYNZrHQ 5gcMT3+fnms32b8Rma1+kdYPpA0jEuHPEgsIbvPbcjOtmmoZeV3aznnXOFgWJ8g2XB6m rCbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=Jc53nrAqFJzWpeihvWmPt4pPa8IxoJsUWg6qJW7UN5M=; b=ldXJPBvZiaaOTCXM1gVPDadCViqUTDxanvDAJD8JHOvRZJLJKfNdoYPF5Js4XgeEnO r2He7cj3bdNGGrWIimr6TDYpVt219if2ID13LQFFAemX9TGuiwjMEVRnHpNwVHsht0aY 56q/WoHXF/uKk0xnh+nXFf6B+Jzb8J6uEh1AC1Ba4j1O1Zny9Zwj+yynwO7nCpa9IOLD iVZ/QzDd7cJ3j5gCONA5fS/s1JGo7yBGyG0xwkFllezs0CGWvxFITXrMCMM1TdhUY4qQ /gh40DH2MEmFNaF+T5kr4JB0Cusn2dZTRMb/LGgsrOQJxn+6Nz2KS7zs0z8FAXzbrA71 MAtw== X-Gm-Message-State: AOAM531wdwaGM+sui7P8aQVwujrIQKgaH8B7cYt3BaFtSYIIllkqT9Ft 388gLBMEgA6qwCYmX+CLKTfXW3Y5U5I93ddJ X-Google-Smtp-Source: ABdhPJw5hlnUT042QEXN8VVf5nfJ/AVjoGh4l18JeMOpvK6M3qXTCL5XvWjntUdRahMujDusex5lVw== X-Received: by 2002:a62:838a:: with SMTP id h132mr15784812pfe.72.1597769699983; Tue, 18 Aug 2020 09:54:59 -0700 (PDT) Received: from [10.0.1.161] (c-73-109-7-33.hsd1.wa.comcast.net. [73.109.7.33]) by smtp.gmail.com with ESMTPSA id n3sm25092146pfq.131.2020.08.18.09.54.59 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 18 Aug 2020 09:54:59 -0700 (PDT) Content-Type: multipart/alternative; boundary=Apple-Mail-601D1338-A5BA-40E5-8410-A242E48E8B36 Content-Transfer-Encoding: 7bit From: Eric Voskuil Mime-Version: 1.0 (1.0) Date: Tue, 18 Aug 2020 09:54:58 -0700 Message-Id: <84F1FD44-0482-4F77-8FFA-65492E445D23@voskuil.org> References: <8a7031f2-a598-ac09-f2df-1672cd82980b@mattcorallo.com> In-Reply-To: <8a7031f2-a598-ac09-f2df-1672cd82980b@mattcorallo.com> To: Matt Corallo , Bitcoin Protocol Discussion X-Mailer: iPhone Mail (17G68) X-Mailman-Approved-At: Tue, 18 Aug 2020 17:28:28 +0000 Subject: Re: [bitcoin-dev] Generalizing feature negotiation when new p2p connections are setup X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 17:16:15 -0000 --Apple-Mail-601D1338-A5BA-40E5-8410-A242E48E8B36 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable =E2=80=9CBitcoin protocol has always expected clients to ignore unknown mess= ages=E2=80=9D This is not true. Bitcoin has long implemented version negotiation, which is= the opposite expectation. Libbitcoin=E2=80=99s p2p protocol implementation i= mmediately drops a peer that sends an invalid message according to the negot= iated version. The fact that a given client does not validate the protocol d= oes not make it an expectation that the protocol not be validated. Features can clearly be optional within an actual protocol. There have been p= ost-handshake negotiations implemented for optional messages which are valid= at the negotiated version. The protocol may be flexible while remaining val= idateable. There is no reason to force a client to accept unknown message tr= affic. A generalized versioning change can be implemented in or after the handshake= . The latter is already done on an ad-hoc basis. The former is possible as l= ong as the peer=E2=80=99s version is sufficient to be aware of the behavior.= This does not imply any need to send invalid messages. The verack itself ca= n simply be extended with a matrix of feature support. There is no reason to= complicate negotiation with an additional message(s). FWIW, bip37 did this poorly, adding a feature field to the version message, r= esulting in bip60. Due to this design, older protocol-validating clients wer= e broken. In this case it was message length that was presumed to not be val= idated. e > On Aug 18, 2020, at 07:59, Matt Corallo via bitcoin-dev wrote: >=20 > =EF=BB=BFThis sounds like a great idea! >=20 > Bitcoin is no longer a homogeneous network of one client - it is many, wit= h different features implemented in each. The Bitcoin protocol hasn't (fully= ) evolved to capture that reality. Initially the Bitcoin protocol had a simp= le numerical version field, but that is wholly impractical for any diverse n= etwork - some clients may not wish to implement every possible new relay mec= hanic, and why should they have to in order to use other new features? >=20 > Bitcoin protocol changes have, many times in recent history, been made via= new dummy "negotiation" messages, which take advantage of the fact that the= Bitcoin protocol has always expected clients to ignore unknown messages. Gi= ven that pattern, it makes sense to have an explicit negotiation phase - aft= er version and before verack, just send the list of features that you suppor= t to negotiate what the connection will be capable of. The exact way we do t= hat doesn't matter much, and sending it as a stream of messages which each i= ndicate support for a given protocol feature perfectly captures the pattern t= hat has been used in several recent network upgrades, keeping consistency. >=20 > Matt >=20 >> On 8/14/20 3:28 PM, Suhas Daftuar via bitcoin-dev wrote: >> Hi, >> Back in February I posted a proposal for WTXID-based transaction relay[1]= (now known as BIP 339), which included a proposal for feature negotiation t= o take place prior to the VERACK message being received by each side. In my= email to this list, I had asked for feedback as to whether that proposal wa= s problematic, and didn't receive any responses. >> Since then, the implementation of BIP 339 has been merged into Bitcoin Co= re, though it has not yet been released. >> In thinking about the mechanism used there, I thought it would be helpful= to codify in a BIP the idea that Bitcoin network clients should ignore unkn= own messages received before a VERACK. A draft of my proposal is available h= ere[2]. >> I presume that software upgrading past protocol version 70016 was already= planning to either implement BIP 339, or ignore the wtxidrelay message prop= osed in BIP 339 (if not, then this would create network split concerns in th= e future -- so I hope that someone would speak up if this were a problem). W= hen we propose future protocol upgrades that would benefit from feature nego= tiation at the time of connection, I think it would be nice to be able to us= e the same method as proposed in BIP 339, without even needing to bump the p= rotocol version. So having an understanding that this is the standard of ho= w other network clients operate would be helpful. >> If, on the other hand, this is problematic for some reason, I look forwar= d to hearing that as well, so that we can be careful about how we deploy fut= ure p2p changes to avoid disruption. >> Thanks, >> Suhas Daftuar >> [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-February= /017648.html >> [2] https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-neg= otiation/bip-p2p-feature-negotiation.mediawiki >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev --Apple-Mail-601D1338-A5BA-40E5-8410-A242E48E8B36 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
=E2=80=9CBitcoin protocol h= as always expected clients to ignore unknown messages=E2=80=9D

This is not true. Bitcoin has long imple= mented version negotiation, which is the opposite expectation. Libbitcoin=E2= =80=99s p2p protocol implementation immediately drops a peer that sends an i= nvalid message according to the negotiated version. The fact that a given client do= es not validate the protocol does not make it an expectation that the protoc= ol not be validated.

Features ca= n clearly be optional within an actual protocol. There have been post-handsh= ake negotiations implemented for optional messages which are valid at the ne= gotiated version. The protocol may be flexible while remaining validateable.= There is no reason to force a client to accept unknown message traffic.

A generalized versioning change can b= e implemented in or after the handshake. The latter is already done on an ad= -hoc basis. The former is possible as long as the peer=E2=80=99s version is s= ufficient to be aware of the behavior. This does not imply any need to send i= nvalid messages. The verack itself can simply be extended with a matrix of f= eature support. There is no reason to complicate negotiation with an additio= nal message(s).
=
FWI= W, bip37 did this poorly, adding a feature field to the version message, res= ulting in bip60. Due to this design, older protocol-validating clients were b= roken. In this case it was message length that was presumed to not be valida= ted.

e
<= div dir=3D"ltr">
On Aug 18, 2020, at 07:59, Mat= t Corallo via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrot= e:

=EF=BB= =BFThis sounds like a great idea!

Bit= coin is no longer a homogeneous network of one client - it is many, with dif= ferent features implemented in each. The Bitcoin protocol hasn't (fully) evo= lved to capture that reality. Initially the Bitcoin protocol had a simple nu= merical version field, but that is wholly impractical for any diverse networ= k - some clients may not wish to implement every possible new relay mechanic= , and why should they have to in order to use other new features?
=
Bitcoin protocol changes have, many times in recent h= istory, been made via new dummy "negotiation" messages, which take advantage= of the fact that the Bitcoin protocol has always expected clients to ignore= unknown messages. Given that pattern, it makes sense to have an explicit ne= gotiation phase - after version and before verack, just send the list of fea= tures that you support to negotiate what the connection will be capable of. T= he exact way we do that doesn't matter much, and sending it as a stream of m= essages which each indicate support for a given protocol feature perfectly c= aptures the pattern that has been used in several recent network upgrades, k= eeping consistency.

Matt

On 8/14/20 3:28 PM, Suhas Daftuar via bitcoin-dev wrote:
Hi,
Back in February I posted a proposal for WTXID-based tr= ansaction relay[1] (now known as BIP 339), which included a proposal for fea= ture negotiation to take place prior to the VERACK message being received by= each side.  In my email to this list, I had asked for feedback as to w= hether that proposal was problematic, and didn't receive any responses.
Since then, the implement= ation of BIP 339 has been merged into Bitcoin Core, though it has not y= et been released.
In t= hinking about the mechanism used there, I thought it would be helpful t= o codify in a BIP the idea that Bitcoin network clients should ignore unknow= n messages received before a VERACK.  A draft of my proposal is availab= le here[2].
I presume= that software upgrading past protocol version 70016 was already planning to= either implement BIP 339, or ignore the wtxidrelay message proposed in BIP 3= 39 (if not, then this would create network split concerns in the future -- s= o I hope that someone would speak up if this were a problem).  When we p= ropose future protocol upgrades that would benefit from feature negotiation a= t the time of connection, I think it would be nice to be able to use the sam= e method as proposed in BIP 339, without even needing to bump the protocol v= ersion.  So having an understanding that this is the standard of how ot= her network clients operate would be helpful.
If, on the other hand, this is problematic for some= reason, I look forward to hearing that as well, so that we can be careful a= bout how we deploy future p2p changes to avoid disruption.
Thanks,
Suhas Daftuar
[1] https://lists.linuxfoundation.org/pipermail/bitcoin-de= v/2020-February/017648.html <https://lists.linuxfoundation.org/pipermail/= bitcoin-dev/2020-February/017648.html>
[2] https://github.com/sdaftuar/bips/blob/2020-08-gener= alized-feature-negotiation/bip-p2p-feature-negotiation.mediawiki <https:/= /github.com/sdaftuar/bips/blob/2020-08-generalized-feature-negotiation/bip-p= 2p-feature-negotiation.mediawiki>
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfounda= tion.org
https://list= s.linuxfoundation.org/mailman/listinfo/bitcoin-dev
_______________________________________________
bitcoin-= dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
= --Apple-Mail-601D1338-A5BA-40E5-8410-A242E48E8B36--