From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id EA302C0051 for ; Tue, 18 Aug 2020 18:16:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id D69178686A for ; Tue, 18 Aug 2020 18:16:59 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id q7kXTUMzkVn3 for ; Tue, 18 Aug 2020 18:16:58 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mail-vs1-f43.google.com (mail-vs1-f43.google.com [209.85.217.43]) by whitealder.osuosl.org (Postfix) with ESMTPS id 167168680C for ; Tue, 18 Aug 2020 18:16:58 +0000 (UTC) Received: by mail-vs1-f43.google.com with SMTP id o184so10613620vsc.0 for ; Tue, 18 Aug 2020 11:16:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=voskuil-org.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=cEtGqtkQWq0l6LICXtCzt1ILvK1uA58i+kdZUP+bVpU=; b=Yx/4gi1UozZ4k35h+A+6ce3cqF4syloJMFw2aHU9Xqg6bZgvWb2rjYFwEJaYxc4sgn r4cfzFkbDfJNLeHTG+aVPX+rSVY/hiawzNJSHVUMGghKbE76IfYB6DpnShSIU7xqW0Sh t6rURh7OzUqwyiBouwG/5ewbBzrZMFDvoKpRKSD4J+7d79gfad/9V9fECGiaObUTyUFA EPfyvltQ1x1CFCzy9kRPpn/J2ksKHO0zUbLIMUMFkGd62iakdkEJgJw5cyZsVGEGgc0p ImZR3NNbEv4ifUuAORXQIPS4QOGfSj4sDcgWfFPW0gyeuEIOn5zwWQ8suBNDWOAnmqxI /WMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=cEtGqtkQWq0l6LICXtCzt1ILvK1uA58i+kdZUP+bVpU=; b=WO3usaC0r4aLei+0RELI23lWzzzaU1rau+BIw7Yt0ZK30VgZxR2M5Uj/Gk7L1yilMY lzLOeioE2d+oplGx66Vk719OD3auBtRb/dIN7RalHaULxsyKfFzQLxwnDIzfoFNRquZr 7jPhcaT4x3d1noHc9TH4NCbZKmCeaJLWYUlWJSZZ+n5ShvmkItebhMj5sqjrROdQIRsc ORbKyIa2AOUbQAcq1Dh6RkYXjwJgeasbGV4/KSLTd+nGkhn+Bu5uU2lF4vFSRtHut7D1 UKSy+XyyDo9LhQzFge+Tcs7BICfZDMf0PfhZNWzcMcqWTcsDyu7NHsLAs4xrS+DHreIX WJMA== X-Gm-Message-State: AOAM533dZuHAwpu63T6gpteKmchCOibCY/GFwdFrORL+lWZOQFzW7zB1 ZzbZOUQgRstDz4VpfqVO/Nk7ZD262skl+g== X-Google-Smtp-Source: ABdhPJxqTCAfm0JM/5un4S64tqPioi2LXjUFFMO6Rq60BWs/Bv74iTCe3LPgpm2uc0Zz2eg5ue/CSQ== X-Received: by 2002:a17:902:b18e:: with SMTP id s14mr8867891plr.160.1597774276214; Tue, 18 Aug 2020 11:11:16 -0700 (PDT) Received: from ?IPv6:2600:380:7066:ce6e:35fb:26cc:838e:c70d? ([2600:380:7066:ce6e:35fb:26cc:838e:c70d]) by smtp.gmail.com with ESMTPSA id q82sm27621823pfc.139.2020.08.18.11.11.14 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 18 Aug 2020 11:11:15 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Eric Voskuil Mime-Version: 1.0 (1.0) Date: Tue, 18 Aug 2020 11:11:12 -0700 Message-Id: <95DD247F-028E-432A-9C26-6337F5819A31@voskuil.org> References: <6ccf6ff8-6b4f-c8f4-0dbb-36c5d076528f@mattcorallo.com> In-Reply-To: <6ccf6ff8-6b4f-c8f4-0dbb-36c5d076528f@mattcorallo.com> To: Matt Corallo X-Mailer: iPhone Mail (17G68) X-Mailman-Approved-At: Tue, 18 Aug 2020 18:21:57 +0000 Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] Generalizing feature negotiation when new p2p connections are setup X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 18:17:00 -0000 > On Aug 18, 2020, at 10:26, Matt Corallo wrote: >=20 > =EF=BB=BFThere are several cases where a new message has been sent as a pa= rt of a negotiation without changing the protocol version. Such as? > You may chose to ignore that, but that doesn't mean that it isn't an under= stood and even relied upon feature of the Bitcoin P2P protocol. If you wish t= o fail connections to new nodes (and risk network splits, as Suhas points ou= t), then you may do so, but that doesn't make it a part of the Bitcoin P2P p= rotocol that you must do so. Of course there is no "official document" by wh= ich we can make a formal appeal, but historical precedent suggests otherwise= . You are misrepresenting =E2=80=9Chistorical precedent=E2=80=9D. I=E2=80=99ve= seen several attempts to require arbitrary traffic over the years and none h= ave been realized. > Still, I think we're talking pedantics here, and not in a useful way. Not to be pedantic, but I don=E2=80=99t know what that means. > Ultimately we need some kind of negotiation which is flexible in allowing d= ifferent software to negotiate different features without a global lock-step= version number increase. I have shown below how that already works. > Or, to put it another way, if a feature is fully optional, why should ther= e be a version number increase for it For the reasons previously given. > - the negotiation of it is independent and a version number only increases= confusion over which change "owns" a given version number. Presumably this is why we have a standards process. Any new message implies o= wnership. Deconflicting that is required, which implies it can easily be ver= sion associated (as it has been). > I presume you'd support a single message that lists the set of features wh= ich a node (optionally) wishes to support on the connection. This proposal i= s fully equivalent to that, instead opting to list them as individual messag= es instead of one message, which is a bit nicer in that they can be handled m= ore independently or by different subsystems including even the message hash= ing. This presumes an implementation. As part of the handshake, collection of an a= rbitrary set of messages is a significant and unnecessary complication *of t= he protocol*. Extension of the verack is not. It is the simplest change poss= ible to implement the desired behavior. Each peer simply supplies the matrix= of sub-protocols it supports and only those supported by both are allowed. T= here is no reason for the protocol to split that matrix up into multiple mes= sages, requiring termination. Independent messages exist because of timing o= r ordering requirements. Implementing dependent messages as if they were ind= ependent is wasteful and complicating. I=E2=80=99m well aware of the inefficiency produced by version linearity in t= he face of optional sub-protocols. The protocol must negotiate to the versio= n where it can then negotiate support, which has been done. I support creati= ng a simpler system, eliminating these extra messages. The existing numeric v= ersion can be reserved exclusively for =E2=80=9Cmust=E2=80=9D implement, and= can be used to signal an extension to the verack. The verack can then carry= a list of =E2=80=9Cmay=E2=80=9D or =E2=80=9Cshould=E2=80=9D sub-protocols f= or final negotiation. The format of the matrix is arbitrary, but the requirement is to list a set o= f optional sub-protocols. This implies a namespace. This implies =E2=80=9Cow= nership=E2=80=9C of names. In other words, that coordination requirement is n= ot eliminated. e > Matt >=20 >> On 8/18/20 12:54 PM, Eric Voskuil wrote: >> =E2=80=9CBitcoin protocol has always expected clients to ignore unknown m= essages=E2=80=9D >> This is not true. Bitcoin has long implemented version negotiation, which= is the opposite expectation. Libbitcoin=E2=80=99s p2p protocol implementati= on immediately drops a peer that sends an invalid message according to the n= egotiated version. The fact that a given client does not validate the protoc= ol does not make it an expectation that the protocol not be validated. >> Features can clearly be optional within an actual protocol. There have be= en post-handshake negotiations implemented for optional messages which are v= alid at the negotiated version. The protocol may be flexible while remaining= validateable. There is no reason to force a client to accept unknown messag= e traffic. >> A generalized versioning change can be implemented in or after the handsh= ake. The latter is already done on an ad-hoc basis. The former is possible a= s long as the peer=E2=80=99s version is sufficient to be aware of the behavi= or. This does not imply any need to send invalid messages. The verack itself= can simply be extended with a matrix of feature support. There is no reason= to complicate negotiation with an additional message(s). >> FWIW, bip37 did this poorly, adding a feature field to the version messag= e, resulting in bip60. Due to this design, older protocol-validating clients= were broken. In this case it was message length that was presumed to not be= validated. >> e >>>> On Aug 18, 2020, at 07:59, Matt Corallo via bitcoin-dev wrote: >>>=20 >>> =EF=BB=BFThis sounds like a great idea! >>>=20 >>> Bitcoin is no longer a homogeneous network of one client - it is many, w= ith different features implemented in each. The Bitcoin protocol hasn't (ful= ly) evolved to capture that reality. Initially the Bitcoin protocol had a si= mple numerical version field, but that is wholly impractical for any diverse= network - some clients may not wish to implement every possible new relay m= echanic, and why should they have to in order to use other new features? >>>=20 >>> Bitcoin protocol changes have, many times in recent history, been made v= ia new dummy "negotiation" messages, which take advantage of the fact that t= he Bitcoin protocol has always expected clients to ignore unknown messages. G= iven that pattern, it makes sense to have an explicit negotiation phase - af= ter version and before verack, just send the list of features that you suppo= rt to negotiate what the connection will be capable of. The exact way we do t= hat doesn't matter much, and sending it as a stream of messages which each i= ndicate support for a given protocol feature perfectly captures the pattern t= hat has been used in several recent network upgrades, keeping consistency. >>>=20 >>> Matt >>>=20 >>> On 8/14/20 3:28 PM, Suhas Daftuar via bitcoin-dev wrote: >>>> Hi, >>>> Back in February I posted a proposal for WTXID-based transaction relay[= 1] (now known as BIP 339), which included a proposal for feature negotiation= to take place prior to the VERACK message being received by each side. In m= y email to this list, I had asked for feedback as to whether that proposal w= as problematic, and didn't receive any responses. >>>> Since then, the implementation of BIP 339 has been merged into Bitcoin C= ore, though it has not yet been released. >>>> In thinking about the mechanism used there, I thought it would be helpf= ul to codify in a BIP the idea that Bitcoin network clients should ignore un= known messages received before a VERACK. A draft of my proposal is availabl= e here[2]. >>>> I presume that software upgrading past protocol version 70016 was alrea= dy planning to either implement BIP 339, or ignore the wtxidrelay message pr= oposed in BIP 339 (if not, then this would create network split concerns in t= he future -- so I hope that someone would speak up if this were a problem). = When we propose future protocol upgrades that would benefit from feature ne= gotiation at the time of connection, I think it would be nice to be able to u= se the same method as proposed in BIP 339, without even needing to bump the p= rotocol version. So having an understanding that this is the standard of ho= w other network clients operate would be helpful. >>>> If, on the other hand, this is problematic for some reason, I look forw= ard to hearing that as well, so that we can be careful about how we deploy f= uture p2p changes to avoid disruption. >>>> Thanks, >>>> Suhas Daftuar >>>> [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-Februa= ry/017648.html >>>> [2] https://github.com/sdaftuar/bips/blob/2020-08-generalized-feature-n= egotiation/bip-p2p-feature-negotiation.mediawiki >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists.linuxfoundation.org >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev