From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 2BC3D258 for ; Tue, 14 Feb 2017 19:54:40 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-oi0-f49.google.com (mail-oi0-f49.google.com [209.85.218.49]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 4F426167 for ; Tue, 14 Feb 2017 19:54:39 +0000 (UTC) Received: by mail-oi0-f49.google.com with SMTP id j15so76615881oih.2 for ; Tue, 14 Feb 2017 11:54:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=voskuil-org.20150623.gappssmtp.com; s=20150623; h=from:subject:to:references:message-id:date:user-agent:mime-version :in-reply-to; bh=7O8n+gQ3HZsuaJOXYUKIL6PZPDaYOykS6GBZresGiuA=; b=cmbWQ7wHf9hC8XBKtXz/XZcaSjYMJrBfU4lRqToS4dOkRdCGT0SDA1mm061OxnRCN/ rYDjG+ubU/4+6HQTfFyxGmxebyEWGa7m8DSckIoHuS0JqBk2effS2V3EAv6KvFcltoNC SXZnjEW0eWNTtzF5mPm+7a3fmDFzMg3xuzcOhVzjIJnO0Ljxi2D0uKR7zqNYvz6BZHSQ G+AgNT9YLq0DsRvinedKyEQvwDLCjnAUvUy4ySNrS/xcRDzjJdSVyQfrDsPHGMJkTADA qRMmxqJtWt4obxDZ7cnMYhuWOKDg/3XC6AAIGca5nBLmyRHB3QAyhywIIq54alBBb+aW Hapw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:references:message-id:date :user-agent:mime-version:in-reply-to; bh=7O8n+gQ3HZsuaJOXYUKIL6PZPDaYOykS6GBZresGiuA=; b=A3Xnn4LD5WUH9thZf8uMfV3q2KIYcjJLJpVyrJrX5/n2aGNSoFcwgCsfiTLSDMSILE tAqKQ4nzge7SWILXK71PjVgM5FOzdxnDefyg19/4s/jd2yzsrCNPhl1gtsqCFzwZMran jXnkjZPiB+xe7e1G799KeYmkH5D/btQjj13mGk8pfG1ohT+kkmlGmRe9mXMWh7xVYqJf FYv4gztWeIh5/1kkenT0J6a8pwre5805fuD6VGX5j7KQ1wXdCakPleUsJzyHgpN+LIrv dxxrRNgh3dH/kaRMVGT4R2efkjbodtyiBs5waNvPvS7t3C4jx8s4lY4vW4GBPiVWINBl zUWw== X-Gm-Message-State: AMke39nIZwmZwCQt1BypNgXFSly3kvmDuSxk8H/5hIFqpp43gKbh1brn6Kq5bO/DSDR+6A== X-Received: by 10.84.130.99 with SMTP id 90mr38735694plc.167.1487102078305; Tue, 14 Feb 2017 11:54:38 -0800 (PST) Received: from ?IPv6:2601:600:9000:d69e:6d41:c58:a546:6c21? ([2601:600:9000:d69e:6d41:c58:a546:6c21]) by smtp.gmail.com with ESMTPSA id h68sm2760417pfj.124.2017.02.14.11.54.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Feb 2017 11:54:37 -0800 (PST) From: Eric Voskuil To: Jonas Schnelli , Bitcoin Protocol Discussion , libbitcoin@lists.dyne.org References: <638deacd-c117-f1a7-10de-a7e36a47c3c7@voskuil.org> X-Enigmail-Draft-Status: N1110 Message-ID: <17f2f92c-68fc-7a7d-c015-9bad810365e3@voskuil.org> Date: Tue, 14 Feb 2017 11:54:37 -0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="B2GJJetgJLbf4XujGK5X1qDXqppMpD8dn" X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Tue, 14 Feb 2017 19:55:12 +0000 Subject: Re: [bitcoin-dev] BIP151 protocol incompatibility X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Feb 2017 19:54:40 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --B2GJJetgJLbf4XujGK5X1qDXqppMpD8dn Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 02/13/2017 03:14 AM, Jonas Schnelli wrote: >>> Look at feefilter BIP 133 >>> (https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki#backw= ard-compatibility) >>> or sendheaders BIP130 >>> (https://github.com/bitcoin/bips/blob/master/bip-0130.mediawiki#backw= ard-compatibility) >>> Isn't it the same there? >> No. This is what I was referring to. These messages are enabled by >> protocol version. If they are received by a node below the version at >> which they are activated, they are unknown messages, implying an inval= id >> peer. The above messages cannot be sent until *after* the version is >> negotiated. BIP151 violates this rule by allowing the new control >> message to be sent *before* the version handshake. > This indeed is not ideal for compatibility checks, but increases securi= ty. The issue I raised is that it is not backward compatible. It sounds like you agree but consider it a fair trade. My suggesting was that the BIP be updated to reflect the lack of compatibility. > I could not find a protocol specification that said communication must > be terminated when messages are transmitted before the version handshak= e > has been done. It doesn't need to be specified, most of Bitcoin is unspecified. The version handshake establishes the negotiated version. It is not possible to determine if a message is of the negotiated version before the version is negotiated. All messages apart from this one have followed that rule. > Also. BIP151 clearly says that the requesting peer needs to initiate th= e > encryption (encinit). An incoming connection will be dropped due to invalid protocol and potentially banned depending on the implementation. > In case of light clients not supporting BIP151 connecting to peers > supporting BIP151, there should never be transmission of new message > types specified in BIP151. Not working with peers not supporting BIP151 is the compatibility issue. But it sort of seems the intent in this case is to rely on that incompatibility (expecting connections to nonsupporting peers to fail as opposed to negotiating). >>> Once BIP151 is implemented, it would make sense to bump the protocol >>> version, but this needs to be done once this has been >>> implemented/deployed. >> There are already nodes out there breaking connections based on the BI= P. > It could very likely be possible that the initial responding peer tries= > to initiate a encryption session which would mean that BIP151 was not > implemented correctly. > Correct me if I'm wrong please. I did consider the possibility, but there's this: "Encryption initialization must happen before sending any other messages to the responding peer (encinit message after a version message must be ignored)." https://github.com/bitcoin/bips/blob/master/bip-0151.mediawiki#specificat= ion The BIP does not define "responding" and "requesting" peers, but: "A peer that supports encryption must accept encryption requests from all peers... The responding peer accepts the encryption request by sending a encack message." This implies the requesting peer is the peer that sends the message. You seem to be saying that the requesting peer is the one that initiated the connection and the responding peer is the connection receiver. If this is the case it should be more clearly documented. But in the case I experienced the "requester" of an encrypted session was also the "receiver" of the connection. >>> Or do I make a mistake somewhere? >> Yes, the ordering of the messages. New messages can only be added afte= r >> the handshake negotiates the higher version. Otherwise the handshake i= s >> both irrelevant (as Pieter is implying) and broken (for all existing >> protocol versions). > I could not find evidence of the protocol specification that would > forbid (=3Dterminate connection) such messages and I think allowing > unknown-messages before the version handshake makes the protocol flexib= le. Flexible is certainly one word for it. Another way to describe it is dirty. Allowing invalid messages in a protocol encourages protocol incompatibility. You end up with various implementations and eventually have no way of knowing how they are impacted by changes. There could be a range of peers inter-operating with the full network while running their own sub-protocols. Given the network is public and strong identification of peers is undesirable, the invalid messages would reasonably just get sent to everyone. So over time, what is the protocol? Due to certain "flexibility" it is already a hassle to properly implement. > Are there any reasons we should drop peers if they send us unknown, but= > correctly formatted p2p packages (magic, checksum, etc.) before the > version handshake, ... but not drop them if we have received unknown > messages after the version handshake? There is no reason to treat invalid messages differently based on where they occur in the communication. After the handshake the agreed version is known to both peers. As a result there is never a reason for an invalid message to be sent. Therefore it is always proper to drop a peer that sends an invalid message. > I can't see that a such spec. would reduce the DOS attack vector? This was previously addressed (immediately below). >>>> As for DOS, waste of bandwidth is not something to be ignored. If a = peer >>>> is flooding a node with addr message the node can manage it because = it >>>> understands the semantics of addr messages. If a node is required to= >>>> allow any message that it cannot understand it has no recourse. It >>>> cannot determine whether it is under attack or if the behavior is >>>> correct and for proper continued operation must be ignored. >>> How do you threat any other not known message types? >> You may be more familiar with non-validating peers. If a message type = is >> not known it is an invalid message and the peer is immediately dropped= =2E >> We started seeing early drops in handshakes with bcoin nodes because o= f >> this issue. > If this had happened, it's very likely because the responding peer trie= d > to initiate a encryption session which is against BIP151 specs. See above. >>> Any peer can send you any type of message anytime. >> Sure, a peer can do what it wants. It can send photos. But I'm not sur= e >> what makes you think it would be correct to maintain the connection wh= en >> an *invalid* message is received. > Check: > https://github.com/bitcoin/bitcoin/blob/a06ede9a138d0fb86b0de17c42b936d= 9fe6e2158/src/net_processing.cpp#L2595 > I think it was a wise implementation decision to allow unknown (not > invalid) messages. > This had allowed us to deploy stuff like compact blocks, feefilter, etc= =2E > without breaking backward compatibility. > IMO, without a such flexibility, the deployment complexity would be > irresponsible high without really solving the DOS problem. This is a misinterpretation. The failure to validate did not enable anything except possibly some broken peers not getting dropped. None of the protocol changes previously deployed require the older version peer to allow invalid messages. While it may appear otherwise, due to a particular implementation, it is never necessary to send a message to a peer that the peer does not understand. The handshake gives each peer the other peer's version. That obligates the newer peer to conform to the older (or disconnect if the old is insufficient). That's the nature of backward compatibility. >>> Why would your implementation how you threat unknown messages be >> different for messages specified in BIP151? >> >> Because it properly validates the protocol. > For feefilter or compact block or sendheaders? Yes, this is the purpose of version negotiation, which is why there are version and verack messages. And this is also why, in the satoshi client, two of the above messages are sent from the verack handler. The feefilter message is sent dynamically but only if the peer's version allows it. > You can't link a (unimplemented) specification (improvement process) to= > a protocol version before deployment. Or can you? I'm not sure I follow your question. The BIP should presumably declare a version number if one is necessary. > Once it has been widely deployed, we should set a protocol minversion > for BIP151, right. In general you should set a version before it's ever live on the network. But if it precedes the protocol version negotiation the protocol version number is moot. I've been asked to throttle the discussion in the interest of reducing list volume. I think the issue is pretty clearly addressed at this point, but feel free to follow up directly and/or via the libbitcoin development list (copied). e --B2GJJetgJLbf4XujGK5X1qDXqppMpD8dn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQEcBAEBAgAGBQJYo2B9AAoJEDzYwH8LXOFOAeUH/iHCE+6McSKBp70spLQ0QDJY qDpx8vX4Dj6sK4lQYARe3rXXFLw07r4ybufV4oiMagRTC9NNeA6ftdtOWTFicfEy tM44rythBUDqw9xtRvw9yjxr7yQCY0G7k+dbnnFqLIHA9Lz6ejNFKI9AK1SN7FYy +/3q0y3QTt0RZQ/FnuQOZOPhzHjnLGsjXlhqocqIZFrUCq62kn91oxHNuRfuSnrs lH0jm+M0Y7Y2c5vPNmYjbJNx2h4R/6P/wkzTIkfH8fE+lm6f6sTAZh1lVkv6pKL2 tXxU4jWmUtA21bCy9Trl3sT87ORpmU9/Ywom2ImZvYllW1iLl6QaTxeyyIreNTc= =BrxR -----END PGP SIGNATURE----- --B2GJJetgJLbf4XujGK5X1qDXqppMpD8dn--