From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 8B78B49B for ; Tue, 12 Dec 2017 21:07:14 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ua0-f179.google.com (mail-ua0-f179.google.com [209.85.217.179]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id CB9D6E7 for ; Tue, 12 Dec 2017 21:07:13 +0000 (UTC) Received: by mail-ua0-f179.google.com with SMTP id h2so154922uae.12 for ; Tue, 12 Dec 2017 13:07:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=//1Re6R81X5QQCMnOO9DcsIhwZRwiRgbv85eSybAFdI=; b=ihX5q6WebLnF199wtxxIzcn+MmSnsstdBhag/an2ut5O0gwIByYA312sOKjMdCAyY8 6ZXGpa79IdMrIdjEi2JBIEtYpE4+eMvr+sqSx+dq39kd/V/OB+QyF/LL1iaQDs17neVL hGJx6kgiY9brCyymbcVM9L/YTm5+BamvXJNRvnSRO9Pz6lNUFQpnMC4sNR+fTjUcE0hI 3XDsXaUM3aF6kTqCqizFM77gnklZAUgesefN7d6AqF29qDv8F7yl1jAUr6Iyk5wnPeQ0 LcPlNNzscMxkVfS8itxAlgnNSIsihCWglZuHs5PcmJ2Y/vaermX211nf951Eb+HHxrjQ 4bsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=//1Re6R81X5QQCMnOO9DcsIhwZRwiRgbv85eSybAFdI=; b=YXIzbkhq193DfDFT336cu2s2O29+006+XNxV8lfkvp+0Ih0SrCrw9yGTDx/IP9kuxO 8nk4XZ+ryPDWWAzvY4F7BIsfXS0CHsJBOrcbUWMrr8yA2LZ8zuA9m5XLLYe7MsL3pyxn zTjqXqdg7GhIpLnLwTgnmrNfBre+N9/WPpHwyMb6gWmya7Ot2X8IWn7TUFmvQMmwH8kQ aOsLYq7pUvfSAQ8CxOs5akwuFIpT79xDzONlpMPd64Ozy0GuynOVwEcFnwx1mzj1uPvS xrs+VcFETm22Qb1nTMXgNYq53DuN2tDG2uL5Ey842ibQRRNgBwjckIBwe84RyHLMfLSt jEog== X-Gm-Message-State: AKGB3mK61knrE7j+5JU82vrmuBB8thcnY4C8i55hLHfuvOeY0kayh/6K 4aNT/9pCFDdS4AO2rYaLH+meFysSnRWxgvlWrDo= X-Google-Smtp-Source: ACJfBovfK5dhAaAX+RbMMKlk5ZRtI79H338AgE+5z1Y/6AH0ycd140Iiwkw+swU+oHYpXqNMtXSM/Zb659TYP1v9UtU= X-Received: by 10.176.80.107 with SMTP id z40mr5777789uaz.82.1513112832345; Tue, 12 Dec 2017 13:07:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.103.73.80 with HTTP; Tue, 12 Dec 2017 13:07:11 -0800 (PST) In-Reply-To: References: From: Suhas Daftuar Date: Tue, 12 Dec 2017 16:07:11 -0500 Message-ID: To: Gregory Maxwell , Bitcoin Protocol Discussion Content-Type: multipart/alternative; boundary="94eb2c1916c6e08d0705602b085a" X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] "Compressed" headers stream X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Dec 2017 21:07:14 -0000 --94eb2c1916c6e08d0705602b085a Content-Type: text/plain; charset="UTF-8" Hi, First, thanks for resurrecting this, I agree this is worth pursuing. On Mon, Dec 11, 2017 at 4:04 PM, Gregory Maxwell via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > > Nbits _never_ needs to be sent even with other consensus rules because > its more or less necessarily a strict function of the prior headers. > This still holds in every clone of Bitcoin I'm aware of; sending it > with the first header in a group probably makes sense so it can be > checked independently. > I think it would be nice, though, to not require the consensus-correct calculation of nBits in order to process p2p messages. For instance, I think there's a use for nBits at the p2p layer for calculating the work on a chain, which can be used as an anti-DoS measure, even without verifying that the difficulty adjustments are following the consensus rules. Moreover I think it's a bit messy if the p2p layer depends on intricate consensus rules in order to reconstruct a message -- either we'd need to interact with existing consensus logic in a potentially new way, or we'd reimplement the same logic in the p2p layer, neither of which is very desirable imo. But I think we should be able to get nearly all the benefit just by including nBits in any messages where the value is ambiguous; ie we include it with the first header in a message, and whenever it changes from the previous header's nBits. I would rather not change the serialization of existing messages, > nodes are going to have to support speaking both messages for a long > time, and I think we already want a different protocol flow for > headers fetching in any case. > I agree with this. Specifically the way I envisioned this working is that we could introduce a new 'cmpctheaders'/'getcmpcthdrs' message pair for syncing using this new message type, while leaving the existing 'headers'/'getheaders' messages unchanged. So when communicating with upgraded peers, we'd never use 'getheaders' messages, and we'd only use 'headers' messages for potentially announcing new blocks. Of course, we'll have to support the existing protocol for a long time. But one downside I've discovered from deploying BIP 130, which introduced block announcements via headers messages, is that overloading a 'headers' message to be either a block announcement or a response to a 'getheaders' message results in p2p-handling logic which is more complicated than it needs to be. So splitting off the headers chain-sync functionality to a new message pair seems like a nice side-effect benefit, in the long run. --94eb2c1916c6e08d0705602b085a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi,<= /div>

First,= thanks for resurrecting this, I agree this is worth pursuing.

On Mon, Dec 11= , 2017 at 4:04 PM, Gregory Maxwell via bitcoin-dev <bi= tcoin-dev@lists.linuxfoundation.org> wrote:

Nbits _never_ needs to be sent even with other consensus rules because
its more or less necessarily a strict function of the prior headers.
This still holds in every clone of Bitcoin I'm aware of; sending it
with the first header in a group probably makes sense so it can be
checked independently.

I think it would= be nice, though, to not require the consensus-correct calculation of nBits= in order to process p2p messages.=C2=A0 For instance, I think there's = a use for nBits at the p2p layer for calculating the work on a chain, which= can be used as an anti-DoS measure, even without verifying that the diffic= ulty adjustments are following the consensus rules.

Moreover I think it's a bit messy if the p2p layer depends on intrica= te consensus rules in order to reconstruct a message -- either we'd nee= d to interact with existing consensus logic in a potentially new way, or we= 'd reimplement the same logic in the p2p layer, neither of which is ver= y desirable imo.

But I think we should be able to = get nearly all the benefit just by including nBits in any messages where th= e value is ambiguous; ie we include it with the first header in a message, = and whenever it changes from the previous header's nBits.
I would rather not change the seriali= zation of existing messages,
nodes are going to have to support speaking both messages for a long
time, and I think we already want a different protocol flow for
headers fetching in any case.

I agree with this.=C2=A0 Specifically the = way I envisioned this working is that we could introduce a new 'cmpcthe= aders'/'getcmpcthdrs' message pair for syncing using this new m= essage type, while leaving the existing 'headers'/'getheaders&#= 39; messages unchanged.=C2=A0 So when communicating with upgraded peers, we= 'd never use 'getheaders' messages, and we'd only use '= headers' messages for potentially announcing new blocks.

=
Of course, we'll have to support the existing protocol for a= long time.=C2=A0 But one downside I've discovered from deploying BIP 1= 30, which introduced block announcements via headers messages, is that over= loading a 'headers' message to be either a block announcement or a = response to a 'getheaders' message results in p2p-handling logic wh= ich is more complicated than it needs to be.=C2=A0 So splitting off the hea= ders chain-sync functionality to a new message pair seems like a nice side-= effect benefit, in the long run.

--94eb2c1916c6e08d0705602b085a--