From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E5B8ABF6 for ; Wed, 13 Dec 2017 00:01:33 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-vk0-f46.google.com (mail-vk0-f46.google.com [209.85.213.46]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 61994411 for ; Wed, 13 Dec 2017 00:01:33 +0000 (UTC) Received: by mail-vk0-f46.google.com with SMTP id x140so402439vke.4 for ; Tue, 12 Dec 2017 16:01:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Ibj/lWoR8nZIV6xBDumeEfxb954H1KnKNn2afglqSJU=; b=sfzONKOpyRwsxCimI2xbNKiZN6QIBpithNHBSD2C1Hv1IYpLiCwF08WDSMyEo89gVZ 67TVhORiekqGjKjcEZqH7rwKBL94K+UDXKaCuyHkA/ojc/36BG7fdJfATi14HTMG+Srn w5FZyX1rfEzw8f4jg8fJFboPgM+DcRkjRGTdYWVSs53t29CvnFnqwmffwooaz45UrqJv NJhPIjHV1a5jYgp+yP2sjGnKIs0dD8wRJHfyrh4/Q8vndtYEfndRJmPLkGVTNsJ0Tswe xc3f0RIc79v0BTj9fGDEsKs4yvkTbmSZI2IEGXhPZOZFTTQXYyVjHXhzu0dxYlxgtQmH H0+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Ibj/lWoR8nZIV6xBDumeEfxb954H1KnKNn2afglqSJU=; b=SdPUP4YGPP1CmsJ2fsvvEZfEZeG6pacM8Snj/1nhOo7ctRJgJKsTBD4oxLA/C2epY7 qlUydk37TMDbJLMFP/j/96nyuk7XlpWTcktm8gNQlpeGz2WiKjbeF+i1XAwapMxdgBQy 56cbi90gMwXURiiDg9m738Ikf/guQ8MbTlKI1ugMDQw5lqwFtQC0IbP3wFqn021frld0 q5RPeHMQm7qxaWIOuXDQ5UoGNi8r2DPr8p+1mn0j0mYLUeEb7siCwSX1E8I1J9hW0FrF FUr7dTxiGafg12cvd3EQbsBYuZz3nmBeQMvr40+gspwPyFF4nZTPEWkutdfTAO4K9FHT M48w== X-Gm-Message-State: AKGB3mLFabdRMnJWgFRjY27uNrZRBbw26DBcyclCXEnaESwx7Om8hyzr lGKY4YxuXvVRDqMfceVudw9f1A7BpgdszFmNkmc= X-Google-Smtp-Source: ACJfBots+EE7JYEGioZH47fjwzJtSebRTGt884uj9eanBBflBz2klkIJdjM3NoYWljMHHBJ3L9m8yTjVCIzIpic3b0Y= X-Received: by 10.31.54.151 with SMTP id d145mr5758494vka.14.1513123292507; Tue, 12 Dec 2017 16:01:32 -0800 (PST) MIME-Version: 1.0 Sender: gmaxwell@gmail.com Received: by 10.103.85.148 with HTTP; Tue, 12 Dec 2017 16:01:32 -0800 (PST) In-Reply-To: References: From: Gregory Maxwell Date: Wed, 13 Dec 2017 00:01:32 +0000 X-Google-Sender-Auth: D_Z2P3Y8hvIVlUh_SZRT85M61AM Message-ID: To: Suhas Daftuar Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] "Compressed" headers stream X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2017 00:01:34 -0000 On Tue, Dec 12, 2017 at 9:07 PM, Suhas Daftuar wrote: > But I think we should be able to get nearly all the benefit just by > including nBits in any messages where the value is ambiguous; ie we include > it with the first header in a message, and whenever it changes from the > previous header's nBits. Yes, that is what I was thinking last time we discussed it, just with each header include a one byte flag that lets you express: bit: meaning (0) if nbits is the same as last, (1) if timestamp is a full field or a small offset, (e.g. two bytes defined as unsigned offset between the last time - 7200 and the new time). (2,3,4) if version is the same as the last distinct value .. 7th last, or a new 32bit distinct value; (5,6) if prev is entirely there, entirely gone, first 4 bytes provided, or first 8 bytes provided. (if provided they override the computed values). That would be 7 bits in total; the 8th could be reserved or use to signal "more headers follow" to make the encoding self-delimiting. The downside with nbits the same as last as the optimization is that if we ever change consensus rules to ones where difficulty management works differently it may be the case that nbits changes every block. Alternatively, nbits could get a differential encoding that could be opted into for small differences-- though I haven't thought much about it to see if a one byte difference would be that useful (e.g. can bch differences usually be expressed with one byte?) I'm kind of dubious of the consensus layer anti-dos separation: nbits minimum is so low compared to the speed of a mining device, virtually any attack that you might do with invalid headers could still be done with headers at the minimum difficulty. But I'm fully willing to accept that simpler is better... >> I would rather not change the serialization of existing messages, >> nodes are going to have to support speaking both messages for a long >> time, and I think we already want a different protocol flow for >> headers fetching in any case. > > > I agree with this. Specifically the way I envisioned this working is that > we could introduce a new 'cmpctheaders'/'getcmpcthdrs' message pair for > syncing using this new message type, while leaving the existing > 'headers'/'getheaders' messages unchanged. So when communicating with > upgraded peers, we'd never use 'getheaders' messages, and we'd only use > 'headers' messages for potentially announcing new blocks. > > Of course, we'll have to support the existing protocol for a long time. But > one downside I've discovered from deploying BIP 130, which introduced block > announcements via headers messages, is that overloading a 'headers' message > to be either a block announcement or a response to a 'getheaders' message > results in p2p-handling logic which is more complicated than it needs to be. > So splitting off the headers chain-sync functionality to a new message pair > seems like a nice side-effect benefit, in the long run. >