From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id EEA9CD3C for ; Mon, 11 Dec 2017 21:04:02 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ua0-f174.google.com (mail-ua0-f174.google.com [209.85.217.174]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 84E9650E for ; Mon, 11 Dec 2017 21:04:02 +0000 (UTC) Received: by mail-ua0-f174.google.com with SMTP id p33so12965899uag.9 for ; Mon, 11 Dec 2017 13:04:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=ywMSPGSBsgf4QryHMoW7PHTC4mV0EqvduXhE2IR3clU=; b=PovK8xerqA1VL9s1bQk1otIylgaNRUZ0XAgl6l8oY+Pgyj1JeCOqSKRSrQXkW7SgRw EPjU4v+0Vxt5WwLz25QXUsXpGw2g0aDVxg1d17dcHRrMOUlO3iP8tvf/PB9AJJnrQb2O Mi4havZLNn53n+dMMO0vyxEyUpAzY9PU43boKx1LjRTJP/W0iUXR9STm1FNF7PlSaIKc DaInwrMpvxoXioeWuJvNpHHVCVuTFY8GutlKYVutKeG43yUCxlFxegvKXL8gSRRQ+RgS jvXFC8f7SDDaUoWmwZzEKAxwnEuVZpAbde0blE8YTV1dIi/8oe6yXPe/YXPxkEqOhxgq ZTfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=ywMSPGSBsgf4QryHMoW7PHTC4mV0EqvduXhE2IR3clU=; b=YBg2LfwGMBzLACfUJ6JyrkCNL6R6DbCY50Rd349i12oTkSm/PuHAATTDvEFmAMfGIy vOkAcFIp7wfP00vKJR2yjqBinoPAPMSl58uJIF40E8RR/+h121NeD5IJPbBmZy8uyQnB l3vvZHeBGMFPWfGIbUYdxSuDTYmyvrIdaMMRXFX6MC8nHGtW79m5X2ZpFO67OjGu7ToA HLn4x3+am028maCz9zG57V3uBxcyd87c74w+Naf7tUiduPKj0yVK8uoOpYmORBUMIaEi TnfkO8Al766Uj0tQnVCkVgIKK+oyn9wM5OYovnjDSTP40OxGfNq/DmedWIK+HSfQvVNG CwsA== X-Gm-Message-State: AKGB3mIeqa4gwAtSFCEnuHiOwBhYv/XnUrPKuqbXttMV3e93RJtbkAAL UH+p5i9zlwkF1hGCLiNPtNE0WI1mKd+1nSVZMgQ= X-Google-Smtp-Source: ACJfBovG4h7mGkqbRjCnM1Tcvj3USMoDFMEE0kfZrQzW/7u876jYhPKC7kzgYN3JMoBHtQ5fe7iu+aycAJVz4M7pPMo= X-Received: by 10.159.60.84 with SMTP id w20mr1947648uah.18.1513026241675; Mon, 11 Dec 2017 13:04:01 -0800 (PST) MIME-Version: 1.0 Sender: gmaxwell@gmail.com Received: by 10.103.85.148 with HTTP; Mon, 11 Dec 2017 13:04:01 -0800 (PST) In-Reply-To: References: From: Gregory Maxwell Date: Mon, 11 Dec 2017 21:04:01 +0000 X-Google-Sender-Auth: eFGtBZXR0tHUfnnppABS8GcSX-g Message-ID: To: Jim Posen , Bitcoin Protocol Discussion Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] "Compressed" headers stream X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Dec 2017 21:04:03 -0000 On Mon, Dec 11, 2017 at 8:40 PM, Jim Posen via bitcoin-dev wrote: > Firstly, I don't like the idea of making the net header encoding dependent > on the specific header validation rules that Bitcoin uses (eg. the fact that > difficulty is only recalculated every 2016 blocks). This would be coupling In the last proposal I recall writing up, there was a one byte flag on each header to indicate what was included. Nbits _never_ needs to be sent even with other consensus rules because its more or less necessarily a strict function of the prior headers. This still holds in every clone of Bitcoin I'm aware of; sending it with the first header in a group probably makes sense so it can be checked independently. > with insufficient benefit. another >18% reduction in size beyond the removal of prev. is not insubstantial by any means. I don't think it should lightly be ignored. Prev omission itself is not, sadly, magically compatible: I am quite confident that if there is a bitcoin hardfork it would recover the nbits/4-guarenteed always-zero bits of prev to use as extra nonce for miners. This has been proposed many times, implemented at least once, and the current requirement for mining infrastructure to reach inside the coinbase txn to increment a nonce has been a reliable source of failures. So I think we'd want to have the encoding able to encode leading prev bits. Many altcoins also change the header structures. If the better thing is altcoin incompatible, we should still do it. Doing otherwise would competitively hobble Bitcoin especially considering the frequent recklessly incompetent moves made by various altcoins and the near total lack of useful novel development we've seen come out of the clones. Probably the most important change in a new header message wouldn't be the encoding, but it would be changing the fetching mechanism so that header sets could be pulled in parallel, etc. I would rather not change the serialization of existing messages, nodes are going to have to support speaking both messages for a long time, and I think we already want a different protocol flow for headers fetching in any case.