From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 03E7836 for ; Fri, 8 Jun 2018 16:14:44 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ua0-f195.google.com (mail-ua0-f195.google.com [209.85.217.195]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6C9D5708 for ; Fri, 8 Jun 2018 16:14:43 +0000 (UTC) Received: by mail-ua0-f195.google.com with SMTP id c23-v6so9248170uan.3 for ; Fri, 08 Jun 2018 09:14:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=H2JeyC9VUr7/EjgpcIyyiQz6GrC67fl99Omzgqhn0lk=; b=Ntg/ymEZlW6lGuLK30ihXyrfC2NTsbpPwRwk0gwfC4auRpNh54+IQcLfOzyjJgdJpP tJnjRFmQH2afguCWzKLUeUN8cmCt2derZrj8zA1ClQnoMn6B+Qphp322dMQVwShCPvr7 2o9VADMCQEyJldyASZbcBNb7oT55xsZYNvGVliQPsl3kGrYl2TLAHVtHQ5Xb9m7N411y O+ftKY8ogoDmUjjOmP8GGiZSTVf3L78eL2bSlf/IzyYwO9RZPKQsmLNjgIKK8XDmcvPA zpMRlcp+8yEMfrTj60zi2wSL7lh/tsp51aXdJk1GyJOwB1T3QliovLK9vRCSzEwDGIJT klzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=H2JeyC9VUr7/EjgpcIyyiQz6GrC67fl99Omzgqhn0lk=; b=ZGrcqz0LCCKq+EyFHNh8pnbQo+u9yjEFa0IHRcjf1A4Xdxuav1MmUiGRT+9zxaQEhR wonI8X2FQZcAI7QYHEjcB8y4p9bBvU//e3x8s5VsC9FHflTQmH2PKa4qFYbSZKGT1OHK HUEn2TAD5KomSHj3/qVTpqn2r9Fsjta2e1QQyWdaFLmb472kYPl0iiFux05hMTxSAtF+ MZ4REBXe58s/WZfx1qv1VZABJdIF+dS/o+M2vwOwq7nh2xzw7yIOZxH5jeh9gt5E42IH UmEZsaf/oY0Bod6sN+RAQ0HThNLR7G/K8N0uP0TJYiZ70oEI9QjuwenWstQR+dBTnZfn 9aVw== X-Gm-Message-State: APt69E3H06LsWrNUc5Np3FEyOTV5PjH1P/caLDJiZfwbVk+mtt+aDxzq jm93q/gKuOppO7B9SP53yCCgNQK0iN+2cR00wcQ= X-Google-Smtp-Source: ADUXVKIRUCYbhjSeioz4jgQkiJ7kS4W3N3/qoLftAJi40zMlsLDPDGjVUB8kLUPnkPPRrdE8DgGW9Exwun+ldr5OB9A= X-Received: by 2002:ab0:1092:: with SMTP id d18-v6mr4531025uab.87.1528474482452; Fri, 08 Jun 2018 09:14:42 -0700 (PDT) MIME-Version: 1.0 Sender: gmaxwell@gmail.com Received: by 2002:a67:5193:0:0:0:0:0 with HTTP; Fri, 8 Jun 2018 09:14:41 -0700 (PDT) In-Reply-To: References: <7E4FA664-BBAF-421F-8C37-D7CE3AA5310A@gmail.com> <20180602124157.744x7j4u7dqtaa43@email> <343A3542-3103-42E9-95B7-640DFE958FFA@gmail.com> <37BECD1A-7515-4081-85AC-871B9FB57772@gmail.com> From: Gregory Maxwell Date: Fri, 8 Jun 2018 16:14:41 +0000 X-Google-Sender-Auth: Ys0NIvNSlPsxNAUSOcC7mVnvoe4 Message-ID: To: Olaoluwa Osuntokun , Bitcoin Protocol Discussion Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] BIP 158 Flexibility and Filter Size X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2018 16:14:44 -0000 On Fri, Jun 8, 2018 at 5:03 AM, Olaoluwa Osuntokun via bitcoin-dev wrote: > As someone who's written and reviews code integrating the proposal all the > way up the stack (from node to wallet, to application), IMO, there's no > immediate cost to deferring the inclusion/creation of a filter that includes > prev scripts (b) instead of the outpoint as the "regular" filter does now. > Switching to prev script in the _short term_ would be costly for the set of > applications already deployed (or deployed in a minimal or flag flip gated > fashion) as the move from prev script to outpoint is a cascading one that > impacts wallet operation, rescans, HD seed imports, etc. It seems to me that you're making the argument against your own case here: I'm reading this as a "it's hard to switch so it should be done the inferior way". That in argument against adopting the inferior version, as that will contribute more momentum to doing it in a way that doesn't make sense long term. > Such a proposal would need to be generalized enough to allow several components to be committed, I don't agree at all, and I can't see why you say so. > likely have versioning, This is inherent in how e.g. the segwit commitment is encoded, the initial bytes are an identifying cookies. Different commitments would have different cookies. > and also provide the necessary extensibility to allow additional items to be committed in the future What was previously proposed is that the commitment be required to be consistent if present but not be required to be present. This would allow changing whats used by simply abandoning the old one. Sparsity in an optional commitment can be addressed when there is less than 100% participation by having each block that includes a commitment commit to the missing filters ones from their immediate ancestors. Additional optionality can be provided by the other well known mechanisms, e.g. have the soft fork expire at a block 5 years out past deployment, and continue to soft-fork it in for a longer term so long as its in use (or eventually without expiration if its clear that it's not going away). > wallets which wish to primarily use the filters for rescan purposes can't > just construct them locally for this particular use case independent of > what's currently deployed on the p2p network. Absolutely, but given the failure of BIP37 on the network-- and the apparent strong preference of end users for alternatives that don't scan (e.g. electrum and web wallets)-- supporting making this available via P2P was already only interesting to many as a nearly free side effect of having filters for local scanning. If it's a different filter, it's no longer attractive. It seems to me that some people have forgotten that this whole idea was originally proposed to be a committed data-- but with an added advantage of permitting expirementation ahead of the commitment. > Maintaining the outpoint also allows us to rely on a "single honest peer"security model in the short term. You can still scan blocks directly when peers disagree on the filter content, regardless of how the filter is constructed-- yes, it uses more bandwidth if you're attacked, but it makes the attack ineffective and using outpoints considerably increases bandwidth for everyone without an attack. These ineffective (except for increasing bandwidth) attacks would have to be common to offset the savings. It seems to me this point is being overplayed, especially considering the current state of non-existing validation in SPV software (if SPV software doesn't validate anything else they could be validating, why would they implement a considerable amount of logic for this?).