From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E5EE389C for ; Fri, 4 Dec 2015 12:45:19 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com [209.85.215.45]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 81DC7126 for ; Fri, 4 Dec 2015 12:45:15 +0000 (UTC) Received: by lffu14 with SMTP id u14so111207718lff.1 for ; Fri, 04 Dec 2015 04:45:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=xmfgTCei0C9wOuJtPoBMdg28mMnaerY5cB01TELbZyk=; b=Vyxt21Y5kWE6zNxJPbene5nc0G13E/KTa7INc+apQEENfLU+CIB5V3h6r154RXo7qR h7+IAlaYuVLn2gRwt0yCLDRi2RSCKHbedhaZe2MQ8eLfxAzqSCmJWHtdqZ2cl4Kj5xLy kUToU48v15V6Pu1lC7b94Zqs7J+a+bCvwk5HJ9r377SQDZ7nAjZrUmZ66SZPnprM8KHk fpnsOelQdfc87pb3AemxiraAtnKgCbHR9enxHwNk3KaKWmReXi2SfSlHqf2TD3WdcWEt kBP6BC2hExbDaC3UtHtHMgc+/aMzUH7nE15DxvwvJ6jtDxJPvvbb64WQnaza8EbaQjtH SggA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=xmfgTCei0C9wOuJtPoBMdg28mMnaerY5cB01TELbZyk=; b=KhlqwsYovhztAhZSAKi/b0grsYWTqKNi4kLTgYVeNS2M7wznJJpnIwheSQYd8n6f+n wIyk7BQ3Mxq5Jmmmr8c/5KShjVIlhPoch/kpc/05lQr014dJz/wMyrr4sCa61VaNVc/M dUp9HFJ4O/RZgH+VXYriTa1wxm7BFZzDzpRHT0c0TCnjyIlqD5T2HKRGQ8adYMUwc7UQ muXmEyq2IPyQBbHE/+hoCSXDQ3EOSC9qR7pcqDrwVXef9o32JT5h0GiOiGUy+5brBGM9 tYwsxRGUEdzAPrnjBvg+5YFdgUkrf+aXg4r7Z5y1pm19SgWlTKKEIimUgZCV44vX8ixo GiMg== X-Received: by 10.25.145.81 with SMTP id t78mr8156647lfd.86.1449233113636; Fri, 04 Dec 2015 04:45:13 -0800 (PST) Received: from mail-lf0-f49.google.com (mail-lf0-f49.google.com. [209.85.215.49]) by smtp.gmail.com with ESMTPSA id s7sm2271095lbo.30.2015.12.04.04.45.12 for (version=TLSv1/SSLv3 cipher=OTHER); Fri, 04 Dec 2015 04:45:13 -0800 (PST) Received: by lfaz4 with SMTP id z4so111345552lfa.0 for ; Fri, 04 Dec 2015 04:45:12 -0800 (PST) X-Gm-Message-State: ALoCoQkTatgaMid17G5bE4BHK2YUVlkdUxnnbxXrlpvbZyRNb2y0J90QT09QiHpgbgP3xsrmXFKI X-Received: by 10.25.64.5 with SMTP id n5mr7274038lfa.18.1449233112607; Fri, 04 Dec 2015 04:45:12 -0800 (PST) MIME-Version: 1.0 Received: by 10.112.157.199 with HTTP; Fri, 4 Dec 2015 04:44:52 -0800 (PST) In-Reply-To: References: From: Jannes Faber Date: Fri, 4 Dec 2015 13:44:52 +0100 X-Gmail-Original-Message-ID: Message-ID: To: Gregory Maxwell Content-Type: multipart/alternative; boundary=001a113eb83edfcb8d052611e04b X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Fri, 04 Dec 2015 13:29:23 +0000 Cc: Bitcoin Dev Subject: Re: [bitcoin-dev] Blockchain verification flag (BIP draft) X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Dec 2015 12:45:20 -0000 --001a113eb83edfcb8d052611e04b Content-Type: text/plain; charset=UTF-8 1) (I would assume this is already current default behaviour, but just in case.) Would it not make sense to *never* send a blockheader to an SPV client unless the node itself fully validated that block? Regardless of who mined the block and whether this verification flag has been set or not. 2) Besides having your verification flag in the block, would it not also make sense to have such a flag in the P2P protocol when blocks (or headers) are communicated? That way a node could simply do some quick sanity checks (difficulty as anti-DOS) on an incoming block and then immediately propagate it to the next (non-SPV) node, but with a flag "Looks good, but I haven't fully validated it myself, so please don't blame me". And if the block does turn out to be invalid, the node does not get banned if it was honest about it. 3) With the above implemented, I can imagine miners running 2 (or more) nodes side by side, one of them doesn't validate in order to reduce latency and orphan rates, but the other one does validate and quickly signals the first one if there's a problem. Both nodes don't necessarily need to be in the same network or even on the same side of the Great Firewall. Of course they would be whitelisting each other for trust, or the signal would need to include some sort of proof. This probably has been suggested many times already, sorry if this is a dumb idea. -- Jannes On 4 December 2015 at 09:26, Gregory Maxwell via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > For discussion, > > A significant fraction of hashrate currently mines blocks without > verifying them for a span of time after a new block shows up on the > network for economically rational reasons. This otherwise harmful > behavior can be made a beneficial to the whole network; but only if it > is communicated. > > This BIP proposal suggests a communication channel and describes its > use and the motivations for it. I wrote it in response to suggestions > that Bitcoin Core add explicit support for this kind of mining, which > could also implement best in class risk mitigations. I believe > signaling the behavior is a necessary component for risk mitigation > here. > > ----------------------------------------------------------------- > > 
>   BIP: draft-maxwell-flagverify
>   Title: Blockchain verification flag
>   Author: Greg Maxwell 
>   Status: Draft
>   Type: Standards Track
>   Created: 2015-12-02
>
> > ==Abstract== > > This BIP describes a flag that the authors of blocks can use to voluntarily > signal that they have completely validated the content of their > block and the blocks before it. > > Correct use of this signaling is not enforced internally to the network > but if used it can act as a hint allowing more intelligent risk analysis. > > If deployed and adhered to, this mechanism turns otherwise harmful > validation skipping by miners into a behavior which benefits the public. > > ==Summary== > > The version field in a Bitcoin block header is a 32-bit signed integer. > > The most significant bit (30) of the block version is defined to signal > that > the author of the block has validated the whole chain up to and including > the > content of the block. > > Conforming miners MUST NOT set this flag when they have not completely > validated the prior block(s) or the content of their own block. Miners > should continue to try to minimize the amount of time spent mining > on a non-validated chain. Blocks which extend an invalid chain will > continue to be rejected and ultimately orphaned as validation catches up. > > It is recommended, but not required, that miners also not set the flag on > blocks > created by the same device which created the block immediately prior. This > will reduce the incorrect implication of independent validation when the > two > most recent blocks are both the product of the same, single, faulty system. > > The set state for the bit is defined as verified so that that > un(der)maintained systems do not falsely signal validation. > > Non-verifying clients of the network may check this bit (e.g. checking > that the version is >= 1073741824) and use it as an input to their risk > modeling. It is recommended that once this BIP is widely accepted by the > network that non-full-node wallets refrain from counting confirmations on > blocks where the bit is not set. > > The authors of non-verifying clients should keep in mind that this flag > is only correct with the cooperation of the block author, and even then > a validating miner may still accidentally accept or produce an invalid > block due to faulty hardware or software. Additionally, any miner which > correctly uses this flag could stop doing so at any time, and might > do so intentionally in order to increase the effectiveness of an attack. > As a result of misunderstanding, misconfiguration, laziness, or other > human factors some miners may falsely set the flag. Because invalid > blocks are rare it may take a long time to detect misuse of the flag. > > As such, the accuracy of this field MUST NOT be strongly relied upon. > > Especially due to the non-enforceability of the flag, the user community > should keep in mind that both setting the flag correctly and mining > without verification (for brief periods of time) are healthy for the > network. If participants are punished for following this specification > they will simply lie, and its utility will be diminished. > > ==Motivation== > > Some applications of the Bitcoin system such as thin-client wallets make > a strong assumption that all the authors of the blocks have faithfully > verified the blockchain. Because many of these applications also take > irreversible actions based on only one or two confirmations and the time > between blocks is often very short, these clients are vulnerable to > even small and short-duration violations of this assumption. > > Processing and propagation delays resulting from increased transaction > load contribute to block orphaning when multiple blocks are found at > close to the same time. This has caused some miners to work on extending > the chain with the most proof-of-work prior to validating the latest > block(s). > > Although this validation skipping undermines the security assumptions > of thin clients, it also has a beneficial effect: these delays also > make the mining process unfair and cause increased rewards for the > largest miners relative to other miners, resulting in a centralization > pressure. Deferring validation can reduce this pressure and improve > the security of the Bitcoin system long term. > > This BIP seeks to mitigate the harm of breaking the thin client > assumption by allowing miners to efficiently provide additional > information on their level of validation. By doing so the > network can take advantage of the benefits of bypassed > validation with minimal collateral damage. > > ==Deployment== > > Because there is no consensus enforced behavior there is no special > deployment strategy required. [BIP 9 will need to be updated.] > > ==Credits== > > Thanks goes to Jeremy Rubin for his two-phase mining suggestion > which inspired this simplified proposal. > > ==Copyright== > > This document is placed in the public domain. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > --001a113eb83edfcb8d052611e04b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
1) (I would assume this is already current default be= haviour, but just in case.) Would it not make sense to *never* send a block= header to an SPV client unless the node itself fully validated that block? = Regardless of who mined the block and whether this verification flag has be= en set or not.

2) Besides having your verification flag i= n the block, would it not also make sense to have such a flag in the P2P pr= otocol when blocks (or headers) are communicated? That way a node could sim= ply do some quick sanity checks (difficulty as anti-DOS) on an incoming blo= ck and then immediately propagate it to the next (non-SPV) node, but with a= flag "Looks good, but I haven't fully validated it myself, so ple= ase don't blame me". And if the block does turn out to be invalid,= the node does not get banned if it was honest about it.

=
3) With the above implemented, I can imagine miners running 2 (o= r more) nodes side by side, one of them doesn't validate in order to re= duce latency and orphan rates, but the other one does validate and quickly = signals the first one if there's a problem. Both nodes don't necess= arily need to be in the same network or even on the same side of the Great = Firewall. Of course they would be whitelisting each other for trust, or the= signal would need to include some sort of proof.

<= div>This probably has been suggested many times already, sorry if this is a= dumb idea.

--
Jannes

On 4 December 2015 at 09:26, Gregory Maxwell= via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.or= g> wrote:
For discussion,
A significant fraction of hashrate currently mines blocks without
verifying them for a span of time after a new block shows up on the
network for economically rational reasons. This otherwise harmful
behavior can be made a beneficial to the whole network; but only if it
is communicated.

This BIP proposal suggests a communication channel and describes its
use and the motivations for it.=C2=A0 I wrote it in response to suggestions=
that Bitcoin Core add explicit support for this kind of mining, which
could also implement best in class risk mitigations. I believe
signaling the behavior is a necessary component for risk mitigation
here.

-----------------------------------------------------------------

<pre>
=C2=A0 BIP: draft-maxwell-flagverify
=C2=A0 Title: Blockchain verification flag
=C2=A0 Author: Greg Maxwell <greg@xiph.= org>
=C2=A0 Status: Draft
=C2=A0 Type: Standards Track
=C2=A0 Created: 2015-12-02
</pre>

=3D=3DAbstract=3D=3D

This BIP describes a flag that the authors of blocks can use to voluntarily=
signal that they have completely validated the content of their
block and the blocks before it.

Correct use of this signaling is not enforced internally to the network
but if used it can act as a hint allowing more intelligent risk analysis.
If deployed and adhered to, this mechanism turns otherwise harmful
validation skipping by miners into a behavior which benefits the public.
=3D=3DSummary=3D=3D

The version field in a Bitcoin block header is a 32-bit signed integer.

The most significant bit (30) of the block version is defined to signal tha= t
the author of the block has validated the whole chain up to and including t= he
content of the block.

Conforming miners MUST NOT set this flag when they have not completely
validated the prior block(s) or the content of their own block.=C2=A0 Miner= s
should continue to try to minimize the amount of time spent mining
on a non-validated chain.=C2=A0 Blocks which extend an invalid chain will continue to be rejected and ultimately orphaned as validation catches up.
It is recommended, but not required, that miners also not set the flag on b= locks
created by the same device which created the block immediately prior.=C2=A0= This
will reduce the incorrect implication of independent validation when the tw= o
most recent blocks are both the product of the same, single, faulty system.=

The set state for the bit is defined as verified so that that
un(der)maintained systems do not falsely signal validation.

Non-verifying clients of the network may check this bit (e.g. checking
that the version is >=3D 1073741824) and use it as an input to their ris= k
modeling.=C2=A0 It is recommended that once this BIP is widely accepted by = the
network that non-full-node wallets refrain from counting confirmations on blocks where the bit is not set.

The authors of non-verifying clients should keep in mind that this flag
is only correct with the cooperation of the block author, and even then
a validating miner may still accidentally accept or produce an invalid
block due to faulty hardware or software.=C2=A0 Additionally, any miner whi= ch
correctly uses this flag could stop doing so at any time, and might
do so intentionally in order to increase the effectiveness of an attack. As a result of misunderstanding, misconfiguration, laziness, or other
human factors some miners may falsely set the flag.=C2=A0 Because invalid blocks are rare it may take a long time to detect misuse of the flag.

As such, the accuracy of this field MUST NOT be strongly relied upon.

Especially due to the non-enforceability of the flag, the user community should keep in mind that both setting the flag correctly and mining
without verification (for brief periods of time) are healthy for the
network.=C2=A0 If participants are punished for following this specificatio= n
they will simply lie, and its utility will be diminished.

=3D=3DMotivation=3D=3D

Some applications of the Bitcoin system such as thin-client wallets make a strong assumption that all the authors of the blocks have faithfully
verified the blockchain.=C2=A0 Because many of these applications also take=
irreversible actions based on only one or two confirmations and the time between blocks is often very short, these clients are vulnerable to
even small and short-duration violations of this assumption.

Processing and propagation delays resulting from increased transaction
load contribute to block orphaning when multiple blocks are found at
close to the same time. This has caused some miners to work on extending the chain with the most proof-of-work prior to validating the latest
block(s).

Although this validation skipping undermines the security assumptions
of thin clients, it also has a beneficial effect: these delays also
make the mining process unfair and cause increased rewards for the
largest miners relative to other miners, resulting in a centralization
pressure.=C2=A0 Deferring validation can reduce this pressure and improve the security of the Bitcoin system long term.

This BIP seeks to mitigate the harm of breaking the thin client
assumption by allowing miners to efficiently provide additional
information on their level of validation.=C2=A0 By doing so the
network can take advantage of the benefits of bypassed
validation with minimal collateral damage.

=3D=3DDeployment=3D=3D

Because there is no consensus enforced behavior there is no special
deployment strategy required.=C2=A0 [BIP 9 will need to be updated.]

=3D=3DCredits=3D=3D

Thanks goes to Jeremy Rubin for his two-phase mining suggestion
which inspired this simplified proposal.

=3D=3DCopyright=3D=3D

This document is placed in the public domain.
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.= linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev

--001a113eb83edfcb8d052611e04b--