Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[NotMike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1422 bytes --]

On 28 September 2015 at 06:48, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> There is no consensus on using a soft fork to deploy this feature. It will
> result in the same problems as all the other soft forks - SPV wallets will
> become less reliable during the rollout period. I am against that, as it's
> entirely avoidable.
>
> Make it a hard fork and my objection will be dropped.
>
> Until then, as there is no consensus, you need to do one of two things:
>
> 1) Drop the "everyone must agree to make changes" idea that people here
like
> to peddle, and do it loudly, so everyone in the community is correctly
> informed
>
> 2) Do nothing
>
>

I agree with Mike Hearn that there is no consensus on using a soft fork to
deploy this feature. Either everyone agrees that we should all agree on
consensus or else there is arbitrary disagreement. You cannot have it both
ways.

It is very important that we reach consensus on consensus or, if you will,
meta0consensus. I think we should Do nothing as that is clearly the choice
that we have taken re: blocksize. If we use one set of rules for that
decision we should use the same set of rules for all decisions and there is
no middle ground.

Thank you.

>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 2038 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[GC]

[-- Attachment #1: Type: text/plain, Size: 2140 bytes --]

Or, you know, enter some discussions on what exactly are the issues that SPV
clients face during soft forks and see if anything can be done (on all
sides) to mitigate the risks.

Crazy stuff, I know Š ;-)

From:  NotMike Hearn via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Reply-To:  NotMike Hearn <not.mike.hearn@gmail.com>
Date:  Friday, 2 October 2015 9:57 am
To:  <bitcoin-dev@lists.linuxfoundation.org>
Subject:  Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

On 28 September 2015 at 06:48, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> There is no consensus on using a soft fork to deploy this feature. It will
> result in the same problems as all the other soft forks - SPV wallets will
> become less reliable during the rollout period. I am against that, as it's
> entirely avoidable.
>
> Make it a hard fork and my objection will be dropped.
>
> Until then, as there is no consensus, you need to do one of two things:
>
> 1) Drop the "everyone must agree to make changes" idea that people here like
> to peddle, and do it loudly, so everyone in the community is correctly
> informed
>
> 2) Do nothing
>
>

I agree with Mike Hearn that there is no consensus on using a soft fork to
deploy this feature. Either everyone agrees that we should all agree on
consensus or else there is arbitrary disagreement. You cannot have it both
ways.

It is very important that we reach consensus on consensus or, if you will,
meta0consensus. I think we should Do nothing as that is clearly the choice
that we have taken re: blocksize. If we use one set of rules for that
decision we should use the same set of rules for all decisions and there is
no middle ground.

Thank you.

>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

_______________________________________________ bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


[-- Attachment #2: Type: text/html, Size: 3803 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 3213 bytes --]

Putting aside stupid arguments about who is older or who starting using the
term SPV wallet first, let me try and make a better suggestion than what's
in the BIP. How about the following:

A new flag is introduced to Core, --scriptchecks=[all,standardonly,none].
The default is all. When set to "standardonly", non-standard scripts are
not checked but others are. This is similar to the behaviour during a soft
fork. In "none" you have something a bit like SPV mode, but still
calculating the UTXO set. This flag is simple and can be implemented in a
few lines of code. Then an unused opcode is used for CLTV, so making it a
hard fork.

This has the following advantages:

   - Nodes that want the pseudo-SPV behaviour of a soft fork can opt in to
   it if they want it. This prioritises availability (in a sense) over
   correctness.

   - But otherwise, nodes will prioritise correctness by default, which is
   how it should be. This isn't PHP where nonsensical code the interpreter
   doesn't understand just does ...... something. This is financial software
   where money is at risk. I feel very strongly about this: undefined
   behaviour is fine *if you opted into getting it. *Otherwise it should be
   avoided whenever possible.

   - SPV wallets do the right thing by default.

   - IsStandard doesn't silently become a part of the consensus rules.

   - All other software gets simpler. It's not just SPV wallets. Block
   explorers, for example, can just add a single line to their opcode map.
   With a soft fork they have to implement the entire soft fork logic just to
   figure out when an opcode transitioned from OP_NOP to CLTV and make sure
   they render old scripts differently to new scripts. And they face tricky
   questions - do they render an opcode as a NOP if the miner who built it was
   un-upgraded, or do they calculate the flag day and change all of them after
   that? It's just an explosion of complexity.

Many people by now have accepted that hard forks are simpler, conceptually
cleaner, and prioritise correctness of results over availability of
results. I think these arguments are strong.

So let me try addressing the counter-arguments one more time:

   - Hard forks require everyone to upgrade and soft forks don't. I still
   feel this one has never actually been explained. There is no difference to
   the level of support required to trigger the change. With the suggestion
   above, if someone can't or won't upgrade their full node but can no longer
   verify the change, they can simply restart with -scriptchecks=standardonly
   and get the soft fork behaviour. Or they can upgrade and get their old
   security level back.

   - Hard forks are somehow bad or immoral or can lead to "schisms". This
   is just saying, if we hold a vote, the people who lose the vote might try
   starting a civil war and refuse to accept the change. That's not a reason
   to not hold votes.

   But at any rate, they can do that with soft forks too: just decide that
   any output that contains OP_CLTV doesn't make it into the UTXO set.
   Eventually coins that trace back to such an output will become unusable in
   the section of the economy that decided to pick a fight.

[-- Attachment #2: Type: text/html, Size: 3446 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 4316 bytes --]

- It is true that hard forks produce a much cleaner outcome, in terms of
well defined behavior across the entire network.

- Replacing an opcode should not result in undefined behavior.  The
non-upgraded behavior is defined and deterministic.

- IsStandard remains an assistant.  Miners may mine non-standard
transactions.

- "Hard forks require everyone to upgrade and soft forks don't"   Doesn't
require tons of explanation:  Non upgraded clients continue working on the
network even after the rules are upgraded.

All those corrections aside, I do think there has been too much hysteria
surrounding hard forks.  Hard forks, when done right, produce a much
cleaner system for users.








On Mon, Oct 5, 2015 at 6:59 AM, Mike Hearn via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Putting aside stupid arguments about who is older or who starting using
> the term SPV wallet first, let me try and make a better suggestion than
> what's in the BIP. How about the following:
>
> A new flag is introduced to Core, --scriptchecks=[all,standardonly,none].
> The default is all. When set to "standardonly", non-standard scripts are
> not checked but others are. This is similar to the behaviour during a soft
> fork. In "none" you have something a bit like SPV mode, but still
> calculating the UTXO set. This flag is simple and can be implemented in a
> few lines of code. Then an unused opcode is used for CLTV, so making it a
> hard fork.
>
> This has the following advantages:
>
>    - Nodes that want the pseudo-SPV behaviour of a soft fork can opt in
>    to it if they want it. This prioritises availability (in a sense) over
>    correctness.
>
>    - But otherwise, nodes will prioritise correctness by default, which
>    is how it should be. This isn't PHP where nonsensical code the interpreter
>    doesn't understand just does ...... something. This is financial software
>    where money is at risk. I feel very strongly about this: undefined
>    behaviour is fine *if you opted into getting it. *Otherwise it should
>    be avoided whenever possible.
>
>    - SPV wallets do the right thing by default.
>
>    - IsStandard doesn't silently become a part of the consensus rules.
>
>    - All other software gets simpler. It's not just SPV wallets. Block
>    explorers, for example, can just add a single line to their opcode map.
>    With a soft fork they have to implement the entire soft fork logic just to
>    figure out when an opcode transitioned from OP_NOP to CLTV and make sure
>    they render old scripts differently to new scripts. And they face tricky
>    questions - do they render an opcode as a NOP if the miner who built it was
>    un-upgraded, or do they calculate the flag day and change all of them after
>    that? It's just an explosion of complexity.
>
> Many people by now have accepted that hard forks are simpler, conceptually
> cleaner, and prioritise correctness of results over availability of
> results. I think these arguments are strong.
>
> So let me try addressing the counter-arguments one more time:
>
>    - Hard forks require everyone to upgrade and soft forks don't. I still
>    feel this one has never actually been explained. There is no difference to
>    the level of support required to trigger the change. With the suggestion
>    above, if someone can't or won't upgrade their full node but can no longer
>    verify the change, they can simply restart with -scriptchecks=standardonly
>    and get the soft fork behaviour. Or they can upgrade and get their old
>    security level back.
>
>    - Hard forks are somehow bad or immoral or can lead to "schisms". This
>    is just saying, if we hold a vote, the people who lose the vote might try
>    starting a civil war and refuse to accept the change. That's not a reason
>    to not hold votes.
>
>    But at any rate, they can do that with soft forks too: just decide
>    that any output that contains OP_CLTV doesn't make it into the UTXO set.
>    Eventually coins that trace back to such an output will become unusable in
>    the section of the economy that decided to pick a fight.
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 5413 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 5476 bytes --]

Well, let's agree to disagree on these two things:

- I define "working" for a full node as verifying everything; if a node
starts skipping bits then I'd say it's not really "working" according to
its original design goals

- Saying the pre-fork behaviour is defined and deterministic is true, but
only in the sense that reading an uninitialised variable in C is defined
and deterministic. It reads whatever happens to be at that stack position:
easily defined. For many programs, that may be the same value each time:
deterministic. Nonetheless, it's considered undefined behaviour by the C
specification and programmers that rely on it can easily create security
holes.

In the same way, I'd consider a node running a script with a NOP and
reaching the opposite conclusion from other nodes to be a case of undefined
behaviour leading to a non-fully-working node.

But these are arguments about the semantics of words. I think we both know
what each other is getting at.

On Mon, Oct 5, 2015 at 1:23 PM, Jeff Garzik <jgarzik@gmail.com> wrote:

>
> - It is true that hard forks produce a much cleaner outcome, in terms of
> well defined behavior across the entire network.
>
> - Replacing an opcode should not result in undefined behavior.  The
> non-upgraded behavior is defined and deterministic.
>
> - IsStandard remains an assistant.  Miners may mine non-standard
> transactions.
>
> - "Hard forks require everyone to upgrade and soft forks don't"   Doesn't
> require tons of explanation:  Non upgraded clients continue working on the
> network even after the rules are upgraded.
>
> All those corrections aside, I do think there has been too much hysteria
> surrounding hard forks.  Hard forks, when done right, produce a much
> cleaner system for users.
>
>
>
>
>
>
>
>
> On Mon, Oct 5, 2015 at 6:59 AM, Mike Hearn via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Putting aside stupid arguments about who is older or who starting using
>> the term SPV wallet first, let me try and make a better suggestion than
>> what's in the BIP. How about the following:
>>
>> A new flag is introduced to Core, --scriptchecks=[all,standardonly,none].
>> The default is all. When set to "standardonly", non-standard scripts are
>> not checked but others are. This is similar to the behaviour during a soft
>> fork. In "none" you have something a bit like SPV mode, but still
>> calculating the UTXO set. This flag is simple and can be implemented in a
>> few lines of code. Then an unused opcode is used for CLTV, so making it a
>> hard fork.
>>
>> This has the following advantages:
>>
>>    - Nodes that want the pseudo-SPV behaviour of a soft fork can opt in
>>    to it if they want it. This prioritises availability (in a sense) over
>>    correctness.
>>
>>    - But otherwise, nodes will prioritise correctness by default, which
>>    is how it should be. This isn't PHP where nonsensical code the interpreter
>>    doesn't understand just does ...... something. This is financial software
>>    where money is at risk. I feel very strongly about this: undefined
>>    behaviour is fine *if you opted into getting it. *Otherwise it should
>>    be avoided whenever possible.
>>
>>    - SPV wallets do the right thing by default.
>>
>>    - IsStandard doesn't silently become a part of the consensus rules.
>>
>>    - All other software gets simpler. It's not just SPV wallets. Block
>>    explorers, for example, can just add a single line to their opcode map.
>>    With a soft fork they have to implement the entire soft fork logic just to
>>    figure out when an opcode transitioned from OP_NOP to CLTV and make sure
>>    they render old scripts differently to new scripts. And they face tricky
>>    questions - do they render an opcode as a NOP if the miner who built it was
>>    un-upgraded, or do they calculate the flag day and change all of them after
>>    that? It's just an explosion of complexity.
>>
>> Many people by now have accepted that hard forks are simpler,
>> conceptually cleaner, and prioritise correctness of results over
>> availability of results. I think these arguments are strong.
>>
>> So let me try addressing the counter-arguments one more time:
>>
>>    - Hard forks require everyone to upgrade and soft forks don't. I
>>    still feel this one has never actually been explained. There is no
>>    difference to the level of support required to trigger the change. With the
>>    suggestion above, if someone can't or won't upgrade their full node but can
>>    no longer verify the change, they can simply restart with
>>    -scriptchecks=standardonly and get the soft fork behaviour. Or they can
>>    upgrade and get their old security level back.
>>
>>    - Hard forks are somehow bad or immoral or can lead to "schisms".
>>    This is just saying, if we hold a vote, the people who lose the vote might
>>    try starting a civil war and refuse to accept the change. That's not a
>>    reason to not hold votes.
>>
>>    But at any rate, they can do that with soft forks too: just decide
>>    that any output that contains OP_CLTV doesn't make it into the UTXO set.
>>    Eventually coins that trace back to such an output will become unusable in
>>    the section of the economy that decided to pick a fight.
>>
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>

[-- Attachment #2: Type: text/html, Size: 6994 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

[-- Attachment #1: Type: text/plain, Size: 1440 bytes --]

On Oct 5, 2015 1:28 PM, "Mike Hearn via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> Well, let's agree to disagree on these two things:
>
> - I define "working" for a full node as verifying everything; if a node
starts skipping bits then I'd say it's not really "working" according to
its original design goals

But assuming the hashrate majority has upgraded (and we're using 95% as the
miner upgrade confirmation threshold to start activation, so that
assumption seems pretty safe), a non-upgraded full node and an upgraded
full will converge on what they see: "the most-work valid chain" will be
the same for both. A non-upgraded full node wallet waiting for several
confirmations (for example, 6 confirmations) will be just as safe as an
upgraded one. In that sense, it keeps working. On top of that, nodes (of
any kind) can use unknown block version numbers to notify the user or even
stop working (the same notification mechanism you would use with hardforks).

I agree that hardforks are necessary and we should deploy a hardfork asap
to show the world they are indeed possible (bip99 proposes a likely
uncontroversial one), but I still believe that is clear that softfork
deployment is preferrable in many cases like this one.

Are you going to produce a bip65 hardfork alternative to try to convince
people of its advantages over bip65 (it is not clear to me how you include
a new script operand via hardfork)?

[-- Attachment #2: Type: text/html, Size: 1673 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Clément Elbaz]

[-- Attachment #1: Type: text/plain, Size: 2080 bytes --]

It will get correct results about :
- the existence every block
- the existence of every transaction

It will get incorrect results :
- about the *nature* of some transactions
- and therefore, about the balances of some wallets.

I fully agree with Mike here.

Le lun. 5 oct. 2015 à 14:04, Jorge Timón <
bitcoin-dev@lists.linuxfoundation.org> a écrit :

>
> On Oct 5, 2015 1:28 PM, "Mike Hearn via bitcoin-dev" <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
> >
> > Well, let's agree to disagree on these two things:
> >
> > - I define "working" for a full node as verifying everything; if a node
> starts skipping bits then I'd say it's not really "working" according to
> its original design goals
>
> But assuming the hashrate majority has upgraded (and we're using 95% as
> the miner upgrade confirmation threshold to start activation, so that
> assumption seems pretty safe), a non-upgraded full node and an upgraded
> full will converge on what they see: "the most-work valid chain" will be
> the same for both. A non-upgraded full node wallet waiting for several
> confirmations (for example, 6 confirmations) will be just as safe as an
> upgraded one. In that sense, it keeps working. On top of that, nodes (of
> any kind) can use unknown block version numbers to notify the user or even
> stop working (the same notification mechanism you would use with hardforks).
>
> I agree that hardforks are necessary and we should deploy a hardfork asap
> to show the world they are indeed possible (bip99 proposes a likely
> uncontroversial one), but I still believe that is clear that softfork
> deployment is preferrable in many cases like this one.
>
> Are you going to produce a bip65 hardfork alternative to try to convince
> people of its advantages over bip65 (it is not clear to me how you include
> a new script operand via hardfork)?
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 2779 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

[-- Attachment #1: Type: text/plain, Size: 458 bytes --]

On Oct 5, 2015 2:08 PM, "Clément Elbaz" <clem.ds@gmail.com> wrote:
>
> It will get correct results about :
> - the existence every block
> - the existence of every transaction
>
> It will get incorrect results :
> - about the nature of some transactions

Given the assumptions above, only of transactions without enough
confirmations.

> - and therefore, about the balances of some wallets.

Not if the wallet waits for enough confirmations.

[-- Attachment #2: Type: text/html, Size: 638 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Clément Elbaz]

[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]

I fail to see how the number of confirmations has anything to do with it.

With a non-upgraded Bitcoin software during a soft fork, you get the same
blocks as everyone else, and you get the same confirmed transactions as
everyone else. So you do have the exact same "writings" as everyone else to
calculate your balance.

The problem is that some transactions that are meaningless to you are
actually meaningful to people using an upgraded Bitcoin software.

Therefore during a softfork, while you can not miss the *existence* of a
transaction, you can miss its *meaning*.

If Bitcoin was just a decentralized whiteboard for people to write on it,
that would be no problem.

But as soon as you try to actually use Bitcoin (that is, calculate the
accurate balance of a wallet in a very broad sense), you can be led a wrong
result if you did not upgrade, which is a critical problem for financial
software.

And because nothing prevent people to send you transactions of a new type,
you have no way to "opt out" of this problem.



Le lun. 5 oct. 2015 à 14:16, Jorge Timón <jtimon@jtimon.cc> a écrit :

>
> On Oct 5, 2015 2:08 PM, "Clément Elbaz" <clem.ds@gmail.com> wrote:
> >
> > It will get correct results about :
> > - the existence every block
> > - the existence of every transaction
> >
> > It will get incorrect results :
> > - about the nature of some transactions
>
> Given the assumptions above, only of transactions without enough
> confirmations.
>
> > - and therefore, about the balances of some wallets.
>
> Not if the wallet waits for enough confirmations.
>

[-- Attachment #2: Type: text/html, Size: 2165 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

On Mon, Oct 5, 2015 at 2:29 PM, Clément Elbaz <clem.ds@gmail.com> wrote:
> The problem is that some transactions that are meaningless to you are
> actually meaningful to people using an upgraded Bitcoin software.
>
> Therefore during a softfork, while you can not miss the existence of a
> transaction, you can miss its meaning.

Why would you care about payments to other people?
The scriptPubKey's that you give to your payers certainly have meaning to you.

> But as soon as you try to actually use Bitcoin (that is, calculate the
> accurate balance of a wallet in a very broad sense), you can be led a wrong
> result if you did not upgrade, which is a critical problem for financial
> software.

What is it important that you are able to calculate balances of
wallets that aren't yours?

> And because nothing prevent people to send you transactions of a new type,
> you have no way to "opt out" of this problem.

Why would anyone "pay you" to a scriptPubKey you don't understand?

I can "pay" the bill of my internet services by burying cash in a park
nearby my house for my provider to pick up later.
But if I don't tell my provider, it will never know. If I inform it, I
will get an answer: "no, sorry, we won't accept this new 'form of
payment' of yours as payment".

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1483 bytes --]

Hi Jorge,

I'm glad we seem to be reaching agreement that hard forks aren't so bad
really and can even have advantages. It seems the remaining area of
disagreement is this rollout specifically.

> a non-upgraded full node and an upgraded full will converge on what they
> see: "the most-work valid chain" will be the same for both.
>
Indeed it will, but the point of fully verifying is to *not* converge with
the miner majority, if something goes wrong and they aren't following the
same rules as you. Defining "work" as "converge with miner majority" is
fine for SPV wallets and a correct or at least reasonable definition. But
not for fully verifying nodes, where non-convergence is an explicit design
goal! That's the only thing that stops miners awarding themselves infinite
free money!

> Are you going to produce a bip65 hardfork alternative to try to convince
> people of its advantages over bip65 (it is not clear to me how you include
> a new script operand via hardfork)?
>
No, I'm focused on the block size issue right now. I don't think there's
much point in improving the block chain protocol if most users are going to
be unable to use it. But the modification is simple, right? You just
replace this bit:

  CHECKLOCKTIMEVERIFY redefines the existing NOP2 opcode

with this

  CHECKLOCKTIMEVERIFY defines a new opcode (0xc0)

and that's it. The section *upgrade and testing plan* only says TBD so that
part doesn't even need to change at all, as it's not written yet.

[-- Attachment #2: Type: text/html, Size: 3272 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

On Mon, Oct 5, 2015 at 2:10 PM, Mike Hearn <hearn@vinumeris.com> wrote:
> Hi Jorge,
>
> I'm glad we seem to be reaching agreement that hard forks aren't so bad
> really and can even have advantages. It seems the remaining area of
> disagreement is this rollout specifically.
>>
>> a non-upgraded full node and an upgraded full will converge on what they
>> see: "the most-work valid chain" will be the same for both.
>
> Indeed it will, but the point of fully verifying is to not converge with the
> miner majority, if something goes wrong and they aren't following the same
> rules as you. Defining "work" as "converge with miner majority" is fine for
> SPV wallets and a correct or at least reasonable definition. But not for
> fully verifying nodes, where non-convergence is an explicit design goal!
> That's the only thing that stops miners awarding themselves infinite free
> money!

As Greg explained to you repeatedly, a softfork won't cause a
non-upgraded full node to start accepting blocks that create more
subsidy than is valid.
It's only the new rule (in this case, BIP65) that they won't validate.
That's very different security from an SPV node, and as Greg also
explained, SPV nodes could be much more secure than bitcoinj nodes
(they could, for example, validate the coinbase transaction of every
block).
If a non-upgraded node it's not a "full node" for you, that's fine,
but it is for everyone else. So please stop confusing other people.
Assuming the majority of the hashrate upgraded, there's almost no risk
for non-upgraded full nodes.

>> Are you going to produce a bip65 hardfork alternative to try to convince
>> people of its advantages over bip65 (it is not clear to me how you include a
>> new script operand via hardfork)?
>
> No, I'm focused on the block size issue right now. I don't think there's
> much point in improving the block chain protocol if most users are going to
> be unable to use it. But the modification is simple, right? You just replace
> this bit:
>
>   CHECKLOCKTIMEVERIFY redefines the existing NOP2 opcode
>
> with this
>
>   CHECKLOCKTIMEVERIFY defines a new opcode (0xc0)
>
> and that's it. The section upgrade and testing plan only says TBD so that
> part doesn't even need to change at all, as it's not written yet.

Thanks, I wasn't aware that there was room for new opcodes that
weren't noops already.
Can you give an example of an attack in which a non-upgraded full node
wallet is defrauded with BIP65 but could not with the hardfork
alternative (that nobody seems to be willing to implement)?
Please, don't assume 0 confirmation transactions or similar
unreasonable assumptions (ie see section 11 "Calculations" of the
Bitcoin whitepaper).

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 3313 bytes --]

>
> As Greg explained to you repeatedly, a softfork won't cause a
> non-upgraded full node to start accepting blocks that create more
> subsidy than is valid.
>

It was an example. Adam Back's extension blocks proposal would, in fact,
allow for a soft forking change that creates more subsidy than is valid (or
does anything else) by hiding one block inside another.

Anyway, I think you got my point.


> That's very different security from an SPV node, and as Greg
> also explained, SPV nodes could be much more secure than bitcoinj
> nodes (they could, for example, validate the coinbase transaction of
> every block).
>

I'm pretty sure Gregory did not use such an example because it's dead
wrong. You cannot verify the size of a coinbase without being a fully
verifying node because you need to know the fees in the block, and
calculating that requires access to the entire UTXO set.

This sort of thing is why I get annoyed when people lecture me about SPV
wallets and the things they "should" do. None of you guys has built one. I
keep seeing wild statements about theoretical unicorn wallets that nobody
has even designed, and how all existing wallets are crappy and insecure
because they don't meet your ever shifting goal posts.

To everyone making such statements I say: go away and build an SPV wallet
of your own from scratch. Then you will understand the engineering
tradeoffs involved much better, and be in a much better position to debate
what they should or should not be doing.

And bear in mind if it weren't for the work myself and a few others did on
SPV wallets, everyone would be using web wallets instead. Then you'd all
just complain about that instead.


> Can you give an example of an attack in which a non-upgraded full node
> wallet is defrauded with BIP65 but could not with the hardfork
> alternative (that nobody seems to be willing to implement)?
>

Making it a hard fork instead is changing one line of code (ignoring the
code to set up the flag day, which can be based on the code for BIP101). If
it comes down to it, then I'll do the work to change that one line. But
obviously I'd need to see agreement from the maintainers that such a pull
req would be merged first.

The example is this: find someone that accepts 1-block confirmed
transactions in return for something valuable. There are plenty of them out
there. Once the soft fork starts, send a P2SH transaction that defines a
new output controlled by OP_CLTV. It will be incorporated into the UTXO set
by all miners because it's opaque (p2sh).

Now send a transaction that pays the merchant, and make it spend your
OP_CLTV output with an invalid script. New nodes will reject it as a rule
violator. Old nodes won't. So at some point an old miner will create a
block containing your invalid transaction, the merchant will think they got
paid, they'll give you the stuff and the fraud is done.


> Please, don't assume 0 confirmation transactions or similar
> unreasonable assumptions (ie see section 11 "Calculations" of the
> Bitcoin whitepaper).
>

This is just embarrassing - do any of you guys at Blockstream actually use
Bitcoin in the real world? Virtually all payments that aren't moving money
into/out of exchange wallets are 0-confirm in reality. I described a
1-confirm attack above, but really ... come on.

[-- Attachment #2: Type: text/html, Size: 4485 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Anthony Towns]

On Mon, Oct 05, 2015 at 06:46:28PM +0200, Mike Hearn via bitcoin-dev wrote:
> The example is this: find someone that accepts 1-block confirmed
> transactions in return for something valuable. There are plenty of them out
> there. Once the soft fork starts, send a P2SH transaction that defines a
> new output controlled by OP_CLTV. It will be incorporated into the UTXO set
> by all miners because it's opaque (p2sh).
> 
> Now send a transaction that pays the merchant, and make it spend your
> OP_CLTV output with an invalid script. New nodes will reject it as a rule
> violator. Old nodes won't.

Old nodes running bitcoind will see it as OP_NOP2, and will reject it
unless they've manually disabled SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS,
which (aiui) has been available since bitcoin 0.10 [0], but not backported
to 0.8 or 0.9.

[0] https://github.com/bitcoin/bitcoin/commit/0391423

That covers about 4700/5880 nodes going by bitnodes.21.co; but I can't
tell how many miners it covers.

Further, AIUI, nodes running 0.8 or 0.9 will still apply IsStandard()
checks to scripts attempting to spend p2sh outputs [1], so will also
fail to either mine or relay your OP_NOP2 payment.

[1] https://github.com/bitcoin/bitcoin/commit/6259937

> So at some point an old miner will create a
> block containing your invalid transaction, the merchant will think they got
> paid, they'll give you the stuff and the fraud is done.

My understanding is that this isn't supposed to be a problem because you
won't be able to find an old miner that will do that; released versions
of bitcoin already block it by default.

Sure, someone could disable those checks and not pay attention to a soft
fork that will cause their blocks to be orphaned, but I'm not seeing why
that's any different a threat compared to someone deliberately mining
invalid blocks to do 1-confirmation doublespends against merchants not
running a full node.

At least, that's my understanding, and I'm not an expert, so corrections
appreciated.

Cheers,
aj

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Micha Bailey]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

On Monday, October 5, 2015, Mike Hearn via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> As Greg explained to you repeatedly, a softfork won't cause a
>> non-upgraded full node to start accepting blocks that create more
>> subsidy than is valid.
>>
>
> It was an example. Adam Back's extension blocks proposal would, in fact,
> allow for a soft forking change that creates more subsidy than is valid (or
> does anything else) by hiding one block inside another.
>

Maybe I'm missing something, but wouldn't this turn into a hard fork the
moment you try to spend an output created in one of these extension blocks?
So sure, the block that contains the extension would be considered valid,
but unupgraded validators will not update the UTXO set accordingly, meaning
that those new TXOs can't be spent because, according to their rules, they
don't exist.

>
>

[-- Attachment #2: Type: text/html, Size: 1588 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

On Fri, Oct 2, 2015 at 4:12 AM, GC via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> Or, you know, enter some discussions on what exactly are the issues that SPV
> clients face during soft forks and see if anything can be done (on all
> sides) to mitigate the risks.

This has already been discussed. The recommended risk mitigation
mechanism for softforks it's just the same as the one for hardforks:
unknown block version notifications.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

"Consensus" it's a term we use for consensus critical code and we
refer to different machines (potentially with different software)
validating in exactly the same way.
I think also using the term for people agreeing on what those
consensus rules is confusing, so in BIP99 I used the term
"uncontroversial" instead.
From BIP99 current's content (improvements welcomed):

"
Uncontroversial consensus upgrades

"Uncontroversial" is something though to define in this context. What
if a single user decides he won't upgrade no matter what and he
doesn't even attempt to explain his decision? Obviously, such a user
should be just ignored. But what if the circumstances are slightly
different? What if they're 2, 10 users? Where's the line? It is
possible that we can never have a better definition than "I know it
when I see it" [citation].
"

The fact that there's at least 3 different proposals for a blocksize
increase, that there's not a lot of data comparing different possible
block sizes and its potential effects on block propagation and that
the development progress has enormously slowed down during months of
discussion are, in my opinion, clear signs that none of the current
proposals are "uncontroversial", even by this vague definition.

I believe BIP65 is uncontroversial since no reasonable objections to
the feature itself have been raised, it has been widely reviewed and
tested. The only complain is about it is it's softfork deployment
mechanism.

Was deployment of bip16, bip30 or bip66 controversial (which were
deployed via softforks, some of them even with people [ie Mike Hearn]
preferring always hardforks over softforks) uncontroversial?
I believe they were all (maybe with the exception of bip16)
uncontroversial. That's the story bip99 is telling, but bip99 is not
finished so we can change that if it makes sense.
We could say that they have been "Unilateral softforks", but I don't
think that would be fair for the miners who helped deploy it. Or we
could always create a new category in bip99 (please, propose a new
category of softforks if you think there's some potential case that's
not covered).

This is not about Mike Hearn or you or any person in particular.
"Uncontroversial" is so far defined in a vague way, if you think you
can put a more formal definition forward, please do so (provided that
it's not an absurd definition which allows any individual to block
everything without reasonable arguments). I'm more than happy
improving bip99 before we move it from its current "draft" status.

If Mike Hearn (and you) are right, I should update bip99 to NEVER
recommend softforks for consensus rule changes.
But I still believe it is uncontroversial that softforks have great
advantages in many cases (even if not everybody understand this).
I want bip99 itself to be uncontroversial, so please nit/nack fast,
nit/nack often and please please please nit/nack on time (while bip99
is still a draft).

[bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 9504 bytes --]

Summary
-------

It's time to deploy BIP65 CHECKLOCKTIMEVERIFY.

I've backported the CLTV op-code and a IsSuperMajority() soft-fork to
the v0.10 and v0.11 branches, pull-reqs #6706 and #6707 respectively. A
pull-req for git HEAD for the soft-fork deployment has been open since
June 28th, #6351 - the opcode implementation itself was merged two
months ago.

We should release a v0.10.3 and v0.11.1 with CLTV and get the ball
rolling on miner adoption. We have consensus that we need CLTV, we have
a well tested implementation, and we have a well-tested deployment
mechanism. We also don't need to wait for other soft-fork proposals to
catch up - starting the CLTV deployment process isn't going to delay
future soft-forks, or for that matter, hard-forks.

I think it's possible to safely get CLTV live on mainnet before the end
of the year. It's time we get this over with and done.


Detailed Rational
-----------------

1) There is a clear need for CLTV

Escrow and payment channels both benefit greatly from CLTV. In
particular, payment channel implementations are made significantly
simpler with CLTV, as well as more secure by removing the malleability
vulnerability.

Why are payment channels important? There's a lot of BTC out there
vulnerable to theft that doesn't have to be. For example, just the other
day I was talking with Nick Sullivan about ChangeTip's vulnerability to
theft, as well as regulatory uncertainty about whether or not they're a
custodian of their users' funds. With payment channels ChangeTip would
only be able to spend as much of a deposit as a user had spent, keeping
the rest safe from theft. Similarly, in the other direction - ChangeTip
to their users - in many cases it is feasible to also use payment
channels to immediately give users control of their funds as they
receive them, again protecting users and helping make the case that
they're not a custodian. In the future I'm sure we'll see fancy
bi-directional payment channels serving this role, but lets not let
perfect be the enemy of good.


2) We have consensus on the semantics of the CLTV opcode

Pull-req #6124 - the implementation of the opcode itself - was merged
nearly three months ago after significant peer review and discussion.
Part of that review process included myself(1) and mruddy(2) writing
actual demos of CLTV. The chance of the CLTV semantics changing now is
near-zero.


3) We have consensus that Bitcoin should adopt CLTV

The broad peer review and discussion that got #6124 merged is a clear
sign that we expect CLTV to be eventually adopted. The question isn't if
CLTV should be added to the Bitcoin protocol, but rather when.


4) The CLTV opcode and IsSuperMajority() deployment code has been
   thoroughly tested and reviewed

The opcode implementation is very simple, yet got significant review,
and it has solid test coverage by a suite of tx-(in)valid.json tests.
The tests themselves have been reviewed by others, resulting in Esteban
Ordano's pull-req #6368 by Esteban Ordano which added a few more cases.

As for the deployment code, both the actual IsSuperMajority() deployment
code and associated unit-tests tests were copied nearly line-by-line
from the succesful BIP66. I did this deliberately to make all the peer
review and testing of the deployment mechanism used in BIP66 be equally
valid for CLTV.


5) We can safely deploy CLTV with IsSuperMajority()

We've done two soft-forks so far with the IsSuperMajority() mechanism,
BIP34 and BIP66. In both cases the IsSuperMajority() mechanism itself
worked flawlessly. As is well-known BIP66 in combination with a large %
of the hashing power running non-validating "SPV" mining operations did
lead to a temporary fork, however the root cause of this issue is
unavoidable and not unique to IsSuperMajority() soft-forks.

Pragmatically speaking, now that miners are well aware of the issue it
will be easy for them to avoid a repeat of that fork by simply adding
IsSuperMajority() rules to their "SPV" mining code. Equally turning off
SPV mining (temporarily) is perfectly feasable.


6) We have the necessary consensus to deploy CLTV via IsSuperMajority()

The various "nVersion bits" proposals - which I am a co-author of - have
the primary advantage of being able to cleanly deal with the case where
a soft-fork fails to get adopted. However, we do have broad consensus,
including across all sides of the blocksize debate, that CLTV should be
adopted. The risk of CLTV failing to get miner adoption, and thus
blocking other soft-forks, is very low.


7) Using IsSuperMajority() to deploy CLTV doesn't limit or delay other upgrades

It _is_ possible for multiple IsSuperMajority() soft-forks to coexist,
in the sense that if one soft-fork is "in flight" that doesn't prevent
another soft-fork from also being deployed simultaneously.

In particular, if we deploy CLTV via IsSuperMajority() that does _not_
impact the adoption schedule for other future soft-forks, including
soft-forks using a future nVersion bits deployment mechanism.

For instance, suppose we start deployment of CLTV right now with
nVersion=4 blocks. In three months we have 25% miner support, and start
deploying CHECKSEQUENCEVERIFY with nVersion=5 blocks. For miners
supporting only OP_CLTV, the nVersion=5 blocks still trigger OP_CLTV;
miners creating nVersion=5 blocks are simply stating that they support
both soft-forks. Equally, if in three months we finish a nVersion bits
proposal, those miners will be advertising nVersion=(1 << 29) blocks,
which also advertise OP_CLTV support.


8) BIP101 miners have not proved to be a problem for CLTV deployment

While there was concern that BIP101's use of nVersion would cause
issues with a IsSuperMajority() softfork, the % of blocks with BIP101
nVersion's never reached more than 1%, and currently is hovering at
around 0.1%

As Gavin Andresen has stated that he is happy to add CLTV to BIP101, and
thus Bitcoin XT, I believe we can expect those miners to safely support
CLTV well before soft-fork enforcement happens. Secondly, the 95%
enforcement threshold means we can tolerate a fairly high % of miners
running pre-CLTV BIP101 implementations without fatal effects in the
unlikely event that those miners don't upgrade.


9) Doing another IsSuperMajority() soft-fork doesn't "burn a bit"

This is a common myth! All nVersion bits proposals involve permanently
setting a high-order bit to 1, which results in nVersion >= all prior
IsSuperMajority() soft-forks. In short, we can do a nearly unlimited
number of IsSuperMajority() soft-forks without affecting future nVersion
bits soft-forks at all.


10) Waiting for nVersion bits and CHECKSEQUENCEVERIFY will significantly
    delay deployment of CLTV

It's been proposed multiple times that we wait until we can do a single
soft-fork with CSV using the nVersion bits mechanism.

nVersion bits doesn't even have an implementation yet, nor has solid
consensus been reached on the exact semantics of how nVersion bits
should work. The stateful nature of nVersion bits soft-forks requires a
significant amount of new code compared to IsSuperMajority() soft-forks,
which in turn will require a significant amount of testing. (again I'll
point out I'm a co-author to all the nVersion bits proposals)

CSV has an implementation, but there is still debate going on about what
the exact semantics of it should be. Getting the semantics right is
especially important as part of CSV includes changing the meaning of
nSequence, restricting future uses of that field. There have been many
proposals to use nSequence, e.g. for proof-of-stake blocksize voting,
and it has the unique capability of being a field that is both unused,
and signed by scriptSigs. We shouldn't take potentially restricting
future uses of it lightly.

CSV is also significantly more complex and invasive than CLTV in terms
of code changes. A large % of the mining power is running forks
of Bitcoin Core with custom changes - modifying these forks with new
features is a labor intensive and slow process.

If CLTV is ready now, why delay it - potentially for 6-12 months - for
other proposals to catch up? Equally if they do catch up, great! As
explained above an in-flight CLTV soft-fork won't delay future upgrades.


11) Even if CLTV is broken/obsoleted there is very little carrying cost
    to having it

Suppose we decide in two years that CLTV was botched and we need to fix
it. What's the "carrying cost" of having implemented CLTV in the first
place?

We'll have used up one of our ten soft-forkable NOPs, but if we ever
"run out" it's easy to use extension NOPs(3). Similarly, future script
improvements like OP_MAST - or even a hard-fork - can easily expand the
range of NOPs to the point where this is a non-issue.

If you don't use OP_CLTV in your scripts there is zero effect on your
transactions; we're not limiting future improvements to Bitcoin in any
way other than using up a NOP by implementing CLTV.


References
----------

1) https://github.com/petertodd/checklocktimeverify-demos
2) https://github.com/mruddy/bip65-demos
3) https://github.com/bitcoin/bitcoin/pull/5496#issuecomment-101293403
4) https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki

-- 
'peter'[:-1]@petertodd.org
000000000000000006a257845da185433cbde54a74be889b1c046a267dcf4ab2

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[jl2012]

+1 for deploying BIP65 immediately without further waiting. Agree with 
all Peter's points.

If BIP65 has to follow the 0.12 schedule, it will take almost 9 months 
from now to complete the softfork. I don't see any good reason to wait 
for that long. We have too much talk, too little action.

Some mining pools hinted that they may adopt BitcoinXT at the end of 
2015. If we could start deploying BIP65 earlier, they will have a 
patched version by the time they switch. Gavin has agreed to support 
BIP65 in XT.

By the way, is there any chance to backport it to 0.9? In the deployment 
of BIP66 some miners requested a backport to 0.9 and that's why we have 
0.9.5.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 576 bytes --]

On Sun, Sep 27, 2015 at 04:26:12PM -0400, jl2012@xbt.hk wrote:
> +1 for deploying BIP65 immediately without further waiting. Agree
> with all Peter's points.

Thanks!

> By the way, is there any chance to backport it to 0.9? In the
> deployment of BIP66 some miners requested a backport to 0.9 and
> that's why we have 0.9.5.

I certainly could, though there's good reasons to move to v0.10.x; I'd
want to first hear from miners as to why they're still on v0.9.x

-- 
'peter'[:-1]@petertodd.org
00000000000000000f8ed62397b82b10e56b9aea309fb18c37985d1405808c4f

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mark Friedenbach]

[-- Attachment #1: Type: text/plain, Size: 10999 bytes --]

Agree with all CLTV and nVersionBits points. We should deploy a lock-time
soft-fork ASAP, using the tried and true IsSuperMajoirty test.

However your information regarding BIPs 68 (sequence numbers), 112
(checksequenceverify) and 113 (median time past) is outdated. Debate
regarding semantics has been settled, and there are working implementations
ready for merge on github. See pull requests #6312, #6564, and #6566. I
don’t know what the hold up has been regarding further reviews and merging,
but it is ready.

If you believe there are reasons #6312, #6564, or #6566 should not be
merged, please speak up. Otherwise it appears there is consensus on these
changes. They are related, and there is no reason not to include them in
the soft-fork, delaying applications using these features by 6-12 months.

On Sun, Sep 27, 2015 at 11:50 AM, Peter Todd via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Summary
> -------
>
> It's time to deploy BIP65 CHECKLOCKTIMEVERIFY.
>
> I've backported the CLTV op-code and a IsSuperMajority() soft-fork to
> the v0.10 and v0.11 branches, pull-reqs #6706 and #6707 respectively. A
> pull-req for git HEAD for the soft-fork deployment has been open since
> June 28th, #6351 - the opcode implementation itself was merged two
> months ago.
>
> We should release a v0.10.3 and v0.11.1 with CLTV and get the ball
> rolling on miner adoption. We have consensus that we need CLTV, we have
> a well tested implementation, and we have a well-tested deployment
> mechanism. We also don't need to wait for other soft-fork proposals to
> catch up - starting the CLTV deployment process isn't going to delay
> future soft-forks, or for that matter, hard-forks.
>
> I think it's possible to safely get CLTV live on mainnet before the end
> of the year. It's time we get this over with and done.
>
>
> Detailed Rational
> -----------------
>
> 1) There is a clear need for CLTV
>
> Escrow and payment channels both benefit greatly from CLTV. In
> particular, payment channel implementations are made significantly
> simpler with CLTV, as well as more secure by removing the malleability
> vulnerability.
>
> Why are payment channels important? There's a lot of BTC out there
> vulnerable to theft that doesn't have to be. For example, just the other
> day I was talking with Nick Sullivan about ChangeTip's vulnerability to
> theft, as well as regulatory uncertainty about whether or not they're a
> custodian of their users' funds. With payment channels ChangeTip would
> only be able to spend as much of a deposit as a user had spent, keeping
> the rest safe from theft. Similarly, in the other direction - ChangeTip
> to their users - in many cases it is feasible to also use payment
> channels to immediately give users control of their funds as they
> receive them, again protecting users and helping make the case that
> they're not a custodian. In the future I'm sure we'll see fancy
> bi-directional payment channels serving this role, but lets not let
> perfect be the enemy of good.
>
>
> 2) We have consensus on the semantics of the CLTV opcode
>
> Pull-req #6124 - the implementation of the opcode itself - was merged
> nearly three months ago after significant peer review and discussion.
> Part of that review process included myself(1) and mruddy(2) writing
> actual demos of CLTV. The chance of the CLTV semantics changing now is
> near-zero.
>
>
> 3) We have consensus that Bitcoin should adopt CLTV
>
> The broad peer review and discussion that got #6124 merged is a clear
> sign that we expect CLTV to be eventually adopted. The question isn't if
> CLTV should be added to the Bitcoin protocol, but rather when.
>
>
> 4) The CLTV opcode and IsSuperMajority() deployment code has been
>    thoroughly tested and reviewed
>
> The opcode implementation is very simple, yet got significant review,
> and it has solid test coverage by a suite of tx-(in)valid.json tests.
> The tests themselves have been reviewed by others, resulting in Esteban
> Ordano's pull-req #6368 by Esteban Ordano which added a few more cases.
>
> As for the deployment code, both the actual IsSuperMajority() deployment
> code and associated unit-tests tests were copied nearly line-by-line
> from the succesful BIP66. I did this deliberately to make all the peer
> review and testing of the deployment mechanism used in BIP66 be equally
> valid for CLTV.
>
>
> 5) We can safely deploy CLTV with IsSuperMajority()
>
> We've done two soft-forks so far with the IsSuperMajority() mechanism,
> BIP34 and BIP66. In both cases the IsSuperMajority() mechanism itself
> worked flawlessly. As is well-known BIP66 in combination with a large %
> of the hashing power running non-validating "SPV" mining operations did
> lead to a temporary fork, however the root cause of this issue is
> unavoidable and not unique to IsSuperMajority() soft-forks.
>
> Pragmatically speaking, now that miners are well aware of the issue it
> will be easy for them to avoid a repeat of that fork by simply adding
> IsSuperMajority() rules to their "SPV" mining code. Equally turning off
> SPV mining (temporarily) is perfectly feasable.
>
>
> 6) We have the necessary consensus to deploy CLTV via IsSuperMajority()
>
> The various "nVersion bits" proposals - which I am a co-author of - have
> the primary advantage of being able to cleanly deal with the case where
> a soft-fork fails to get adopted. However, we do have broad consensus,
> including across all sides of the blocksize debate, that CLTV should be
> adopted. The risk of CLTV failing to get miner adoption, and thus
> blocking other soft-forks, is very low.
>
>
> 7) Using IsSuperMajority() to deploy CLTV doesn't limit or delay other
> upgrades
>
> It _is_ possible for multiple IsSuperMajority() soft-forks to coexist,
> in the sense that if one soft-fork is "in flight" that doesn't prevent
> another soft-fork from also being deployed simultaneously.
>
> In particular, if we deploy CLTV via IsSuperMajority() that does _not_
> impact the adoption schedule for other future soft-forks, including
> soft-forks using a future nVersion bits deployment mechanism.
>
> For instance, suppose we start deployment of CLTV right now with
> nVersion=4 blocks. In three months we have 25% miner support, and start
> deploying CHECKSEQUENCEVERIFY with nVersion=5 blocks. For miners
> supporting only OP_CLTV, the nVersion=5 blocks still trigger OP_CLTV;
> miners creating nVersion=5 blocks are simply stating that they support
> both soft-forks. Equally, if in three months we finish a nVersion bits
> proposal, those miners will be advertising nVersion=(1 << 29) blocks,
> which also advertise OP_CLTV support.
>
>
> 8) BIP101 miners have not proved to be a problem for CLTV deployment
>
> While there was concern that BIP101's use of nVersion would cause
> issues with a IsSuperMajority() softfork, the % of blocks with BIP101
> nVersion's never reached more than 1%, and currently is hovering at
> around 0.1%
>
> As Gavin Andresen has stated that he is happy to add CLTV to BIP101, and
> thus Bitcoin XT, I believe we can expect those miners to safely support
> CLTV well before soft-fork enforcement happens. Secondly, the 95%
> enforcement threshold means we can tolerate a fairly high % of miners
> running pre-CLTV BIP101 implementations without fatal effects in the
> unlikely event that those miners don't upgrade.
>
>
> 9) Doing another IsSuperMajority() soft-fork doesn't "burn a bit"
>
> This is a common myth! All nVersion bits proposals involve permanently
> setting a high-order bit to 1, which results in nVersion >= all prior
> IsSuperMajority() soft-forks. In short, we can do a nearly unlimited
> number of IsSuperMajority() soft-forks without affecting future nVersion
> bits soft-forks at all.
>
>
> 10) Waiting for nVersion bits and CHECKSEQUENCEVERIFY will significantly
>     delay deployment of CLTV
>
> It's been proposed multiple times that we wait until we can do a single
> soft-fork with CSV using the nVersion bits mechanism.
>
> nVersion bits doesn't even have an implementation yet, nor has solid
> consensus been reached on the exact semantics of how nVersion bits
> should work. The stateful nature of nVersion bits soft-forks requires a
> significant amount of new code compared to IsSuperMajority() soft-forks,
> which in turn will require a significant amount of testing. (again I'll
> point out I'm a co-author to all the nVersion bits proposals)
>
> CSV has an implementation, but there is still debate going on about what
> the exact semantics of it should be. Getting the semantics right is
> especially important as part of CSV includes changing the meaning of
> nSequence, restricting future uses of that field. There have been many
> proposals to use nSequence, e.g. for proof-of-stake blocksize voting,
> and it has the unique capability of being a field that is both unused,
> and signed by scriptSigs. We shouldn't take potentially restricting
> future uses of it lightly.
>
> CSV is also significantly more complex and invasive than CLTV in terms
> of code changes. A large % of the mining power is running forks
> of Bitcoin Core with custom changes - modifying these forks with new
> features is a labor intensive and slow process.
>
> If CLTV is ready now, why delay it - potentially for 6-12 months - for
> other proposals to catch up? Equally if they do catch up, great! As
> explained above an in-flight CLTV soft-fork won't delay future upgrades.
>
>
> 11) Even if CLTV is broken/obsoleted there is very little carrying cost
>     to having it
>
> Suppose we decide in two years that CLTV was botched and we need to fix
> it. What's the "carrying cost" of having implemented CLTV in the first
> place?
>
> We'll have used up one of our ten soft-forkable NOPs, but if we ever
> "run out" it's easy to use extension NOPs(3). Similarly, future script
> improvements like OP_MAST - or even a hard-fork - can easily expand the
> range of NOPs to the point where this is a non-issue.
>
> If you don't use OP_CLTV in your scripts there is zero effect on your
> transactions; we're not limiting future improvements to Bitcoin in any
> way other than using up a NOP by implementing CLTV.
>
>
> References
> ----------
>
> 1) https://github.com/petertodd/checklocktimeverify-demos
> 2) https://github.com/mruddy/bip65-demos
> 3) https://github.com/bitcoin/bitcoin/pull/5496#issuecomment-101293403
> 4) https://github.com/bitcoin/bips/blob/master/bip-0112.mediawiki
>
> --
> 'peter'[:-1]@petertodd.org
> 000000000000000006a257845da185433cbde54a74be889b1c046a267dcf4ab2
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 12728 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Btc Drak]

[-- Attachment #1: Type: text/plain, Size: 1360 bytes --]

On Sun, Sep 27, 2015 at 7:50 PM, Peter Todd via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> 10) Waiting for nVersion bits and CHECKSEQUENCEVERIFY will significantly
>     delay deployment of CLTV
>
> It's been proposed multiple times that we wait until we can do a single
> soft-fork with CSV using the nVersion bits mechanism.
>
> nVersion bits doesn't even have an implementation yet, nor has solid
> consensus been reached on the exact semantics of how nVersion bits
> should work.


Small correction, the suggestion is to aim to roll out CLTV+CSV together by
0.12 release, using IsSuperMajority() (or versionbits if it is ready by
then). If CSV is not ready by then, we'd just roll out CLTV.

However, the CSV related pull requests are ready for final review and if
that can happen soon I don't see why we wouldn't roll CLTV+CSV out together
before 0.12. A considerable amount of time, discussion and iterations have
occurred for the related PRs and I believe they are at the point of
consensus modulo final review before merging.

References:

Mempool-only sequence number constraint verification
https://github.com/bitcoin/bitcoin/pull/6312

Mempool-only CHECKSEQUENCEVERIFY
https://github.com/bitcoin/bitcoin/pull/6564

Mempool-only Median time-past as endpoint for lock-time calculations
https://github.com/bitcoin/bitcoin/pull/6566

[-- Attachment #2: Type: text/html, Size: 2397 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[s7r]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

+1
I was actually waiting for this. It makes 'smart contracts' simpler
and better from many points of view.

Pete I don't see anything about RCLTV in BIP65, was that a separate
BIP? Which one is it and are we also deploying it via
IsSuperMajority()? RCLTV in addition to CLTV would be trivial so maybe
we can take the pain just once?

On 9/27/2015 9:50 PM, Peter Todd via bitcoin-dev wrote:
> Summary -------
> 
> It's time to deploy BIP65 CHECKLOCKTIMEVERIFY.
> 
> I've backported the CLTV op-code and a IsSuperMajority() soft-fork
> to the v0.10 and v0.11 branches, pull-reqs #6706 and #6707
> respectively. A pull-req for git HEAD for the soft-fork deployment
> has been open since June 28th, #6351 - the opcode implementation
> itself was merged two months ago.
> 
> We should release a v0.10.3 and v0.11.1 with CLTV and get the ball 
> rolling on miner adoption. We have consensus that we need CLTV, we
> have a well tested implementation, and we have a well-tested
> deployment mechanism. We also don't need to wait for other
> soft-fork proposals to catch up - starting the CLTV deployment
> process isn't going to delay future soft-forks, or for that matter,
> hard-forks.
> 
> I think it's possible to safely get CLTV live on mainnet before the
> end of the year. It's time we get this over with and done.
> 
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBCAAGBQJWCRIeAAoJEIN/pSyBJlsRlqgH/iir3Ao99WMNV0xC5RL+fv/Q
J1az1dXif9w9sTaCZMkENyIH9B2kwmOcPX/pU+p75qNvhQi9OrNMNRE8Wlwa+tcL
DD9DbyiQvxKdXjCnZqUyyIgBjuFbiF5VNQ67B1faEnzvmX81PoDjd2FPC51WChjZ
j7xPcJ73d23OPXpsKtyaUwn1QbGwprhFCEkcqjC50gw/IQkMJiqZ6pMepDVSyGKl
RpOsWCyCVoTJtM5NFk7wXg5LBFA7rXXQL56M00YJKLJAx/ooGb2T4ZRX0GeEWX8/
wquNA9Bj7picIr20sPohGE0cr2QiD3gmL9qLT2ZDlrFFDk8thL8afOx00Z6ih3I=
=dPXd
-----END PGP SIGNATURE-----

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 548 bytes --]

There is *no* consensus on using a soft fork to deploy this feature. It
will result in the same problems as all the other soft forks - SPV wallets
will become less reliable during the rollout period. I am against that, as
it's entirely avoidable.

Make it a hard fork and my objection will be dropped.

Until then, as there is no consensus, you need to do one of two things:

1) Drop the "everyone must agree to make changes" idea that people here
like to peddle, and do it loudly, so everyone in the community is correctly
informed

2) Do nothing

[-- Attachment #2: Type: text/html, Size: 946 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Adam Back]

I wonder what Gavin's views are, he's usually constructive, and see if
he'll include it in XT - I think he may have said he was supportive.

The rationale for soft vs hard-forks is well known, so I wont go over them.

Adam


On 28 September 2015 at 06:48, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> There is no consensus on using a soft fork to deploy this feature. It will
> result in the same problems as all the other soft forks - SPV wallets will
> become less reliable during the rollout period. I am against that, as it's
> entirely avoidable.
>
> Make it a hard fork and my objection will be dropped.
>
> Until then, as there is no consensus, you need to do one of two things:
>
> 1) Drop the "everyone must agree to make changes" idea that people here like
> to peddle, and do it loudly, so everyone in the community is correctly
> informed
>
> 2) Do nothing
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 571 bytes --]

>
> The rationale for soft vs hard-forks is well known, so I wont go over them.
>

The rationale of "backwards compatibility" is well known, yet wrong. I've
gone over the arguments here and explained why the concept makes no sense:

https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7

Eric - no, it's not sophisticated humour. I've been objecting to soft forks
since this idea first appeared.

There is no consensus. Now pick. Lose the requirement that everyone agree
for consensus changes, and tell people you've done it. Change the spec. Or
do nothing.

[-- Attachment #2: Type: text/html, Size: 1036 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Eric Lombrozo]

[-- Attachment #1: Type: text/plain, Size: 2439 bytes --]

Perhaps Adam won't go into the rationale...but I think it is important we clarify this.

For better or worse, the only "voting" system available to Bitcoin that cannot be trivially attacked is hashing power. Soft forks are essentially miner-enforced rule changes...rules they could have decided to enforce without the consensus of anyone else. For instance, as far as old nodes are concerned, a p2sh output can be redeemed by a simple preimage of the hash...with no signatures. The point, however, is that as long as the majority of hashpower enforces the new rule, such attempts to redeem the output will never end up on the blockchain. Therefore, transactions that attempt to redeem the output with a simple preimage are as good as invalid...and effectively have become invalid.

I concede that this mechanism has some issues. Moreover, I agree that it is important that the Bitcoin community be aware of these things. I've been proposing making these rule changes explicit in the BIPs (https://github.com/CodeShark/bips/blob/BIP_Classification/bip-layers.mediawiki). I believe it is important that people weigh in on such rule changes. However, the above stated mechanism does not fall under the definition of "hard fork" we've come to accept.

Go ahead and object to soft forks...but at least try not to make arguments based on changing the definitions of terms we all generally agree upon.

- Eric

On September 28, 2015 4:40:35 AM PDT, Mike Hearn via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> The rationale for soft vs hard-forks is well known, so I wont go over
>them.
>>
>
>The rationale of "backwards compatibility" is well known, yet wrong.
>I've
>gone over the arguments here and explained why the concept makes no
>sense:
>
>https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7
>
>Eric - no, it's not sophisticated humour. I've been objecting to soft
>forks
>since this idea first appeared.
>
>There is no consensus. Now pick. Lose the requirement that everyone
>agree
>for consensus changes, and tell people you've done it. Change the spec.
>Or
>do nothing.
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 3367 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 989 bytes --]

>
> Go ahead and object to soft forks...but at least try not to make arguments
> based on changing the definitions of terms we all generally agree upon.
>

I don't intend to do that, and I don't think I am - I know what the
difference between a soft and hard fork is and am not trying to confuse or
blur the two.

To reiterate: this current BIP implements a soft fork. I am not debating
that. I am saying it should use a hard fork instead. This will ensure no
repeat of the P2SH case where invalid blocks were being found for weeks (or
was it months?) after the new rules kicked in, thus exposing SPV wallets
and old nodes to unnecessary risk for no benefit.

Additionally, I am making it clear that there's no consensus for rolling
out the new opcode in this way. As you say, the mechanism has issues. If
you read the comments when I wrote my article, you can see that others
share the same concerns:

https://www.reddit.com/r/Bitcoin/comments/3griiv/on_consensus_and_forks_by_mike_hearn

[-- Attachment #2: Type: text/html, Size: 1494 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Eric Lombrozo]

[-- Attachment #1: Type: text/plain, Size: 1630 bytes --]

SPV wallets in their current form are inherently insecure. Moreover, while we at least have a soft fork mechanism that is not trivially exploitable (yes, it's got issues...but unlike SPV wallets, it isn't so easily exploitable), we have NO hard fork mechanism in place that isn't highly prone to systemic consensus failure.

But I think pretty much anyone who hasn't been in a coma for the last several years knows this...and I'll stop repeating the obvious.

On September 28, 2015 5:26:17 AM PDT, Mike Hearn <hearn@vinumeris.com> wrote:
>>
>> Go ahead and object to soft forks...but at least try not to make
>arguments
>> based on changing the definitions of terms we all generally agree
>upon.
>>
>
>I don't intend to do that, and I don't think I am - I know what the
>difference between a soft and hard fork is and am not trying to confuse
>or
>blur the two.
>
>To reiterate: this current BIP implements a soft fork. I am not
>debating
>that. I am saying it should use a hard fork instead. This will ensure
>no
>repeat of the P2SH case where invalid blocks were being found for weeks
>(or
>was it months?) after the new rules kicked in, thus exposing SPV
>wallets
>and old nodes to unnecessary risk for no benefit.
>
>Additionally, I am making it clear that there's no consensus for
>rolling
>out the new opcode in this way. As you say, the mechanism has issues.
>If
>you read the comments when I wrote my article, you can see that others
>share the same concerns:
>
>https://www.reddit.com/r/Bitcoin/comments/3griiv/on_consensus_and_forks_by_mike_hearn

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 2380 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

>
> we have NO hard fork mechanism in place that isn't highly prone to
> systemic consensus failure.
>

Just use an opcode that isn't currently defined. Done. What about that
mechanism is prone to failure?

Re: coma. No need for insults. Please read my article and address the
points raised there, which, by the way, do not include any mention of SPV
wallets. Although your belief that SPV wallets are "inherently insecure"
seems needlessly trollish - I certainly would disagree, but it's a
different debate.

[-- Attachment #2: Type: text/html, Size: 783 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Eric Lombrozo]

[-- Attachment #1: Type: text/plain, Size: 1210 bytes --]

Mike,

Insults were not really my intention. Let's set aside our differences regarding SPV security and assume you understand the different implications for soft forks and hard forks.

Other than the fact that doing this as a soft fork requires an extra OP_DROP, how would doing this as a hard fork make any difference to SPV clients? If, as others have suggested, all clients warn the user on unrecognized nVersion and make unknown noops nonstandard, would this satisfy your concerns? The logic seems pretty straightforward.

- Eric

On September 28, 2015 5:54:33 AM PDT, Mike Hearn <hearn@vinumeris.com> wrote:
>>
>> we have NO hard fork mechanism in place that isn't highly prone to
>> systemic consensus failure.
>>
>
>Just use an opcode that isn't currently defined. Done. What about that
>mechanism is prone to failure?
>
>Re: coma. No need for insults. Please read my article and address the
>points raised there, which, by the way, do not include any mention of
>SPV
>wallets. Although your belief that SPV wallets are "inherently
>insecure"
>seems needlessly trollish - I certainly would disagree, but it's a
>different debate.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 1732 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1523 bytes --]

>
> Other than the fact that doing this as a soft fork requires an extra
> OP_DROP, how would doing this as a hard fork make any difference to SPV
> clients? If, as others have suggested, all clients warn the user on
> unrecognized nVersion
>

All clients do *not* do this. Why would they? What action would they take?
Try and simulate a hard fork in some complicated roundabout manner? Why not
just do the real thing and keep things simple?


> and make unknown noops nonstandard
>

They are already non-standard. That change was made last time I brought up
the problems with soft forks. It brought soft forks that use OP_NOPs a bit
closer to the ideal of a hard fork, but didn't go all the way. I pointed
that out above in my reply to Peter's mail.

So to answer your question, no, it wouldn't satisfy my concerns. My logic
is this:

Hard forks - simple, well understood, SPV friendly, old full nodes do not
calculate incorrect ledgers whilst telling their users (via UI, RPC) that
they are fully synced. Emphasis on simple: simple is good.

Soft forks - to get the benefits of a hard fork back requires lots of extra
code, silently makes IsStandard() effectively a part of the consensus rules
when in the past it hasn't been, SPV unfriendly. Benefits? As far as I can
tell, there are none.

If someone could elucidate *what* the benefits actually are, that would be
a good next step. So far everyone who tried to answer this question gave a
circular answer of the form "soft forks are good because they are soft
forks".

[-- Attachment #2: Type: text/html, Size: 2035 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Btc Drak]

[-- Attachment #1: Type: text/plain, Size: 2061 bytes --]

On Mon, Sep 28, 2015 at 12:40 PM, Mike Hearn via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> There is no consensus. Now pick. Lose the requirement that everyone agree
> for consensus changes, and tell people you've done it. Change the spec. Or
> do nothing.
>

Of course there is good technical consensus for CLTV by IsSuperMajority()
in the same way as BIP66 was rolled out. I believe the only open question
is whether we have to account for XT's use of versionbits (because the
standard has not been finalised). One can take the view that it is a non
issue given the almost negligible number of BIP101 blocks, but it certainly
goes away if XT also merges BIP65/CLTV.

As for risks, I think we learned a lot from BIP66:

1. miners are now aware of the risks of SPV mining near activation and are
financially incentivised not to during that period.
2. As for SPV wallets need to handle awareness of the new blocks. BitcoinJ
can play a pivotal role: as far as I am aware if we'd thought about adding
handling to BitcoinJ before activation rather than after activation[1][2],
the SPV issues would have been mitigated for the vast majority who rely on
the library. To me, this particular issue highlights our collective failure
to communicate the necessity for additional SPV handling requirements and
other preparation the ecosystem should engage in during a soft fork. This
is something we should definitely add to the release notes for the next
soft fork and advertise widely. Certainly it MUST be well documented in the
BIP65 deployment section, which it is currently not.

Lastly your objections came across very strongly (at least to my
understanding) so I am curious: Peter stated Gavin is OK with adding CLTV
support to XT, and assuming that is the case, will you object to merging it
or similarly object to adding the necessary block handling to BitcoinJ?

[1]
https://github.com/bitcoinj/bitcoinj/commit/6f03669fbd6c368961a25dfd772751d1ca2a1b5b
[2]
https://github.com/bitcoinj/bitcoinj/commit/d3d11df6d71ff11cef2dc0caa8263daa641fe118

[-- Attachment #2: Type: text/html, Size: 2880 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1076 bytes --]

>
> 2. As for SPV wallets need to handle awareness of the new blocks.
>

There is simply no need for any wallets to change. Making the spec a hard
fork instead of a soft fork means all existing software does the right
thing automatically.

To repeat, please bear in mind that bitcoinj is no longer the only SPV
wallet implementation. BreadWallet has its own code in Objective-C and is
the second most popular SPV implementation (and growing). Additionally,
bitcoinj is incorporated into lots of apps that'd have to have new versions
released, some of which don't have any way to force a user to update.

So it's not just my time you'd waste: it's lots of different people's.

One thing I haven't seen yet is the justification for why a soft fork
should be used here. There's no requirement that it be so, and there are
real downsides. As Eric said, the fact that the mechanism has issues is not
under dispute.

The normal justification for this it's that it's forwards compatible. But
that's not a justification, that's a description.

Re: XT, I already addressed this above.

[-- Attachment #2: Type: text/html, Size: 1546 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[odinn]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

And still no movement on BIP 63...

https://bitcointalk.org/index.php?topic=1083961.20

Apart from that,

All my prior objections to XT still hold as expressed on this list.
XT is not acceptable.

On the topic of consensus:

Reaching consensus, I hope, is something that developers can
accomplish by refining and adjusting the BIPS and coming to agreement
upon them.  This should be something that can be done in a few months
time, before the end of the year.

Cheers,

- - O

Adam Back via bitcoin-dev:
> I wonder what Gavin's views are, he's usually constructive, and see
> if he'll include it in XT - I think he may have said he was
> supportive.
> 
> The rationale for soft vs hard-forks is well known, so I wont go
> over them.
> 
> Adam
> 
> 
> On 28 September 2015 at 06:48, Mike Hearn via bitcoin-dev 
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>> There is no consensus on using a soft fork to deploy this
>> feature. It will result in the same problems as all the other
>> soft forks - SPV wallets will become less reliable during the
>> rollout period. I am against that, as it's entirely avoidable.
>> 
>> Make it a hard fork and my objection will be dropped.
>> 
>> Until then, as there is no consensus, you need to do one of two
>> things:
>> 
>> 1) Drop the "everyone must agree to make changes" idea that
>> people here like to peddle, and do it loudly, so everyone in the
>> community is correctly informed
>> 
>> 2) Do nothing
>> 
>> 
>> 
>> _______________________________________________ bitcoin-dev
>> mailing list bitcoin-dev@lists.linuxfoundation.org 
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>> 
> _______________________________________________ bitcoin-dev mailing
> list bitcoin-dev@lists.linuxfoundation.org 
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> 

- -- 
http://abis.io ~
"a protocol concept to enable decentralization
and expansion of a giving economy, and a new social good"
https://keybase.io/odinn
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJWCa0jAAoJEGxwq/inSG8CuCUIALiRt6cE3b+9f+l9m6aMTjIR
vTEIM/7B4dIZW9eatXmkxyd44uz5YoN93SlZtV62c90HCqqpFRBCfyXRyXzQ11E7
0i70or5LnWDOqrD1bSsCEdrQxPIpAQnv101UHe3iyn/uHAVBiz/HfqvGMruNt0r1
4sMecp+LedWpy6/p9c6iMHV1rhtYRfmRfJHj+9KlSn+in5PQKx2kieWqpfqjmlNs
J/UNoLvRuF0YxDcqEdp2BAaI0s+NyXBo3YDi4R77U9YPRj/cYuWHh/yPKAvFW+2K
0d9NNuKSKEY/m4uW3ghPEJL7OxlGbOoNWFS3kcKYr+BanfsPTov7yHQhBuRBRPw=
=hd0W
-----END PGP SIGNATURE-----

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Dave Scotese]

[-- Attachment #1: Type: text/plain, Size: 3121 bytes --]

Why are they called soft forks when they are really hidden forks?  Isn't
the point of a soft fork to prevent old clients from rejecting what they
don't have the code to validate?  That seems dangerous.

notplato

On Mon, Sep 28, 2015 at 2:12 PM, odinn via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> And still no movement on BIP 63...
>
> https://bitcointalk.org/index.php?topic=1083961.20
>
> Apart from that,
>
> All my prior objections to XT still hold as expressed on this list.
> XT is not acceptable.
>
> On the topic of consensus:
>
> Reaching consensus, I hope, is something that developers can
> accomplish by refining and adjusting the BIPS and coming to agreement
> upon them.  This should be something that can be done in a few months
> time, before the end of the year.
>
> Cheers,
>
> - - O
>
> Adam Back via bitcoin-dev:
> > I wonder what Gavin's views are, he's usually constructive, and see
> > if he'll include it in XT - I think he may have said he was
> > supportive.
> >
> > The rationale for soft vs hard-forks is well known, so I wont go
> > over them.
> >
> > Adam
> >
> >
> > On 28 September 2015 at 06:48, Mike Hearn via bitcoin-dev
> > <bitcoin-dev@lists.linuxfoundation.org> wrote:
> >> There is no consensus on using a soft fork to deploy this
> >> feature. It will result in the same problems as all the other
> >> soft forks - SPV wallets will become less reliable during the
> >> rollout period. I am against that, as it's entirely avoidable.
> >>
> >> Make it a hard fork and my objection will be dropped.
> >>
> >> Until then, as there is no consensus, you need to do one of two
> >> things:
> >>
> >> 1) Drop the "everyone must agree to make changes" idea that
> >> people here like to peddle, and do it loudly, so everyone in the
> >> community is correctly informed
> >>
> >> 2) Do nothing
> >>
> >>
> >>
> >> _______________________________________________ bitcoin-dev
> >> mailing list bitcoin-dev@lists.linuxfoundation.org
> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >>
> > _______________________________________________ bitcoin-dev mailing
> > list bitcoin-dev@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> >
>
> - --
> http://abis.io ~
> "a protocol concept to enable decentralization
> and expansion of a giving economy, and a new social good"
> https://keybase.io/odinn
> -----BEGIN PGP SIGNATURE-----
>
> iQEcBAEBCgAGBQJWCa0jAAoJEGxwq/inSG8CuCUIALiRt6cE3b+9f+l9m6aMTjIR
> vTEIM/7B4dIZW9eatXmkxyd44uz5YoN93SlZtV62c90HCqqpFRBCfyXRyXzQ11E7
> 0i70or5LnWDOqrD1bSsCEdrQxPIpAQnv101UHe3iyn/uHAVBiz/HfqvGMruNt0r1
> 4sMecp+LedWpy6/p9c6iMHV1rhtYRfmRfJHj+9KlSn+in5PQKx2kieWqpfqjmlNs
> J/UNoLvRuF0YxDcqEdp2BAaI0s+NyXBo3YDi4R77U9YPRj/cYuWHh/yPKAvFW+2K
> 0d9NNuKSKEY/m4uW3ghPEJL7OxlGbOoNWFS3kcKYr+BanfsPTov7yHQhBuRBRPw=
> =hd0W
> -----END PGP SIGNATURE-----
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 4842 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Eric Lombrozo]

[-- Attachment #1: Type: text/plain, Size: 1109 bytes --]

My initial reaction is just HUH?!?!? Is this some sophisticated form of humor I'm just not getting?

On September 28, 2015 3:48:57 AM PDT, Mike Hearn via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>There is *no* consensus on using a soft fork to deploy this feature. It
>will result in the same problems as all the other soft forks - SPV
>wallets
>will become less reliable during the rollout period. I am against that,
>as
>it's entirely avoidable.
>
>Make it a hard fork and my objection will be dropped.
>
>Until then, as there is no consensus, you need to do one of two things:
>
>1) Drop the "everyone must agree to make changes" idea that people here
>like to peddle, and do it loudly, so everyone in the community is
>correctly
>informed
>
>2) Do nothing
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 1819 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Tier Nolan]

[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]

On Mon, Sep 28, 2015 at 11:48 AM, Mike Hearn via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> 1) Drop the "everyone must agree to make changes" idea that people here
> like to peddle, and do it loudly, so everyone in the community is correctly
> informed
>

There never was a rule that soft-forks require total consensus.  It is
desirable but not mandatory.

A majority of miners can inherently implement a soft fork against the
wishes of the rest of the users.

Merchant/exchange/user checkpointing is the defense and therefore is a
perfectly valid response to miners taking such an action.  If a soft fork
is opposed by a large section of the users, then threatening (and
implementing) a checkpoint is the correct response.

No group can force through a hard fork, it inherently requires buy-in from
a large portion of the userbase.  That is where the "total consensus"
requirement comes from.  Naturally, absolute total consensus isn't actually
required but you do need very large consensus and also consensus across the
various sub-groups.

[-- Attachment #2: Type: text/html, Size: 1529 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 1569 bytes --]

I think three things need to happen:

1) Stop pretending that "everyone must agree to make consensus rule
changes." "Rough consensus" is what we've always gone with, and is good
enough.

2) Mr. Todd (or somebody) needs to write up a risk/benefit security
tradeoff analysis doo-hickey document and publish it. I'm reasonably
confident that the risks to SPV nodes can be mitigated (e.g. by deploying
mempool-only first, before the soft fork rolls out), but as somebody who
has only been moderately paying attention, BETTER COMMUNICATION is needed.
What should SPV wallet authors be doing right now, if anything? Once the
soft fork starts to roll out or activates, what do miners need to be aware
of? SPV wallet authors?

3) I agree CLTV is ready to roll out, that there is rough consensus a soft
fork is a reasonable way to do it, and that it should happen ASAP.

On Mon, Sep 28, 2015 at 6:48 AM, Mike Hearn via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> There is *no* consensus on using a soft fork to deploy this feature. It
> will result in the same problems as all the other soft forks - SPV wallets
> will become less reliable during the rollout period. I am against that, as
> it's entirely avoidable.
>
> Make it a hard fork and my objection will be dropped.
>
> Until then, as there is no consensus, you need to do one of two things:
>
> 1) Drop the "everyone must agree to make changes" idea that people here
> like to peddle, and do it loudly, so everyone in the community is correctly
> informed
>
> 2) Do nothing
>
>
-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 2381 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1359 bytes --]

On Mon, Sep 28, 2015 at 09:01:02AM -0400, Gavin Andresen wrote:
> I think three things need to happen:
> 
> 1) Stop pretending that "everyone must agree to make consensus rule
> changes." "Rough consensus" is what we've always gone with, and is good
> enough.
> 
> 2) Mr. Todd (or somebody) needs to write up a risk/benefit security
> tradeoff analysis doo-hickey document and publish it. I'm reasonably
> confident that the risks to SPV nodes can be mitigated (e.g. by deploying
> mempool-only first, before the soft fork rolls out), but as somebody who
> has only been moderately paying attention, BETTER COMMUNICATION is needed.
> What should SPV wallet authors be doing right now, if anything? Once the
> soft fork starts to roll out or activates, what do miners need to be aware
> of? SPV wallet authors?

Do you have such a document for your BIP101? That would save me a lot of
time, and the need for that kind of document is significantly higher
with BIP101 anyway.


Re: mempool, CLTV-using transactions are non-standard and are not
relayed in all Bitcoin Core releases. (see my pull-req #5000 -
Discourage NOPs reserved for soft-fork upgrades - for why) I believe
that meets your suggestion of deploying mempool-only first.

-- 
'peter'[:-1]@petertodd.org
0000000000000000032420ad2987adc954df855f9ae10cf608e911b431f640e0

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 1524 bytes --]

On Mon, Sep 28, 2015 at 9:28 AM, Peter Todd <pete@petertodd.org> wrote:

> > 2) Mr. Todd (or somebody) needs to write up a risk/benefit security
> > tradeoff analysis doo-hickey document and publish it. I'm reasonably
> > confident that the risks to SPV nodes can be mitigated (e.g. by deploying
> > mempool-only first, before the soft fork rolls out), but as somebody who
> > has only been moderately paying attention, BETTER COMMUNICATION is
> needed.
> > What should SPV wallet authors be doing right now, if anything? Once the
> > soft fork starts to roll out or activates, what do miners need to be
> aware
> > of? SPV wallet authors?
>
> Do you have such a document for your BIP101? That would save me a lot of
> time, and the need for that kind of document is significantly higher
> with BIP101 anyway.
>

Hmmm?  When I asked YOU for that kind of security analysis document, you
said you'd see if any of your clients would be willing to let you publish
one you'd done in the past. Then I never heard back from you.

So, no, I don't have one for BIP 101, but unless you were lying and just
trying to add Yet Another Hoop for BIP 101 to jump through, you should
already have something to start from.

RE: mempool only: yes, pull-req 5000 satisfies (and that's what I was
thinking of). There should be a nice, readable blog post explaining to
other full node implementors and wallet implementors why that was done for
Core and what they should do to follow 'best practices to be soft-fork
ready.'

-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 2078 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2590 bytes --]

On Mon, Sep 28, 2015 at 09:43:42AM -0400, Gavin Andresen wrote:
> On Mon, Sep 28, 2015 at 9:28 AM, Peter Todd <pete@petertodd.org> wrote:
> 
> > > 2) Mr. Todd (or somebody) needs to write up a risk/benefit security
> > > tradeoff analysis doo-hickey document and publish it. I'm reasonably
> > > confident that the risks to SPV nodes can be mitigated (e.g. by deploying
> > > mempool-only first, before the soft fork rolls out), but as somebody who
> > > has only been moderately paying attention, BETTER COMMUNICATION is
> > needed.
> > > What should SPV wallet authors be doing right now, if anything? Once the
> > > soft fork starts to roll out or activates, what do miners need to be
> > aware
> > > of? SPV wallet authors?
> >
> > Do you have such a document for your BIP101? That would save me a lot of
> > time, and the need for that kind of document is significantly higher
> > with BIP101 anyway.
> >
> 
> Hmmm?  When I asked YOU for that kind of security analysis document, you
> said you'd see if any of your clients would be willing to let you publish
> one you'd done in the past. Then I never heard back from you.

I don't remember what you are referring to at all. Was this a private
email? IRC chat? In person discussion?

> So, no, I don't have one for BIP 101, but unless you were lying and just
> trying to add Yet Another Hoop for BIP 101 to jump through, you should
> already have something to start from.

"unless you were lying"

Please keep the discussion on the development mailing list civil and
respectful.

> RE: mempool only: yes, pull-req 5000 satisfies (and that's what I was
> thinking of). There should be a nice, readable blog post explaining to
> other full node implementors and wallet implementors why that was done for
> Core and what they should do to follow 'best practices to be soft-fork
> ready.'

Actually, that sounds like the kind of thing that should be in the
bitcoin.org developer documentation; IMO for the audience of competent
full node developers the comments in the pull-req code itself and
associated discussion covers everything they need to know. Without that
background though, this is something that'd fit well in the category of
general education to get new developers to a good state of competence.

As for wallets specifically, that's pretty much all covered by SPV
wallets based on bitcoinj, and Mike Hearn has different views on the
subject which need to be resolved first.

-- 
'peter'[:-1]@petertodd.org
0000000000000000102f6eb0772c453a0ad0e10a6f720f41a7f008a7d329ef66

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 4538 bytes --]

On Mon, Sep 28, 2015 at 12:48:57PM +0200, Mike Hearn wrote:
> There is *no* consensus on using a soft fork to deploy this feature. It
> will result in the same problems as all the other soft forks - SPV wallets
> will become less reliable during the rollout period. I am against that, as
> it's entirely avoidable.
> 
> Make it a hard fork and my objection will be dropped.
> 
> Until then, as there is no consensus, you need to do one of two things:
> 
> 1) Drop the "everyone must agree to make changes" idea that people here
> like to peddle, and do it loudly, so everyone in the community is correctly
> informed
> 
> 2) Do nothing

Hmm? You didn't quote any of my email, so I'll remind you what I did say
we had consensus about:

    2) We have consensus on the semantics of the CLTV opcode

and

    3) We have consensus that Bitcoin should adopt CLTV

    The broad peer review and discussion that got #6124 merged is a clear
    sign that we expect CLTV to be eventually adopted.  __The question isn't
    if CLTV should be added to the Bitcoin protocol, but rather when.__

(emphasis mine)

Both those statements of consensus are *not* about how CLTV is to be
deployed. I did discuss deployment later:

    6) We have the __necessary consensus__ to deploy CLTV via IsSuperMajority()

    The various "nVersion bits" proposals - which I am a co-author of - have
    the primary advantage of being able to cleanly deal with the case where
    a soft-fork fails to get adopted. However, we do have broad consensus,
    including across all sides of the blocksize debate, that CLTV should be
    adopted. __The risk of CLTV failing to get miner adoption, and thus
    blocking other soft-forks, is very low.__

I probably could have worded this section a bit more clearly; when I say
"necessary consensus" I'm referring to the consensus required for a
soft-fork deployment. At minimum a simple majority of hashing power -
your approval isn't required.

For a safe soft-fork, we'd like a super majority of miners to be on
board. For a IsSuperMajority() soft-fork - as opposed to nVersion bits -
we also need the probability of the soft-fork being rejected to be very
low. To achieve that, having consensus that CLTV is a good idea is the
best situation to be in. But that's not to say that a few dissenting
voices should be seen as a blocker to progress - rather is just makes
the deployment a bit more risky, being a sign that the consensus may
change in the future, with the soft-fork being later rejected. For
example strong objections by a respected Bitcoin developer who has made
significant contributions to the consensus codebase and protocol
development would be a strong sign that a IsSuperMajority() soft-fork
might fail, and deployment via nVersion bits is probably a better
approach. Fortunately we're not in that situation.

Hard-forks are a very different situation, with significantly more need
for very broad consensus, but that's been well discussed elsewhere.


I have three questions to you:

1) Do you agree that CLTV should be added to the Bitcoin protocol?

Ignoring the question how exactly it is added, hard-fork or soft-fork.


2) Will you add a IsSuperMajority() CLTV soft-fork to Bitcoin XT if it
   is added to Bitcoin Core?

If you refuse to do this the risk of the soft-fork is increased a bit,
although miner support for XT has remained extremely low, and the 95%
switch-over threshold has a significant margin for error. (there's a 75%
threshold to consider as well, however as XT has adopted my pull-req
#5000 - Discourage NOPs reserved for soft-fork upgrades - those miners
will only produce valid blocks under CLTV rules)


3) Will you add soft-fork detection to bitcoinj, to allow SPV clients to
   detect advertised soft-forks and correctly handle them?

Notably, if you do this your objections against soft-forks will be met,
as the behavior of a SPV client with soft-fork detection during a
soft-fork will be identical to that client during a hard-fork. In
particular, the SPV client will correct reject invalid blocks, and
continue to follow only the longest valid chain. (modulo unadvertised
forks of course, an inherently unavoidable problem with the SPV security
model) Secondly, that code should also detect forks it doesn't know
about - as is done in Bitcoin Core already - and warn the user.

-- 
'peter'[:-1]@petertodd.org
00000000000000000d74f5def1087f3ec1571cb468e471e71f96063253988c78

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1317 bytes --]

>
> 1) Do you agree that CLTV should be added to the Bitcoin protocol?
>
> Ignoring the question how exactly it is added, hard-fork or soft-fork.
>

The opcode definition seems OK.


> 2) Will you add a IsSuperMajority() CLTV soft-fork to Bitcoin XT if it
>    is added to Bitcoin Core?
>

Yes. It might be worth putting the version bit change behind a command line
flag though: the BIP, as written, has problems (with deployment).


> 3) Will you add soft-fork detection to bitcoinj, to allow SPV clients to

   detect advertised soft-forks and correctly handle them?
>

I'd really hate to do that. It'd be a Rube Goldberg machine:

   https://krypt3ia.files.wordpress.com/2011/11/rube.jpg

There's no really good way to do what you propose, and we already have a
perfectly workable mechanism to tell SPV clients about chain forks: the
block chain itself. This has the advantage of being already implemented,
already deployed, and it works correctly.

Attempting to strap a different mechanism on top to try and make soft forks
more like hard forks would be a large and pointless waste of people's time
and effort, not just mine (bitcoinj is not the only widely used SPV
implementation nowadays). You may as well go straight to the correct
outcome instead of trying to simulate it with ever more complex mechanisms.

[-- Attachment #2: Type: text/html, Size: 2371 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 3007 bytes --]

On Mon, Sep 28, 2015 at 03:41:56PM +0200, Mike Hearn wrote:
> >
> > 1) Do you agree that CLTV should be added to the Bitcoin protocol?
> >
> > Ignoring the question how exactly it is added, hard-fork or soft-fork.
> >
> 
> The opcode definition seems OK.

Good!

> > 2) Will you add a IsSuperMajority() CLTV soft-fork to Bitcoin XT if it
> >    is added to Bitcoin Core?
> >
> 
> Yes. It might be worth putting the version bit change behind a command line
> flag though: the BIP, as written, has problems (with deployment).

Could you elaborate on what exatly you mean by this.

> > 3) Will you add soft-fork detection to bitcoinj, to allow SPV clients to
> 
>    detect advertised soft-forks and correctly handle them?
> >
> 
> I'd really hate to do that. It'd be a Rube Goldberg machine:
> 
>    https://krypt3ia.files.wordpress.com/2011/11/rube.jpg
> 
> There's no really good way to do what you propose, and we already have a
> perfectly workable mechanism to tell SPV clients about chain forks: the
> block chain itself. This has the advantage of being already implemented,
> already deployed, and it works correctly.

SPV wallets can't detect hard-forks, so in both cases you will have
invalid blocks be accepted by SPV clients; there's no deployment
scenario for either hard or soft forks that guarantees all miners adopt
a fork.

What does prevent invalid blocks being accepted by SPV clients is
checking the block nVersion field and applying forking logic. Of course,
that only works for advertised forks, but again, that's equally true for
soft and hard forks.

> Attempting to strap a different mechanism on top to try and make soft forks
> more like hard forks would be a large and pointless waste of people's time
> and effort, not just mine (bitcoinj is not the only widely used SPV
> implementation nowadays). You may as well go straight to the correct
> outcome instead of trying to simulate it with ever more complex mechanisms.

Again, in neither case do you get the "correct outcome" of SPV clients
accepting no invalid blocks without nVersion field checking.

However, in the hard-fork case, because the non-adopting miners reject
the fork, they build a chain which could be used to attack SPV clients
with false confirmations by sybil attacking those clients. In the
soft-fork case, the non-adopting miners keep accepting the longer chain
built by the adopting miners, preventing the creation of a chain that
could be used to attack SPV miners.


BTW, what's the other widely used SPV implementation you're thinking of?
I'll contact them directly and help them implement proper SPV fork
protections if they haven't already; if bitcoinj is unwilling to do this
at least we could have an alternative implementation that does.
(equally, if anyone wants to fork bitcoinj and correct this flaw I'd be
happy to help advise)

-- 
'peter'[:-1]@petertodd.org
00000000000000000ca626374f25dadbbb9245e60563a4d876f3c73070ad3849

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 516 bytes --]

>
> SPV wallets can't detect hard-forks


They don't have to - they pick the highest work chain. Any miner who hasn't
upgraded makes blocks on the shorter chain that are then ignored (or
rather, stored for future reorgs). After the fork point, there won't be any
blocks in the main chain that violate the rules and end up being doomed to
being orphaned, which is the underlying problem.

And I think you know this already. There is no "flaw" in bitcoinj in this
respect. It works exactly as it was designed to work.

[-- Attachment #2: Type: text/html, Size: 784 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1305 bytes --]

On Mon, Sep 28, 2015 at 04:33:23PM +0200, Mike Hearn wrote:
> >
> > SPV wallets can't detect hard-forks
> 
> 
> They don't have to - they pick the highest work chain. Any miner who hasn't
> upgraded makes blocks on the shorter chain that are then ignored (or
> rather, stored for future reorgs). After the fork point, there won't be any
> blocks in the main chain that violate the rules and end up being doomed to
> being orphaned, which is the underlying problem.
> 
> And I think you know this already. There is no "flaw" in bitcoinj in this
> respect. It works exactly as it was designed to work.

Ok, so again, if that's your security criteria, what's the issue with
soft-forks? With soft-forks, the result of a SPV wallet following the
highest work chain is the same: eventually invalid blocks are reorged
out.

However, because soft-forks make it less likely that a long invalid
chain will be generated, an attacker sybil attacking your SPV wallet has
a much harder time tricking it into accepting a transaction. (they might
get one or two confirmations, rather than dozens)

What's the scenario where soft-forks are worse than hard-forks from a
SPV wallet's perspective?

-- 
'peter'[:-1]@petertodd.org
00000000000000000368227ec1de9c27c14d23cb7be9e9f38c0082db79a87c49

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1175 bytes --]

>
> Ok, so again, if that's your security criteria, what's the issue
> with soft-forks?


Please read my article as it's all explained there.

But to reiterate: the risk is that miners will build invalid blocks on top
of the best work chain, instead of an ignored lower work side chain. This
opens users to payment fraud. With a hard fork, all the blocks by miners
that aren't checking all the rules anymore get neatly collected together on
a side chain after the split, and wallets all know how to ignore that chain.

Yes, you made OP_NOPs be non-standard. So out of the box, miners won't
create invalid blocks, as long as they're running Core past that version.
But this makes the IsStandard function very much like a part of the
consensus rules, as bypassing it can result in invalid blocks being
created. Miners have always understood that they can modify this function,
or even bypass it entirely, without affecting the validity of their blocks.
And some miners do exactly that.

So I'll repeat the question that I posed before - given that there are
clear, explicit downsides, what is the purpose of doing things this way?
Where is the gain for ordinary Bitcoin users?

[-- Attachment #2: Type: text/html, Size: 1492 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2762 bytes --]

On Mon, Sep 28, 2015 at 04:51:22PM +0200, Mike Hearn wrote:
> >
> > Ok, so again, if that's your security criteria, what's the issue
> > with soft-forks?
> 
> 
> Please read my article as it's all explained there.

I have read your article. In fact we reviewed it at a NY BitDevs meetup
that I attended.

> But to reiterate: the risk is that miners will build invalid blocks on top
> of the best work chain, instead of an ignored lower work side chain. This
> opens users to payment fraud. With a hard fork, all the blocks by miners
> that aren't checking all the rules anymore get neatly collected together on
> a side chain after the split, and wallets all know how to ignore that chain.

Can you explain exactly how you think wallets will "know" how to ignore
the invalid chain?

With an advertised soft-fork, e.g. the IsSuperMajority() mechanism,
ignoring the invalid chain is easy: use nVersion to detect invalid
blocks when you know what soft-forks are coming up, and if presented
with an unknown - but advertised - soft-fork at minimum loudly warn the
user. In the case of a hard-fork identical logic can be used. (BIP101
being an example of a hard-fork triggered in a way that can be detected
by SPV clients, both explicitly (BIP101 specific) and implicitly
(general unknown block nVersion warnings))

> Yes, you made OP_NOPs be non-standard. So out of the box, miners won't
> create invalid blocks, as long as they're running Core past that version.
> But this makes the IsStandard function very much like a part of the
> consensus rules, as bypassing it can result in invalid blocks being
> created.

How so? Miners can always choose to create invalid blocks, thus
attacking SPV wallets; my statement with regard to pull-req #5000 comes
from a risk-based approach, knowing that every invalid block is
expensive and the new concern created by a soft-fork is whether or not
miners will create them accidentally; miners can always create invalid
blocks delibrately.

> Miners have always understood that they can modify this function,
> or even bypass it entirely, without affecting the validity of their blocks.
> And some miners do exactly that.

That's incorrect: Miners bypassing IsStandard() risk creating invalid
blocks in the event of a soft-fork. Equally, we design soft-forks to
take advantage of this.

> So I'll repeat the question that I posed before - given that there are
> clear, explicit downsides, what is the purpose of doing things this way?
> Where is the gain for ordinary Bitcoin users?

We seem to be in strong disagreement about which option has "clear,
explicit downsides"

-- 
'peter'[:-1]@petertodd.org
0000000000000000006f2abe95e361b73289e4a79ba3124801896f6b7dc8d977

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 2073 bytes --]

>
> Can you explain exactly how you think wallets will "know" how to ignore
> the invalid chain?
>

I'm confused - I already said this. For a fork to work, hard or soft, there
must be support from a majority of the hash power.

Therefore, the usual SPV technique of following the highest work chain
results in ignoring the minority chain produced by the hard fork.

BIP 101 is SPV friendly because the wallets would simply follow the 75%
chain and never even be aware anything has changed. It's backwards
compatible with them in this respect: they already know how to ignore the
no-bigger-blocks fork that'd be created if some miners didn't upgrade
during the grace period.

My point about IsStandard is that miners can and do bypass it, without
expecting that to carry financial consequences or lower the security of
other users. By making it so a block which includes non-standard
transactions can end up being seen as invalid, you are increasing the risk
of accidents that carry financial consequences.

That's incorrect: Miners bypassing IsStandard() risk creating invalid
> blocks in the event of a soft-fork. Equally, we design soft-forks to
> take advantage of this.
>

Gah. You repeated what I just said. Yes, I know miners face that risk, my
point is that they do NOT face such a risk when there's no soft fork in
action and have historically NOT faced that risk at all, hence the
widespread practice of bypassing or modifying this function.

All this approach does is make changing IsStandard() the same as changing
AcceptBlock(), except without the advantage of telling anyone about it.


> > So I'll repeat the question that I posed before - given that there are
> > clear, explicit downsides, what is the purpose of doing things this way?
> > Where is the gain for ordinary Bitcoin users?
>
> We seem to be in strong disagreement about which option has "clear,
> explicit downsides"


Obviously. So please enlighten me.

How do ordinary Bitcoin users benefit from this rollout strategy? Put
simply, what is the point of this whole complex soft fork endeavour?

[-- Attachment #2: Type: text/html, Size: 2818 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[jl2012]

Mike Hearn via bitcoin-dev 於 2015-09-28 11:38 寫到:

> My point about IsStandard is that miners can and do bypass it,
> without expecting that to carry financial consequences or lower the
> security of other users. By making it so a block which includes
> non-standard transactions can end up being seen as invalid, you are
> increasing the risk of accidents that carry financial consequences.

Bypassing IsStandard should be considered as an "expert mode". The 
message should be "don't bypass it unless you understand what you are 
doing".

By the way, miners are PAID to protect the network. It is their greatest 
responsibility to follow the development and keep their software up to 
date.



> How do ordinary Bitcoin users benefit from this rollout strategy? Put
> simply, what is the point of this whole complex soft fork endeavour?

Let me try to answer this question. Softfork is beneficial to non-mining 
full nodes as they will follow the majority chain. In the case of a 
hardfork (e.g. BIP101), non-upgrading full nodes will insist to follow 
the minority chain. (unless you believe that all non-miner should use an 
SPV client)

Put it in a different angle. In a softfork, the new fork is a persistent 
95% attack against the old fork, which will force all in-cooperating 
miners to join (or leave). In a hardfork, however, there is no mechanism 
to stop the old fork and we may have 2 chains co-exist for a long time.

Although it is not mentioned in the whitepaper, the ability to softfork 
is a feature of Bitcoin. Otherwise, we won't have these OP_NOPs and the 
original OP_RETURN.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1227 bytes --]

>
> Let me try to answer this question. Softfork is beneficial to non-mining
> full nodes as they will follow the majority chain.


That is not a benefit. That is a description of what the software will do,
but not why you would want it.

In case this seems like a pedantic point, consider the consequences of
following a chain you aren't checking properly. You get SPV level security
and might calculate a corrupted ledger.

In the case of P2SH, I could make a transaction that spends someone elses
money to myself. In the case of CLTV, I could ignore the locktime
requirement.

Now yes, eventually, the miner majority will correct and uncorrupt your
ledger for you. But by then it might be too late, you may have already
acted upon the incorrect data by e.g. selling me lots of stuff that I paid
for with somebody else's coins. If you don't care about that risk, hey,
switch to an SPV wallet and save yourself a lot of disk space.


> In a hardfork, however, there is no mechanism to stop the old fork and we
> may have 2 chains co-exist for a long time.
>

There isn't any difference in how long the divergent state exists for. That
depends only on how fast people upgrade, which is unaffected by the rollout
strategy used.

[-- Attachment #2: Type: text/html, Size: 1803 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

[-- Attachment #1: Type: text/plain, Size: 1709 bytes --]

On Sep 28, 2015 7:14 PM, "Mike Hearn via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:
>> In a hardfork, however, there is no mechanism to stop the old fork and
we may have 2 chains co-exist for a long time.
>
>
> There isn't any difference in how long the divergent state exists for.
That depends only on how fast people upgrade, which is unaffected by the
rollout strategy used.
>

Yes, there is a difference. Assuming the hashrate majority upgrades, in the
case of a softfork non-upgraded miners will try to build on top of the
longest chain (the upgraded one) but their blocks will get consistently
orphaned for having a too old block version (and if they just increment the
version without implementing the new restrictions, then their blocks will
be orphaned when they fail to enforce the new restrictions). In the case of
a hardfork, the non-upgraded miners will keep on building their own longest
valid chain (the upgraded chain is not valid in their eyes), potentially
forever.
That's not to say softforks are always preferrable. There's cases when a
feature can be implemented as a softfork or a hardfork, but the softfork
solution is clearly inferior and introduces technical debt.
In those cases I prefer a hardfork, but this is not one of those cases.

In any case, maybe you want to provide some feedback to bip99, which is
about possible consensus rule changes scenarios and a recommended
deployment path for each of them (softforks and hardforks are subdivided in
several types). This discussion about the general desirability of softforks
seems offtopic for the concrete cltv deployment discussion, which assumes
softforks as deployment mechanism (just like bip66 assumed it).

[-- Attachment #2: Type: text/html, Size: 1901 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1537 bytes --]

Hi Jorge,

Yes, there is a difference. Assuming the hashrate majority upgrades, in the
> case of a softfork [snip] ...... In the case of a hardfork [snip]
>
Yes, I know what the difference between them is at a technical level. You
didn't explain why this would make any difference to how fast miners
upgrade. The amount of money they lose in both cases is identical: they are
equally incentivised to upgrade with both fork types.

Additionally, you say in a hard fork the other chain may "continue
forever". Why do you think this is not true for miners building invalid
blocks on top of the main chain? Why would that not continue forever?

There just isn't any difference between the two fork types in terms of how
fast miners would upgrade. Heck if anything, a hard fork should promote
faster upgrades, because if a miner isn't paying attention to their
debug.log they might miss the warnings. A soft fork would then look
identical to a run of really bad luck, which can legitimately happen from
time to time. A hard fork results in your node having a different height to
everyone else, which is easily detectable by just checking a block explorer.

> This discussion about the general desirability of softforks seems offtopic
> for the concrete cltv deployment discussion, which assumes softforks as
> deployment mechanism (just like bip66 assumed it).
>
Isn't that circular? This thread is about deployment of CLTV, but the BIP
assumes a particular mechanism, so pointing out problems with it is off
topic? Why have a thread at all?

[-- Attachment #2: Type: text/html, Size: 1970 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jonathan Toomim (Toomim Bros)]

[-- Attachment #1: Type: text/plain, Size: 3579 bytes --]


On Sep 28, 2015, at 7:43 AM, Peter Todd via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

> 
> Ok, so again, if that's your security criteria, what's the issue with
> soft-forks? With soft-forks, the result of a SPV wallet following the
> highest work chain is the same: eventually invalid blocks are reorged
> out.
> 
> However, because soft-forks make it less likely that a long invalid
> chain will be generated, an attacker sybil attacking your SPV wallet has
> a much harder time tricking it into accepting a transaction. (they might
> get one or two confirmations, rather than dozens)
> 
> What's the scenario where soft-forks are worse than hard-forks from a
> SPV wallet's perspective?


I don't think this was addressed clearly, so here's my attempt.

With a soft fork, miners who have not upgraded append their blocks to the longest block chain. To SPV clients and to old fully-validating clients, it appears to be a valid block that inevitably gets orphaned. SPV clients will be tricked to follow these blocks every time they appear, since every time they appear they will have a PoW advantage for a few minutes. SPV clients will appear to behave normally, and will continue to show new transactions and get confirmations in a timely fashion. However, they will be systematically susceptible to attack from double-spends that attempt to spend funds in a way that the upgraded nodes will reject. These transactions will appear to get 1 confirmation, then regress to zero conf, every single time. These attacks can be performed for as long as someone mines with the old version. If an attacker thinks he could get more than 25 BTC of double-spends per block, he might even choose to mine with the obsolete version in order to get predictable orphans and to trick SPV clients and fully verifying wallets on the old version.

With a hard fork, miners who have not upgraded will append their blocks on the shorter fork. SPV clients will ignore this fork unless Sybil attacked. If an SPV node only connects to one full node server, that's equivalent to a Sybil attack.  In that case, transactions on the long chain will often not be present on the short chain due to its shortness. Confirmations will be slow, and will be shown to be very different from what's shown on block explorers. Displayed transaction dates and times will be off, when they show up at all. Any transactions that have been contaminated by recent mining revenue will not show up at all. SPV client users will probably notice something is wrong. If the SPV client connects to several full nodes, then this should rarely happen. For example, if 5% of full nodes are still on the old version, and an SPV wallet connects to 2 nodes at a time, there is a 0.05**2 = 0.25% chance. If the SPV client has headers cached on disk from a previous connection to the longer chain, then that chance effectively drops to zero. As a further benefit to hard forks, anybody who is ideologically opposed to the change can continue to use the old version successfully, as long as there are enough miners to keep the fork alive.

In short: soft forks mean frequent predictable and manipulable orphan blocks that SPV clients will always follow, with transactions that get confirmed once and then perma-orphaned. Hard forks mean that SPV clients will almost always work flawlessly, and will occasionally give very strange and noticeably wrong results. For fully-verifying nodes, soft forks make old versions insecure, but hard forks allow new and old versions to operate in parallel.


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[jl2012]

Jonathan Toomim (Toomim Bros) via bitcoin-dev 於 2015-09-29 09:30 寫到:
> SPV clients will appear to behave normally, and
> will continue to show new transactions and get confirmations in a
> timely fashion. However, they will be systematically susceptible to
> attack from double-spends that attempt to spend funds in a way that
> the upgraded nodes will reject. These transactions will appear to get
> 1 confirmation, then regress to zero conf, every single time. These
> attacks can be performed for as long as someone mines with the old
> version.

1. Who told you to accept 1-confirmation tx? Satoshi recommended 6 
confirmations in the whitepaper. Take your own risk if you do not follow 
his advice.

2. This is true only if your SPV client naively follows the longest 
chain without even looking at the block version. This might be good 
enough for the 1st generation SPV client, but future generations should 
at least have basic fraud detecting mechanism.



> If an attacker thinks he could get more than 25 BTC of
> double-spends per block, he might even choose to mine with the
> obsolete version in order to get predictable orphans and to trick SPV
> clients and fully verifying wallets on the old version.

This point is totally irrelevant. No matter there is a softfork or not, 
SPV users are always vulnerable to such double-spending attack if they 
blindly follow the longest chain AND accept 1-confirmation. The fiat 
currency system might be safer for them.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[odinn]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello, (see my remarks below)

jl2012 via bitcoin-dev:
> Jonathan Toomim (Toomim Bros) via bitcoin-dev 於 2015-09-29 09:30 寫
> 到:
>> SPV clients will appear to behave normally, and will continue to
>> show new transactions and get confirmations in a timely fashion.
>> However, they will be systematically susceptible to attack from
>> double-spends that attempt to spend funds in a way that the
>> upgraded nodes will reject. These transactions will appear to
>> get 1 confirmation, then regress to zero conf, every single time.
>> These attacks can be performed for as long as someone mines with
>> the old version.
> 
> 1. Who told you to accept 1-confirmation tx? Satoshi recommended 6 
> confirmations in the whitepaper. Take your own risk if you do not
> follow his advice.
> 
> 2. This is true only if your SPV client naively follows the
> longest chain without even looking at the block version. This might
> be good enough for the 1st generation SPV client, but future
> generations should at least have basic fraud detecting mechanism.
> 
> 

Regarding "basic fraud detecting mechanism" of which you speak, being
as I personally enjoy SPV for the time being (Electrum), and I know
that people will continue to keep using SPV wallets because they are
light and handy, I think that you make a good point that "basic fraud
detecting mechanism" is needed, but how to verify that such a
mechanism in an SPV wallet is good, and/or that the software and
version information provided by the server admins via the banner is
valid (being as it's not validated)?  I have made a thread on this
conundrum.  Which is posted here if you are interested.

https://bitcointalk.org/index.php?topic=1157545.0

So as to avoid repeating stuff please read whole thead before
answering in it or posting back to list.  It seems that there are
definitely unanswered questions...

I may open this up as an issue on https://github.com/spesmilo/electrum
about this stuff, but I wanted to post comment here also, for the record
.


> 
>> If an attacker thinks he could get more than 25 BTC of 
>> double-spends per block, he might even choose to mine with the 
>> obsolete version in order to get predictable orphans and to trick
>> SPV clients and fully verifying wallets on the old version.
> 
> This point is totally irrelevant. No matter there is a softfork or
> not, SPV users are always vulnerable to such double-spending attack
> if they blindly follow the longest chain AND accept 1-confirmation.
> The fiat currency system might be safer for them. 
> _______________________________________________ bitcoin-dev mailing
> list bitcoin-dev@lists.linuxfoundation.org 
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

- -- 
http://abis.io ~
"a protocol concept to enable decentralization
and expansion of a giving economy, and a new social good"
https://keybase.io/odinn
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJWCuyBAAoJEGxwq/inSG8CtpAH/R6N1QYzMFWPo75RsP46VYbi
k33QbsbhlEznEEWX/ayKEzmnbt7DkXFXQtesuabongFr9UpwxED0OGQJztyRz5NC
iS8ty+Kfi9/Aq/e79A6IPSYfRCPB1w+oP/cEsV/LB4BPkut2mdpMbdwDZ3TQuLRq
LnFLmz8tY+CUqSbyrPUx/FKJ7ZbQsAlammMTKoUYaAYRytDBPzW4PdYtTyrK2QTK
jjt11n5U8ShmXdsCo/E0pWVbggQlhFgrCoIYjGNfmDyK/eYaskD5O6czIdqd5WPs
P+2zMC1Cukkr5l8BQXiSedVXGpMyaYhMgWB7MD6sNDIAE9IFbfEpkse/Ek4aJII=
=Ud4C
-----END PGP SIGNATURE-----

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Gregory Maxwell]

On Mon, Sep 28, 2015 at 10:48 AM, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> There is no consensus on using a soft fork to deploy this feature. It will
> result in the same problems as all the other soft forks - SPV wallets will
> become less reliable during the rollout period. I am against that, as it's
> entirely avoidable.
>
> Make it a hard fork and my objection will be dropped.

I'm surprised to see this response-- BIP65 is a year old now which is
plenty of time to mature and for issues to be uncovered. It (and its
predecessors) have had extensive discussion-- with no controversy
exposed during its entire lifetime, but in any case...

I am having a little difficulty making sense of this complaint. For
all any of us know miners are already enforcing the validity of CLTV,
it's indistinguishable on the visible behavior.  At the same time in
BitcoinXT's 101 proposal the change in system rules is similarly
"invisible" to existing "SPV" wallets in the same way that enforcement
of CLTV is "invisible": both are no change from their perspective.
Have I missed a proposal to change BIP101 to be a real hardfork (e.g.
be invalid from the perspective of historical bitcoinj clients too)?
---- I'd think it to be completely reasonable to do so, even while not
thinking that it would be reasonable here:  Softforks and hardforks
are not the same thing, not technically, and not politically. Miners
can collectively, at their whim, impose any kind of soft fork they
want, at any time and you won't even necessarily be able to tell...
that is just how the system works. Hardforks on the other hand, can
only happen with the consent of the participants-- they can directly
violate system properties that the participants believe to be largely
nonvolatile, and they _force__ software upgrades, so I think having a
higher bar makes good sense there.

The particular mechanism used in the proposal as-is has been used many
times before (and has been refined over time) and we have considerable
experience with it. The behavior is not, in fact, truly invisible to
non-upgraded participants: it's is visible by way of the block version
changing.  Bitcoin Core, going back years, responds by issuing a
warning-- "%s: %d of last 100 blocks above version %d\n" which then
becomes "Warning: This version is obsolete; upgrade required!".  Users
of the software (directly or via automation) are free to decide to
take whatever policy action they wish to take, delay accepting
transactions, patching software, etc..  The same could be done by any
client of the system if they cared to do so.

I believe the versionbits mechanism will be superior, but-- among
other things-- its deployment has been complicated by BitcoinXT
deploying an incomplete approximation of it.  Versionbits primary
advantage is related to having multiple concurrent proposals in
flight, which will be good to have but isn't itself a reason not to
pull a proposal up ahead of versionbits.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1565 bytes --]

Hi Gregory,


> I'm surprised to see this response


Why? I have objected to the idea of soft forks many times. I wrote an
entire article about it in August. I also objected in April 2014, for
instance, where Pieter agreed with me that soft forks can result in ugly
hacks, and that they are "not nice philosophically because they reduce the
security model of former full nodes to SPV without their knowledge" (he
thought they were worth it anyway).

This is not a new debate. If you're surprised, it means only you weren't
paying attention to all the previous times people raised this issue.


> Have I missed a proposal to change BIP101 to be a real hardfork


There's no such thing as a "real" hard fork - don't try and move the goal
posts. SPV clients do not need any changes to do the right thing with BIP
101, they will follow the new chain automatically, so it needs no changes.

Several people have asked several times now: given the very real and widely
acknowledged downsides that come with a soft fork, *what* is the specific
benefit to end users of doing them?

Until that question is answered to my satisfaction I continue to object to
this BIP on the grounds that the deployment creates financial risk
unnecessarily. To repeat: *CLTV does not have consensus at the moment*.

BTW, in the April 2014 thread Pieter's argument was that hard forks are
more risky, which is at least an answer to my question. But he didn't
explain why he thought that. I disagree: the risk level seems lower with a
hard fork because it doesn't lower anyone's security level.

[-- Attachment #2: Type: text/html, Size: 2122 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

On Wed, Sep 30, 2015 at 7:11 PM, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> Several people have asked several times now: given the very real and widely
> acknowledged downsides that come with a soft fork, what is the specific
> benefit to end users of doing them?

As previously explained, the biggest advantage of softforks is that
assuming the hasrate majority upgrades, network convergence is
guaranteed.
I don't know of anyone else (apart from you) that believes that the
advantages of softforks are generally worse than those of hardforks.
I'm attempting to clarify everything related to consensus rule changes
in BIP99.

> Until that question is answered to my satisfaction I continue to object to
> this BIP on the grounds that the deployment creates financial risk
> unnecessarily. To repeat: CLTV does not have consensus at the moment.

But your argument is flawed because it assumes softforks are more
risky than hardforks.
You've been explained why this is not the case, so unless you can
explain what's more important for a consensus system than network
convergence I think we can still consider this consensus rule change
uncontroversial, just like BIP66 was (even if you were also unable to
understand the advantages of softforks back then, just like you are
unable to understand them now, as you just proved in your answer to my
explanation). Using BIP99's terminology, this is an "uncontroversial
softfork" and it's therefore the safest option for consensus rule
changes deployment.
I should definitely improve my explanation on why uncontroversial
softforks are preferrable to uncontroversial hardforks in most cases
(and maybe try to come up with an example in which a hardfork is
preferable). I should also explain the disadvantages of
uncontroversial softforks that you have pointed out several times. So
I will mention you in BIP99's PR once I update it with a new section
that talks about the trade offs of uncontroversial softforks vs
uncontroversial hardforks.
In the meantime I believe that we can safely move forwards with BIP65
(again, just like we did with BIP66 ) and I also believe that you, as
an expert in Bitcoin, will eventually be able to understand the
advantages of uncontroversial softforks.
With all due respect, I don't think we need to wait for you to
understand the advantages of softforks to move forward with BIP65,
just like we didn't need to wait for every developer and user to
understand BIP66 to deploy it.
You don't have specific complaints against the new script operator,
and you don't have an uncontroversial hardfork alternative design (or
implementation).
This is a feature that enables new contracts that are important to
Bitcoin. Please don't try to block it just to make a point about what
"uncontroversial" means.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Tom Harding]

On 9/30/2015 10:58 AM, Jorge Timón via bitcoin-dev wrote:

> I don't think we need to wait for you to understand the advantages of
> softforks to move forward with BIP65, just like we didn't need to wait
> for every developer and user to understand BIP66 to deploy it.

What a bad example.  BIP66 deployment failed, and was rescued by
centralized intervention.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Adam Back]

On 30 September 2015 at 13:11, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
>> Have I missed a proposal to change BIP101 to be a real hardfork
>
> There's no such thing as a "real" hard fork - don't try and move the goal
> posts. SPV clients do not need any changes to do the right thing with BIP
> 101, they will follow the new chain automatically, so it needs no changes.

BIP101 is a hybrid: in some ways it is a hard-fork and in other ways
it is a soft-fork.  It is a hard-fork to full-nodes, but also a
soft-fork to SPV clients, as by definition the SPV miners are having
changes made whether they approve or not as they are not even aware of
the change.

> To repeat: CLTV does not have consensus at the moment.

I think people are saying CLTV is long discussed and does have consensus.

> Several people have asked several times now: given the very real and widely
> acknowledged downsides that come with a soft fork, what is the specific
> benefit to end users of doing them?
>
> Until that question is answered to my satisfaction I continue to object to
> this BIP on the grounds that the deployment creates financial risk
> unnecessarily.

Let's not conflate CLTV with a discussion about future possible
deployment methods.  Forks are an interesting but different topic.

Soft-forks have a lot of mileage on them at this point, hard-forks do
not, and are anyway inherently higher riskier, even ignoring our lack
of practical experience with planned hard-forks.

With a soft-fork, while it's clear there is a temporary security model
reduction for SPV nodes (and non-upgraded full nodes) in the period
before they upgrade, this is preferable to the risks of a system-wide
coordinated hard-fork upgrade.  There is some limit if the complexity
of soft-forking a feature is quite complicated (eg one could argue
that with soft-fork extension-blocks vs hard-fork method of increasing
block-size for example).  So the balance, which I think is easily met
with CLTV, is that soft-fork is simple-enough technically and the
feature is entirely non-controversial and additive functionality
improvement without downside or reason for dissent.

To my view this is an answer to your question "what is the specific
benefit to end users of doing [soft-forks]" -- it is a lower risk, and
therefore faster way to deploy non-controversial (additive) changes.

Given the CLTV is useful for improving lightning efficiency this is
good for improving Bitcoin's scalability.

Adam

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 413 bytes --]

On Wed, Sep 30, 2015 at 1:11 PM, Mike Hearn via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Several people have asked several times now: given the very real and
> widely acknowledged downsides that come with a soft fork, *what* is the
> specific benefit to end users of doing them?
>

Field experience shows it successfully delivers new features to end users
without a global software upgrade.

[-- Attachment #2: Type: text/html, Size: 999 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 1848 bytes --]

>
> Field experience shows it successfully delivers new features to end users
> without a global software upgrade.
>

The global upgrade is required for all full nodes in both types. If a full
node doesn't upgrade then it no longer does what it was designed to do; if
the user is OK with that, they should just run an SPV wallet or use
blockchain.info or some other mechanism that consumes way fewer resources.

But if you want the software you installed to achieve its stated goal, you
*must* upgrade. There is no way around that.

Jorge has said soft forks always lead to network convergence. No, they
don't. You get constant mini divergences until everyone has upgraded, as
opposed to a single divergence with a hard fork (until everyone has
upgraded). The quantity of invalid blocks mined, on the other hand, is
identical in both types.

Adam has said "there is actually consensus", although I just said there
isn't. Feel free to say what you really mean here Adam - there's consensus
if you ignore people who don't agree, i.e. the concept of "developer
consensus" doesn't actually mean anything. This would contradict your prior
statements about how Bitcoin Core makes decisions, but alright ....

Finally John, I fully agree with what you wrote. Debates that never end are
bad news all round. Bitcoin Core has told the world it uses "developer
consensus" to make decisions. I don't agree that's a good way to do things,
but if Core wants to stick with it then there is no choice - as I am a
developer, and I do not agree with the change, there is no consensus and
the debate is over.

Hey, I have an idea. Maybe we should organise a conference about soft vs
hard forks. Let's have it down the road from where I live, a couple of
weeks from now. Please submit your talk titles to me so I can vet them to
ensure nobody does an offtopic talk ;)

[-- Attachment #2: Type: text/html, Size: 2344 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

[-- Attachment #1: Type: text/plain, Size: 831 bytes --]

On Sep 30, 2015 9:56 PM, "Mike Hearn via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> Jorge has said soft forks always lead to network convergence. No, they
don't. You get constant mini divergences until everyone has upgraded, as
opposed to a single divergence with a hard fork (until everyone has
upgraded). The quantity of invalid blocks mined, on the other hand, is
identical in both types.

Exactly, all those "mini divergences" eventually disappear (because we're
assuming the hashrate majority has upgraded and non-upgraded miners accept
upgraded blocks as valid), even if the hashrate minority never upgrades.
On the other hand, the "single divergence" in the hardfork keeps growing
forever (unless all miners evetually upgrade.
With softforks, we maintain eventual consistency, with hardforks we don't.

[-- Attachment #2: Type: text/html, Size: 1010 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 778 bytes --]

>
> Exactly, all those "mini divergences" eventually disappear
>
A miner that has accepted a newly invalid transaction into its memory pool
and is trying to mine it, will keep producing invalid blocks forever until
the owner shuts it down and upgrades. This was happening for weeks after
P2SH triggered.

For instance, any miner that has modified/bypassed IsStandard() can do
this, or any miner that accepts direct transaction submission, or any miner
that runs an old node from before OP_NOPs were made non-standard.

> On the other hand, the "single divergence" in the hardfork keeps growing
> forever (unless all miners evetually upgrade.
>
Which they do, because they will eventually notice they are burning money.

Sorry Jorge, but I don't think your argument makes sense.

[-- Attachment #2: Type: text/html, Size: 1208 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

On Wed, Sep 30, 2015 at 11:06 PM, Mike Hearn <hearn@vinumeris.com> wrote:
>> Exactly, all those "mini divergences" eventually disappear
>
> A miner that has accepted a newly invalid transaction into its memory pool
> and is trying to mine it, will keep producing invalid blocks forever until
> the owner shuts it down and upgrades. This was happening for weeks after
> P2SH triggered.
>
> For instance, any miner that has modified/bypassed IsStandard() can do this,
> or any miner that accepts direct transaction submission, or any miner that
> runs an old node from before OP_NOPs were made non-standard.

That is correct. But doesn't seem to contradict anything I said.

>> On the other hand, the "single divergence" in the hardfork keeps growing
>> forever (unless all miners evetually upgrade.
>
> Which they do, because they will eventually notice they are burning money.

Assuming it is an uncontroversial hardfork (unlike bip101 in its
current form), miners will eventually upgrade because all users will
eventually upgrade as well.
Softfork-caused forks will live shortly because non-upgraded miners
will build on top of the longest upgraded chain.
In contrast, non-upgraded miners will not build on top of the longest
chain (the upgraded one assuming hashrate majority) and a parallel
chain will be built for some time. This chain can be used to defraud
non-upgraded or SPV users by isolating them and showing them only the
non-upgraded chain, which keeps growing but will eventually be
abandoned.
In the case of a Schism hardfork, some users may never want to
"upgrade" and if there's demand for the "old coins" there will be
miners for the "old chain".

> Sorry Jorge, but I don't think your argument makes sense.

I think my argument makes a lot of sense, it's just that for some
reason you don't think guaranteed eventual consistency has any value
because you are ok with miners abandoning the old rules chain only
eventually (and you don't believe that "eventually" can be far in the
future in practice).

On Wed, Sep 30, 2015 at 9:56 PM, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> Adam has said "there is actually consensus", although I just said there
> isn't. Feel free to say what you really mean here Adam - there's consensus
> if you ignore people who don't agree, i.e. the concept of "developer
> consensus" doesn't actually mean anything. This would contradict your prior
> statements about how Bitcoin Core makes decisions, but alright ....

BIP99 doesn't talk about "developer consensus", but rather
"uncontroversial consensus rule changes".
Obviously a patch in which developers steal everybody else's coins
wouldn't be "uncontroversial" even if "developer consensus" is
reached.
We don't need to ignore anyone to consider BIP65 an uncontroversial
softfork: we just need to ignore fallacious and unreasonable
arguments.
As far as I can tell, you are the only person opposing BIP65 (even if
you keep talking about "several people") and I would like to think
that you are aren't being obstinate on purpose only to make your point
about "developer consensus not meaning anything", but you are making
it very hard.

On Wed, Sep 30, 2015 at 11:01 PM, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> I coined the term SPV so I know exactly what it means, and bitcoinj
> implements it, as does BreadWallet (the other big SPV implementation).

No, you didn't. "Simplified Payment Verification" is section 8 in the
Bitcoin whitepaper that you like to cite so much.

> I'm going to ignore the rest of the stuff you wrote about "design decisions
> to lack security" or "cheaply avoidable lack of validation". When you have
> sat down and written an SPV implementation by yourself, then shipped it to a
> couple of million users, you might have better insight into basic
> engineering costs. Until then, I find your criticisms of code you think was
> missing due to "stonewalling" and so on to be seriously lacking real world
> experience.

Please study this page carefully and hopefully one day you will stop
using logical fallacies as often as you currently do:
https://en.wikipedia.org/wiki/List_of_fallacies
In this case you manage to combine ad hominem and appeal to authority
(maybe false authority is more accurate?).
Once again, please, stop using fallacies to try to convince people
that you are right. No offense, but being warned publicly about the
use of logical fallacies so often would be extremely embarrassing to
me.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

[-- Attachment #1: Type: text/plain, Size: 875 bytes --]

On Oct 1, 2015 12:14 AM, "Jorge Timón" <jtimon@jtimon.cc> wrote:
>
> On Wed, Sep 30, 2015 at 11:06 PM, Mike Hearn <hearn@vinumeris.com> wrote:
> >> Exactly, all those "mini divergences" eventually disappear
> >
> > A miner that has accepted a newly invalid transaction into its memory
pool
> > and is trying to mine it, will keep producing invalid blocks forever
until
> > the owner shuts it down and upgrades. This was happening for weeks after
> > P2SH triggered.
> >
> > For instance, any miner that has modified/bypassed IsStandard() can do
this,
> > or any miner that accepts direct transaction submission, or any miner
that
> > runs an old node from before OP_NOPs were made non-standard.
>
> That is correct. But doesn't seem to contradict anything I said.

Actually, no, sorry, the second paragraph is not correct as explained by
Greg Maxwell.

[-- Attachment #2: Type: text/html, Size: 1122 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 1339 bytes --]

On Wed, Sep 30, 2015 at 3:56 PM, Mike Hearn <hearn@vinumeris.com> wrote:

> Field experience shows it successfully delivers new features to end users
>> without a global software upgrade.
>>
>
> The global upgrade is required for all full nodes in both types. If a full
> node doesn't upgrade then it no longer does what it was designed to do; if
> the user is OK with that, they should just run an SPV wallet or use
> blockchain.info or some other mechanism that consumes way fewer resources.
>
> But if you want the software you installed to achieve its stated goal, you
> *must* upgrade. There is no way around that.
>

It is correct that security is slightly reduced for full nodes that have
not upgraded.  It is not correct that the choice is binary, full node or
SPV.

Any user running a not-upgraded full node still retains protection against
many attacks outside the subset related to the feature being introduced.

The field observable end result is that we do receive new features, secured
by hashpower and user full nodes, via soft fork, without a global flag day
upgrade.

Yes, a flag day hard fork upgrade is cleaner and results in higher security
due to the entire network validating the new rules.  However, the
difficulty of executing hard forks would mean fewer total features to
users, if that route were chosen instead.

[-- Attachment #2: Type: text/html, Size: 2113 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Gregory Maxwell]

On Wed, Sep 30, 2015 at 10:17 PM, Jeff Garzik <jgarzik@gmail.com> wrote:
> It is correct that security is slightly reduced for full nodes that have not
> upgraded.  It is not correct that the choice is binary, full node or SPV.
>
> Any user running a not-upgraded full node still retains protection against
> many attacks outside the subset related to the feature being introduced.

An extra way to look at this is that even absent any rule changes--
users who are asleep at the switch may lose effective security over
time because attackers learn new tricks against existing
vulnerabilities. Security requires a bit of vigilance, inherently.

In many specific cases I think it's hard-to-impossible to articulate a
concrete way that security is lost by users at all, excluding some
small amplification of orphan blocks.


On Wed, Sep 30, 2015 at 9:06 PM, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> and is trying to mine it, will keep producing invalid blocks forever until
> the owner shuts it down and upgrades.

This is the outcome guaranteed for absentee miners with a hard fork,
but it is not guaranteed for a soft fork.

> For instance, any miner that has modified/bypassed IsStandard() can do this,

Miners who have changed their code in inadvisable ways can produce
invalid blocks as a result. There are many seemingly innocuous ways
one can produce invalid blocks, and miners have stumbled on a few of
them over the years.

Pedantically, modifying IsStandard() will not have this effect:
Unknown NOPs are now handled via a script validation flag--
SCRIPT_VERIFY_DISCOURAGE_UPGRADABLE_NOPS.  Experience (e.g. with
STRICTDER) has show that script validation flags are much more robust
to casual twiddling than IsStandard is.

The only way that script validation flags have been observed getting
bypassed in the field was a miner that had disabled all signature
validation completely (and whom had a not-completely-negligible amount
of hashpower. :( )... as it's a lot more clear that you might be
exposing yourself to trouble if you mess with the validation flags.

> runs an old node from before OP_NOPs were made non-standard.

IIRC; There is no released version of Bitcoin that has IsStandard
which has failed failed to treat the NOPs as non-standard.

There was a brief time in git master between when IsStandardness was
relaxed and NOPs were addressed via a validation flag but I am
reasonably confident that didn't make it into a release.

Regardless, anyone actually running that code of that vintage would
already be incompatible with the current network already due to prior
soft forks.

And as a matter of fact, invalid CLTVs don't currently appear to get
mined. Checking this again pre-release would be a good checklist item.
For prior soft-forks we've monitored and tested for this (with the
goal of going and yelling at any broken miners to fix their behavior).

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Gregory Maxwell]

On Wed, Sep 30, 2015 at 5:11 PM, Mike Hearn <hearn@vinumeris.com> wrote:
> Hi Gregory,
>
>>
>> I'm surprised to see this response
>
>
> Why? I have objected to the idea of soft forks many times. I wrote an entire
> article about it in August.

Yes, your article contained numerous factual and logical inaccuracies
which I corrected (many of which you had been previously corrected on
as well by others). (For example:
https://www.reddit.com/r/Bitcoin/comments/3griiv/on_consensus_and_forks_by_mike_hearn/cu0yv0r
)

I would have hoped that after so many corrections you would have
updated your beliefs.

> There's no such thing as a "real" hard fork - don't try and move the goal
> posts. SPV clients do not need any changes to do the right thing with BIP
> 101, they will follow the new chain automatically, so it needs no changes.

Yes, because what 101 does is not a hard-fork from the perspective of
BitcoinJ clients. Please do not conflate BitcoinJ with all of SPV; a
SPV client could validate the information received more extensively or
respond to alerts in reject rule violating blocks--  BitcoinJ does
not, but this is BitcoinJ's design decision to lack security in this
respect and not something inherent to SPV).

Directly fixing the time-warp attack, for example, would be a hard
fork from the perspective of BitcoinJ clients.  Recovering the fixed 0
bits in the header for use as extra-nonce would be a hard fork from
the perspective of BitcoinJ clients. Changing the transaction format
to include an explicit nonce for ECDH (e.g. stealth addresses) would
be a hard fork from the perspective of BitcoinJ clients. Increasing
the precision of Bitcoin by 1000 would be a hard fork from the
perspective of BitcoinJ. As would adjusting the hashtree to commit to
fees, including fees under OP_CHECKSIGs hash, or switching to the
segregated witness commitment structure from elements alpha that
allows syncing the chain without fetching signatures... all that would
be hardforks from the perspective of BitcoinJ.

Because of an cheaply avoidable the lack of validation in BitcoinJ no
increase of the blocksize is a hard-fork from its perspective. Nor
would increasing the subsidy to miners, or allowing third parties to
confiscate coins. But other SPV clients could, if they wanted to,
reject blocks the violated most of these criteria.

The argument you are presenting against BIP65 is that it is bad
because it is silently accepted. But this applies no less to 101 for
SPV clients, and in 101's case it's a failure to enforce pre-existing
rules which the users might care a great deal about. Worse, counting
on this kind of behavior can build a dependence on weak security forms
of SPV and inhibits the use of full security SPV.

In truth, both of BIP101 and BIP65 are detectable even by the most
simplistic and pre-change clients due to the voluntarily use of block
version signaling. Any participant in the network is free to take
whatever action they choose to take in response to such an event.
Bitcoin Core's behavior is to issue alerts to the user when unexpected
block versions show up on the network.  Users and implementer are free
to turn changes like BIP65 into hardforks from the perspective of
their own system, necessitating manual intervention, by simply forcing
the block version to be a particular value (or shutting down when
there are many blocks of a new version; until manually authorized to
continue).

> Several people have asked several times now: given the very real and widely
> acknowledged downsides that come with a soft fork, what is the specific
> benefit to end users of doing them?

For many changes, including CLTV the actual soft fork change is by far
the most natural way of implementing the change itself. One simply
takes an existing non-standard placeholder op code sequence and
assigns it the new VERIFY style meaning. It is clean, tidy, and the
result is nearly as if the system has had it all along. The only
complexity is around the activation and can be dropped in future code.

Beyond that, the primary upside is no forced industry wide "flag day"
where everyone is _forced_ to modify their software arises, taking
considerable cost.  People who care about the new rule can use it,
people who don't don't. All the rules that you care about enforcing
remain in force-- you still prevent inflation, you still will not
tolerate the theft of your own coins (or those of most other people),
etc.. No one is necessarily caught by surprise since the block
versions communicate that something is happening, allowing network
participants to choose to act (or not).

For example, for years you stonewalled P2SH and multi-signature.  You
didn't care about it. You didn't think it was valuable. You didn't add
it to your software, even after it was well specified and deployed in
production. Could it have been done as a hard-fork?  Likely not: you
would have prevented it. But as a soft-fork you were free to ignore it
with no ill-effect for a long time existing for those who cared about
it, and not for you, until widespread use resulted in demand enough to
justify accepting a patch that permitted sending to it.

What if we'd needed a hard fork to enable CoinJoin or other privacy
features?  I think would have blocked that too.

The relative ease of handling soft-forks which you are indifferent to
means that there is little reason to object to a compatible change
that gives other people flexibility they care about greatly but which
you are indifferent to; and it forces people who would oppose a
functionality because they don't want others to have some piece of
freedom to try to frame justifications in language other than "I don't
think it's worth the cost" since they have the nearly free option of
ignoring the change-- they're forced to actually argue against other
people having that freedom.

Soft-forks also allows us to deploy fixes to the Bitcoin protocol
which are more important-to-have but not urgently critical (like
height in coinbase),  or sometimes to deploy fixes to critical
vulnerabilities without first handing everyone excruciatingly detailed
instructions on exploiting them, simply by closing off an pattern of
protocol which is obviously bad and risky.

The primary cost of a soft-fork for non-participants is simply some
risk of increased network instability around the change-- but short
lived forks happen every day, and longer lived ones happen from time
to time. Larger amounts of instability occur from time to time due to
network partitioning, misconfiguration, and software bugs-- and client
software must be prepared to cope with it; this is a fact of bitcoin
and decenteralized systems in general. Upgraded, change enforcing,
client software is not exposed to the this instability, and
non-upgraded software could choose to mitigate any exposure by
monitoring the block versions. This is a far better situation that the
natural instability that will happen from time to time in a
decentralized system.

By contrast, the programmed activation point of BIP101 at 75% almost
guarantees activation among considerable controversy, promising
network instability which BitcoinJ clients would experience upgraded
or not, even if the larger block side was ultimately the losing side
in the switch. I find it more than a little strange that you think the
instability of a 75%-version-hashpower cut is acceptable but the a
95%-version-hashpower compatible change is not.

Finally, there is the demonstrated track record: They work; they
deliver new features to people. Our experience in the half dozen or
more soft-forks in the system so far is that in practice do not cause
significant problems, including financial losses for SPV wallet users.
Even with that complete success there has been room for improvement,
which is why the process has evolved over time to feature things like
preemptive non-standardness, high switchover thresholds, etc.  and
these will continue to evolve over time.

I hope that you can put aside your effort to force a blocksize
increase on others for a moment and add functionality, of the kind the
Bitcoin Core has had for years, to BitcoinJ to improve the experience
with soft-forks if you think it isn't good enough as is...

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 4144 bytes --]

tl;dr Nothing I have read here has changed my mind. There is still no
consensus to deploy CLTV in this way.


> Yes, your article contained numerous factual and logical inaccuracies
> which I corrected
>

I responded to your response several times. It was not convincing, and I do
not think you corrected factual inaccuracies. I mean, you said yourself you
once used the correct terminology of forwards compatibility but stopped
only because the term "backwards compatibility" is more common. But that's
not a good reason to use a term with the opposite meaning and is certainly
not a factual correction!


> Yes, because what 101 does is not a hard-fork from the perspective of
> BitcoinJ clients. Please do not conflate BitcoinJ with all of SPV;


I coined the term SPV so I know exactly what it means, and bitcoinj
implements it, as does BreadWallet (the other big SPV implementation).

Yes, SPV wallets will follow the mining hashpower instead of doing a hard
reject for bigger blocks, because they deliberately check a subset of the
rules: block size is not and never has been one of them. Indeed it's not
even included in the protocol messages. Users have no expectation that SPV
wallets would check that, as it's never been claimed they do.

On the other hand, full nodes all claim they run scripts. Users expect that
and may be relying on it. The unstated assumption here is that the nodes
run them correctly. A soft fork breaks this assumption.

I'm going to ignore the rest of the stuff you wrote about "design decisions
to lack security" or "cheaply avoidable lack of validation". When you have
sat down and written an SPV implementation by yourself, then shipped it to
a couple of million users, you might have better insight into basic
engineering costs. Until then, I find your criticisms of code you think was
missing due to "stonewalling" and so on to be seriously lacking real world
experience.

Yes, a hypothetical full node could fork on the version bits. I would be
quite happy with the version number in the header being an enforced
consensus rule: it'd make hard forks easier to trigger. But it hasn't been
done that way, and wishing away the behaviour of existing software in the
field is no good. Luckily, for introducing a new opcode, the same effect
can be achieved by using a non-allocated opcode number.


> For many changes, including CLTV the actual soft fork change is by far
> the most natural way of implementing the change itself.


This is subjective. I'd say picking an entirely new opcode number is most
natural.

The rest of your argument boils down to "people don't have to upgrade if
they don't want to", which is addressed in the article I wrote already, and
multiple responses on this thread. Yes, they do, otherwise they aren't
getting the security level they were before.


> Could [P2SH] have been done as a hard-fork?  Likely not: you would have
> prevented it.


What? This is nonsensical. P2SH was added to the full verification code
quite quickly, but it didn't matter much because nobody uses bitcoinj for
mining. The docs explicitly tell people, in fact, not to mine on top of
bitcoinj:

https://bitcoinj.github.io/full-verification

So no, bitcoinj+P2SH was irrelevant from a fork type perspective. It just
had no effect at all. This entire section of your message is completely
wrong.

The code that did take longer was for wallet support. And the reason it
came later was resource prioritisation: there were more important issues to
resolve. Like I said - write the amount of code I've written, unpaid in
your evenings and weekends, and then you can criticise bitcoinj for lacking
features.

75% is a fine activation threshold. By definition if support is at 75% then
bigger blocks is "winning", but if support fell, then the SPV wallets would
just reorg back onto the 1mb-blocks chain.

Re: demonstrated track record. They "work" only if you ignore the actual
problems that have resulted. P2SH-invalid blocks were being mined for weeks
after the flag day. That's not good no matter how you slice it: even if you
didn't hear about any fraud resulting, it is still risk that can be avoided.

[-- Attachment #2: Type: text/html, Size: 5525 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Gregory Maxwell]

On Wed, Sep 30, 2015 at 9:01 PM, Mike Hearn <hearn@vinumeris.com> wrote:
> I coined the term SPV so I know exactly what it means, and bitcoinj

The term comes from the Bitcoin whitepaper.

> On the other hand, full nodes all claim they run scripts. Users expect that
> and may be relying on it. The unstated assumption here is that the nodes run
> them correctly. A soft fork breaks this assumption.

They run them correctly with respect to the rules that they implement,
nothing about a soft-fork changes that.

The system could have been designed in a way that wasn't full of nice
compatibility features. The history of bitcoin could have been that
past improvements were all performed with hard forks instead of soft
forks.

But neither of these things are true. (And I think it's very likely
that there would have been fewer improvements if that were the case).

> I'm going to ignore the rest of the stuff you wrote  about "design decisions
> to lack security" or "cheaply avoidable lack of validation". When you have
> sat down and written an SPV implementation by yourself, then shipped it to a
> couple of million users, you might have better insight into basic
> engineering costs.

At the end of the day we need to deliver software to our users that
delivers on their needs and doesn't undermine their privacy or
security;  even if its really hard.  So when someone calls out
something that I'm working on that could use improvement, my response
isn't to tell them how much I'm not going to listen to them because
I've accomplished some long list of things that they haven't; that
software I've written runs on hundreds of millions of devices; ...
rather my response is to hear out their concerns, even when due to
extensive context I'm confident that they are probably confused or
dishonestly motivated; because there is always a potential to learn,
and always a potential to do better.  I have found this to be pretty
productive, as even when both parties walk away with the same
positions they started with, I usually learn something along the way
just because I paid attention.

> Until then, I find your criticisms of code you think was
> missing due to "stonewalling" and so on to be seriously lacking real world
> experience.

BIP16 was published on 2012-01-01. Enforcement on the network began on
April 1st 2012.

Support for merely sending to P2SH addresses was merged into BitcoinJ,
Nov 30th _2013_, after it was written by Mike Belshe.

In the interim you spent considerable time arguing against
implementing it, e.g. in one example incident:

--- Day changed Thu Sep 12 2013
10:03 < TD> heck, if a recipient really really wants to receive a p2sh
payment for some reason, they can just put the p2sh output into the
payment request message
....
10:17 <@gmaxwell> TD: In any case, P2SH has been deployed for
something like two years now. Your arguement seems to be basically we
should be creating a false tying between payment protocol messages and
things like escrow usage in order to coerce people to adopt the
payment protocol in places where an address would do.
10:17 <@gmaxwell> I think thats a bit cruddy.
10:19 < TD> no, i'm saying p2sh is a feature that just isn't usable. i
did point this out at the time it was merged - gavin believed that
apps to do complicated multi-device wallets would appear before a
payment protocol did, and people wouldn't like the look of long
addresses. that was pretty much the rationale given. that didn't
happen, obviously
10:20 < TD> developing features that are used only by bitcoind
developers, isn't the right way to go, and p2sh definitely falls into
that category
...
10:35 < TD> i'm not going to do it myself because anyone who is
capable of producing and running something that uses p2sh is capable
of working with the payment protocol as well, and that gives a better
user experience overall. but if someone else wants to, go for it.

I don't think there is anything wrong about calling this stonewalling.
At least thats how it came across to myself and others. I'm sorry if I
judged too harshly there.

To be clear, by pointing out your past opposition and non-deployment
in this message I am _not_ trying to attacking you for failing to
support P2SH.

I pointing out that right or wrong..... That you actively argued
against it. That you chose not to implement it, and only accepted a
patch for it a year and a half later. From your own words it seems
clear that you didn't implement it due to actual opposition, but even
if the non-implementation was simply engineering priorities, the fact
remains that you didn't implement for a very long time.

And that is _okay_, we still got it anyways, and today tens of
thousands of transactions per day use it and P2SH secures about 10% of
all Bitcoin value.  This is possible because with a soft fork users
using other software can gain functionality which might be critical to
them (As Jgarzik was saying about Bitpay in the discussion I was
quoting from) that you don't have the time or interest to implement in
your own software.


> What? This is nonsensical. P2SH was added to the full verification code
> quite quickly,

Yes, Matt Corallo added it to code which by your admission no one was
using.  I agree this is not relevant.

> resolve. Like I said - write the amount of code I've written, unpaid in your
> evenings and weekends, and then you can criticise bitcoinj for lacking
> features.

I think it's likely that I've spent significant more time unpaid on my
evenings and weekends creating software for others than you have (and
continue to do so; as well as having donated years worth of income
supporting other people's Free Software work), but it's a bit of an
unfair comparison: I'm a fair bit older than you. :)

And yet, I think that is all irrelevant here because I'm not current
criticizing bitcoinj for lacking features!   Quite the opposite, I am
pointing out that the advantage of soft-forks is that its OKAY for
software to lack soft-fork features,  which means that participants
who code only on evenings and weeks are free to continue participating
with the priorities they choose!

Forcing _all_ upgrades to be via hard-fork takes away the freedom to
make that trade-off; and concurrently reduces the collection of fixes
upgrades we could potentially deploy; because will always be
implementations out there like BitcoinJ in 2012 that didn't have the
resources (or interest) to fully implement this feature or that
feature, at least not right away.

And for many things, they simply don't have to, and that should be okay.

> Yes, a hypothetical full node could fork on the version bits. I would be
> quite happy with the version number in the header being an enforced
> consensus rule: it'd make hard forks easier to trigger. But it hasn't been
> done that way, and wishing away the behaviour of existing software in the
> field is no good. Luckily, for introducing a new opcode, the same effect can
> be achieved by using a non-allocated opcode number.

We handle this in Bitcoin Core. Our chosen and intentional way to
handle this is setting a notice. This gives users the freedom to do
what they like, while also behaving in a reasonably sane way by
default.

You don't have to like it, you can behave differently in your own
software or on your own hosts-- all the data is available to you.

(I wouldn't object out of principle to a default config option to take
more aggressive action on unexpected versions... but no one has ever
asked for one... and I'm doubtful anyone would ever do so.)

> The rest of your argument boils down to "people don't have to upgrade if
> they don't want to", which is addressed in the article I wrote already, and
> multiple responses on this thread. Yes, they do, otherwise they aren't
> getting the security level they were before.

They continue to enforce all the same rules as before. With the soft
fork Bitcoin Core users are informed that unexpected things are going
on, and they are free to look at whats going on and decide how to
handle it, or just accept that the new thing is almost certainly
something they don't care about (after all, the rules they signed up
for before are all still in effect, and at any time miners could be
silently imposing new 'soft fork' like rules without their knowledge--
having a big reaction to ones the network was kind enough to tell them
about doesn't seem that reasonable).

For many users and many soft-forks there is no substantial security
implication, and you cannot say that they were not getting the
security level they were getting before. But regardless, even what it
is different, they're free to decide on the cost tradeoff with
upgrading, and they're not forced onto an upgrade hamsterwheel that
disenfranchises their role in the system.

If you have a specific generalized security implication in mind,
you're failing to state it. In your writings you like to assert that I
 "did not respond" or were "not convincing"-- that is not generally my
style, I don't usually think anyone owes me point by point answers,
but I think on this point it seems clear that there is some
implication which is in your head that is a mystery to at least myself
and Jgarzik.

> 75% is a fine activation threshold. By definition if support is at 75% then
> bigger blocks is "winning", but if support fell, then the SPV wallets would
> just reorg back onto the 1mb-blocks chain.

A 75% measurement doesn't actually mean 75% support, due to variance.
Even ignoring that-- you recognize the acceptability of reorgs. The
situation is no worse for an SPV client for a soft-fork; and it's
better because (1) convergence is still guaranteed with exponential
probability (a hard fork can be mutual and no convergence may be
possible-- as is the case for more conceivable hard forks), and (2)
for BIP65 (and current soft forks generally) a _much_ more
conservative threshold is set (because in Bitcoin Core and the general
community around here considers 75% to be too low to achieve high
stability, based on our past experience).

> Re: demonstrated track record. They "work" only if you ignore the actual
> problems that have resulted. P2SH-invalid blocks were being mined for weeks
> after the flag day. That's not good no matter how you slice it: even if you
> didn't hear about any fraud resulting, it is still risk that can be avoided.

A couple points:

That same invalid blocks for weeks (actually months) from BIP16 is the
behavior you will get with a hard fork, for at least the same reasons
(miners asleep at the switch). Much more for a controversial hard-fork
as there will be principled objections.

Blocks get produced that get orphaned every day and this is
unavoidable, so users already must deal with occasional cases where
confirmations get undone.

More recent soft-forks have reduced the incidence of invalid blocks by
substantially increasing the threshold, including better notification
in Bitcoin core, communicating directly with miners more, and making
non-conforming transactions non-standard in advance. These mitigations
have been effective in practice; and we have not seen the same
behavior (which, as, noted is not known to have enabled any fraud in
any case -- in part because to non-upgraded wallets it looks just like
the orphaning that normally happens but with somewhat increased
frequency.).  I think it's unfortunate that people proposing hard
forks have not learned the same lessons, even though the stakes are
higher and the self-resolution of the system is greatly diminished.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Anthony Towns]

On Tue, Sep 29, 2015 at 06:31:28PM +0000, Gregory Maxwell via bitcoin-dev wrote:
> On Mon, Sep 28, 2015 at 10:48 AM, Mike Hearn via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
> > There is no consensus on using a soft fork to deploy this feature. It will
> > result in the same problems as all the other soft forks - SPV wallets will
> > become less reliable during the rollout period. I am against that, as it's
> > entirely avoidable.
> > Make it a hard fork and my objection will be dropped.
> I'm surprised to see this response-- [...]
> I am having a little difficulty making sense of this complaint. [...]

I think I finally understand this objection.

For a hard fork, activated by a majority of nodes/hashpower upgrading
to a new bitcoin release, the behaviour is:

 - upgraded bitcoin nodes: everything works fine

 - non-upgraded bitcoin nodes: total breakage. there will be a push
   alert telling you to upgrade. anyone who doesn't will think they're
   tracking "bitcoin" but will actually be tracking a new "bitcoin-old"
   altcoin. most non-upgraded miners will presumably realise they're
   wasting hashpower and stop doing this pretty quick; and remaining
   miners will only create blocks very slowly due to sudden reduced
   hashpower, without possibility of difficulty adjustment. users who
   don't uprade will try to do transactions, but won't see them confirm
   for hours or days due to lack of hashpower.

 - SPV nodes: they track the upgraded majority, everything works fine
   even if they don't upgrade

For a soft fork, again activated by the majority of upgraded hashpower,
the behaviour is:

 - upgraded bitcoin nodes: everything works fine

 - non-upgraded bitcoin miners willing to mine newly unacceptable txs:
   may produce orphaned blocks; may be able to be forced into producing
   blocks that will be orphaned

 - other non-upgraded bitcoin nodes: everything works fine

 - SPV nodes: partial breakage -- may track invalid blocks for 1-2
   confirmations until the set of "non-upgraded bitcoin miners willing
   to produce newly unacceptable txs" becomes vanishingly few.

In the hard fork case, all non-upgraded nodes get a DoS attack, but
aren't likey to be hit by doublespends. That's inconvenient, but it's
not too bad.

In the soft fork case, if there's likely to be old nodes mining
previously invalid transactions, SPV clients become very unreliable,
to the point of possibly seeing semi-regular double-spends with 1 or
2 confirmation, until miners that aren't paying attention notice their
blocks are getting orphaned and upgrade. That is pretty bad IMHO; and
there are a lot more *people* running SPV clients than bitcoin nodes,
so its impact is potentially worse in both ways.

Comparing generic hard forks versus generic soft forks, the above says
to me that a hard fork would be less harmful to users in general, and
thus a better approach.

*But* a soft fork that only forbids transactions that would previously
not have been mined anyway should be the best of both worlds, as it
automatically reduces the liklihood of old miners building newly invalid
blocks to a vanishingly small probability; which means that upgraded
bitcoin nodes, non-upgraded bitcoin nodes, /and/ SPV clients *all*
continuing to work fine during the upgrade.

AFAICS, that's what BIP65 achieves, as will similar OP_NOP* replacements
like BIP112.

But that only applies to a subset of potential soft forks, not every
soft fork.

Maybe a good way to think about it is something like this.  Consensus
(IsValid) is always less restrictive than (default) policy (previously
IsStandard, not sure how to summarise it now, maybe it's just OP_NOP
redefinition?).  So choosing a new consensus rule will be one of:

  * even less restrictive than consensus (hard fork)

  * more restrictive than consensus, but less restrictive than policy
    (safe soft fork)

  * more restrictive than IsStandard etc (damaging soft fork)

Hmm, in particular, following this line of thinking it's not clear to
me that BIP68 is actually less restrictive than current policy? At
least, I can't see anything that prevents txs with nSequence set to
something other than 0 or ~0 from being relayed?

If it's not, and nodes currently happily mine and relay transactions
with nSequence set without caring what it's set to, doesn't this mean
BIP68 is of the "damaging soft fork" variety? That is, if it activated
as a soft-fork with a majority of miners using it, but a minority of ~5%
not upgraded, then

 - someone could construct an tx with nSequence set to sometime in
   the future, but not using OP_CSV

 - this tx would get relayed by old nodes (but not upgraded nodes
   due to CheckLockTime)

 - non-upgraded miners would mine it into a block immediately, which
   would then get orphaned by majority hashpower

 - before it got orphaned, non-upgraded nodes and SPV clients would
   be misled and vulnerable to double spend attacks of txs with 0, 1 or
   maybe 2 confirmations

(BIP65 with OP_CLTV and BIP112 with OP_CSV don't have that problem as
they both redefine a non-standard opcode and would not get relayed or
mined by old, non-upgraded nodes, and are thus "safe soft forks" per
above terminology. This is just BIP68)

Can anyone confirm or refute the above?

Cheers,
aj

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jonathan Toomim (Toomim Bros)]

[-- Attachment #1.1: Type: text/plain, Size: 2167 bytes --]


On Oct 7, 2015, at 8:00 AM, Anthony Towns via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

> *But* a soft fork that only forbids transactions that would previously
> not have been mined anyway should be the best of both worlds, as it
> automatically reduces the liklihood of old miners building newly invalid
> blocks to a vanishingly small probability; which means that upgraded
> bitcoin nodes, non-upgraded bitcoin nodes, /and/ SPV clients *all*
> continuing to work fine during the upgrade.

I agree with pretty much everything you wrote except the above paragraph.

An attacker can create a transaction that would be valid if it were an OP_NOP, but not valid if it were any more restrictive transaction. For example, an attacker might send 1 BTC to an address with  . An old node would consider that OP_CLTV to be OP_NOP, so no signature is necessary for old nodes. Then the attacker buys something from a merchant running old node code or an SPV client, and spends the 1 BTC in that address in a way that is invalid according to OP_CLTV but valid according to OP_NOP, and includes a hefty fee. A miner on the old version includes this transaction into a block, thereby making the block invalid according to the new rules, and rejected by new-client miners. The merchant sees the 1-conf, and maybe even 2-conf, rejoices, and ships. The attacker then has until the OP_CLTV matures to double-spend the coin with new nodes using a valid signature.

Basically, it's trivial to create transactions that exploit the difference in validation rules as long as miners are still on the old version to mine them. Transactions can be created that are guaranteed to be orphaned and trivially double-spendable. Attackers never have to risk actual losses. This can be done as long as miners continue to mine old-version blocks, regardless of their frequency.

Those of you who know Script better than me: would this be an example of a transaction that would be spendable with a valid sig XOR with (far future date OR old code)?

OP_DUP OP_HASH160 <pubkeyhash> OP_EQUALVERIFY OP_CHECKSIGVERIFY OP_PUSHDATA <locktime far in the future> OP_CLTV

[-- Attachment #1.2: Type: text/html, Size: 6781 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Eric Lombrozo]

[-- Attachment #1: Type: text/plain, Size: 3149 bytes --]

That's why it's important to measure miner adoptance. Note that this isn't a vote - it's an adoption metric for what is presumably a fairly uncontroversial upgrade. If there's contentious controversy amongst miner all bets are off.

Our current mechanisms are imperfect in this regard...as we've seen in the past, miners have deliberately disabled checks despite signaling adoption in their blocks. But a real hashpower supermajority would make such attacks hard to pull off in practice.

- Eric

On October 7, 2015 8:46:08 AM PDT, "Jonathan Toomim (Toomim Bros) via bitcoin-dev" <bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>On Oct 7, 2015, at 8:00 AM, Anthony Towns via bitcoin-dev
><bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> *But* a soft fork that only forbids transactions that would
>previously
>> not have been mined anyway should be the best of both worlds, as it
>> automatically reduces the liklihood of old miners building newly
>invalid
>> blocks to a vanishingly small probability; which means that upgraded
>> bitcoin nodes, non-upgraded bitcoin nodes, /and/ SPV clients *all*
>> continuing to work fine during the upgrade.
>
>I agree with pretty much everything you wrote except the above
>paragraph.
>
>An attacker can create a transaction that would be valid if it were an
>OP_NOP, but not valid if it were any more restrictive transaction. For
>example, an attacker might send 1 BTC to an address with  . An old node
>would consider that OP_CLTV to be OP_NOP, so no signature is necessary
>for old nodes. Then the attacker buys something from a merchant running
>old node code or an SPV client, and spends the 1 BTC in that address in
>a way that is invalid according to OP_CLTV but valid according to
>OP_NOP, and includes a hefty fee. A miner on the old version includes
>this transaction into a block, thereby making the block invalid
>according to the new rules, and rejected by new-client miners. The
>merchant sees the 1-conf, and maybe even 2-conf, rejoices, and ships.
>The attacker then has until the OP_CLTV matures to double-spend the
>coin with new nodes using a valid signature.
>
>Basically, it's trivial to create transactions that exploit the
>difference in validation rules as long as miners are still on the old
>version to mine them. Transactions can be created that are guaranteed
>to be orphaned and trivially double-spendable. Attackers never have to
>risk actual losses. This can be done as long as miners continue to mine
>old-version blocks, regardless of their frequency.
>
>Those of you who know Script better than me: would this be an example
>of a transaction that would be spendable with a valid sig XOR with (far
>future date OR old code)?
>
>OP_DUP OP_HASH160 <pubkeyhash> OP_EQUALVERIFY OP_CHECKSIGVERIFY
>OP_PUSHDATA <locktime far in the future> OP_CLTV
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 8100 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Eric Lombrozo]

[-- Attachment #1: Type: text/plain, Size: 4141 bytes --]

You're right about the potential for 1 bad confirmation even with very low frequency...but with an overwhelming supermajority of hashpower, 2 bad confirmations become quite unlikely, n bad confirmations becomes exponentially unlikely in n.

As part of such soft fork deployments, it's true that old nodes might see a bad confirmation on occasion (even assuming overwhelming supermajority hashpower adoptance). So yes, old nodes and SPV clients should probably require more confirmations right around such a transition...or should upgrade. It is entirely possible to make clients warn the user if the block version is unrecognized, which will help to prevent anyone from accepting bad blocks (although SPV security necessarily relies on miners to validate for them).

On October 7, 2015 9:02:14 AM PDT, Eric Lombrozo <elombrozo@gmail.com> wrote:
>That's why it's important to measure miner adoptance. Note that this
>isn't a vote - it's an adoption metric for what is presumably a fairly
>uncontroversial upgrade. If there's contentious controversy amongst
>miner all bets are off.
>
>Our current mechanisms are imperfect in this regard...as we've seen in
>the past, miners have deliberately disabled checks despite signaling
>adoption in their blocks. But a real hashpower supermajority would make
>such attacks hard to pull off in practice.
>
>- Eric
>
>On October 7, 2015 8:46:08 AM PDT, "Jonathan Toomim (Toomim Bros) via
>bitcoin-dev" <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>On Oct 7, 2015, at 8:00 AM, Anthony Towns via bitcoin-dev
>><bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>>> *But* a soft fork that only forbids transactions that would
>>previously
>>> not have been mined anyway should be the best of both worlds, as it
>>> automatically reduces the liklihood of old miners building newly
>>invalid
>>> blocks to a vanishingly small probability; which means that upgraded
>>> bitcoin nodes, non-upgraded bitcoin nodes, /and/ SPV clients *all*
>>> continuing to work fine during the upgrade.
>>
>>I agree with pretty much everything you wrote except the above
>>paragraph.
>>
>>An attacker can create a transaction that would be valid if it were an
>>OP_NOP, but not valid if it were any more restrictive transaction. For
>>example, an attacker might send 1 BTC to an address with  . An old
>node
>>would consider that OP_CLTV to be OP_NOP, so no signature is necessary
>>for old nodes. Then the attacker buys something from a merchant
>running
>>old node code or an SPV client, and spends the 1 BTC in that address
>in
>>a way that is invalid according to OP_CLTV but valid according to
>>OP_NOP, and includes a hefty fee. A miner on the old version includes
>>this transaction into a block, thereby making the block invalid
>>according to the new rules, and rejected by new-client miners. The
>>merchant sees the 1-conf, and maybe even 2-conf, rejoices, and ships.
>>The attacker then has until the OP_CLTV matures to double-spend the
>>coin with new nodes using a valid signature.
>>
>>Basically, it's trivial to create transactions that exploit the
>>difference in validation rules as long as miners are still on the old
>>version to mine them. Transactions can be created that are guaranteed
>>to be orphaned and trivially double-spendable. Attackers never have to
>>risk actual losses. This can be done as long as miners continue to
>mine
>>old-version blocks, regardless of their frequency.
>>
>>Those of you who know Script better than me: would this be an example
>>of a transaction that would be spendable with a valid sig XOR with
>(far
>>future date OR old code)?
>>
>>OP_DUP OP_HASH160 <pubkeyhash> OP_EQUALVERIFY OP_CHECKSIGVERIFY
>>OP_PUSHDATA <locktime far in the future> OP_CLTV
>>
>>
>>------------------------------------------------------------------------
>>
>>_______________________________________________
>>bitcoin-dev mailing list
>>bitcoin-dev@lists.linuxfoundation.org
>>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>-- 
>Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 9164 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jonathan Toomim (Toomim Bros)]

[-- Attachment #1.1: Type: text/plain, Size: 472 bytes --]



On Oct 7, 2015, at 9:02 AM, Eric Lombrozo <elombrozo@gmail.com> wrote:

> But a real hashpower supermajority would make such attacks hard to pull off in practice.

If you had a 99% hashpower supermajority on the new version, an attacker would still be able to perform this attack once per day. Since the attacker is creating a transaction which is invalid according to new clients, it will just sit around in old clients' mempool until one of them mines a block.

[-- Attachment #1.2: Type: text/html, Size: 1215 bytes --]

[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 496 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Anthony Towns]

On Wed, Oct 07, 2015 at 08:46:08AM -0700, Jonathan Toomim (Toomim Bros) via bitcoin-dev wrote:
> On Oct 7, 2015, at 8:00 AM, Anthony Towns via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
> > *But* a soft fork that only forbids transactions that would previously
> > not have been mined anyway should be the best of both worlds, ...
> I agree with pretty much everything you wrote except the above paragraph.
> An attacker can create a transaction that [...] A miner on the old version 
> includes this transaction into a block, [...]

The point of that case is that there aren't such miners, so that exploit
doesn't apply.

In particular, AIUI, you'll have a hard job right now finding someone to
mine an OP_NOP2 transaction -- eligius might do it, but I don't think many
others will. And you also need your currently OP_NOP2-friendly miner not
to upgrade to an OP_CLTV-validating codebase, so I don't think eligius
will qualify there.

> Those of you who know Script better than me: would this be an example of a transaction that would be spendable with a valid sig XOR with (far future date OR old code)?
> 
> OP_DUP OP_HASH160 <pubkeyhash> OP_EQUALVERIFY OP_CHECKSIGVERIFY OP_PUSHDATA <locktime far in the future> OP_CLTV

If you want XOR, you'd need something more like:

 OP_IF OP_DUP OP_HASH160 <pubkeyhash> OP_EQUALVERIFY OP_CHECKSIGVERIFY
 OP_ELSE <locktime> OP_CLTV
 OP_ENDIF

But that' still fail IsStandard and DISCOURAGE_UPGRADABLE_NOPS checks
if you tried spending without a valid sig, so wouldn't be mined by
current nodes. (Not having a sig would also allow anyone to spend it to
themselves, so that might make it hard to use as a basis for double
spends anyway...)

Cheers,
aj

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Anthony Towns]

On Thu, Oct 08, 2015 at 01:00:14AM +1000, Anthony Towns via bitcoin-dev wrote:
> *But* a soft fork that only forbids transactions that would previously
> not have been mined anyway should be the best of both worlds, [...]

>   * more restrictive than consensus, but less restrictive than policy
>     (safe soft fork)

> Hmm, in particular, following this line of thinking it's not clear to
> me that BIP68 is actually less restrictive than current policy?

As was discussed on the weekly meeting [0], turns out it *is* less
restrictive than current policy. IsStandardTx currently returns a failure
if the tx version is greater than 1, and per BIP68, nSequence will only
be inforced with tx version of 2 or greater.

So afaics, BIP 65 (OP_CLTV), BIP 68 (nSequence) and BIP 112 (OP_CSV)
are all "safe soft forks", and if activated won't cause SPV nodes to
see a significant uptick in reorgs, double-spends etc. (They'll still
be vulnerable to people deliberately spending hashpower to mine invalid
blocks, but that's a problem at any point, independent of whether a
soft-fork is underway)

[0] http://www.erisian.com.au/meetbot/bitcoin-dev/2015/bitcoin-dev.2015-10-08-18.59.log.html#l-312

Cheers,
aj

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[digitsu]

[-- Attachment #1: Type: text/plain, Size: 1748 bytes --]

Thanks for that great breakdown Anthony. I think it helps a lot of us get a better handle on the matter without getting too technical. 


A couple of questions on some of the points you made I'd like to put out there:




First I think your unsaid assumption about the fragility of a soft fork showing incorrect confirmations is dependent on the percentage of hash power that didn't upgrade.  If using your same numbers this was only 5% of the hash power, the attack is effectively not effective (u less the attacker knew an exact merchant that was unfortunately on the minority of the network. 




-- snip --


 - non-upgraded bitcoin nodes: total breakage. there will be a push

   alert telling you to upgrade. anyone who doesn't will think they're

   tracking "bitcoin" but will actually be tracking a new "bitcoin-old"

   altcoin. most non-upgraded miners will presumably realise they're

   wasting hashpower and stop doing this pretty quick; and remaining

   miners will only create blocks very slowly due to sudden reduced

   hashpower, without possibility of difficulty adjustment. 

----




Is this true? I thought that un-upgraded nodes would just dump the new blocks from the upgraded miner majority as invalid. This how would they even know (besides the PSA) that they were on the wrong side? 




----snip---

users who

   don't uprade will try to do transactions, but won't see them confirm

   for hours or days due to lack of hashpower.






----




But only for txns for users who are using the new OP code right? Regular transactions will get relayed by both upgraded and in-upgraded nodes and miners alike. 








—
Regards,

[-- Attachment #2: Type: text/html, Size: 2507 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Anthony Towns]

On Mon, Oct 12, 2015 at 12:02:51AM -0700, digitsu412 via bitcoin-dev wrote:
> Thanks for that great breakdown Anthony. I think it helps a lot of us
> get a better handle on the matter without getting too technical.

Glad you found it useful; there's a lot of subtleties in how this
stuff works, and I'm certainly still figuring it out.

> A couple of questions on some of the points you made I'd like to put
> out there:
>
> First I think your unsaid assumption about the fragility of a soft
> fork showing incorrect confirmations is dependent on the percentage
> of hash power that didn't upgrade.  If using your same numbers this
> was only 5% of the hash power, the attack is effectively not effective
> (u less the attacker knew an exact merchant that was unfortunately on
> the minority of the network. 

Absolutely. So I think there are three scenarios where SPV clients get
affected by orphans, where full nodes wouldn't be worried:

 1. Independent of any soft/hard fork, someone just mines a completely
    invalid block containing invalid transactions (eg, spending a
    non-existant transaction) based of the best current valid block,
    and presents it to SPV clients.

    An SPV client will accept this as valid until it's orphaned by the
    rest of the network building two valid blocks.

    This is expensive since you could have mined a valid block and
    got 25+ bitcoin legitimately, and it also doesn't last very long
    (around 20m on average, but less if you're unlucky), and the timing
    is unpredictable. It also won't get relayed by nodes, so you have
    to do a Sybil attack against the SPV client as well. It's also only
    good for places that accept 1-confirmation transactions; so you're
    probably better off finding somewhere that accepts 0-confirmation
    transactions and doing a Finney attack, where you at least get to keep
    the 25+ bitcoin from the subsidy/fees.

    So it's possible in theory, but seems pretty unlikely to be worth
    the hassle in practice.


 2. There's a "damaging soft-fork", with lock-in occurring immediately
    after 95% of blocks claim to support the soft-fork.

    In this case upgraded miners will only build on blocks from other
    upgraded miners, but non-upgraded miners will build on blocks from
    upgraded miners or non-upgraded miners. But with a ratio of 95:5,
    upgraded miners will tend to find a block every 10.5 minutes, while
    non-upgraded miners will only find one every 3h20m. SPV clients will
    accept the latest block from whichever miner most recently found a
    block, upgraded or not.

    If one of the blocks mined by a non-upgraded node includes a
    transaction that was valid under the old rules, but is not under the
    new rules (which is entirely likely in the damaging soft-fork case),
    that transaction will be trivially vulnerable to double-spends. So
    until the remaining 5% of hashpower upgrades, SPV clients will
    easily be able to be spoofed roughly once every 3h20m (when a
    non-upgraded miner finds a block) for about 20m (until two upgraded
    blocks are found, and the non-upgraded block is orphaned).

    The cost of this attack is borne entirely by the non-upgraded
    miners, who are mining blocks that will always be orphaned -- ie,
    they're still spending electricity on maintaining 5% of hashpower,
    but their "blocks" are all orphaned so they're getting a return of
    0 BTC per day, instead of roughly 180 BTC per day. So presumably
    they won't continue wasting power too long, and this will only be a
    problem for a few days at most, but during that time SPV clients are
    quite vulnerable.

    Also, that assumes that the soft-fork activates immediately on
    hitting 95%. The versionbits proposal will add two weeks' delay
    between seeing 95% of nodes supporting the soft-fork, and enforcing
    the rules, which gives the 5% additional time to upgrade, which
    probably means there won't be much of a window left at all.


 3. AIUI, without versionbits soft-forks are done by bumping the
    block nVersion field (ie, from "2" currently to "3"); then enforcing
    the new rules on blocks with a bumped version; and finally orphaning
    blocks with the old version once 95% of the last 1000 blocks use
    the bumped version. (See BIP 34)

    This means that even with a "safe soft-fork", non-upgraded miners
    will have their blocks orphaned by upgraded miners immediately after
    the soft-fork is activated, and SPV clients will see a similar orphan
    rate (up to 1 in 20 blocks, seeing an block that will get orphaned
    for about ~20 minutes every 3h20m).

    However, in the "safe soft-fork" case, all the transactions included
    in the invalid blocks will also be acceptable to upgraded miners,
    and will likely be included in the replacement blocks anyway. So
    this should be an annoyance, rather than an actual problem. (In
    a safe soft-fork, any transaction you attempt to get only mined
    by non-upgraded nodes will be picked up by upgraded nodes anyway;
    and any transaction you attempt to get mined only be new nodes will
    be mined very quickly so non-upgraded nodes won't have a chance to
    mine a block that double-spends)

    Further while I don't think they actually do it currently, SPV
    clients *could* monitor the block version (they download it anyway),
    and simply decline to accept new version "2" blocks once version "3"
    is seen for 95% of the last 1000 blocks. Then they would not see any
    blocks that will get orphaned, in either safe or damaging soft-forks.

    They couldn't do something similar with versionbits in place, since
    the corresponding bit is cleared once the soft-fork activates. However
    this also means that with versionbits in use, upgraded miners have
    no reason to orphan blocks built by non-upgraded miners; so this
    isn't a problem in the first place...

> -- snip --
>  - non-upgraded bitcoin nodes: total breakage. there will be a push
>    alert telling you to upgrade. anyone who doesn't will think they're
>    tracking "bitcoin" but will actually be tracking a new "bitcoin-old"
>    altcoin.

This might have been confusing. A hard fork creates two separate
blockchains both starting from the genesis block. The old one that
obeys the old rules, call it "bitcoin-old" and the new one obeying the
new rules, call it "bitcoin-new". The first block making use of the new
features will be unacceptablee on "bitcoin-old", and will be the point
of divergence. At that point, with 95% of hashpower on bitcoin-new,
it will see new blocks every 10.5 minutes, while with 5% of hashpower
bitcoin-old will only see new blocks every 3h20m. (With 75% hashpower,
bitcoin-new would see new blocks every 13m20s, while bitcoin-old would
see new blocks every 40m)

I'm assuming that as far as almost everyone is concerned, the blockchain
with the most hashpower (bitcoin-new in this case) would be called
"bitcoin", but I'm sure people would argue over it.

>             most non-upgraded miners will presumably realise they're
>    wasting hashpower and stop doing this pretty quick; and remaining
>    miners will only create blocks very slowly due to sudden reduced
>    hashpower, without possibility of difficulty adjustment. 
> ----
> Is this true? I thought that un-upgraded nodes would just dump the new
> blocks from the upgraded miner majority as invalid. This how would they
> even know (besides the PSA) that they were on the wrong side? 

Since a majority of hashpower switched to a different chain, anyone
running non-upgraded nodes after a hard fork was activated would see far
fewer blocks being found (ie, with 5% of hashpower, that would be every 3
hours, rather than every 10 minutes). This would resolve itself when the
difficulty next reset, but that would be after 2016 blocks, which at 3h20m
per block would take about 9 months rather than the standard 2 weeks.
(With 25% of hashpower, bitcoin-old would see new blocks every 40
minutes, and difficulty would be reset after about 8 weeks)

Until the difficulty reset, if they were mining, they'd also see more
blocks being found by them (eg, if they had 2% of hashpower previously,
instead of finding 2% of blocks, they'd now be finding 40% of blocks,
ie 2/5 instead of 2/100).

Because a hard fork doesn't invalidate any transactions, non-upgraded
nodes would still see almost all the transactions intended for bitcoin-new
(excepting those from miners working on the fork or that explicitly use
new features enabled by the fork) so the mempool would grow pretty large,
given the low rate at which blocks are mined. (If the new feature is
something like a bigger blocksize, which then increases the rate of
transactions in bitcoin-new, then that's even worse for bitcoin-old
nodes!)

And, of course, those numbers get worse if the 5% of hashpower mining
bitcoin-old reduces as miners write it off as not-profitable.

(Maybe full bitcoin nodes should emit a warning that you've probably
been hard-forked off the main chain if they see, say, 4 or fewer blocks
in 5 hours -- with normal hashpower, I think that should only happen
once in something like 6000 years, but if you've got less than about
13% of hashpower left on your chain will happen about 50% of the time.
I don't know if that would actually be helpful compared to a pushed GPG
signed announcement of the hard-fork though)

> ----snip---
>    users who
>    don't uprade will try to do transactions, but won't see them confirm
>    for hours or days due to lack of hashpower.
>
> But only for txns for users who are using the new OP code right? Regular
> transactions will get relayed by both upgraded and in-upgraded nodes
> and miners alike. 

Hmm. Depends what you mean by "users". If the user is running an
SPV wallet, they'll be following the most hashpower and will see
bitcoin-new. With 12 random connections (bitcoinj's default), I think
you'd just need ~1350 of 6000 nodes to have upgraded to have a 95%
chance of seeing bitcoin-new somewhere. So I don't see SPV users having
any problems.

But in the quote above I was talking about users who are running bitcoin
core rather than an SPV client. *They* would have the same problems
mentioned above, because non-upgraded bitcoin core would just totally
ignore the bitcoin-new blockchain.

So if they published a transaction, it would get confirmed quickly on
bitcoin-new, sure, but they wouldn't see that confirmation because their
software is deliberately ignoring bitcoin-new. They'd instead see it as
unconfirmed until it was included in a bitcoin-old block, but those only
come every 40m (25% hashpower) or every 3h20m (5% hashpower).

Worse, most of the bitcoin-new transactions are still valid for
bitcoin-old, so those transactions might get included by the remaining
miners in bitcoin-old -- so you'd have to pay higher fees to get
confirmed in 3h40m in bitcoin-old than you would to get confirmed in
13m in bitcoin-new...


TL;DR: I think my conclusions are:

 - gads this stuff is complicated

 - "safe soft-forks" really are safe (this covers BIP 65 (OP_CLTV) and
   BIPs 68 and 112 (OP_CSV))

 - as currently proposed, versionbits will actually make "damaging
   soft-forks" pretty safe too

 - if there's a hard fork in the wind, and you're running a full node,
   make sure you're on the latest version. maybe run an SPV client as well
   and check you're getting the same answers from both, just to be safe.

Cheers,
aj

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Anthony Towns]

On Mon, Oct 12, 2015 at 12:02:51AM -0700, digitsu412 via bitcoin-dev wrote:
> First I think your unsaid assumption about the fragility of a soft
> fork showing incorrect confirmations is dependent on the percentage
> of hash power that didn't upgrade.  If using your same numbers this
> was only 5% of the hash power, the attack is effectively not effective
> (u less the attacker knew an exact merchant that was unfortunately on
> the minority of the network. 

Actually, just to take this scenario more explicitly...

Say you've got 5% of hashpower running on old software, along with,
say, 1500 nodes; and meanwhile you've got 95% of hashpower running new
software, along with 4000 nodes.

There's still about 750 nodes running 0.9 or 0.8 of 5400 total according
to bitnodes.21.co/nodes, so those numbers seems at least plausible to
me for the first week or two after a soft-fork is activated.

Eventually an old-rules block gets found by the 5% hashpower. The 4000
new nodes and 95% of hashpower ignore it, of course. With 8 random
connections, old nodes should have 92% chance of seeing an old node
as a peer, so I think around ~1300 of them should still be a connected
subgraph, and the old-rules block should get propogated amongst them
(until two new-rules blocks come along and orphan it).

An SPV client with 12 random connections here has 96% chance of having one
of the ~1300 old nodes as a peer, and if so, will see the old-rules block,
that will be orphaned, and may be at risk from double-spends as a result.

So I think even with just 5% hashpower and ~30% of nodes left running
the old version, a "damaging soft fork" still poses a fairly high risk to
someone receiving payments via an SPV client, and trusting transactions
with few confirmations.

Cheers,
aj

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[digitsu]

[-- Attachment #1: Type: text/plain, Size: 3272 bytes --]

Thanks AJ,




That is a must more concise example, which I think makes it very clear all the variables at play. 




I agree with its conclusion. 




Though I'm wondering about its actual significance in ability to do harm as with 5% hash power we would have to wait quite a long time before such a block was created and it would be unpredictable when exactly this would occur, and in order to actually execute such a double spend maliciously you would have to 1) notice that such a block was mined and 2) be in a position to double spend a payment with a merchant for physical goods who you would know was using an SPV wallet at that exact time, correct? (By deliberately publishing a txn which would be blocked by the upgraded network)

Isn't that in itself unlikely enough to make this form of double spend unlikely to be exploitable?




Perhaps with malicious wallet software which always publishes "bad" (will be mostly rejected) txn first and then retries with a normal one?




But I agree with you that if the risk is there why not avoid it if possible.  



—
Regards,

On Tue, Oct 13, 2015 at 2:06 AM, Anthony Towns via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Mon, Oct 12, 2015 at 12:02:51AM -0700, digitsu412 via bitcoin-dev wrote:
>> First I think your unsaid assumption about the fragility of a soft
>> fork showing incorrect confirmations is dependent on the percentage
>> of hash power that didn't upgrade.  If using your same numbers this
>> was only 5% of the hash power, the attack is effectively not effective
>> (u less the attacker knew an exact merchant that was unfortunately on
>> the minority of the network. 
> Actually, just to take this scenario more explicitly...
> Say you've got 5% of hashpower running on old software, along with,
> say, 1500 nodes; and meanwhile you've got 95% of hashpower running new
> software, along with 4000 nodes.
> There's still about 750 nodes running 0.9 or 0.8 of 5400 total according
> to bitnodes.21.co/nodes, so those numbers seems at least plausible to
> me for the first week or two after a soft-fork is activated.
> Eventually an old-rules block gets found by the 5% hashpower. The 4000
> new nodes and 95% of hashpower ignore it, of course. With 8 random
> connections, old nodes should have 92% chance of seeing an old node
> as a peer, so I think around ~1300 of them should still be a connected
> subgraph, and the old-rules block should get propogated amongst them
> (until two new-rules blocks come along and orphan it).
> An SPV client with 12 random connections here has 96% chance of having one
> of the ~1300 old nodes as a peer, and if so, will see the old-rules block,
> that will be orphaned, and may be at risk from double-spends as a result.
> So I think even with just 5% hashpower and ~30% of nodes left running
> the old version, a "damaging soft fork" still poses a fairly high risk to
> someone receiving payments via an SPV client, and trusting transactions
> with few confirmations.
> Cheers,
> aj
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

[-- Attachment #2: Type: text/html, Size: 3826 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Wladimir J. van der Laan]

On Sun, Sep 27, 2015 at 02:50:31PM -0400, Peter Todd via bitcoin-dev wrote:

> It's time to deploy BIP65 CHECKLOCKTIMEVERIFY.

There appears to be common agreement on that.

The only source of some controversy is how to deploy: versionbits versus
IsSuperMajority. I think the versionbits proposal should first have code
out there for longer before we consider it for concrete softforks. Haste-ing
along versionbits because CLTV is wanted would be risky.

> I've backported the CLTV op-code and a IsSuperMajority() soft-fork to
> the v0.10 and v0.11 branches, pull-reqs #6706 and #6707 respectively. A
> pull-req for git HEAD for the soft-fork deployment has been open since
> June 28th, #6351 - the opcode implementation itself was merged two
> months ago.

> We should release a v0.10.3 and v0.11.1 with CLTV and get the ball
> rolling on miner adoption. We have consensus that we need CLTV, we have
> a well tested implementation, and we have a well-tested deployment
> mechanism. 

As you say, the underlying code has been merged for months in master, and #6351
seems to have had quite some eyes on it already.

It does need to be made sure that the backports are correct, however.
Although the tests do provide some assurance, I think those two pulls
require more review.

After they are merged, a 0.10.3 and 0.11.1 release can be rolled out (with RC
cycle).

> We also don't need to wait for other soft-fork proposals to
> catch up - starting the CLTV deployment process isn't going to delay
> future soft-forks, or for that matter, hard-forks.
> 
> I think it's possible to safely get CLTV live on mainnet before the end
> of the year. It's time we get this over with and done.

Wladimir

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Rusty Russell]

"Wladimir J. van der Laan via bitcoin-dev"
<bitcoin-dev@lists.linuxfoundation.org> writes:
> On Sun, Sep 27, 2015 at 02:50:31PM -0400, Peter Todd via bitcoin-dev wrote:
>
>> It's time to deploy BIP65 CHECKLOCKTIMEVERIFY.
>
> There appears to be common agreement on that.
>
> The only source of some controversy is how to deploy: versionbits versus
> IsSuperMajority. I think the versionbits proposal should first have code
> out there for longer before we consider it for concrete softforks. Haste-ing
> along versionbits because CLTV is wanted would be risky.

Agreed.  Unfortunately, a simple "block version >= 4" check is
insufficient, due to XT which sets version bits 001....111.

Given that, I suggest using the simple test:

        if (pstart->nVersion & 0x8)
            ++nFound;

Which means:
1) XT won't trigger it.
2) It won't trigger XT.
3) You can simply set block nVersion to 8 for now.
4) We can still use versionbits in parallel later.

Cheers,
Rusty.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Adam Back]

I think from discussion with Gavin sometime during the montreal
scaling bitcoin workshop, XT maybe willing to make things easy and
adapt what it's doing.  For example in relation to versionBits Gavin
said he'd be willing to update XT with an updated/improved
versionBits, for example.

It seems more sensible to do what is simple and clean and have both
core do that, and XT follow if there is no particular philosophy
debate on a given technical topic.  This seems a quite constructive
approach.

Adam

On 30 September 2015 at 00:05, Rusty Russell via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> "Wladimir J. van der Laan via bitcoin-dev"
> <bitcoin-dev@lists.linuxfoundation.org> writes:
>> On Sun, Sep 27, 2015 at 02:50:31PM -0400, Peter Todd via bitcoin-dev wrote:
>>
>>> It's time to deploy BIP65 CHECKLOCKTIMEVERIFY.
>>
>> There appears to be common agreement on that.
>>
>> The only source of some controversy is how to deploy: versionbits versus
>> IsSuperMajority. I think the versionbits proposal should first have code
>> out there for longer before we consider it for concrete softforks. Haste-ing
>> along versionbits because CLTV is wanted would be risky.
>
> Agreed.  Unfortunately, a simple "block version >= 4" check is
> insufficient, due to XT which sets version bits 001....111.
>
> Given that, I suggest using the simple test:
>
>         if (pstart->nVersion & 0x8)
>             ++nFound;
>
> Which means:
> 1) XT won't trigger it.
> 2) It won't trigger XT.
> 3) You can simply set block nVersion to 8 for now.
> 4) We can still use versionbits in parallel later.
>
> Cheers,
> Rusty.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 447 bytes --]

>
> I think from discussion with Gavin sometime during the montreal
> scaling bitcoin workshop, XT maybe willing to make things easy and
> adapt what it's doing.


If Core ships CLTV as is, then XT will have to adopt it - such is the
nature of a consensus system.

This will not change the fact that the rollout strategy is bad and nobody
has answered my extremely basic question: *why* is it being done in this
way, given the numerous downsides?

[-- Attachment #2: Type: text/html, Size: 735 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Jorge Timón]

On Tue, Sep 29, 2015 at 2:07 PM, Mike Hearn <hearn@vinumeris.com> wrote:
> Hi Jorge,
>
>> Yes, there is a difference. Assuming the hashrate majority upgrades, in
>> the case of a softfork [snip] ...... In the case of a hardfork [snip]
>
> Yes, I know what the difference between them is at a technical level. You
> didn't explain why this would make any difference to how fast miners
> upgrade. The amount of money they lose in both cases is identical: they are
> equally incentivised to upgrade with both fork types.
>
> Additionally, you say in a hard fork the other chain may "continue forever".
> Why do you think this is not true for miners building invalid blocks on top
> of the main chain? Why would that not continue forever?

I didn't talked about how fast miners would upgrade, please read again
because I believe I was extremely precise.
In both cases I'm assuming there's a minority of the hasrate which
doesn't upgrade.
In the softfork case, the minority will always build on top of the
longest chain (which is valid to them). There may be many little
alternative chains that are ignored (and orphaned) by the upgraded
miners, but non-upgraded miners will always build on top of the
longest chain.

In the hardfork case, non-upgraded miners will reject the upgraded
chain because it is invalid to them, so they will build on top of the
longest non-upgraded chains.
Two alternative chains will continue growing forever unless the
non-upgraded miners eventually upgrade.
In contrast, there won't be 2 alternative chains growing forever in
the softfork case even if the minority miners never upgrade.

> There just isn't any difference between the two fork types in terms of how
> fast miners would upgrade. Heck if anything, a hard fork should promote
> faster upgrades, because if a miner isn't paying attention to their
> debug.log they might miss the warnings. A soft fork would then look
> identical to a run of really bad luck, which can legitimately happen from
> time to time. A hard fork results in your node having a different height to
> everyone else, which is easily detectable by just checking a block explorer.
>>
>> This discussion about the general desirability of softforks seems offtopic
>> for the concrete cltv deployment discussion, which assumes softforks as
>> deployment mechanism (just like bip66 assumed it).
>
> Isn't that circular? This thread is about deployment of CLTV, but the BIP
> assumes a particular mechanism, so pointing out problems with it is off
> topic? Why have a thread at all?

BIP99 recommends an uncontroversial softfork for this kind of case.
You seem to be contradicting BIP99 in many other places. Maybe you
want to complain about some of the recommendations in BIP99 (instead
of everywhere else):

https://github.com/bitcoin/bips/pull/181

On Wed, Sep 30, 2015 at 2:30 PM, Mike Hearn via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> This will not change the fact that the rollout strategy is bad and nobody
> has answered my extremely basic question: why is it being done in this way,
> given the numerous downsides?

You seem to be the only one who thinks that softforks have "numerous
downsides" over hardforks.
So everybody just basically disagrees with the assumption in your
question and thus nobody can answer it.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[John Winslow]

Two observations from a Bitcoin investor and non-programmer:

1) I think it's possible that those who are adamantly opposed to a soft 
fork may be largely (if not completely) correct on purely technical 
terms, but that they also may be underestimating the risk of a 
contentious hardfork.

2) The downsides of a softfork are unclear because they seem to be based 
primarily on inelegant coding, not that it couldn't be made to work.

As a Bitcoin investor, I am becoming increasingly concerned that the 
rancorous and mostly unproductive debates occurring here daily are 
slowly closing the window of opportunity for Bitcoin to succeed. If this 
were a start-up or public company, the stock would be plunging. Why? 
Simple. Uncertainty. While I think (and I'm sure most here would agree) 
that these debates are necessary (and due to Bitcoin's decentralized 
nature perhaps even necessary to have in a public forum) but when these 
debates go on and on indefinitely thereby reducing confidence in 
Bitcoin's future something different needs to be done. In a public 
company or startup these debates would be happening in private with a an 
eye on competition, public/market perception, timing and anticipation of 
a shareholder-value-increasing outcome followed by a press release or 
marketing campaign. And the clock is always ticking.

My suggestion is the top devs from both sides need to get together 
offline and decide what the best compromise would be and then publicly 
promote a non-contentious solution that balances the technical with 
market concerns that everyone can get behind. Continuing to debate 
technical issues ad-infinitum without compromise or waiting until the 
Hong Kong conference in December to start making a decision while 
Bitcoin dies on the vine should not be an option. If anything, the 
conference should be the time at the end of which a confidence-inspiring 
technical roadmap is announced.

Further, I would like to add that in my perception what Bitcoin needs to 
and can easily become is essentially a public utility/backbone 
blockchain (like IP is to the internet) upon which all current and 
future blockchain stakeholders should see as their best and cheapest 
option for entering the space. For this to happen Bitcoin, from a user's 
standpoint needs to be simple and reliable, and from an 
investor/developer standpoint cost-effective and scalable. I don't see 
why this can't happen.

JTW














On 9/30/2015 8:55 AM, Jorge Timón via bitcoin-dev wrote: On Wed, Sep 30, 
2015 at 2:30 PM, Mike Hearn via bitcoin-dev 
<bitcoin-dev@lists.linuxfoundation.org> wrote:
>> This will not change the fact that the rollout strategy is bad and nobody
>> has answered my extremely basic question: why is it being done in this way,
>> given the numerous downsides?
> You seem to be the only one who thinks that softforks have "numerous
> downsides" over hardforks.
> So everybody just basically disagrees with the assumption in your
> question and thus nobody can answer it.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Rusty Russell]

John Winslow via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
writes:
> Two observations from a Bitcoin investor and non-programmer:

Please take this off the -dev list.

Thanks,
Rusty.

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Adam Back]

[-- Attachment #1: Type: text/plain, Size: 585 bytes --]

I was talking about the versionBits from Rusty's email (pasted below) and
simplifying that by XT adopting the patch as Gavin had seemed agreeable to.

Adam

Rusty wrote:
> Agreed.  Unfortunately, a simple "block version >= 4" check is
> insufficient, due to XT which sets version bits 001....111.
>
> Given that, I suggest using the simple test:
>
>         if (pstart->nVersion & 0x8)
>             ++nFound;
>
> Which means:
> 1) XT won't trigger it.
> 2) It won't trigger XT.
> 3) You can simply set block nVersion to 8 for now.
> 4) We can still use versionbits in parallel later.

[-- Attachment #2: Type: text/html, Size: 817 bytes --]

Re: [bitcoin-dev] Let's deploy BIP65 CHECKLOCKTIMEVERIFY!

[Rusty Russell]

Adam Back <adam@cypherspace.org> writes:
> I think from discussion with Gavin sometime during the montreal
> scaling bitcoin workshop, XT maybe willing to make things easy and
> adapt what it's doing.  For example in relation to versionBits Gavin
> said he'd be willing to update XT with an updated/improved
> versionBits, for example.
>
> It seems more sensible to do what is simple and clean and have both
> core do that, and XT follow if there is no particular philosophy
> debate on a given technical topic.  This seems a quite constructive
> approach.

That too, but let's not break existing software.  This proposal allows
that, and is trivial.

Cheers,
Rusty.