[Bitcoin-development] BIP62 and future script upgrades

[Bitcoin-development] BIP62 and future script upgrades

[Pieter Wuille]

Hi all,

one of the rules in BIP62 is the "clean stack" requirement, which
makes passing more inputs to a script than necessary illegal.

Unfortunately, this rule needs an exception for P2SH scripts: the test
can only be done after (and not before) the second stage evaluation.
Otherwise it would reject all spends from P2SH (which rely on
"superfluous" inputs to pass data to the second stage).

I submitted a Pull Request to clarify this in BIP62:
https://github.com/bitcoin/bips/pull/115

However, this also leads to the interesting observation that the
clean-stack rule is incompatible with future P2SH-like constructs -
which would be very useful if we'd ever want to deploy a "Script 2.0".
Any such upgrade would suffer from the same problem as P2SH, and
require an exception in the clean-stack rule, which - once deployed -
is no longer a softfork.

Luke suggested on the pull request to not apply this rule on every
transaction with nVersion >= 3, which indeed solves the problem. I
believe this can easily be generalized: make the (non mandatory) BIP62
rules only apply to transaction with strict nVersion==3, and not to
higher ones. The higher ones are non-standard anyway, and shouldn't be
used before there is a rule that applies to them anyway - which could
include some or all of BIP62 if wanted at that point still.

Opinions?

Re: [Bitcoin-development] BIP62 and future script upgrades

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 184 bytes --]

This is another problem that only exists because of the desire to soft
fork. If "script 2.0" is a hard fork upgrade, you no longer need weird
hacks like scripts-which-are-not-scripts.

[-- Attachment #2: Type: text/html, Size: 215 bytes --]

Re: [Bitcoin-development] BIP62 and future script upgrades

[Pieter Wuille]

On Tue, Nov 4, 2014 at 5:38 AM, Mike Hearn <mike@plan99.net> wrote:
> This is another problem that only exists because of the desire to soft fork.
> If "script 2.0" is a hard fork upgrade, you no longer need weird hacks like
> scripts-which-are-not-scripts.

I agree.
I also agree that the desire for softforks sometimes lead to ugly hacks.
I also that they are not "nice" philosophically because they reduce
the security model of former full nodes to SPV wrt. the new rules
without their knowledge.
I also agree that hardforks should be possible when they're useful.

But in practice, hardforks have a much larger risk which just isn't
justified for everything. Especially when it's about introducing a new
transaction type that won't be used before the softfork takes place
anyway.

And to keep the option for doing future softforks open, I believe we
need to be aware of the effects of changes like this.

-- 
Pieter

Re: [Bitcoin-development] BIP62 and future script upgrades

[Gavin Andresen]

[-- Attachment #1: Type: text/plain, Size: 491 bytes --]

On Tue, Nov 4, 2014 at 8:29 AM, Pieter Wuille <pieter.wuille@gmail.com>
wrote:

> Luke suggested on the pull request to not apply this rule on every
> transaction with nVersion >= 3, which indeed solves the problem. I
> believe this can easily be generalized: make the (non mandatory) BIP62
> rules only apply to transaction with strict nVersion==3, and not to
> higher ones.
>

I agree; soft-forking is a useful way of rolling out upgrades, we shouldn't
prohibit it.

-- 
--
Gavin Andresen

[-- Attachment #2: Type: text/html, Size: 970 bytes --]

Re: [Bitcoin-development] BIP62 and future script upgrades

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2014 bytes --]

On Tue, Nov 04, 2014 at 05:29:46AM -0800, Pieter Wuille wrote:
> one of the rules in BIP62 is the "clean stack" requirement, which
> makes passing more inputs to a script than necessary illegal.
> 
> Unfortunately, this rule needs an exception for P2SH scripts: the test
> can only be done after (and not before) the second stage evaluation.
> Otherwise it would reject all spends from P2SH (which rely on
> "superfluous" inputs to pass data to the second stage).
> 
> I submitted a Pull Request to clarify this in BIP62:
> https://github.com/bitcoin/bips/pull/115
> 
> However, this also leads to the interesting observation that the
> clean-stack rule is incompatible with future P2SH-like constructs -
> which would be very useful if we'd ever want to deploy a "Script 2.0".
> Any such upgrade would suffer from the same problem as P2SH, and
> require an exception in the clean-stack rule, which - once deployed -
> is no longer a softfork.
> 
> Luke suggested on the pull request to not apply this rule on every
> transaction with nVersion >= 3, which indeed solves the problem. I
> believe this can easily be generalized: make the (non mandatory) BIP62
> rules only apply to transaction with strict nVersion==3, and not to
> higher ones. The higher ones are non-standard anyway, and shouldn't be
> used before there is a rule that applies to them anyway - which could
> include some or all of BIP62 if wanted at that point still.
> 
> Opinions?

I agree with Luke: make the rules only apply to transactions with a
strict nVersion==3. If we want to extend that later we can do so in
another soft-fork.


On another topic, I'm skeptical of the choice of nVersion==3 - we'll
likely end up doing more block.nVersion increases in the future, and
there's no reason to think they'll have anything to do with
transactions. No sense creating a rule that'll be so quickly broken.

-- 
'peter'[:-1]@petertodd.org
000000000000000002986604739bc94cc42d5c6adf75186e80ba3dbb501a076d

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [Bitcoin-development] BIP62 and future script upgrades

[Jeff Garzik]

On Tue, Nov 4, 2014 at 8:13 PM, Peter Todd <pete@petertodd.org> wrote:
> On another topic, I'm skeptical of the choice of nVersion==3 - we'll
> likely end up doing more block.nVersion increases in the future, and
> there's no reason to think they'll have anything to do with
> transactions. No sense creating a rule that'll be so quickly broken.

Moderately agreed.

Earlier in BIP 62 lifetime, I had commented on ambiguity that arose
from bumping tx version simply because we were bumping block version.
The ambiguity was corrected, but IMO remains symptomatic of potential
problems and confusion down the road.

Though I ACK'd the change, my general preference remains to disconnect
TX and block version.

-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/

Re: [Bitcoin-development] BIP62 and future script upgrades

[Pieter Wuille]

On Tue, Nov 4, 2014 at 11:56 AM, Jeff Garzik <jgarzik@bitpay.com> wrote:
> On Tue, Nov 4, 2014 at 8:13 PM, Peter Todd <pete@petertodd.org> wrote:
>> On another topic, I'm skeptical of the choice of nVersion==3 - we'll
>> likely end up doing more block.nVersion increases in the future, and
>> there's no reason to think they'll have anything to do with
>> transactions. No sense creating a rule that'll be so quickly broken.
>
> Moderately agreed.
>
> Earlier in BIP 62 lifetime, I had commented on ambiguity that arose
> from bumping tx version simply because we were bumping block version.
> The ambiguity was corrected, but IMO remains symptomatic of potential
> problems and confusion down the road.
>
> Though I ACK'd the change, my general preference remains to disconnect
> TX and block version.

I prefer to see consensus rules as one set of rules (especially
because they only really apply to blocks - the part for lone
transactions is just policy), and thus have a single numbering. Still,
I have no strong opinion about it and have now heard 3 'moderately
against' comments. I'm fine with using nVersion==2 for transactions.

-- 
Pieter

Re: [Bitcoin-development] BIP62 and future script upgrades

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 2132 bytes --]

On Tue, Nov 04, 2014 at 12:00:43PM -0800, Pieter Wuille wrote:
> On Tue, Nov 4, 2014 at 11:56 AM, Jeff Garzik <jgarzik@bitpay.com> wrote:
> > On Tue, Nov 4, 2014 at 8:13 PM, Peter Todd <pete@petertodd.org> wrote:
> >> On another topic, I'm skeptical of the choice of nVersion==3 - we'll
> >> likely end up doing more block.nVersion increases in the future, and
> >> there's no reason to think they'll have anything to do with
> >> transactions. No sense creating a rule that'll be so quickly broken.
> >
> > Moderately agreed.
> >
> > Earlier in BIP 62 lifetime, I had commented on ambiguity that arose
> > from bumping tx version simply because we were bumping block version.
> > The ambiguity was corrected, but IMO remains symptomatic of potential
> > problems and confusion down the road.
> >
> > Though I ACK'd the change, my general preference remains to disconnect
> > TX and block version.
> 
> I prefer to see consensus rules as one set of rules (especially
> because they only really apply to blocks - the part for lone
> transactions is just policy), and thus have a single numbering. Still,
> I have no strong opinion about it and have now heard 3 'moderately
> against' comments. I'm fine with using nVersion==2 for transactions.

Keep in mind that we may even have a circumstance where we need to
introduce *two* different new tx version numbers in a single soft-fork,
say because we find an exploit that has two different fixes, each of
which breaks something.

I don't think we have any certainty how new features will be added in
the future - just look at how we only recently realised new opcodes
won't be associated with tx version number bumps - so I'm loath to setup
expectations.

Besides, transactions can certainly be verified for correctness in a
stand-alone fashion outside a block; CHECKLOCKTIMEVERIFY was
specifically designed so that verifying scripts containing it could be
done in a self-contained manner only referencing the transaction the
script was within.

-- 
'peter'[:-1]@petertodd.org
0000000000000000036655c955dd94ba7f9856814f3cb87f003e311566921807

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [Bitcoin-development] BIP62 and future script upgrades

[Pieter Wuille]

Ok, addressed these (and a few other things) in
https://github.com/bitcoin/bips/pull/117:
* Better names for the rules.
* Clarify interaction of BIP62 with P2SH.
* Clarify that known hashtypes are required, despite not being part of DER.
* Use v2 transactions instead of v3 transactions.
* Apply the optional rules only to strict v2, and not higher or lower.


On Tue, Nov 4, 2014 at 12:07 PM, Peter Todd <pete@petertodd.org> wrote:
> On Tue, Nov 04, 2014 at 12:00:43PM -0800, Pieter Wuille wrote:
>> On Tue, Nov 4, 2014 at 11:56 AM, Jeff Garzik <jgarzik@bitpay.com> wrote:
>> > On Tue, Nov 4, 2014 at 8:13 PM, Peter Todd <pete@petertodd.org> wrote:
>> >> On another topic, I'm skeptical of the choice of nVersion==3 - we'll
>> >> likely end up doing more block.nVersion increases in the future, and
>> >> there's no reason to think they'll have anything to do with
>> >> transactions. No sense creating a rule that'll be so quickly broken.
>> >
>> > Moderately agreed.
>> >
>> > Earlier in BIP 62 lifetime, I had commented on ambiguity that arose
>> > from bumping tx version simply because we were bumping block version.
>> > The ambiguity was corrected, but IMO remains symptomatic of potential
>> > problems and confusion down the road.
>> >
>> > Though I ACK'd the change, my general preference remains to disconnect
>> > TX and block version.
>>
>> I prefer to see consensus rules as one set of rules (especially
>> because they only really apply to blocks - the part for lone
>> transactions is just policy), and thus have a single numbering. Still,
>> I have no strong opinion about it and have now heard 3 'moderately
>> against' comments. I'm fine with using nVersion==2 for transactions.
>
> Keep in mind that we may even have a circumstance where we need to
> introduce *two* different new tx version numbers in a single soft-fork,
> say because we find an exploit that has two different fixes, each of
> which breaks something.
>
> I don't think we have any certainty how new features will be added in
> the future - just look at how we only recently realised new opcodes
> won't be associated with tx version number bumps - so I'm loath to setup
> expectations.
>
> Besides, transactions can certainly be verified for correctness in a
> stand-alone fashion outside a block; CHECKLOCKTIMEVERIFY was
> specifically designed so that verifying scripts containing it could be
> done in a self-contained manner only referencing the transaction the
> script was within.
>
> --
> 'peter'[:-1]@petertodd.org
> 0000000000000000036655c955dd94ba7f9856814f3cb87f003e311566921807