[bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 4744 bytes --]

Deployment of the proposed CLTV, CSV, etc. soft-forks has been recently
complicated by the existence of XT(1) and Not-Bitcoin-XT(2) miners. Both
mine blocks with nVersion=0x20000007, which would falsely trigger the
previously suggested implementation using the IsSuperMajority()
mechanism and nVersion=4 blocks. Additionally while the
XT/Not-Bitcoin-XT software claims to support Wuille/Todd/Maxwell's
nVersion soft-fork mechanism(3) a key component of it - fork
deadlines(3) - is not implemented.


XT/Not-Bitcoin-XT behavior
--------------------------

Both implementations produce blocks with nVersion=0x20000007,
or in binary: 0b001...111

Neither implementation supports a fork deadline; both Not-Bitcoin-XT and
XT will produce blocks with those bits set indefinitely under any
circumstance, with the proviso that while XT has a hashing power
majority, blocks it produces might not be part of the Bitcoin blockchain
after Jan 11th 2016. (though this can flap back and forth if reorgs
happen)

Curiously the BIP101 draft was changed(4) at the last minute from using
the nVersion bits compliant 0x20000004 block nVersion, to using two more
bits unnecessarily. The rational for doing this is unknown; the git
commit message associated with the change suggested "compatibility
concerns", but what the concerns actually were isn't specified. Equally
even though implementing the fork deadline would be very each in the XT
implementation, this was not done. (the XT codebase has had almost no
peer review)


Options for CLTV/CSV/etc. deployment
------------------------------------

1) Plain IsSuperMajority() with nVersion=4

This option can be ruled out immediately due to the high risk of
premature triggering, without genuine 95% miner support.


2) nVersion mask, with IsSuperMajority()

In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would
be masked away, prior to applying standard IsSuperMajority() logic:

    block.nVersion & ~0x20000007

This means that CLTV/CSV/etc. miners running Bitcoin Core would create
blocks with nVersion=8, 0b1000. From the perspective of the
CLTV/CSV/etc.  IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be
advertising blocks that do not trigger the soft-fork.

For the perpose of soft-fork warnings, the highest known version can
remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks
as well as a future nVersion bits implementation. Equally,
XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an
unknown bit set.

When nVersion bits is implemented by the Bitcoin protocol, the plan of
setting the high bits to 0b001 still works. The three lowest bits will
be unusable for some time, but will be eventually recoverable as
XT/Not-Bitcoin-XT mining ceases.

Equally, further IsSuperMajority() softforks can be accomplished with
the same masking technique.

This option does complicate the XT-coin protocol implementation in the
future. But that's their problem, and anyway, the maintainers
(Hearn/Andresen) has strenuously argued(5) against the use of soft-forks
and/or appear to be in favor of a more centralized mandatory update
schedule.(6)


3) Full nVersion bits implementation

The most complex option would be to deploy via full nVersion bits
implementation using flag bit #4 to trigger the fork. Compliant miners
would advertise 0x20000008 initially, followed by 0x20000000 once the
fork had triggered. The lowest three bits would be unusable for forks
for some time, although they could be eventually recovered as
XT/Not-Bitcoin-XT mining ceases.

The main disadvantage of this option is high initial complexity - the
reason why IsSuperMajority() was suggested for CLTV/CSV in the first
place. That said, much of the code required has been implemented in XT
for the BIP101 hard-fork logic, although as mentioned above, the code
has had very little peer review.


References
----------

1) https://github.com/bitcoinxt/bitcoinxt

2) https://github.com/xtbit/notbitcoinxt

3) "Version bits proposal",
    Pieter Wuille, May 26th 2015, Bitcoin-development mailing list,
    http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008282.html,
    https://gist.github.com/sipa/bf69659f43e763540550

4) https://github.com/bitcoin/bips/commit/3248c9f67bd7fcd1d05b8db7c5c56e4788deebfe

5) "On consensus and forks - What is the difference between a hard and soft fork?",
   Mike Hearn, Aug 12th 2015,
   https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7

6) 2013 San Jose Bitcoin conference developer round-table

-- 
'peter'[:-1]@petertodd.org
00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Mark Friedenbach]

[-- Attachment #1: Type: text/plain, Size: 5282 bytes --]

We can use nVersion & 0x8 to signal support, while keeping the consensus
rule as nVersion >= 4, right? That way we don't waste a bit after this all
clears up.
On Aug 18, 2015 10:50 PM, "Peter Todd via bitcoin-dev" <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Deployment of the proposed CLTV, CSV, etc. soft-forks has been recently
> complicated by the existence of XT(1) and Not-Bitcoin-XT(2) miners. Both
> mine blocks with nVersion=0x20000007, which would falsely trigger the
> previously suggested implementation using the IsSuperMajority()
> mechanism and nVersion=4 blocks. Additionally while the
> XT/Not-Bitcoin-XT software claims to support Wuille/Todd/Maxwell's
> nVersion soft-fork mechanism(3) a key component of it - fork
> deadlines(3) - is not implemented.
>
>
> XT/Not-Bitcoin-XT behavior
> --------------------------
>
> Both implementations produce blocks with nVersion=0x20000007,
> or in binary: 0b001...111
>
> Neither implementation supports a fork deadline; both Not-Bitcoin-XT and
> XT will produce blocks with those bits set indefinitely under any
> circumstance, with the proviso that while XT has a hashing power
> majority, blocks it produces might not be part of the Bitcoin blockchain
> after Jan 11th 2016. (though this can flap back and forth if reorgs
> happen)
>
> Curiously the BIP101 draft was changed(4) at the last minute from using
> the nVersion bits compliant 0x20000004 block nVersion, to using two more
> bits unnecessarily. The rational for doing this is unknown; the git
> commit message associated with the change suggested "compatibility
> concerns", but what the concerns actually were isn't specified. Equally
> even though implementing the fork deadline would be very each in the XT
> implementation, this was not done. (the XT codebase has had almost no
> peer review)
>
>
> Options for CLTV/CSV/etc. deployment
> ------------------------------------
>
> 1) Plain IsSuperMajority() with nVersion=4
>
> This option can be ruled out immediately due to the high risk of
> premature triggering, without genuine 95% miner support.
>
>
> 2) nVersion mask, with IsSuperMajority()
>
> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would
> be masked away, prior to applying standard IsSuperMajority() logic:
>
>     block.nVersion & ~0x20000007
>
> This means that CLTV/CSV/etc. miners running Bitcoin Core would create
> blocks with nVersion=8, 0b1000. From the perspective of the
> CLTV/CSV/etc.  IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be
> advertising blocks that do not trigger the soft-fork.
>
> For the perpose of soft-fork warnings, the highest known version can
> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks
> as well as a future nVersion bits implementation. Equally,
> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an
> unknown bit set.
>
> When nVersion bits is implemented by the Bitcoin protocol, the plan of
> setting the high bits to 0b001 still works. The three lowest bits will
> be unusable for some time, but will be eventually recoverable as
> XT/Not-Bitcoin-XT mining ceases.
>
> Equally, further IsSuperMajority() softforks can be accomplished with
> the same masking technique.
>
> This option does complicate the XT-coin protocol implementation in the
> future. But that's their problem, and anyway, the maintainers
> (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks
> and/or appear to be in favor of a more centralized mandatory update
> schedule.(6)
>
>
> 3) Full nVersion bits implementation
>
> The most complex option would be to deploy via full nVersion bits
> implementation using flag bit #4 to trigger the fork. Compliant miners
> would advertise 0x20000008 initially, followed by 0x20000000 once the
> fork had triggered. The lowest three bits would be unusable for forks
> for some time, although they could be eventually recovered as
> XT/Not-Bitcoin-XT mining ceases.
>
> The main disadvantage of this option is high initial complexity - the
> reason why IsSuperMajority() was suggested for CLTV/CSV in the first
> place. That said, much of the code required has been implemented in XT
> for the BIP101 hard-fork logic, although as mentioned above, the code
> has had very little peer review.
>
>
> References
> ----------
>
> 1) https://github.com/bitcoinxt/bitcoinxt
>
> 2) https://github.com/xtbit/notbitcoinxt
>
> 3) "Version bits proposal",
>     Pieter Wuille, May 26th 2015, Bitcoin-development mailing list,
>
> http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008282.html
> ,
>     https://gist.github.com/sipa/bf69659f43e763540550
>
> 4)
> https://github.com/bitcoin/bips/commit/3248c9f67bd7fcd1d05b8db7c5c56e4788deebfe
>
> 5) "On consensus and forks - What is the difference between a hard and
> soft fork?",
>    Mike Hearn, Aug 12th 2015,
>    https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7
>
> 6) 2013 San Jose Bitcoin conference developer round-table
>
> --
> 'peter'[:-1]@petertodd.org
> 00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 6872 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Jorge Timón]

Seems like 3 is something we want to do no matter what and therefore
is the "most future-proof" solution.
I wonder if I can help with that (and I know there's more people that
would be interested).
Where's the current "non-full" nVersion bits implementation?
Why implement a "non-full" version instead of going with the full
implementation directly?


On Wed, Aug 19, 2015 at 8:10 AM, Mark Friedenbach via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> We can use nVersion & 0x8 to signal support, while keeping the consensus
> rule as nVersion >= 4, right? That way we don't waste a bit after this all
> clears up.
>
> On Aug 18, 2015 10:50 PM, "Peter Todd via bitcoin-dev"
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> Deployment of the proposed CLTV, CSV, etc. soft-forks has been recently
>> complicated by the existence of XT(1) and Not-Bitcoin-XT(2) miners. Both
>> mine blocks with nVersion=0x20000007, which would falsely trigger the
>> previously suggested implementation using the IsSuperMajority()
>> mechanism and nVersion=4 blocks. Additionally while the
>> XT/Not-Bitcoin-XT software claims to support Wuille/Todd/Maxwell's
>> nVersion soft-fork mechanism(3) a key component of it - fork
>> deadlines(3) - is not implemented.
>>
>>
>> XT/Not-Bitcoin-XT behavior
>> --------------------------
>>
>> Both implementations produce blocks with nVersion=0x20000007,
>> or in binary: 0b001...111
>>
>> Neither implementation supports a fork deadline; both Not-Bitcoin-XT and
>> XT will produce blocks with those bits set indefinitely under any
>> circumstance, with the proviso that while XT has a hashing power
>> majority, blocks it produces might not be part of the Bitcoin blockchain
>> after Jan 11th 2016. (though this can flap back and forth if reorgs
>> happen)
>>
>> Curiously the BIP101 draft was changed(4) at the last minute from using
>> the nVersion bits compliant 0x20000004 block nVersion, to using two more
>> bits unnecessarily. The rational for doing this is unknown; the git
>> commit message associated with the change suggested "compatibility
>> concerns", but what the concerns actually were isn't specified. Equally
>> even though implementing the fork deadline would be very each in the XT
>> implementation, this was not done. (the XT codebase has had almost no
>> peer review)
>>
>>
>> Options for CLTV/CSV/etc. deployment
>> ------------------------------------
>>
>> 1) Plain IsSuperMajority() with nVersion=4
>>
>> This option can be ruled out immediately due to the high risk of
>> premature triggering, without genuine 95% miner support.
>>
>>
>> 2) nVersion mask, with IsSuperMajority()
>>
>> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would
>> be masked away, prior to applying standard IsSuperMajority() logic:
>>
>>     block.nVersion & ~0x20000007
>>
>> This means that CLTV/CSV/etc. miners running Bitcoin Core would create
>> blocks with nVersion=8, 0b1000. From the perspective of the
>> CLTV/CSV/etc.  IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be
>> advertising blocks that do not trigger the soft-fork.
>>
>> For the perpose of soft-fork warnings, the highest known version can
>> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks
>> as well as a future nVersion bits implementation. Equally,
>> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an
>> unknown bit set.
>>
>> When nVersion bits is implemented by the Bitcoin protocol, the plan of
>> setting the high bits to 0b001 still works. The three lowest bits will
>> be unusable for some time, but will be eventually recoverable as
>> XT/Not-Bitcoin-XT mining ceases.
>>
>> Equally, further IsSuperMajority() softforks can be accomplished with
>> the same masking technique.
>>
>> This option does complicate the XT-coin protocol implementation in the
>> future. But that's their problem, and anyway, the maintainers
>> (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks
>> and/or appear to be in favor of a more centralized mandatory update
>> schedule.(6)
>>
>>
>> 3) Full nVersion bits implementation
>>
>> The most complex option would be to deploy via full nVersion bits
>> implementation using flag bit #4 to trigger the fork. Compliant miners
>> would advertise 0x20000008 initially, followed by 0x20000000 once the
>> fork had triggered. The lowest three bits would be unusable for forks
>> for some time, although they could be eventually recovered as
>> XT/Not-Bitcoin-XT mining ceases.
>>
>> The main disadvantage of this option is high initial complexity - the
>> reason why IsSuperMajority() was suggested for CLTV/CSV in the first
>> place. That said, much of the code required has been implemented in XT
>> for the BIP101 hard-fork logic, although as mentioned above, the code
>> has had very little peer review.
>>
>>
>> References
>> ----------
>>
>> 1) https://github.com/bitcoinxt/bitcoinxt
>>
>> 2) https://github.com/xtbit/notbitcoinxt
>>
>> 3) "Version bits proposal",
>>     Pieter Wuille, May 26th 2015, Bitcoin-development mailing list,
>>
>> http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008282.html,
>>     https://gist.github.com/sipa/bf69659f43e763540550
>>
>> 4)
>> https://github.com/bitcoin/bips/commit/3248c9f67bd7fcd1d05b8db7c5c56e4788deebfe
>>
>> 5) "On consensus and forks - What is the difference between a hard and
>> soft fork?",
>>    Mike Hearn, Aug 12th 2015,
>>    https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7
>>
>> 6) 2013 San Jose Bitcoin conference developer round-table
>>
>> --
>> 'peter'[:-1]@petertodd.org
>> 00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Btc Drak]

[-- Attachment #1: Type: text/plain, Size: 1418 bytes --]

On Wed, Aug 19, 2015 at 10:34 AM, Jorge Timón <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Seems like 3 is something we want to do no matter what and therefore
> is the "most future-proof" solution.
> I wonder if I can help with that (and I know there's more people that
> would be interested).
> Where's the current "non-full" nVersion bits implementation?
> Why implement a "non-full" version instead of going with the full
> implementation directly?


There is a simple answer to this, convenience: versionbits has not been
implemented yet, and I believe the BIP is still in review stage. As it
seems likely the remaining locktime pull requests will be ready by or
before the next major release, we need a deployment method if versionbits
is not ready (which is unlikely because no-one appears to be working on it
at the moment). Pieter indicated he is OK with another IsSuperMajority()
rollout in the interim. Personally, I dont think we should let perfection
be the enemy of progress here because at the end of the day, the deployment
method is less important than the actual featureset being proposed.

That said, the features in the next soft fork proposal are all related and
best deployed as one featureset softfork, but moving forward, versionbits
seems essential to be able to roll out multiple features in parallel
without waiting for activation and enforcement each time.

[-- Attachment #2: Type: text/html, Size: 1872 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Jorge Timón]

I don't think just using version=4 for cltv and friends would be a
problem if it wasn't for the XT/nonXT issue.

On Wed, Aug 19, 2015 at 12:20 PM, Btc Drak <btcdrak@gmail.com> wrote:
> On Wed, Aug 19, 2015 at 10:34 AM, Jorge Timón
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> Seems like 3 is something we want to do no matter what and therefore
>> is the "most future-proof" solution.
>> I wonder if I can help with that (and I know there's more people that
>> would be interested).
>> Where's the current "non-full" nVersion bits implementation?
>> Why implement a "non-full" version instead of going with the full
>> implementation directly?
>
>
> There is a simple answer to this, convenience: versionbits has not been
> implemented yet, and I believe the BIP is still in review stage. As it seems
> likely the remaining locktime pull requests will be ready by or before the
> next major release, we need a deployment method if versionbits is not ready
> (which is unlikely because no-one appears to be working on it at the
> moment). Pieter indicated he is OK with another IsSuperMajority() rollout in
> the interim. Personally, I dont think we should let perfection be the enemy
> of progress here because at the end of the day, the deployment method is
> less important than the actual featureset being proposed.
>
> That said, the features in the next soft fork proposal are all related and
> best deployed as one featureset softfork, but moving forward, versionbits
> seems essential to be able to roll out multiple features in parallel without
> waiting for activation and enforcement each time.
>
>
>
>
>

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Btc Drak]

[-- Attachment #1: Type: text/plain, Size: 579 bytes --]

On Wed, Aug 19, 2015 at 11:31 AM, Jorge Timón <jtimon@jtimon.cc> wrote:

> I don't think just using version=4 for cltv and friends would be a
> problem if it wasn't for the XT/nonXT issue.


What problem am I missing if we just mask of the offending bits. For my own
project which uses auxpow (and thus has weird nVersion), I also used the
bitmasking method to get rid of auxpow version bits before making the
standard integer comparisons to deploy BIP66 using IsSuperMajority():

    if ((block.nVersion & 0xff) >= 4 && CBlockIndex::IsSuperMajority(...))
{ //...}

[-- Attachment #2: Type: text/html, Size: 923 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Tier Nolan]

[-- Attachment #1: Type: text/plain, Size: 611 bytes --]

On Wed, Aug 19, 2015 at 2:15 PM, Btc Drak via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

>
> What problem am I missing if we just mask of the offending bits. For my
> own project which uses auxpow (and thus has weird nVersion), I also used
> the bitmasking method to get rid of auxpow version bits before making the
> standard integer comparisons to deploy BIP66 using IsSuperMajority():
>
>     if ((block.nVersion & 0xff) >= 4 && CBlockIndex::IsSuperMajority(...))
> { //...}
>

What if version number 257 is used in the future?  That would appear to be
a version 1 block and fail the test.

[-- Attachment #2: Type: text/html, Size: 1063 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Btc Drak]

On Wed, Aug 19, 2015 at 2:24 PM, Tier Nolan via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> On Wed, Aug 19, 2015 at 2:15 PM, Btc Drak via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>> What problem am I missing if we just mask of the offending bits. For my own project which uses auxpow (and thus has weird nVersion), I also used the bitmasking method to get rid of auxpow version bits before making the standard integer comparisons to deploy BIP66 using IsSuperMajority():
>>
>>     if ((block.nVersion & 0xff) >= 4 && CBlockIndex::IsSuperMajority(...)) { //...}
>
> What if version number 257 is used in the future?  That would appear to be a version 1 block and fail the test.

To clarify this is the code example for how the same problem occurs
with auxpow bits when running a an IsSuperMajority() softfork and how
it's solved in that instance.

In our case for Bitcoin Core, option 2 we use nVersion=8, apply a
bitmask of 0xdffffff8 thus:

    if ((block.nVersion & ~0x20000007) >= 4 &&
CBlockIndex::IsSuperMajority(...)) { //...}

With nVersion=8, but using comparison >=4 allows us to recover the
bit later, assuming we want it (otherwise we use version >=8).

This should, unless I am missing something, be forward compatible with
future softforks. My understanding was if "versionbits softfork" code
is not ready by the time we're ready for the deployment,
IsSuperMajority() would be acceptable; and this is how we could deploy
it in the wake of the XT developers' carelessness.

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Tier Nolan]

[-- Attachment #1: Type: text/plain, Size: 961 bytes --]

On Wed, Aug 19, 2015 at 6:25 PM, Btc Drak <btcdrak@gmail.com> wrote:

> In our case for Bitcoin Core, option 2 we use nVersion=8, apply a
> bitmask of 0xdffffff8 thus:
>
>     if ((block.nVersion & ~0x20000007) >= 4 &&
> CBlockIndex::IsSuperMajority(...)) { //...}
>
> With nVersion=8, but using comparison >=4 allows us to recover the
> bit later, assuming we want it (otherwise we use version >=8).
>

That is the 75% "activation" rule portion?  The 95% rule has to apply to
all blocks.

The supermajority applies to unmasked blocks?

I think you want it so that a sequence of blocks with version 8 can be
followed by version 4 blocks?

If 950 of the last 1000 blocks have bit 0x08 set, then reject any block
with a version less than 4.

This means transitioning to the version bits BIP just requires dropping the
version back to 4 and adding a rule enforcing the BIPs for version 4 and
higher blocks.

This would be part of the version bits BIP enforcement.

[-- Attachment #2: Type: text/html, Size: 1429 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Matt Corallo]

Wait, why did Bitcoin-XT use that nVersion???

Definitely option 3 is much cleaner technically, and it would be nice to have that code implemented, but I'd be rather concerned about the size of the fork ballooning. It's already four separate features in one fork, which seems pretty big, even if the code isn't too huge. I'd probably prefer slightly to waste another two version bits than add a bunch of code to the fork now (maybe we can take just a part of the bit-based approach and allow lower version numbers again after the fork reaches 95%?). Either way, a proper version of the bit-based soft forking mechanism should be implemented sooner rather than later (maybe merged unused, even).

Still, it is entirely possible that there is relatively little uptake on XT and a competing proposal has a lot of support before we want to ship a LTV fork, so it may all be moot (or we could choose option 1 to fix the XT fork for them - easily forking off XT miners when either fork happens so XT isn't vulnerable to the 25% attack without needing to mine a massive transaction on Bitcoin first :p).

Matt

On August 19, 2015 11:34:54 AM GMT+02:00, "Jorge Timón via bitcoin-dev" <bitcoin-dev@lists.linuxfoundation.org> wrote:
>Seems like 3 is something we want to do no matter what and therefore
>is the "most future-proof" solution.
>I wonder if I can help with that (and I know there's more people that
>would be interested).
>Where's the current "non-full" nVersion bits implementation?
>Why implement a "non-full" version instead of going with the full
>implementation directly?
>
>
>On Wed, Aug 19, 2015 at 8:10 AM, Mark Friedenbach via bitcoin-dev
><bitcoin-dev@lists.linuxfoundation.org> wrote:
>> We can use nVersion & 0x8 to signal support, while keeping the
>consensus
>> rule as nVersion >= 4, right? That way we don't waste a bit after
>this all
>> clears up.
>>
>> On Aug 18, 2015 10:50 PM, "Peter Todd via bitcoin-dev"
>> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>>
>>> Deployment of the proposed CLTV, CSV, etc. soft-forks has been
>recently
>>> complicated by the existence of XT(1) and Not-Bitcoin-XT(2) miners.
>Both
>>> mine blocks with nVersion=0x20000007, which would falsely trigger
>the
>>> previously suggested implementation using the IsSuperMajority()
>>> mechanism and nVersion=4 blocks. Additionally while the
>>> XT/Not-Bitcoin-XT software claims to support Wuille/Todd/Maxwell's
>>> nVersion soft-fork mechanism(3) a key component of it - fork
>>> deadlines(3) - is not implemented.
>>>
>>>
>>> XT/Not-Bitcoin-XT behavior
>>> --------------------------
>>>
>>> Both implementations produce blocks with nVersion=0x20000007,
>>> or in binary: 0b001...111
>>>
>>> Neither implementation supports a fork deadline; both Not-Bitcoin-XT
>and
>>> XT will produce blocks with those bits set indefinitely under any
>>> circumstance, with the proviso that while XT has a hashing power
>>> majority, blocks it produces might not be part of the Bitcoin
>blockchain
>>> after Jan 11th 2016. (though this can flap back and forth if reorgs
>>> happen)
>>>
>>> Curiously the BIP101 draft was changed(4) at the last minute from
>using
>>> the nVersion bits compliant 0x20000004 block nVersion, to using two
>more
>>> bits unnecessarily. The rational for doing this is unknown; the git
>>> commit message associated with the change suggested "compatibility
>>> concerns", but what the concerns actually were isn't specified.
>Equally
>>> even though implementing the fork deadline would be very each in the
>XT
>>> implementation, this was not done. (the XT codebase has had almost
>no
>>> peer review)
>>>
>>>
>>> Options for CLTV/CSV/etc. deployment
>>> ------------------------------------
>>>
>>> 1) Plain IsSuperMajority() with nVersion=4
>>>
>>> This option can be ruled out immediately due to the high risk of
>>> premature triggering, without genuine 95% miner support.
>>>
>>>
>>> 2) nVersion mask, with IsSuperMajority()
>>>
>>> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners
>would
>>> be masked away, prior to applying standard IsSuperMajority() logic:
>>>
>>>     block.nVersion & ~0x20000007
>>>
>>> This means that CLTV/CSV/etc. miners running Bitcoin Core would
>create
>>> blocks with nVersion=8, 0b1000. From the perspective of the
>>> CLTV/CSV/etc.  IsSuperMajority() test, XT/Not-Bitcoin-XT miners
>would be
>>> advertising blocks that do not trigger the soft-fork.
>>>
>>> For the perpose of soft-fork warnings, the highest known version can
>>> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT
>blocks
>>> as well as a future nVersion bits implementation. Equally,
>>> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an
>>> unknown bit set.
>>>
>>> When nVersion bits is implemented by the Bitcoin protocol, the plan
>of
>>> setting the high bits to 0b001 still works. The three lowest bits
>will
>>> be unusable for some time, but will be eventually recoverable as
>>> XT/Not-Bitcoin-XT mining ceases.
>>>
>>> Equally, further IsSuperMajority() softforks can be accomplished
>with
>>> the same masking technique.
>>>
>>> This option does complicate the XT-coin protocol implementation in
>the
>>> future. But that's their problem, and anyway, the maintainers
>>> (Hearn/Andresen) has strenuously argued(5) against the use of
>soft-forks
>>> and/or appear to be in favor of a more centralized mandatory update
>>> schedule.(6)
>>>
>>>
>>> 3) Full nVersion bits implementation
>>>
>>> The most complex option would be to deploy via full nVersion bits
>>> implementation using flag bit #4 to trigger the fork. Compliant
>miners
>>> would advertise 0x20000008 initially, followed by 0x20000000 once
>the
>>> fork had triggered. The lowest three bits would be unusable for
>forks
>>> for some time, although they could be eventually recovered as
>>> XT/Not-Bitcoin-XT mining ceases.
>>>
>>> The main disadvantage of this option is high initial complexity -
>the
>>> reason why IsSuperMajority() was suggested for CLTV/CSV in the first
>>> place. That said, much of the code required has been implemented in
>XT
>>> for the BIP101 hard-fork logic, although as mentioned above, the
>code
>>> has had very little peer review.
>>>
>>>
>>> References
>>> ----------
>>>
>>> 1) https://github.com/bitcoinxt/bitcoinxt
>>>
>>> 2) https://github.com/xtbit/notbitcoinxt
>>>
>>> 3) "Version bits proposal",
>>>     Pieter Wuille, May 26th 2015, Bitcoin-development mailing list,
>>>
>>>
>http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008282.html,
>>>     https://gist.github.com/sipa/bf69659f43e763540550
>>>
>>> 4)
>>>
>https://github.com/bitcoin/bips/commit/3248c9f67bd7fcd1d05b8db7c5c56e4788deebfe
>>>
>>> 5) "On consensus and forks - What is the difference between a hard
>and
>>> soft fork?",
>>>    Mike Hearn, Aug 12th 2015,
>>>   
>https://medium.com/@octskyward/on-consensus-and-forks-c6a050c792e7
>>>
>>> 6) 2013 San Jose Bitcoin conference developer round-table
>>>
>>> --
>>> 'peter'[:-1]@petertodd.org
>>> 00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d
>>>
>>> _______________________________________________
>>> bitcoin-dev mailing list
>>> bitcoin-dev@lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>_______________________________________________
>bitcoin-dev mailing list
>bitcoin-dev@lists.linuxfoundation.org
>https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Tier Nolan]

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On Wed, Aug 19, 2015 at 7:10 AM, Mark Friedenbach via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> We can use nVersion & 0x8 to signal support, while keeping the consensus
> rule as nVersion >= 4, right? That way we don't waste a bit after this all
> clears up.
>
What happens if XT gets 40% and this BIP gets 55%?  That gives 95% that
accept the upgrade.  Version 3 and lower blocks need to be rejected after
that.

By advertising 0x7 for the last 3 bits, XT is effectively claiming to
support the check locktime verify BIPs but they don't have the code.

This sequence could be used, without a specific version-bits proposal.

Until height = N + 5000, if 950 of the last 1000 blocks have the 0x8 bit
set, then reject blocks with version numbers less than 8.

At height N, if the above rule is active, then the BIP is permanent.

It applies to any block with bit 0x8 set, once the 75% threshold is met.

From height N + 5000 to N + 10000, reject blocks which have bit 0x8 set.

N could be set 1 year from now, or so.


This gives a period of time after lock that bit 8 is kept and then a period
where is is guaranteed to be zero.

This gives software that is only watching the bit time to be upgraded and
similarly time where the bit is set to zero before it can be reused.

[-- Attachment #2: Type: text/html, Size: 1880 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 2042 bytes --]

A lot of this detail and worry is eliminated by simply asking BIP 102
maintainers to change to a different bit that works best for everyone.
Existing nVersion garbage will get buried in sufficient time window to
prevent anything from triggering.

Just settle on a specific standard, reserve bits for feature upgrade forks,
and move on.  There is already a rough standard proposed in sipa's gist.





On Wed, Aug 19, 2015 at 9:22 AM, Tier Nolan via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Wed, Aug 19, 2015 at 7:10 AM, Mark Friedenbach via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> We can use nVersion & 0x8 to signal support, while keeping the consensus
>> rule as nVersion >= 4, right? That way we don't waste a bit after this all
>> clears up.
>>
> What happens if XT gets 40% and this BIP gets 55%?  That gives 95% that
> accept the upgrade.  Version 3 and lower blocks need to be rejected after
> that.
>
> By advertising 0x7 for the last 3 bits, XT is effectively claiming to
> support the check locktime verify BIPs but they don't have the code.
>
> This sequence could be used, without a specific version-bits proposal.
>
> Until height = N + 5000, if 950 of the last 1000 blocks have the 0x8 bit
> set, then reject blocks with version numbers less than 8.
>
> At height N, if the above rule is active, then the BIP is permanent.
>
> It applies to any block with bit 0x8 set, once the 75% threshold is met.
>
> From height N + 5000 to N + 10000, reject blocks which have bit 0x8 set.
>
> N could be set 1 year from now, or so.
>
>
> This gives a period of time after lock that bit 8 is kept and then a
> period where is is guaranteed to be zero.
>
> This gives software that is only watching the bit time to be upgraded and
> similarly time where the bit is set to zero before it can be reused.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 3181 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Mark Friedenbach]

[-- Attachment #1: Type: text/plain, Size: 2267 bytes --]

I think you misunderstand my suggestion Tier. I am suggesting the same as
BtcDrak, I think:

Modify IsSuperMajority to take an (optional) mask field. Bits set in that
mask are zero'd for the purpose of the IsSuperMajority calculation. For
this vote we mask bits 0x20000007.

Thus to signal support for CLTV/CSV/etc. (on Core, XT, or not-XT), you set
bit 4. On Core this would mean a minimum version of 0x8, on XT/not-XT a
minimum version of 0x20000008.

However the vote is still over whether to enforce BIP 65, 68, etc. for
blocks with nVersion>=4. So when this all clears up we haven't needlessly
wasted an additional bit.

On Wed, Aug 19, 2015 at 6:22 AM, Tier Nolan via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Wed, Aug 19, 2015 at 7:10 AM, Mark Friedenbach via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> We can use nVersion & 0x8 to signal support, while keeping the consensus
>> rule as nVersion >= 4, right? That way we don't waste a bit after this all
>> clears up.
>>
> What happens if XT gets 40% and this BIP gets 55%?  That gives 95% that
> accept the upgrade.  Version 3 and lower blocks need to be rejected after
> that.
>
> By advertising 0x7 for the last 3 bits, XT is effectively claiming to
> support the check locktime verify BIPs but they don't have the code.
>
> This sequence could be used, without a specific version-bits proposal.
>
> Until height = N + 5000, if 950 of the last 1000 blocks have the 0x8 bit
> set, then reject blocks with version numbers less than 8.
>
> At height N, if the above rule is active, then the BIP is permanent.
>
> It applies to any block with bit 0x8 set, once the 75% threshold is met.
>
> From height N + 5000 to N + 10000, reject blocks which have bit 0x8 set.
>
> N could be set 1 year from now, or so.
>
>
> This gives a period of time after lock that bit 8 is kept and then a
> period where is is guaranteed to be zero.
>
> This gives software that is only watching the bit time to be upgraded and
> similarly time where the bit is set to zero before it can be reused.
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 3385 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 3201 bytes --]

On Wed, Aug 19, 2015 at 09:32:49AM -0700, Mark Friedenbach via bitcoin-dev wrote:
> I think you misunderstand my suggestion Tier. I am suggesting the same as
> BtcDrak, I think:
> 
> Modify IsSuperMajority to take an (optional) mask field. Bits set in that
> mask are zero'd for the purpose of the IsSuperMajority calculation. For
> this vote we mask bits 0x20000007.
> 
> Thus to signal support for CLTV/CSV/etc. (on Core, XT, or not-XT), you set
> bit 4. On Core this would mean a minimum version of 0x8, on XT/not-XT a
> minimum version of 0x20000008.
> 
> However the vote is still over whether to enforce BIP 65, 68, etc. for
> blocks with nVersion>=4. So when this all clears up we haven't needlessly
> wasted an additional bit.

Ah, I see your point now re: wasting bits; my post was a bit incorrect
on that point.

So a subtle thing with the IsSuperMajority() mechanism, and the nVersion
bits proposal alternative, is one of the main objectives of the latter
proposal proposal is to allow forks to *fail* to be adopted cleanly.

To illustrate the problem, consider a hypothetical CLTV soft-fork,
implemented with IsSuperMajority() nVersion >= 4. We release Bitcoin
Core with that code, call it Bitcoin Core v0.12.0, however some
substantial fraction of the mining community refuses to upgrade,
believing CLTV to be a bad idea. This forms the community into Bitcoin
Core and Bitcoin Not-CLTV camps. The Not-CLTV camp then wants to do a
new soft-fork upgrade, say for CHECKSIG2

What now? If CHECKSIG2 is implemented via IsSuperMajority, nVersion >=
5, that'll falsely trigger Core nodes to think the upgrade has gone
though. You could safely define >= 5 semantics to be "OP_CLTV is now
disabled", but that's pretty ugly and unnecessarily uses up a NOP.

You can avoid this problem by assigning one bit out of nVersion for
every soft-fork, but then you can only do ~29 more soft-forks - ugly!


Come to think of it, if you're Really Sure™ the soft-fork will be
adopted, you can recycle those bits by using the following rule:

    if (IsFlagBitMaskSuperMajority(1 << 4, pindex->pprev) || block.nMedianTime > CLTV_SOFTFORK_DEADLINE) {
        flags |= SCRIPT_VERIFY_CLTV;
    }

IsFlagBitMaskSuperMajority() is a masked version of the existing
IsSuperMajority() logic. CLTV_SOFTFORK_DEADLINE would be set some
reasonable amount of time in the future, perhaps 3 years.

This would probably be ok for non-controversial forks - implementing
DERSIG this way would have been fine - and an unlimited number of
soft-forks can be done this way safely. (even in parallel)

However, this idea still causes problems if forks ever fail to get
adoption, something the nVersion bits proposal handles cleanly, albeit
at the cost of a significantly more complex implementation. With a
sufficiently far off fork deadline in practice that may not be a big
issue - nearly everyone would have upgraded their software anyway -  but
it's still technically creating hard-fork scenarios with respect to
older software whenever forks fail to get adoption.

-- 
'peter'[:-1]@petertodd.org
00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[jl2012]

Peter Todd via bitcoin-dev 於 2015-08-19 01:50 寫到:

> 
> 
> 2) nVersion mask, with IsSuperMajority()
> 
> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would
> be masked away, prior to applying standard IsSuperMajority() logic:
> 
>     block.nVersion & ~0x20000007
> 
> This means that CLTV/CSV/etc. miners running Bitcoin Core would create
> blocks with nVersion=8, 0b1000. From the perspective of the
> CLTV/CSV/etc.  IsSuperMajority() test, XT/Not-Bitcoin-XT miners would 
> be
> advertising blocks that do not trigger the soft-fork.
> 
> For the perpose of soft-fork warnings, the highest known version can
> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks
> as well as a future nVersion bits implementation. Equally,
> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an
> unknown bit set.
> 
> When nVersion bits is implemented by the Bitcoin protocol, the plan of
> setting the high bits to 0b001 still works. The three lowest bits will
> be unusable for some time, but will be eventually recoverable as
> XT/Not-Bitcoin-XT mining ceases.
> 
> Equally, further IsSuperMajority() softforks can be accomplished with
> the same masking technique.
> 
> This option does complicate the XT-coin protocol implementation in the
> future. But that's their problem, and anyway, the maintainers
> (Hearn/Andresen) has strenuously argued(5) against the use of 
> soft-forks
> and/or appear to be in favor of a more centralized mandatory update
> schedule.(6)
> 

If you are going to mask bits, would you consider to mask all bits 
except the 4th bit? So other fork proposals may use other bits for 
voting concurrently.

And as I understand, the masking is applied only during the voting 
stage? After the softfork is fully enforced with 95% support, the 
nVersion will be simply >=8, without any masking?

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Mark Friedenbach]

[-- Attachment #1: Type: text/plain, Size: 2320 bytes --]

No, the nVersion would be >= 4, so that we don't waste any version values.

On Thu, Aug 20, 2015 at 10:32 AM, jl2012 via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Peter Todd via bitcoin-dev 於 2015-08-19 01:50 寫到:
>
>
>>
>> 2) nVersion mask, with IsSuperMajority()
>>
>> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would
>> be masked away, prior to applying standard IsSuperMajority() logic:
>>
>>     block.nVersion & ~0x20000007
>>
>> This means that CLTV/CSV/etc. miners running Bitcoin Core would create
>> blocks with nVersion=8, 0b1000. From the perspective of the
>> CLTV/CSV/etc.  IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be
>> advertising blocks that do not trigger the soft-fork.
>>
>> For the perpose of soft-fork warnings, the highest known version can
>> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks
>> as well as a future nVersion bits implementation. Equally,
>> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an
>> unknown bit set.
>>
>> When nVersion bits is implemented by the Bitcoin protocol, the plan of
>> setting the high bits to 0b001 still works. The three lowest bits will
>> be unusable for some time, but will be eventually recoverable as
>> XT/Not-Bitcoin-XT mining ceases.
>>
>> Equally, further IsSuperMajority() softforks can be accomplished with
>> the same masking technique.
>>
>> This option does complicate the XT-coin protocol implementation in the
>> future. But that's their problem, and anyway, the maintainers
>> (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks
>> and/or appear to be in favor of a more centralized mandatory update
>> schedule.(6)
>>
>>
> If you are going to mask bits, would you consider to mask all bits except
> the 4th bit? So other fork proposals may use other bits for voting
> concurrently.
>
> And as I understand, the masking is applied only during the voting stage?
> After the softfork is fully enforced with 95% support, the nVersion will be
> simply >=8, without any masking?
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[-- Attachment #2: Type: text/html, Size: 3074 bytes --]

Re: [bitcoin-dev] CLTV/CSV/etc. deployment considerations due to XT/Not-BitcoinXT miners

[Btc Drak]

I have changed BIPS 112 and 113 to reflect this amended deployment
strategy. I'm beginning to think the issues created by Bitcoin XT are
so serious it probably deserves converting OPs text into an
informational BIP.

On Thu, Aug 20, 2015 at 6:42 PM, Mark Friedenbach via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> No, the nVersion would be >= 4, so that we don't waste any version values.
>
> On Thu, Aug 20, 2015 at 10:32 AM, jl2012 via bitcoin-dev
> <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>
>> Peter Todd via bitcoin-dev 於 2015-08-19 01:50 寫到:
>>
>>>
>>>
>>> 2) nVersion mask, with IsSuperMajority()
>>>
>>> In this option the nVersion bits set by XT/Not-Bitcoin-XT miners would
>>> be masked away, prior to applying standard IsSuperMajority() logic:
>>>
>>>     block.nVersion & ~0x20000007
>>>
>>> This means that CLTV/CSV/etc. miners running Bitcoin Core would create
>>> blocks with nVersion=8, 0b1000. From the perspective of the
>>> CLTV/CSV/etc.  IsSuperMajority() test, XT/Not-Bitcoin-XT miners would be
>>> advertising blocks that do not trigger the soft-fork.
>>>
>>> For the perpose of soft-fork warnings, the highest known version can
>>> remain nVersion=8, which is triggered by both XT/Not-Bitcoin-XT blocks
>>> as well as a future nVersion bits implementation. Equally,
>>> XT/Not-Bitcoin-XT soft-fork warnings will be triggered, by having an
>>> unknown bit set.
>>>
>>> When nVersion bits is implemented by the Bitcoin protocol, the plan of
>>> setting the high bits to 0b001 still works. The three lowest bits will
>>> be unusable for some time, but will be eventually recoverable as
>>> XT/Not-Bitcoin-XT mining ceases.
>>>
>>> Equally, further IsSuperMajority() softforks can be accomplished with
>>> the same masking technique.
>>>
>>> This option does complicate the XT-coin protocol implementation in the
>>> future. But that's their problem, and anyway, the maintainers
>>> (Hearn/Andresen) has strenuously argued(5) against the use of soft-forks
>>> and/or appear to be in favor of a more centralized mandatory update
>>> schedule.(6)
>>>
>>
>> If you are going to mask bits, would you consider to mask all bits except
>> the 4th bit? So other fork proposals may use other bits for voting
>> concurrently.
>>
>> And as I understand, the masking is applied only during the voting stage?
>> After the softfork is fully enforced with 95% support, the nVersion will be
>> simply >=8, without any masking?
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

[bitcoin-dev] BIP: Using Median time-past as endpoint for locktime calculations

[Thomas Kerin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi all,

In collaboration with Mark Friedenbach, we have drawn up a proposal for
using
the median time of the past 11 blocks in locktime calculations.

<pre>
  BIP: XX
  Title: Median time-past as endpoint for lock-time calculations
  Author: Thomas Kerin <me@thomaskerin.io>
          Mark Friedenbach <mark@friedenbach.org>    
  Status: Draft
  Type: Standards Track
  Created: 2015-08-10
</pre>


==Abstract==

This BIP is a proposal to redefine the semantics used in determining a
time-locked transaction's eligibility for inclusion in a block. The
median of the last 11 blocks is used instead of the block's timestamp,
ensuring that it increases monotonically with each block.


==Motivation==

At present, transactions are excluded from inclusion in a block if the
present time or block height is less than or equal to that specified
in the locktime. Since the consensus rules do not mandate strict
ordering of block timestamps, this has the unfortunate outcome of
creating a perverse incentive for miners to lie about the time of
their blocks in order to collect more fees by including transactions
that by wall clock determination have not yet matured.

This BIP proposes comparing the locktime against the median of the
past 11 block's timestamps, rather than the timestamp of the block
including the transaction. Existing consensus rules guarantee this
value to monotonically advance, thereby removing the capability for
miners to claim more transaction fees by lying about the timestamps of
their block.

This proposal seeks to ensure reliable behaviour in locktime calculations as
required by BIP65, BIP68, and BIPXX (OP_CHECKSEQUENCEVERIFY).


==Specification==

The values for transaction locktime remain unchanged. The difference is
only in
the calculation determining whether a transaction can be included.
Instead of
an unreliable timestamp, the following function is used to determine the
current
block time for the purpose of checking lock-time constraints:

    enum { nMedianTimeSpan=11 };
    
    int64_t GetMedianTimePast(const CBlockIndex* pindex)
    {
        int64_t pmedian[nMedianTimeSpan];
        int64_t* pbegin = &pmedian[nMedianTimeSpan];
        int64_t* pend = &pmedian[nMedianTimeSpan];
        for (int i = 0; i < nMedianTimeSpan && pindex; i++, pindex =
pindex->pprev)
             *(--pbegin) = pindex->GetBlockTime();
        std::sort(pbegin, pend);
        return pbegin[(pend - pbegin)/2];
    }

Lock-time constraints are checked by the consensus method IsFinalTx(),
or LockTime() under BIP68. These methods take the block time as one
parameter. This BIP proposes that after activation calls to
IsFinalTx() or LockTime() within consensus code use the return value
of `GetMedianTimePast(pindexPrev)` instead.

A reference implementation of this proposal is provided in the
following git repository:

https://github.com/maaku/bitcoin/tree/medianpasttimelock


==Deployment==

We reuse the double-threshold switchover mechanism from BIPs 34 and 66,
with the
same thresholds, but for block.nVersion = 4. The new rules are in effect for
every block (at height H) with nVersion = 4 and at least 750 out of 1000
blocks
preceding it (with heights H-1000...H-1) also have nVersion = 4.
Furthermore,
when 950 out of the 1000 blocks preceding a block do have nVersion = 4,
nVersion = 3 blocks become invalid, and all further blocks enforce the
new rules.

It is recommended that this soft-fork deployment trigger include other
related
proposals for improving Bitcoin's lock-time capabilities, such as BIP
65, BIP68
and CHECKSEQUENCEVERIFY.


==Acknowledgements==

Mark Friedenbach for designing and authoring the reference
implementation of this BIP.

Thomas Kerin authored this BIP document.


==Compatibility==

Transactions generated using time-based lock-time will take
approximately an hour longer to confirm than would be expected under
the old rules. This is not known to introduce any compatibility
concerns with existing protocols.


==References==
[https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki BIP65:
OP_CHECKLOCKTIMEVERIFY]

[https://github.com/bitcoin/bips/blob/master/bip-0068.mediawiki BIP68:
Consensus-enforced transaction replacement signaled via sequence numbers]

[https://github.com/bitcoin/bips/blob/master/bip-00.mediawiki BIPXX:
CHECKSEQUENCEVERIFY]


==Copyright==

This document is placed in the public domain.




- -- 
My PGP key can be found here: <https://thomaskerin.io/me.pub.asc>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJV0oi8AAoJEAiDZR291eTl2soP/1MOjgQDncoUdMptqfeqMLfU
ewENNPLQwXXje7PFn/gIVa+Ghxu+f9rrRHt6v8Udd4wsnDTqhz2gV6dKCyF0K4IS
seLTH2kyTfPGm1KOp6WSwvxoyc5iWLBH4wkSm4oI9WmXkLzDq0yEYUDE8t9yNYwf
0Fgrg1KPIP4bhoxWchEa237rrH/qTh0Zdxdj/N0YCrX9u4fBy+xoTM6gnt0bFCK2
SaGXvC8PsA23gkJjjwFnWh/JU0Q5BJTElUsq1re3gmwcnLNKyB5cx0bFephk2pFd
NC3rqEIIVPd7aLs+lWmD4/NXdm+VtUEQo3MmQ1YW5zwjeoJxZhfMfXwmQw3vw2f7
FSyExUXNNwh2lMoLCcWvWWEOKYaSV9iLX4TacvpbOSDQgz3rDl3iqeLmSgp3S8M3
Se1S9AzilJsT0jIe2Ob2hu/gXEXeBmI9k2kRJELSaIFgCWadUky63NwNNfRipiBq
USroBIym2dpXFLygcwgwf6F/yAYYg6/5QiUKclhqvxArxVEcijw18SHGZVYpW83S
Q0mzJnRVGF7yscJl84zHyAj5QMWoMFgKSqFbOLcmNDUPLoaFJxAGezGCLXNaHinA
LY5Qp0t0Vg4hXi6QcCiWv2U8E1K4oN5VZNSlagUyXsAHd3c4icZTVj+TTWKJ7GLB
Gmbe3i9G90rpgDbHWXFq
=EQdY
-----END PGP SIGNATURE-----

Re: [bitcoin-dev] BIP: Using Median time-past as endpoint for locktime calculations

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 640 bytes --]

On Tue, Aug 18, 2015 at 02:22:10AM +0100, Thomas Kerin via bitcoin-dev wrote:
> ==Acknowledgements==
> 
> Mark Friedenbach for designing and authoring the reference
> implementation of this BIP.
> 
> Thomas Kerin authored this BIP document.

IIRC Gregory Maxwell came up with the actual idea in question, during a
discussion between myself and Luke-Jr about incentives related to nTime
on #bitcoin-dev or #bitcoin-wizards. He should get credit for it; I'll
see if I can dig up a link to the discussion for historical interest.

-- 
'peter'[:-1]@petertodd.org
00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] BIP: Using Median time-past as endpoint for locktime calculations

[Mark Friedenbach]

[-- Attachment #1: Type: text/plain, Size: 1088 bytes --]

You are absolutely correct! My apologies for the oversight in editing. If
you could dig up the link though that would be really helpful.

On Tue, Aug 18, 2015 at 6:04 PM, Peter Todd via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> On Tue, Aug 18, 2015 at 02:22:10AM +0100, Thomas Kerin via bitcoin-dev
> wrote:
> > ==Acknowledgements==
> >
> > Mark Friedenbach for designing and authoring the reference
> > implementation of this BIP.
> >
> > Thomas Kerin authored this BIP document.
>
> IIRC Gregory Maxwell came up with the actual idea in question, during a
> discussion between myself and Luke-Jr about incentives related to nTime
> on #bitcoin-dev or #bitcoin-wizards. He should get credit for it; I'll
> see if I can dig up a link to the discussion for historical interest.
>
> --
> 'peter'[:-1]@petertodd.org
> 00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

[-- Attachment #2: Type: text/html, Size: 1811 bytes --]

Re: [bitcoin-dev] BIP: Using Median time-past as endpoint for locktime calculations

[Thomas Kerin]

[-- Attachment #1: Type: text/plain, Size: 2476 bytes --]


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

I submitted the pull-request for the median-past timelock BIP just now

https://github.com/bitcoin/bips/pull/182

Any luck finding the link to this discussion? It would be nice to
include this for posterity.


On 19/08/15 02:08, Mark Friedenbach wrote:
> You are absolutely correct! My apologies for the oversight in editing. If you could dig up the link
though that would be really helpful.
>
> On Tue, Aug 18, 2015 at 6:04 PM, Peter Todd via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org
<mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>
>     On Tue, Aug 18, 2015 at 02:22:10AM +0100, Thomas Kerin via
bitcoin-dev wrote:
>     > ==Acknowledgements==
>     >
>     > Mark Friedenbach for designing and authoring the reference
>     > implementation of this BIP.
>     >
>     > Thomas Kerin authored this BIP document.
>
>     IIRC Gregory Maxwell came up with the actual idea in question,
during a
>     discussion between myself and Luke-Jr about incentives related to
nTime
>     on #bitcoin-dev or #bitcoin-wizards. He should get credit for it; I'll
>     see if I can dig up a link to the discussion for historical interest.
>
>     --
>     'peter'[:-1]@petertodd.org <http://petertodd.org>
>     00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d
>
>     _______________________________________________
>     bitcoin-dev mailing list
>     bitcoin-dev@lists.linuxfoundation.org
<mailto:bitcoin-dev@lists.linuxfoundation.org>
>     https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

- -- 
My PGP key can be found here: <https://thomaskerin.io/me.pub.asc>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJV1we8AAoJEAiDZR291eTlkdwP/Rv3wE1kUSGe62Im9VIm49ZW
hL71+9+C9BMyRWfUjizqGi/FgWzKlS5dVrKJdNnzV5g1KjZYIWF7Zy824Gnv8hrE
zW2pJc4vgu8igjtO+Qm/Wi+yy5WNjQ9qhRO+chELx4JLvD5JalagrBP2V693gNNd
g2K6fjXlogx/Rw4mK2pxYlUxgaWf2j+iJ2v09238+aWuXruTirpf/HJ70Pxwm4oq
LvJervkEUtUJLiwqsiwAQwqjesFPfuV1nYCi48G0/QfYK7tpajGD7NkO89RBNnP8
P6XGouBqulSvzc3QgtQqPIfOdEtV2Qi8Aohv+ebttaxFM8LSycvG9wxrQ+5Y/F4w
h7HDkn7ziM8MVS+KfFuuyfKxhDKdfpgu61r1YTzt7TyMSzdvQ8c9XhYCN3/AvL9S
6kwOeWTONVNpQbhI3WkzRPpTpaInLm1LkeXWuZVy9jbUtD8tlI0a6HmLzIrhcpdS
keOZiH1eV1nritrEFeToaTNL09MjyfdJk0kN9dr6fI3eOV/cPQkY52iCHWLMj+WY
X2wWwMPiNzfkgmMC+GvcQNmGSklaIr2OoUuT92q4M1pVi24vPNSvVy00W4+klXgK
UsCIMzo+n1OLRV8I1zuYkCAwOKzzJlvMw/7bggLxhuNsLubtM0HvDjq0uquE+zu6
BO12PmW8qOA2KqyeYEVn
=1Fst
-----END PGP SIGNATURE-----


[-- Attachment #2: Type: text/html, Size: 4193 bytes --]

Re: [bitcoin-dev] BIP: Using Median time-past as endpoint for locktime calculations

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 12063 bytes --]

On Fri, Aug 21, 2015 at 12:13:09PM +0100, Thomas Kerin wrote:
> 
> I submitted the pull-request for the median-past timelock BIP just now
> 
> https://github.com/bitcoin/bips/pull/182
> 
> Any luck finding the link to this discussion? It would be nice to
> include this for posterity.

Found it! From #bitcoin-wizards, 2013-07-16:

23:57 < petertodd> See, it'd be possible for nLockTime w/ time-based locks to create some really ugly incentives for miners to mine blocks at thelimit of the 2hr window - a timestamping chain could provide a way for nodes to at least detect that their clocks are off, especially given how peers can mess with them.
23:58 < petertodd> It's still dodgy though... I was thinking if nLockTime-by-time inclusion was based on the previous block timestamp it'd be ok, but that still leaves large miners with incentives to screw with the 2hr window, never mind how it can reduce competition if there exists clock skew in the mining nodes.
--- Log closed Wed Jul 17 00:00:57 2013
--- Log opened Wed Jul 17 00:00:57 2013
00:01 < petertodd> (remember that if this is a timestamping facility any node wanting to know the current time simply gets a nonce timestamped, and then they know what time it is!)
00:11 < Luke-Jr> I don't see how nLockTime can discourage forward-dating blocks
00:11 < Luke-Jr> and there is no 2hr window backward..
00:12 < Luke-Jr> well, I guess if miners are behaving there is <.<
00:19 < petertodd> The problem is a block being created with nTime > actual time, and the incentive is to get a head start on other miners to put, say, a high-fee nLockTime in the block you are creating.
00:21 < Luke-Jr> petertodd: but nLockTime only sets a minimum time, it cannot set a maximum
00:22 < petertodd> but that's it, if I have a 1BTC fee tx, with nLockTime expiring in two hours, why not take the increased orphan chance and set nTime on my block to two hours ahead/
00:22 < petertodd> ?
00:22 < petertodd> yet if we allow that incentive, it's very bad for consensus
00:23 < gmaxwell> ha. We can fix.
00:23 < gmaxwell> it's a soft forking fix.
00:23 < gmaxwell> use the last blocks ntime, not this one.
00:23 < Luke-Jr> is sipa's secp256k1 branch reasonably stable?
00:23 < petertodd> gmaxwell: that's what I said...
00:24 < gmaxwell> petertodd: sorry I just read the last couple lines.
00:24 < Luke-Jr> petertodd: AFAIK we already don't relay transactions with time in the future?
00:24 < gmaxwell> petertodd: well I agree. (or not even the last block— it could use the minimum time)
00:24 < petertodd> gmaxwell: The problem is, that's only a fix if mining power is well distributed, it actually makes things worse because if there is a lot of profit to be gained the miners with a lot of hashing power still have the incentive, and it's to a much greater degree. (their orphan rate is less)
00:24 < Luke-Jr> gmaxwell: the minimum time will be earlier than the last block's :p
00:25 < gmaxwell> Luke-Jr: sure, but that doesn't change it really. Presumably if people start locking in the future miners will run nodes that take what they get and selfishly horde them, creating incentives for all miners to run good collection networks.
00:25 < petertodd> Luke-Jr: sure, but there are lots of ways to learn that a tx exists
00:26 < gmaxwell> petertodd: one of the reasons that the min is important there is because (1) it's hard to advance, and (2) when you advance it you raise the difficulty.
00:26 < petertodd> gmaxwell: I was working on figuring out the expected return - the math is really ugly
00:27 < gmaxwell> petertodd: a worst case expected return may be easier.
00:27 < petertodd> gmaxwell: Worst case is easy - your block is orphaned.
00:28 < petertodd> gmaxwell: See the issue is that once I find a block, the other side needs to find two blocks to beat me. As time goes on more of the other sides hashing power will accept my from the future block as valid, so then you get the next level where the remainder needs three blocks and so on.
00:28 < petertodd> gmaxwell: Pretty sure it can't be done as a closed-form equation.
00:30 < petertodd> gmaxwell: I don't think minimum time works either, because you still get to manipulate it by creating blocks in the future, although the ability too is definitely less. If I could show you'd need >50% hashing power to do anything interesting I'd be set.
00:31 < Luke-Jr> petertodd: hmm, is block-uneconomic-utxo-creation basically just an older revision of what Gavin did in 0.8.2?
00:31 < gmaxwell> petertodd: moving the minimum time forward needs the coperation of >50% of the hashpower over the small median window.
00:32 < petertodd> Luke-Jr: It's what Gavin did but non-hardcoded. I'd emphasize the better, not the older. :P
00:32 < Luke-Jr> petertodd: will you be rebasing it despite its closed status?
00:32 < Luke-Jr> actually, what about Gavin's is hardcoded? <.<
00:33 < petertodd> gmaxwell: Yeah, but you have to assume a steady stream of these incentives.
00:33 < gmaxwell> petertodd: right, so you have some force that turns all miners into a conspiracy.
00:34 < petertodd> gmaxwell: exactly
00:34 < petertodd> gmaxwell: nLockTime by time should have never been added in the first place, but it's such a nice idea on the face of it
00:35 -!- realazthat is now known as rudeasthat
00:35 -!- rudeasthat is now known as rudest
00:35 < Luke-Jr> softfork so nLockTime requires data on what block a transaction was created at, and enforces the 10 min per block <.<
00:36 -!- rudest is now known as heh
00:36 < petertodd> Luke-Jr: ?
00:36 -!- heh is now known as realz
00:36 < Luke-Jr> petertodd: for example, if you nLockTime for 1 day from now, it also enforces 144 blocks passing too
00:37 < Luke-Jr> so block count must be >now+144 AND time must be >now+24h
00:37 < Luke-Jr> not perfect, but might help
00:37 < petertodd> Still doesn't help in the usual case where mean interval is < 10 minutes, because you're back to only caring about time.
00:38 < Luke-Jr> usual now, but not eventually
00:38 < petertodd> Right, you've solved half the problem, when measured over the entire lifespan of Bitcoin, and only approximately half. :P
00:39 < Luke-Jr> theory is so much nicer than practice <.<
00:39 < gmaxwell> I'm forgetting why this is a problem again?  If miners mine blocks early, people will just artifically inflate their times or switch to height locking.
00:39 < petertodd> The problem is you're incentivising miners to make the 2hr window for block acceptance effectively shorter.
00:39 < petertodd> Thus requiring accurate clocks for consensus.
00:39 < gmaxwell> if miners do this consistently they'll drive difficulty up too which wouldn't be in their interest.
00:39 < Luke-Jr> ^
00:40 < petertodd> gmaxwell: It's only a fixed 2hr offset, that just drives difficulty up by 0.5%
00:40 < Luke-Jr> and on top of that, you'd just end up treating nTime with a minus-2-hours :p
00:41 < Luke-Jr> if everyone does it, it's predictable.
00:41 < petertodd> More to the point for any individual miner the marginal difference if they do it is effectively zero.
00:41 < gmaxwell> consider, why why cant the 2 hour window be 24 hours?
00:41 < petertodd> Luke-Jr: But that's the problem, if everyone does it, and people respond, then you can extend the interval even further!
00:41 < Luke-Jr> petertodd: how?
00:41 < petertodd> gmaxwell: It should have been more like 24 hours in the first place...
00:42 < Luke-Jr> you don't change the 2h rule
00:42 < Luke-Jr> you just assume miner times will always be up against it
00:42 < gmaxwell> Luke-Jr: move your clock/window forward so you dont reject stupid blocks.
00:42 < petertodd> Luke-Jr: Again, the issue is the effect on *consusus*. I don't care when the tx gets mined, I care that miners are incentivised to break consunsus for anyone without NTP.
00:43 < petertodd> The problem is no matter *what* the window is, there is an incentive to mine as close to the window as possible to accept a tx sooner than your competitors.
00:43 < petertodd> It could be a week and people would still have an incentive to set nTime + 1 week - 1 second
00:44 < Luke-Jr> if nTime is future, wait until that time before relaying it? <.<
00:44 -!- realz is now known as pleasedont
00:44 < gmaxwell> and once people did that, you'd want to start accepting blocks that where nTime + 1 week because god knows you don't want to reject a block if your clock was 2 seconds slow and most hashpower accepted it.
00:44 < petertodd> About the only thing that might change that is if the rule was nLockTime > nTime of last block, and then after that being allowed to include a tx was based on H(txhash, last hash) or similar
00:45 < petertodd> gmaxwell: exactly, the fundemental issue is there is no good incentive to set nTime accurately other than miners rejecting your blocks, and nLockTime sabotages that
00:45 -!- pleasedont is now known as realzies
00:45 < petertodd> gmaxwell: (timestamping could do, but the cause->effect is less obvious)
00:45 < Luke-Jr> I guess I just incentivized always setting nTime to the minimum then
00:45 < Luke-Jr> [04:32:26] <Luke-Jr> petertodd: will you be rebasing it despite its closed status? (block-uneconomic-utxo-creation)
00:46 < petertodd> Luke-Jr: again, relaying does nothing - consider the case of nLockTime'd fidelity bonds where it's guaranteed 100% of the hashing power know (why I wrote the spec as by-block-height in the first place)
00:46 < petertodd> Luke-Jr: sure
00:46 < Luke-Jr> petertodd: I mean delaying relaying the BLOCK
00:46 < Luke-Jr> ie, increasing the risk of it being stale
00:47 < petertodd> Luke-Jr: then you have your mining pool connect directly to other mining pools playing the same game
00:47 < petertodd> you have to assume perfect information knowledge in this stuff, at least if you're writing worst-case academic papers
00:48 < gmaxwell> petertodd: so ... prior block vs minimum time.
00:48 < petertodd> see, that's why I was talking about timestamping, because it provides a way for all users to set their clocks to what the majority of hashing power thinks nTime is, sidestepping the problem
00:48 < gmaxwell> petertodd: what are your arguments there?
00:48 < petertodd> gmaxwell: minimum time is definitely stronger because it involves more hashing power
00:49 < petertodd> gmaxwell: users would prefer minimum time - easier to understand why the tx isn't getting mined
00:49 < gmaxwell> sidestepping the problem < that doesn't sidestep the problem, it would allow the majority of hashpower to mine difficulty down to 1; also moots nlocktime as _time_ being more reliable than a height.
00:49 < gmaxwell> petertodd: plus, you can just add a constant offset to your nlocktime to adjust for the expected minimum lag.
00:51 < petertodd> gmaxwell: yes, it creates a new problem, but it did sidestep the existing one :P
00:51 < gmaxwell> petertodd: yea, lol, creates an inflation attack. Keep it up and you'll be qualified to create an altcoin. :P
00:52 < gmaxwell> (sorry, hah, I'm not poking fun at you, I'm poking fun at all the altcoins that "solved the Foo problem" where foo is something no one else thinks is a problem and they totally broke security as a side effect)
00:52 < petertodd> gmaxwell: yup, now you see how it only sidesteps the problem truly when there is enough hashing power setting their clocks back, IE 50% honest, which is better
00:53 < petertodd> gmaxwell: without the timestamping, nodes have the consensus failures, which can be attacked, likely it trades off one risk for a more existential risk
00:53 < petertodd> gmaxwell: and it's a good excuse for timestamping, lol
00:54 < gmaxwell> I thin the min solves the consensus failure so long as hashpower is well distributed.
00:54 < petertodd> yeah, I'm thinking min is probably the best we can do

https://download.wpsoftware.net/bitcoin/wizards/2013/07/13-07-16.log

-- 
'peter'[:-1]@petertodd.org
00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] BIP: Using Median time-past as endpoint for locktime calculations

[Btc Drak]

This BIP was assigned number 113.

I have updated the text accordingly and added credits to Gregory Maxwell.

Please see the changes in the pull request:
https://github.com/bitcoin/bips/pull/182

On Sat, Aug 22, 2015 at 1:57 AM, Peter Todd via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> On Fri, Aug 21, 2015 at 12:13:09PM +0100, Thomas Kerin wrote:
>>
>> I submitted the pull-request for the median-past timelock BIP just now
>>
>> https://github.com/bitcoin/bips/pull/182
>>
>> Any luck finding the link to this discussion? It would be nice to
>> include this for posterity.
>
> Found it! From #bitcoin-wizards, 2013-07-16:
>
> 23:57 < petertodd> See, it'd be possible for nLockTime w/ time-based locks to create some really ugly incentives for miners to mine blocks at thelimit of the 2hr window - a timestamping chain could provide a way for nodes to at least detect that their clocks are off, especially given how peers can mess with them.
> 23:58 < petertodd> It's still dodgy though... I was thinking if nLockTime-by-time inclusion was based on the previous block timestamp it'd be ok, but that still leaves large miners with incentives to screw with the 2hr window, never mind how it can reduce competition if there exists clock skew in the mining nodes.
> --- Log closed Wed Jul 17 00:00:57 2013
> --- Log opened Wed Jul 17 00:00:57 2013
> 00:01 < petertodd> (remember that if this is a timestamping facility any node wanting to know the current time simply gets a nonce timestamped, and then they know what time it is!)
> 00:11 < Luke-Jr> I don't see how nLockTime can discourage forward-dating blocks
> 00:11 < Luke-Jr> and there is no 2hr window backward..
> 00:12 < Luke-Jr> well, I guess if miners are behaving there is <.<
> 00:19 < petertodd> The problem is a block being created with nTime > actual time, and the incentive is to get a head start on other miners to put, say, a high-fee nLockTime in the block you are creating.
> 00:21 < Luke-Jr> petertodd: but nLockTime only sets a minimum time, it cannot set a maximum
> 00:22 < petertodd> but that's it, if I have a 1BTC fee tx, with nLockTime expiring in two hours, why not take the increased orphan chance and set nTime on my block to two hours ahead/
> 00:22 < petertodd> ?
> 00:22 < petertodd> yet if we allow that incentive, it's very bad for consensus
> 00:23 < gmaxwell> ha. We can fix.
> 00:23 < gmaxwell> it's a soft forking fix.
> 00:23 < gmaxwell> use the last blocks ntime, not this one.
> 00:23 < Luke-Jr> is sipa's secp256k1 branch reasonably stable?
> 00:23 < petertodd> gmaxwell: that's what I said...
> 00:24 < gmaxwell> petertodd: sorry I just read the last couple lines.
> 00:24 < Luke-Jr> petertodd: AFAIK we already don't relay transactions with time in the future?
> 00:24 < gmaxwell> petertodd: well I agree. (or not even the last block— it could use the minimum time)
> 00:24 < petertodd> gmaxwell: The problem is, that's only a fix if mining power is well distributed, it actually makes things worse because if there is a lot of profit to be gained the miners with a lot of hashing power still have the incentive, and it's to a much greater degree. (their orphan rate is less)
> 00:24 < Luke-Jr> gmaxwell: the minimum time will be earlier than the last block's :p
> 00:25 < gmaxwell> Luke-Jr: sure, but that doesn't change it really. Presumably if people start locking in the future miners will run nodes that take what they get and selfishly horde them, creating incentives for all miners to run good collection networks.
> 00:25 < petertodd> Luke-Jr: sure, but there are lots of ways to learn that a tx exists
> 00:26 < gmaxwell> petertodd: one of the reasons that the min is important there is because (1) it's hard to advance, and (2) when you advance it you raise the difficulty.
> 00:26 < petertodd> gmaxwell: I was working on figuring out the expected return - the math is really ugly
> 00:27 < gmaxwell> petertodd: a worst case expected return may be easier.
> 00:27 < petertodd> gmaxwell: Worst case is easy - your block is orphaned.
> 00:28 < petertodd> gmaxwell: See the issue is that once I find a block, the other side needs to find two blocks to beat me. As time goes on more of the other sides hashing power will accept my from the future block as valid, so then you get the next level where the remainder needs three blocks and so on.
> 00:28 < petertodd> gmaxwell: Pretty sure it can't be done as a closed-form equation.
> 00:30 < petertodd> gmaxwell: I don't think minimum time works either, because you still get to manipulate it by creating blocks in the future, although the ability too is definitely less. If I could show you'd need >50% hashing power to do anything interesting I'd be set.
> 00:31 < Luke-Jr> petertodd: hmm, is block-uneconomic-utxo-creation basically just an older revision of what Gavin did in 0.8.2?
> 00:31 < gmaxwell> petertodd: moving the minimum time forward needs the coperation of >50% of the hashpower over the small median window.
> 00:32 < petertodd> Luke-Jr: It's what Gavin did but non-hardcoded. I'd emphasize the better, not the older. :P
> 00:32 < Luke-Jr> petertodd: will you be rebasing it despite its closed status?
> 00:32 < Luke-Jr> actually, what about Gavin's is hardcoded? <.<
> 00:33 < petertodd> gmaxwell: Yeah, but you have to assume a steady stream of these incentives.
> 00:33 < gmaxwell> petertodd: right, so you have some force that turns all miners into a conspiracy.
> 00:34 < petertodd> gmaxwell: exactly
> 00:34 < petertodd> gmaxwell: nLockTime by time should have never been added in the first place, but it's such a nice idea on the face of it
> 00:35 -!- realazthat is now known as rudeasthat
> 00:35 -!- rudeasthat is now known as rudest
> 00:35 < Luke-Jr> softfork so nLockTime requires data on what block a transaction was created at, and enforces the 10 min per block <.<
> 00:36 -!- rudest is now known as heh
> 00:36 < petertodd> Luke-Jr: ?
> 00:36 -!- heh is now known as realz
> 00:36 < Luke-Jr> petertodd: for example, if you nLockTime for 1 day from now, it also enforces 144 blocks passing too
> 00:37 < Luke-Jr> so block count must be >now+144 AND time must be >now+24h
> 00:37 < Luke-Jr> not perfect, but might help
> 00:37 < petertodd> Still doesn't help in the usual case where mean interval is < 10 minutes, because you're back to only caring about time.
> 00:38 < Luke-Jr> usual now, but not eventually
> 00:38 < petertodd> Right, you've solved half the problem, when measured over the entire lifespan of Bitcoin, and only approximately half. :P
> 00:39 < Luke-Jr> theory is so much nicer than practice <.<
> 00:39 < gmaxwell> I'm forgetting why this is a problem again?  If miners mine blocks early, people will just artifically inflate their times or switch to height locking.
> 00:39 < petertodd> The problem is you're incentivising miners to make the 2hr window for block acceptance effectively shorter.
> 00:39 < petertodd> Thus requiring accurate clocks for consensus.
> 00:39 < gmaxwell> if miners do this consistently they'll drive difficulty up too which wouldn't be in their interest.
> 00:39 < Luke-Jr> ^
> 00:40 < petertodd> gmaxwell: It's only a fixed 2hr offset, that just drives difficulty up by 0.5%
> 00:40 < Luke-Jr> and on top of that, you'd just end up treating nTime with a minus-2-hours :p
> 00:41 < Luke-Jr> if everyone does it, it's predictable.
> 00:41 < petertodd> More to the point for any individual miner the marginal difference if they do it is effectively zero.
> 00:41 < gmaxwell> consider, why why cant the 2 hour window be 24 hours?
> 00:41 < petertodd> Luke-Jr: But that's the problem, if everyone does it, and people respond, then you can extend the interval even further!
> 00:41 < Luke-Jr> petertodd: how?
> 00:41 < petertodd> gmaxwell: It should have been more like 24 hours in the first place...
> 00:42 < Luke-Jr> you don't change the 2h rule
> 00:42 < Luke-Jr> you just assume miner times will always be up against it
> 00:42 < gmaxwell> Luke-Jr: move your clock/window forward so you dont reject stupid blocks.
> 00:42 < petertodd> Luke-Jr: Again, the issue is the effect on *consusus*. I don't care when the tx gets mined, I care that miners are incentivised to break consunsus for anyone without NTP.
> 00:43 < petertodd> The problem is no matter *what* the window is, there is an incentive to mine as close to the window as possible to accept a tx sooner than your competitors.
> 00:43 < petertodd> It could be a week and people would still have an incentive to set nTime + 1 week - 1 second
> 00:44 < Luke-Jr> if nTime is future, wait until that time before relaying it? <.<
> 00:44 -!- realz is now known as pleasedont
> 00:44 < gmaxwell> and once people did that, you'd want to start accepting blocks that where nTime + 1 week because god knows you don't want to reject a block if your clock was 2 seconds slow and most hashpower accepted it.
> 00:44 < petertodd> About the only thing that might change that is if the rule was nLockTime > nTime of last block, and then after that being allowed to include a tx was based on H(txhash, last hash) or similar
> 00:45 < petertodd> gmaxwell: exactly, the fundemental issue is there is no good incentive to set nTime accurately other than miners rejecting your blocks, and nLockTime sabotages that
> 00:45 -!- pleasedont is now known as realzies
> 00:45 < petertodd> gmaxwell: (timestamping could do, but the cause->effect is less obvious)
> 00:45 < Luke-Jr> I guess I just incentivized always setting nTime to the minimum then
> 00:45 < Luke-Jr> [04:32:26] <Luke-Jr> petertodd: will you be rebasing it despite its closed status? (block-uneconomic-utxo-creation)
> 00:46 < petertodd> Luke-Jr: again, relaying does nothing - consider the case of nLockTime'd fidelity bonds where it's guaranteed 100% of the hashing power know (why I wrote the spec as by-block-height in the first place)
> 00:46 < petertodd> Luke-Jr: sure
> 00:46 < Luke-Jr> petertodd: I mean delaying relaying the BLOCK
> 00:46 < Luke-Jr> ie, increasing the risk of it being stale
> 00:47 < petertodd> Luke-Jr: then you have your mining pool connect directly to other mining pools playing the same game
> 00:47 < petertodd> you have to assume perfect information knowledge in this stuff, at least if you're writing worst-case academic papers
> 00:48 < gmaxwell> petertodd: so ... prior block vs minimum time.
> 00:48 < petertodd> see, that's why I was talking about timestamping, because it provides a way for all users to set their clocks to what the majority of hashing power thinks nTime is, sidestepping the problem
> 00:48 < gmaxwell> petertodd: what are your arguments there?
> 00:48 < petertodd> gmaxwell: minimum time is definitely stronger because it involves more hashing power
> 00:49 < petertodd> gmaxwell: users would prefer minimum time - easier to understand why the tx isn't getting mined
> 00:49 < gmaxwell> sidestepping the problem < that doesn't sidestep the problem, it would allow the majority of hashpower to mine difficulty down to 1; also moots nlocktime as _time_ being more reliable than a height.
> 00:49 < gmaxwell> petertodd: plus, you can just add a constant offset to your nlocktime to adjust for the expected minimum lag.
> 00:51 < petertodd> gmaxwell: yes, it creates a new problem, but it did sidestep the existing one :P
> 00:51 < gmaxwell> petertodd: yea, lol, creates an inflation attack. Keep it up and you'll be qualified to create an altcoin. :P
> 00:52 < gmaxwell> (sorry, hah, I'm not poking fun at you, I'm poking fun at all the altcoins that "solved the Foo problem" where foo is something no one else thinks is a problem and they totally broke security as a side effect)
> 00:52 < petertodd> gmaxwell: yup, now you see how it only sidesteps the problem truly when there is enough hashing power setting their clocks back, IE 50% honest, which is better
> 00:53 < petertodd> gmaxwell: without the timestamping, nodes have the consensus failures, which can be attacked, likely it trades off one risk for a more existential risk
> 00:53 < petertodd> gmaxwell: and it's a good excuse for timestamping, lol
> 00:54 < gmaxwell> I thin the min solves the consensus failure so long as hashpower is well distributed.
> 00:54 < petertodd> yeah, I'm thinking min is probably the best we can do
>
> https://download.wpsoftware.net/bitcoin/wizards/2013/07/13-07-16.log
>
> --
> 'peter'[:-1]@petertodd.org
> 00000000000000000402fe6fb9ad613c93e12bddfc6ec02a2bd92f002050594d
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>

Re: [bitcoin-dev] BIP: Using Median time-past as endpoint for locktime calculations

[Peter Todd]

[-- Attachment #1: Type: text/plain, Size: 1917 bytes --]

On Thu, Aug 27, 2015 at 11:08:32PM +0100, Btc Drak wrote:
> This BIP was assigned number 113.
> 
> I have updated the text accordingly and added credits to Gregory Maxwell.
> 
> Please see the changes in the pull request:
> https://github.com/bitcoin/bips/pull/182

On Thu, Aug 27, 2015 at 11:11:10PM +0100, Btc Drak via bitcoin-dev wrote:
> I have changed BIPS 112 and 113 to reflect this amended deployment
> strategy. I'm beginning to think the issues created by Bitcoin XT are
> so serious it probably deserves converting OPs text into an
> informational BIP.

I thought we had decided that the masking thing doesn't work as
intended?

To recap, XT nodes are producing blocks with nVersion=0b001...111

You're suggesting that we apply a mask of ~0b001...111 then trigger the
soft-fork on nVersion >= 0b0...100 == 4, with miners producing blocks with
nVersion=0b0...1000

That will work, but it still uses up a version bit. The reason why is
blocks with nVersion=0b001...000 - the intended deployment of the
nVersion bits proposal - will be rejected by the nVersion >= 4 rule,
hard-forking them off the network. In short, we have in fact "burnt" a
version bit unnecessarily.

If you're going to accept hard-forking some people off the network, why
not just go with my stateless nVersion bits w/ time-expiration proposal
instead? The only case where it leads to a hard-fork is if a soft-fork
has been rejected by the time the upgrade deadline is reached. It's easy
to set this multiple years into the future, so I think in practice it
won't be a major issue for non-controversial soft-forks.

Equally, spending the time to implement the original stateful nVersion
bits proposal is possible as well, though higher risk due to the extra
complexity of tracking soft-fork state.

-- 
'peter'[:-1]@petertodd.org
000000000000000008ba8215b2b644e33a98a762fd40710bc5e8c7f1b0e78375

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 650 bytes --]

Re: [bitcoin-dev] BIP: Using Median time-past as endpoint for locktime calculations

[jl2012]

I have an ugly solution to this problem, with minimal change to the 
current softfork logic, and still allows all features described in 
sipa's Version bits BIP

1. xVersion = nVersion AND 0b10000000000000000000000000011000
2. miners supporting BIP65 will set xVersion = 8 or greater
3. If 750 of the last 1000 blocks are xVersion >= 8, reject invalid 
xVersion 8 (or greater) blocks
4. If 950 of the last 1000 blocks are xVersion >= 8, reject all blocks 
with xVersion < 8

So the logic is exactly the same as BIP66, with the AND masking in step 
1. After the BIP65 softfork is implied, xVersion may take only one of 
the following values: 8, 16, 24

This is basically moving the "high bits" in sipa's proposal to the 
middle of the nVersion field. After the BIP65 softfork, this will still 
leave 29 available bits for parallel soft forks, just in different 
position.

This is ugly, but I believe this is the easiest solution

Ref: https://gist.github.com/sipa/bf69659f43e763540550

Peter Todd via bitcoin-dev 於 2015-08-27 19:19 寫到:
> On Thu, Aug 27, 2015 at 11:08:32PM +0100, Btc Drak wrote:
>> This BIP was assigned number 113.
>> 
>> I have updated the text accordingly and added credits to Gregory 
>> Maxwell.
>> 
>> Please see the changes in the pull request:
>> https://github.com/bitcoin/bips/pull/182
> 
> On Thu, Aug 27, 2015 at 11:11:10PM +0100, Btc Drak via bitcoin-dev 
> wrote:
>> I have changed BIPS 112 and 113 to reflect this amended deployment
>> strategy. I'm beginning to think the issues created by Bitcoin XT are
>> so serious it probably deserves converting OPs text into an
>> informational BIP.
> 
> I thought we had decided that the masking thing doesn't work as
> intended?
> 
> To recap, XT nodes are producing blocks with nVersion=0b001...111
> 
> You're suggesting that we apply a mask of ~0b001...111 then trigger the
> soft-fork on nVersion >= 0b0...100 == 4, with miners producing blocks 
> with
> nVersion=0b0...1000
> 
> That will work, but it still uses up a version bit. The reason why is
> blocks with nVersion=0b001...000 - the intended deployment of the
> nVersion bits proposal - will be rejected by the nVersion >= 4 rule,
> hard-forking them off the network. In short, we have in fact "burnt" a
> version bit unnecessarily.
> 
> If you're going to accept hard-forking some people off the network, why
> not just go with my stateless nVersion bits w/ time-expiration proposal
> instead? The only case where it leads to a hard-fork is if a soft-fork
> has been rejected by the time the upgrade deadline is reached. It's 
> easy
> to set this multiple years into the future, so I think in practice it
> won't be a major issue for non-controversial soft-forks.
> 
> Equally, spending the time to implement the original stateful nVersion
> bits proposal is possible as well, though higher risk due to the extra
> complexity of tracking soft-fork state.
> 
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev