[bitcoin-dev] New BIP: Hardfork warning system

[bitcoin-dev] New BIP: Hardfork warning system

[Johnson Lau]

This BIP defines a change in consensus rules regarding to block nVersion, and define a concept of generalized block header to implement a hardfork warning system for full nodes and light nodes.

For better formatting, visit github
https://github.com/jl2012/bips/blob/hfwarning/bip-hfwarning.mediawiki



BIP: ?
Title: Hardfork warning system
Author: Johnson Lau <jl2012@xbt.hk>
Status: Draft
Type: Standard
Created: 2016-12-01

Abstract

This BIP defines a change in consensus rules regarding to block nVersion, and define a concept of generalized block header to implement a hardfork warning system for full nodes and light nodes.

Motivation

Softfork and hardfork are the 2 majors categories of consensus rules change. Generally, softforks make some previously valid blocks invalid, while hardforks make some previously invalid blocks valid. Bitcoin has successfully introduced a number of new functions through softforks. A built-in warning system is also available in many implementations to warn users for the activation of any unknown softforks.

Some features, however, may not be easily introduced with a softfork. Examples include expanding maximum block resources limits, and changing the average block time interval. When such features are implemented with a hardfork, existing full node implementations would consider such blocks as invalid, and may even ban a peer for relaying such blocks. They are effectively blind to such hardfork rule changes, leaving users to unknowingly transact on a system with potentially different token value. On the other hand, light nodes may blindly follow a hardfork with unknown rule changes and lose the right to choose the previous system.

This BIP defines a change in consensus rules regarding to block nVersion, and define a concept of generalized block header to implement a hardfork warning system for full nodes and light nodes.

Definitions

Valid block
A block that satisfies all the consensus rules being enforced by a bitcoin protocol implementation. An implementation may intentionally (e.g. a light node) or unintentionally (e.g. unaware of a softfork) not enforcing any part of the current netwrok rules.

Valid blockchain
A blockchain constituting of only valid blocks.

Best valid blockchain
The valid blockchain with highest total proof-of-work.

Valid blockchain fork
A valid blockchain sharing a common ancestral block with the best valid blockchain, but with less total proof-of-work

Generalized block header
Any serialized hexadecimal data with exactly 80 bytes (byte 0 to byte 79). The bytes 4 to 35 are the double-SHA256 hash of another generalized block header. The bytes 72 to 75 are nBits, the target of this generalized block header encoded in the same way as normal bitcoin block header. The 2 most significant bits of the byte 3 are the hardfork notification bits. The semantics of other data in a generalized block header is not defined in any general way. It should be noted that a normal bitcoin block header is a special case of generalized block header.

Generalized block header chain
A chain of generalized block header. A header chain of valid blocks is a special case of a generalized block header chain.


Specifications


Block nVersion softfork

A softfork is deployed to restrict the valid value of block nVersion. Upon activation, any block with the second highest nVersion bit set becomes invalid (nVersion & 0x40000000)

This softfork will be deployed by "version bits" BIP9 with the name "hfbit" and using bit 2.

For Bitcoin mainnet, the BIP9 starttime will be midnight TBC UTC (Epoch timestamp TBC) and BIP9 timeout will be midnight TBC UTC (Epoch timestamp TBC).

For Bitcoin testnet, the BIP9 starttime will be midnight TBC UTC (Epoch timestamp TBC) and BIP9 timeout will be midnight TBC UTC (Epoch timestamp TBC).

Any bitcoin implementation (full nodes and light nodes) supporting this softfork should also implement a hardfork warning system described below.


Validation of generalized block header

A bitcoin protocol implementation should consider a generalized block header as valid if it satisfies all of the following criteria:

	• It is a descendant of the header of a valid block in a valid blockchain (the best valid blockchain or a valid blockchain fork).
	• It satisfies the proof-of-work requirement: its double-SHA256 value MUST be smaller than its target (encoded as nBits).
	• Its target MUST NOT be greater than the target of its last ancestral valid block by more than 1024 times. An implementation may decide to use a different threshold (or dynamic threshold), depending on its tolerance against potential DoS attacks by generating many low difficulty headers. However, if the value is set too low, a hardfork with lower difficulty may not be detected.[1]
In general, a bitcoin protocol implementation should keep an index of all known generalized block header chains, along with the valid blockchain(s). However, if a generalized block header chain is grown on top of a very old valid block, with total proof-of-work much lower than the current best valid bloackchain, it may be safely discarded.


Hardfork warning system in full nodes

Hardfork with unknown rules
If a generalized block header chain with non-trivial total proof-of-work is emerging, and is not considered as a valid blockchain, a hardfork with unknown rules may be happening.

A wallet implementation should issue a warning to its users and stop processing incoming and outgoing transactions, until further instructions are given. It should not attempt to conduct transactions on or otherwise interpreting any block data of the hardfork with unknown rules.

A mining implementation should issue a warning to its operator. Until further instructions are given, it may either stop mining, or ignore the hardfork with unknown rules. It should not attempt to confirm a generalized block header with unknown rules.

Setting of one or both hardfork notification bits is, as defined by BIP34 and this BIP, a hardfork, and should be considered as an indication of a planned hardfork. If a hardfork with unknown rules is happening without any hardfork notification bits set, it is probably an accidental consensus failure, such as the March 2013 fork due to a block database bug (BIP50), and the July 2015 fork following the BIP66 activation.[2]


Hardfork with multiple valid blockchains
If a valid blockchain fork is emerging with non-trivial total proof-of-work, a consensus disagreement may be happening among different miners.

A wallet implementation should issue a warning to its users and stop processing incoming and outgoing transactions, until further instructions are given.

A mining implementation should issue a warning to its operator. Until further instructions are given, it may either stop mining, or mine on top of the best valid chain by its own standard.

Hardfork warning system in light nodes

Light node (usually wallet implementations) is any bitcoin protocol implementations that intentionally not fully enforcing the network rules. As an important part of the hardfork warning system, a light node should observe the hardfork notification bits in block header, along with any other rules it opts to validate. If any of the hardfork notification bits is set, it should issue a warning to its users and stop processing incoming and outgoing transactions, until further instructions are given. It should not attempt to conduct transactions on or otherwise interpreting any block data of the hardfork blockchain, even if it might be able to decode the block data.


Applications

Hardfork notification bits
There are 2 hardfork notification bits defined in this BIP. The higher bit has been forbidden since BIP34, and the lower bit is disabled by this BIP. For nodes supporting this BIP, the semantics of the 2 bits are the same: a hardfork is happening. For legacy node, however, setting the higher bit would make them fail to follow the hardforking chain. In a soft-hardfork design (described below), the lower notification bit should be used.
The hardfork warning system is able to detect the following types of hardforks:

Soft-hardfork (with the lower hardfork notification bit)
A soft-hardfork is a technique to implement a hardfork by pretending to create blocks with only a zero output value coinbase transaction, and commit the real transaction Merkle root in the coinbase scriptSig field. With the lower hardfork notification bit set, a node following this BIP will consider this as a hardfork and enter the safe mode, while a legacy node not following this BIP will be effectively broken due to seeing the continuously empty blockchain.

Redefining the nTime field
As the warning system does not interpret the nTime field, redefining it through a hardfork would be detectable. For example, overflow may be allowed to overcome the year 2106 problem.

Redefining the Merkle root hash field and changing block content validation rules
The 32-byte Merkle root hash could be redefined, for example, with a different hashing algorithm. Any block resources limitation and transaction validation rules may also be changed. All such hardforks would be detected by the warning system.

Changing average block interval or difficulty reset
Since the warning system is not bound to a particular proof-of-work target adjustment schedule, a hardfork changing the average block interval or resetting the difficulty will be detectable.

Introducing secondary proof-of-work
Introducing secondary proof-of-work (with non-SHA256 algorithm or fixing the block withholding attack against mining pools) may be detectable, as long as the generalized block header format is preserved.

Accidental hardfork
An accidental hardfork may be detectable, if the generalized block headers in both forks are valid but no hardfork notification bit is set.


Limitations

The only function of this system is to inform the users that a hardfork might be happening and prompt for further instructions. It does not guarantee that the hardfork will be successful and not end up with two permanent incompatible forks. This requires broad consensus of the whole community and is not solvable with technical means alone.

The following types of hardfork are not detectable with this warning system:

	• Changing of proof-of-work algorithm
	• Changing the encoding of previous block header hash or nBits
	• A coercive soft-hardfork without setting any hardfork notification bit


Backward compatibility

The softfork described in the BIP would only affect miners. As the disabled nVersion bit is never used in the main network, it is unlikely that any miner would unintentionally find an invalid block due to the new rules.

BIP9 is disabled when any of the hardfork notification bits is set, which may interrupt any ongoing softfork support signalling process. Developers should pay attention to this when desinging a hardfork. For example, they may redefine the counting of signal, or move the signalling bitfield to a different location.

Legacy nodes would not be benefited from this softfork and warning system. However, no additional risks are introduced to legacy node either.


Reference implementation

To be done


Footnotes


	• ^ Please note that a hardfork may have lower difficulty but higher total proof-of-work, such as by decreasing the average block interval.
	• ^ In the March 2013 fork, pre-0.8 full nodes would see that as a hardfork with unknown rules, while light nodes and 0.8.0 full nodes would see that as multiple valid blockchains. In the July 2015 fork, BIP66-complying full nodes would see that as a hardfok with unknown rules, while legacy full nodes would see that as multiple valid blockchains.


Copyright

This document is placed in the public domain.

Re: [bitcoin-dev] New BIP: Hardfork warning system

[Tom Zander]

I am failing to see how you actually will detect a hard fork with this 
system.

Maybe its because of this sentence not being very clear to me;
 «If a generalized block header chain with non-trivial total proof-of-work
   is emerging»

Also, can you explain what you are actually trying to accomplish?

I want to point out that the following part from your motivation is 
incorrect. A full node would reject a hard forked chain (by definition), 
there is no risk of them transacting on it.

«When such features are implemented with a hardfork, existing full node 
implementations would consider such blocks as invalid, and may even ban a 
peer for relaying such blocks. They are effectively blind to such hardfork 
rule changes, leaving users to unknowingly transact on a system with 
potentially different token value. »



On Friday, 2 December 2016 01:20:31 CET Johnson Lau via bitcoin-dev wrote:
> This BIP defines a change in consensus rules regarding to block nVersion,
> and define a concept of generalized block header to implement a hardfork
> warning system for full nodes and light nodes.
> 
> For better formatting, visit github
> https://github.com/jl2012/bips/blob/hfwarning/bip-hfwarning.mediawiki
> 
> 
> 
> BIP: ?
> Title: Hardfork warning system
> Author: Johnson Lau <jl2012@xbt.hk>
> Status: Draft
> Type: Standard
> Created: 2016-12-01
> 
> Abstract
> 
> This BIP defines a change in consensus rules regarding to block nVersion,
> and define a concept of generalized block header to implement a hardfork
> warning system for full nodes and light nodes.
> 
> Motivation
> 
> Softfork and hardfork are the 2 majors categories of consensus rules
> change. Generally, softforks make some previously valid blocks invalid,
> while hardforks make some previously invalid blocks valid. Bitcoin has
> successfully introduced a number of new functions through softforks. A
> built-in warning system is also available in many implementations to warn
> users for the activation of any unknown softforks.
> 
> Some features, however, may not be easily introduced with a softfork.
> Examples include expanding maximum block resources limits, and changing
> the average block time interval. When such features are implemented with
> a hardfork, existing full node implementations would consider such blocks
> as invalid, and may even ban a peer for relaying such blocks. They are
> effectively blind to such hardfork rule changes, leaving users to
> unknowingly transact on a system with potentially different token value.
> On the other hand, light nodes may blindly follow a hardfork with unknown
> rule changes and lose the right to choose the previous system.
> 
> This BIP defines a change in consensus rules regarding to block nVersion,
> and define a concept of generalized block header to implement a hardfork
> warning system for full nodes and light nodes.
> 
> Definitions
> 
> Valid block
> A block that satisfies all the consensus rules being enforced by a bitcoin
> protocol implementation. An implementation may intentionally (e.g. a
> light node) or unintentionally (e.g. unaware of a softfork) not enforcing
> any part of the current netwrok rules.
> 
> Valid blockchain
> A blockchain constituting of only valid blocks.
> 
> Best valid blockchain
> The valid blockchain with highest total proof-of-work.
> 
> Valid blockchain fork
> A valid blockchain sharing a common ancestral block with the best valid
> blockchain, but with less total proof-of-work
> 
> Generalized block header
> Any serialized hexadecimal data with exactly 80 bytes (byte 0 to byte 79).
> The bytes 4 to 35 are the double-SHA256 hash of another generalized block
> header. The bytes 72 to 75 are nBits, the target of this generalized
> block header encoded in the same way as normal bitcoin block header. The
> 2 most significant bits of the byte 3 are the hardfork notification bits.
> The semantics of other data in a generalized block header is not defined
> in any general way. It should be noted that a normal bitcoin block header
> is a special case of generalized block header.
> 
> Generalized block header chain
> A chain of generalized block header. A header chain of valid blocks is a
> special case of a generalized block header chain.
> 
> 
> Specifications
> 
> 
> Block nVersion softfork
> 
> A softfork is deployed to restrict the valid value of block nVersion. Upon
> activation, any block with the second highest nVersion bit set becomes
> invalid (nVersion & 0x40000000)
> 
> This softfork will be deployed by "version bits" BIP9 with the name
> "hfbit" and using bit 2.
> 
> For Bitcoin mainnet, the BIP9 starttime will be midnight TBC UTC (Epoch
> timestamp TBC) and BIP9 timeout will be midnight TBC UTC (Epoch timestamp
> TBC).
> 
> For Bitcoin testnet, the BIP9 starttime will be midnight TBC UTC (Epoch
> timestamp TBC) and BIP9 timeout will be midnight TBC UTC (Epoch timestamp
> TBC).
> 
> Any bitcoin implementation (full nodes and light nodes) supporting this
> softfork should also implement a hardfork warning system described below.
> 
> 
> Validation of generalized block header
> 
> A bitcoin protocol implementation should consider a generalized block
> header as valid if it satisfies all of the following criteria:
> 
> 	• It is a descendant of the header of a valid block in a valid 
blockchain
> (the best valid blockchain or a valid blockchain fork). • It satisfies
> the proof-of-work requirement: its double-SHA256 value MUST be smaller
> than its target (encoded as nBits). • Its target MUST NOT be greater than
> the target of its last ancestral valid block by more than 1024 times. An
> implementation may decide to use a different threshold (or dynamic
> threshold), depending on its tolerance against potential DoS attacks by
> generating many low difficulty headers. However, if the value is set too
> low, a hardfork with lower difficulty may not be detected.[1] In general,
> a bitcoin protocol implementation should keep an index of all known
> generalized block header chains, along with the valid blockchain(s).
> However, if a generalized block header chain is grown on top of a very
> old valid block, with total proof-of-work much lower than the current
> best valid bloackchain, it may be safely discarded.
> 
> 
> Hardfork warning system in full nodes
> 
> Hardfork with unknown rules
> If a generalized block header chain with non-trivial total proof-of-work
> is emerging, and is not considered as a valid blockchain, a hardfork with
> unknown rules may be happening.
> 
> A wallet implementation should issue a warning to its users and stop
> processing incoming and outgoing transactions, until further instructions
> are given. It should not attempt to conduct transactions on or otherwise
> interpreting any block data of the hardfork with unknown rules.
> 
> A mining implementation should issue a warning to its operator. Until
> further instructions are given, it may either stop mining, or ignore the
> hardfork with unknown rules. It should not attempt to confirm a
> generalized block header with unknown rules.
> 
> Setting of one or both hardfork notification bits is, as defined by BIP34
> and this BIP, a hardfork, and should be considered as an indication of a
> planned hardfork. If a hardfork with unknown rules is happening without
> any hardfork notification bits set, it is probably an accidental
> consensus failure, such as the March 2013 fork due to a block database
> bug (BIP50), and the July 2015 fork following the BIP66 activation.[2]
> 
> 
> Hardfork with multiple valid blockchains
> If a valid blockchain fork is emerging with non-trivial total
> proof-of-work, a consensus disagreement may be happening among different
> miners.
> 
> A wallet implementation should issue a warning to its users and stop
> processing incoming and outgoing transactions, until further instructions
> are given.
> 
> A mining implementation should issue a warning to its operator. Until
> further instructions are given, it may either stop mining, or mine on top
> of the best valid chain by its own standard.
> 
> Hardfork warning system in light nodes
> 
> Light node (usually wallet implementations) is any bitcoin protocol
> implementations that intentionally not fully enforcing the network rules.
> As an important part of the hardfork warning system, a light node should
> observe the hardfork notification bits in block header, along with any
> other rules it opts to validate. If any of the hardfork notification bits
> is set, it should issue a warning to its users and stop processing
> incoming and outgoing transactions, until further instructions are given.
> It should not attempt to conduct transactions on or otherwise
> interpreting any block data of the hardfork blockchain, even if it might
> be able to decode the block data.
> 
> 
> Applications
> 
> Hardfork notification bits
> There are 2 hardfork notification bits defined in this BIP. The higher bit
> has been forbidden since BIP34, and the lower bit is disabled by this
> BIP. For nodes supporting this BIP, the semantics of the 2 bits are the
> same: a hardfork is happening. For legacy node, however, setting the
> higher bit would make them fail to follow the hardforking chain. In a
> soft-hardfork design (described below), the lower notification bit should
> be used. The hardfork warning system is able to detect the following
> types of hardforks:
> 
> Soft-hardfork (with the lower hardfork notification bit)
> A soft-hardfork is a technique to implement a hardfork by pretending to
> create blocks with only a zero output value coinbase transaction, and
> commit the real transaction Merkle root in the coinbase scriptSig field.
> With the lower hardfork notification bit set, a node following this BIP
> will consider this as a hardfork and enter the safe mode, while a legacy
> node not following this BIP will be effectively broken due to seeing the
> continuously empty blockchain.
> 
> Redefining the nTime field
> As the warning system does not interpret the nTime field, redefining it
> through a hardfork would be detectable. For example, overflow may be
> allowed to overcome the year 2106 problem.
> 
> Redefining the Merkle root hash field and changing block content
> validation rules The 32-byte Merkle root hash could be redefined, for
> example, with a different hashing algorithm. Any block resources
> limitation and transaction validation rules may also be changed. All such
> hardforks would be detected by the warning system.
> 
> Changing average block interval or difficulty reset
> Since the warning system is not bound to a particular proof-of-work target
> adjustment schedule, a hardfork changing the average block interval or
> resetting the difficulty will be detectable.
> 
> Introducing secondary proof-of-work
> Introducing secondary proof-of-work (with non-SHA256 algorithm or fixing
> the block withholding attack against mining pools) may be detectable, as
> long as the generalized block header format is preserved.
> 
> Accidental hardfork
> An accidental hardfork may be detectable, if the generalized block headers
> in both forks are valid but no hardfork notification bit is set.
> 
> 
> Limitations
> 
> The only function of this system is to inform the users that a hardfork
> might be happening and prompt for further instructions. It does not
> guarantee that the hardfork will be successful and not end up with two
> permanent incompatible forks. This requires broad consensus of the whole
> community and is not solvable with technical means alone.
> 
> The following types of hardfork are not detectable with this warning
> system:
> 
> 	• Changing of proof-of-work algorithm
> 	• Changing the encoding of previous block header hash or nBits
> 	• A coercive soft-hardfork without setting any hardfork notification bit
> 
> 
> Backward compatibility
> 
> The softfork described in the BIP would only affect miners. As the
> disabled nVersion bit is never used in the main network, it is unlikely
> that any miner would unintentionally find an invalid block due to the new
> rules.
> 
> BIP9 is disabled when any of the hardfork notification bits is set, which
> may interrupt any ongoing softfork support signalling process. Developers
> should pay attention to this when desinging a hardfork. For example, they
> may redefine the counting of signal, or move the signalling bitfield to a
> different location.
> 
> Legacy nodes would not be benefited from this softfork and warning system.
> However, no additional risks are introduced to legacy node either.
> 
> 
> Reference implementation
> 
> To be done
> 
> 
> Footnotes
> 
> 
> 	• ^ Please note that a hardfork may have lower difficulty but higher
> total proof-of-work, such as by decreasing the average block interval. •
> ^ In the March 2013 fork, pre-0.8 full nodes would see that as a hardfork
> with unknown rules, while light nodes and 0.8.0 full nodes would see that
> as multiple valid blockchains. In the July 2015 fork, BIP66-complying
> full nodes would see that as a hardfok with unknown rules, while legacy
> full nodes would see that as multiple valid blockchains.
> 
> 
> Copyright
> 
> This document is placed in the public domain.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


-- 
Tom Zander
Blog: https://zander.github.io
Vlog: https://vimeo.com/channels/tomscryptochannel

Re: [bitcoin-dev] New BIP: Hardfork warning system

[Luke Dashjr]

On Thursday, December 01, 2016 5:20:31 PM Johnson Lau via bitcoin-dev wrote:
> Any bitcoin implementation (full nodes and light nodes) supporting this
> softfork should also implement a hardfork warning system described below.

I think this "should" needs to be a "must" be make this useful.

> Hardfork with unknown rules
> If a generalized block header chain with non-trivial total proof-of-work is
> emerging, and is not considered as a valid blockchain, a hardfork with
> unknown rules may be happening.
> 
> A wallet implementation should issue a warning to its users and stop
> processing incoming and outgoing transactions, until further instructions
> are given. It should not attempt to conduct transactions on or otherwise
> interpreting any block data of the hardfork with unknown rules.

This seems too unclear. Specifically, if an invalid chain with *equivalent or 
better* work than the best valid chain exists, nodes ought to treat all blocks 
following the common chain (between the better-invalid and best-valid chains) 
as suspect.

So if we have two chains:

    A->B->C->D (valid)
    A->B->X->Y (invalid)

The node should consider block B as the tip until the valid chain becomes and 
stays longer than the invalid one.

> A mining implementation should issue a warning to its operator. Until
> further instructions are given, it may either stop mining, or ignore the
> hardfork with unknown rules. It should not attempt to confirm a
> generalized block header with unknown rules.

I think we need to decide more specifically which behaviour is sane here.

> Hardfork warning system in light nodes
> 
> Light node (usually wallet implementations) is any bitcoin protocol
> implementations that intentionally not fully enforcing the network rules.
> As an important part of the hardfork warning system, a light node should
> observe the hardfork notification bits in block header, along with any
> other rules it opts to validate. If any of the hardfork notification bits
> is set, it should issue a warning to its users and stop processing
> incoming and outgoing transactions, until further instructions are given.
> It should not attempt to conduct transactions on or otherwise interpreting
> any block data of the hardfork blockchain, even if it might be able to
> decode the block data.

Light nodes should probably not be specified here differently than full nodes. 
If they detect an invalid block through *any* means, they should react the 
same as a full node would.

> Redefining the Merkle root hash field and changing block content validation
> rules The 32-byte Merkle root hash could be redefined, for example, with a
> different hashing algorithm. Any block resources limitation and
> transaction validation rules may also be changed. All such hardforks would
> be detected by the warning system.

Note, some changes may be needed to current nodes for this to work. I think at 
this time this would cause a "deserialisation" error, and not accept NOR 
reject the block...

> Introducing secondary proof-of-work
> Introducing secondary proof-of-work (with non-SHA256 algorithm or fixing
> the block withholding attack against mining pools) may be detectable, as
> long as the generalized block header format is preserved.

Not necessarily. A secondary PoW might drastically change the measurement of 
work. Fixing block withholding may result in block hashes that meet a preimage 
rather than bits directly. I think it may be important to fix the latter 
problem for this BIP.

> Accidental hardfork
> An accidental hardfork may be detectable, if the generalized block headers
> in both forks are valid but no hardfork notification bit is set.

Probably best to not call this a hardfork, since it is a break without 
consensus.

Luke

Re: [bitcoin-dev] New BIP: Hardfork warning system

[Jorge Timón]

We can already warn users of a hardfork when a block is invalid (at
least) because of the highest bit in nVersion (as you say, because it
is forbidden since bip34 was deployed). It seems the softfork serves
only to warn about soft-hardforks, assuming it chooses to use this
mechanism (which a malicious soft hardfork may not do). In fact, you
could reuse another of the prohibited bits to signal a soft-hardfork
while distinguishing it from a regular hardfork. And this will also
serve for old nodes that have not upgraded to the softfork. But, wait,
if you signal a soft-hardfork with an invalid bit, it's not a
soft-hardfork anymore, is it? It's simply a hardfork.
Your softfork would result in soft hardforks being hardforks for nodes
that upgraded to this softfork, but softforks for older nodes.
Is this the intended behaviour? if so, why?

I would rather have a simpler BIP that doesn't require a softfork
(whether it recommends soft-hardforks to use one of the currently
invalid bits, but a different one than from hardforks or not, but I
also don't see the reason why soft-hardforks should appear as invalid
blocks for older nodes instead of using regular softfork warning
[besides, in this case, after the "unkown softfork" warning you will
get only empty blocks, which may make you suspicious]).

Re: [bitcoin-dev] New BIP: Hardfork warning system

[Luke Dashjr]

On Friday, December 02, 2016 1:42:46 AM Jorge Timón via bitcoin-dev wrote:
> We can already warn users of a hardfork when a block is invalid (at
> least) because of the highest bit in nVersion (as you say, because it
> is forbidden since bip34 was deployed).

The difference is that right now, full nodes will happily follow a shorter 
best-valid chain. This BIP would require them to hold back at the best-common 
block between the best-valid chain and the invalid chain, forcing the user to 
make a decision whether to reject the invalid chain permanently, or upgrade to 
a version which can understand that chain as valid.

> It seems the softfork serves only to warn about soft-hardforks, assuming it
> chooses to use this mechanism (which a malicious soft hardfork may not do).

Note: a malicious "SHF" is not a SHF at all, but an "evil fork".

> In fact, you could reuse another of the prohibited bits to signal a soft-
> hardfork while distinguishing it from a regular hardfork. And this will also
> serve for old nodes that have not upgraded to the softfork. But, wait,
> if you signal a soft-hardfork with an invalid bit, it's not a
> soft-hardfork anymore, is it? It's simply a hardfork.

Nodes implementing this BIP will see it as a simple hardfork, but will 
intentionally provide equivalent behaviour as older nodes which see it as a 
soft-hardfork. In other words, all [compatible] hardforks will now behave like 
a soft-hardfork without any special DMMS design.

If Bitcoin's eventual hardfork is far enough down the road (such that no nodes 
remain from before this BIP are adopted), the SHF design could be safely done 
away with entirely. And either way, it makes it easier to resist an un-
consented-to hardfork.

Luke

Re: [bitcoin-dev] New BIP: Hardfork warning system

[Jorge Timón]

On Fri, Dec 2, 2016 at 5:18 AM, Luke Dashjr <luke@dashjr.org> wrote:
> On Friday, December 02, 2016 1:42:46 AM Jorge Timón via bitcoin-dev wrote:
>> We can already warn users of a hardfork when a block is invalid (at
>> least) because of the highest bit in nVersion (as you say, because it
>> is forbidden since bip34 was deployed).
>
> The difference is that right now, full nodes will happily follow a shorter
> best-valid chain. This BIP would require them to hold back at the best-common
> block between the best-valid chain and the invalid chain, forcing the user to
> make a decision whether to reject the invalid chain permanently, or upgrade to
> a version which can understand that chain as valid.

Ok, so the goal of the softfork then is to hold on is there is to
"hold on" on the most-work valid chain while there's an even-more-work
invalid chain and for new nodes force a response from the user. This
could be clearer in the motivation section.

We can still notify and force a response from the user with a single
invalid block (or N, or W accumulated work). Note that we don't need
to "hold on" while waiting for the user's response. Therefore I insist
there could be a PIB with all the recommended warnings but not
softfork (which this one could extend from).

Thinking as a full node user, I'm not sure I want my node to "hold on"
on validating new valid blocks just because there's some "longest"
invalid chain out there and I may chose to follow that instead later.
Before paying or copying another address to receive, I would
definitely would want the warning though.

Particularly as a miner (not that I'm one), I think validationless
mining shows us that some miners prefer to throw energy to the abyss
of validation uncertainty rather than stop their mining hardware.
What if when I give the response to the system I decide to pass on the
HF but it means my hardware have been not mining in the valid chain
for hours?
I would account that as "money lost thanks to a 'friendly' interface".
Specially if we're talking about a controversial hardfork.

If we're talking about an uncontroversial hardfork, I would definitely
prefer BIP9 coordination.
I would prefer to receive the warning when, say, 30% of the hashrate
is supporting an unknown change to the consensus rules (regardless of
it being a softfork or a hardfork, which I don't know yet until the hf
bit is used because the change is unknown to me), way before I need to
decide what branch to mine.

In fact, if I was a miner but not a user at the same time, after
knowing about the unknown hardfork and if I consider the hardfork to
be potentially controversial, I would try to coordinate with exchanges
(and pools if no solo mining) to be able to write a program that
chooses the chain likely to be most profitable depending on difficulty
and price feeds for every block.

In the case of a SHF, even more reason to keep mining on the old
chain, perhaps I mine one empty block (assuming that's a rule in the
SHF) out of luck, or maybe I should just start mining empty blocks
whenever I see the SHF bit active for a block whose chain keeps
growing.
Perhaps for a SHF we should use a valid bit instead of an invalid one
(clearing all possible fears with old and older nodes perceiving SHFs
differently as HFs and SFs respectively). We can trivially make old an
older nodes coincide in their perception of good-willing SHFs as
either HFs or SFs as we wish. Choosing divergence of perception from
the 2 versions we're considering makes no sense to me.
I'm reserving my judgement for which one I prefer just in case there's
actually an advantage in this divergence, but I've missed it.

>> It seems the softfork serves only to warn about soft-hardforks, assuming it
>> chooses to use this mechanism (which a malicious soft hardfork may not do).
>
> Note: a malicious "SHF" is not a SHF at all, but an "evil fork".

Terminology, I think you get my point. I'm all for formalizing
definitions but please let's not slow down discussion unnecessarily.
I'm fine saying "evil fork" instead of "malicious SHF" if you prefer
that, but they're just the same thing.

>> In fact, you could reuse another of the prohibited bits to signal a soft-
>> hardfork while distinguishing it from a regular hardfork. And this will also
>> serve for old nodes that have not upgraded to the softfork. But, wait,
>> if you signal a soft-hardfork with an invalid bit, it's not a
>> soft-hardfork anymore, is it? It's simply a hardfork.
>
> Nodes implementing this BIP will see it as a simple hardfork, but will
> intentionally provide equivalent behaviour as older nodes which see it as a
> soft-hardfork. In other words, all [compatible] hardforks will now behave like
> a soft-hardfork without any special DMMS design.

Right, and those same mechanisms could be implemented using one of the
already prohibited bits (for example, just like the higher weight bit
in nHeight, the lowest value one was prohibited when BIP34 was
activated).
There's no need to invalidate another bit in the softfork (repeat: bit
1 got invalidated when bip34 was activated as well; or we could just
use a valid bit for SHFs).

> If Bitcoin's eventual hardfork is far enough down the road (such that no nodes
> remain from before this BIP are adopted), the SHF design could be safely done
> away with entirely. And either way, it makes it easier to resist an un-
> consented-to hardfork.

Right, I'm also under the assumption that a HF (or a SHF) would give
plenty of/enough time (to be defined, I suggest at least 1 year but we
really shouldn't get into this in this thread) in their
BIP9Deployment::nStartTime (or equivalent if BIP9 is not reused for
HF/SHF). Otherwise I would consider any HF or SHF controversial in
itself regardless of what it does (for not giving enough time to users
to adapt).
Therefore we can assume that all the warnings would be deployed in
advance to any HF or SHF, with or without a previous softfork.

I strongly disagree that the proposed softfork "[makes] it easier to
resist an un-consented-to hardfork". If anything, it makes it easier
to disrupt the old network if it doesn't fully consent to the HF. For
"un-consented-to SHFs" (or "evil HFs" if you prefer) I don't think
it's a safe to assume they will use an invalid bit to signal their
intend. At least if I was an "evil softhardforker" just interested in
disruption, I wouldn't do it (just like if I was an "evil hardforker"
I wouldn't use the normal hardfork bit).