From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 8398E516 for ; Mon, 6 Feb 2017 18:06:32 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from sender-of-o52.zoho.com (sender-of-o52.zoho.com [135.84.80.217]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 012031CA for ; Mon, 6 Feb 2017 18:06:30 +0000 (UTC) Received: from [10.8.8.2] (119246245241.ctinets.com [119.246.245.241]) by mx.zohomail.com with SMTPS id 1486404387103321.3966464720522; Mon, 6 Feb 2017 10:06:27 -0800 (PST) From: Johnson Lau Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Date: Tue, 7 Feb 2017 02:06:22 +0800 References: <15A0A5F0-726F-4E8F-9F16-F33D5A30C5F4@xbt.hk> To: Johnson Lau , bitcoin-dev In-Reply-To: <15A0A5F0-726F-4E8F-9F16-F33D5A30C5F4@xbt.hk> Message-Id: X-Mailer: Apple Mail (2.3259) X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] Spoonnet: another experimental hardfork X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Feb 2017 18:06:32 -0000 I fixed a flaw in my original design. It allowed a miner to create a = fake transaction to cheat light wallets. Also, with a second thought, I removed the backward compatibility with = light wallets. Everyone, light or full, should opt-in to a hardfork. The extended header is amended as follow * There must be exactly 2 witness items in coinbase witness (Header1; = Header2) **Header1 is the miner dedicated space. Nothing here has no consensus = meaning and is used for extranonce and merge mining. However, if the = size is larger than 14 bytes, the first 6 bytes must be 0. For a legacy = light node, this will look like a version 0 transaction with zero input = and output. [No need for 14 bytes or below: even with the most broken = design, a wallet can=E2=80=99t misinterpret data with this small size as = a valid tx] **Header2 is the secondary header. It must be 70-128 bytes. The first 4 = bytes must be little-endian encoded number of transactions (minimum 1). = The next 2 bytes must be 0. The next 32 bytes must be the transaction = Merkle root. The next 32 bytes must be the witness Merkle root (to be = defined later). The rest, if any, has no consensus meaning. However, = miners MUST NOT use this space of non-bitcoin purpose [the additional = space allows non-censensus enforced data to be included, easily = accessible to light wallets; the structure of the first 6 bytes make it = looks like an empty tx] * The main header commitment is H(H(Header1)|H(Header2)) H() =3D = dSHA256() [legacy light wallets are broken and they must upgrade to join = the new network] The implementation is updated accordingly > On 6 Feb 2017, at 20:39, Johnson Lau via bitcoin-dev = wrote: >=20 > Finally got some time over the Chinese New Year holiday to code and = write this up. This is not the same as my previous forcenet ( = https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/01347= 2.html ). It is much simpler. Trying to activate it on testnet will get = you banned. Trying to activate it on mainnet before consensus is reached = will make you lose money. >=20 > This proposal includes the following features: >=20 > 1. A fixed starting time. Not dependent on miner signalling. However, = it requires at least 51% of miners to actually build the new block = format in order to get activated. >=20 > 2. It has no mechanism to prevent a split. If 49% of miners insist on = the original chain, they could keep going. Split prevention is a social = problem, not a technical one. >=20 > 3. It is compatible with existing Stratum mining protocol. Only pool = software upgrade is needed >=20 > 4. A new extended and flexible header is located at the witness field = of the coinbase transaction >=20 > 5. It is backward compatible with existing light wallets >=20 > 6. Dedicated space for miners to put anything they want, which bitcoin = users could completely ignore. Merge-mining friendly. >=20 > 7. Small header space for miners to include non-consensus enforced = bitcoin related data, useful for fee estimation etc. >=20 > 8. A new transaction weight formula to encourage responsible use of = UTXO >=20 > 9. A linear growth of actual block size until certain limit >=20 > 10. Sighash O(n^2) protection for legacy (non-segwit) outputs >=20 > 11. Optional anti-transaction replay >=20 > 12. A new optional coinbase tx format that allows additional inputs, = including spending of immature previous coinbase outputs >=20 >=20 >=20 > Specification [Rationales]: >=20 >=20 > Activation: >=20 > * A "hardfork signalling block" is a block with the sign bit of header = nVersion is set [Clearly invalid for old nodes; easy opt-out for light = wallets] >=20 > * If the median-time-past of the past 11 blocks is smaller than the = HardForkTime (exact time to be determined), a hardfork signalling block = is invalid. >=20 > * Child of a hardfork signalling block MUST also be a hardfork = signalling block >=20 > * Initial hardfork signalling is optional, even if the HardForkTime = has past [requires at least 51% of miners to actually build the new = block format] >=20 > * HardForkTime is determined by a broad consensus of the Bitcoin = community. This is the only way to prevent a split. >=20 >=20 > Extended header: >=20 > * Main header refers to the original 80 bytes bitcoin block header >=20 > * A hardfork signalling block MUST have a additional extended header >=20 > * The extended header is placed at the witness field of the coinbase = transaction [There are 2 major advantages: 1. coinbase witness is = otherwise useless; 2. Significantly simply the implementation with its = stack structure] >=20 > * There must be exactly 3 witness items (Header1; Header2 ; Header3) > **Header1 must be exactly 32 bytes of the original transaction hash = Merkle root. > **Header2 is the secondary header. It must be 36-80 bytes. The first 4 = bytes must be little-endian encoded number of transactions (minimum 1). = The next 32 bytes must be the witness Merkle root (to be defined later). = The rest, if any, has no consensus meaning. However, miners MUST NOT use = this space of non-bitcoin purpose [the additional space allows = non-censensus enforced data to be included, easily accessible to light = wallets] > **Header3 is the miner dedicated space. It must not be larger than 252 = bytes. Anything put here has no consensus meaning [space for merge = mining; non-full nodes could completely ignore data in this space; 252 = is the maximum size allowed for signal byte CompactSize] >=20 > * The main header commitment is H(Header1|H(H(Header2)|H(Header3))) = H() =3D dSHA256() [The hardfork is transparent to light wallets, except = one more 32-byte hash is needed to connect a transaction to the root] >=20 > * To place the ext header, segwit becomes mandatory after hardfork >=20 >=20 > A =E2=80=9Cbackdoor=E2=80=9D softfork the relax the size limit of = Header 2 and Header 3: >=20 > * A special BIP9 softfork is defined with bit-15. If this softfork is = activated, full nodes will not enforce the size limit for Header 2 and = Header 3. [To allow header expansion without a hardfork. Avoid miner = abuse while providing flexibility. Expansion might be needed for new = commitments like fraud proof commitments] >=20 >=20 > Anti-tx-replay: >=20 > * Hardfork network version bit is 0x02000000. A tx is invalid if the = highest nVersion byte is not zero, and the network version bit is not = set. >=20 > * Masked tx version is nVersion with the highest byte masked. If = masked version is 3 or above, sighash for OP_CHECKSIG alike is = calculated using BIP143, except 0x02000000 is added to the nHashType = (the nHashType in signature is still a 1-byte value) [ensure a clean = split of signatures; optionally fix the O(n^2) problem] >=20 > * Pre-hardfork policy change: nVersion is determined by the masked tx = version for policy purpose. Setting of Pre-hardfork network version bit = 0x01000000 is allowed. >=20 > * Details: = https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/01347= 3.html >=20 >=20 > Sighash limitation: >=20 > * Sighash impact is estimated by =E2=80=9CLoose estimation=E2=80=9D in = https://github.com/jl2012/bips/blob/065ea7429035d43ff90965f42b086fb7e15172= 91/bip-sighash.mediawiki >=20 > * Only txs with masked version below 3 are counted. [because they are = fixed by the BIP-143 like signature] >=20 > * Each SigHashSize is defined as 1 tx weight (defined later). >=20 > * SIGHASH_SCALE_FACTOR is 90 (see the BIP above) >=20 >=20 > New tx weight definition: >=20 > * Weight of a transaction is the maximum of the 4 following metrics: >=20 > ** The total serialised size * 2 * SIGHASH_SCALE_FACTOR (size defined = by the witness tx format in BIP144) >=20 > ** The adjusted size =3D (Transaction weight by BIP141 - (number of = inputs - number of non-OP_RETURN outputs) * 41) * SIGHASH_SCALE_FACTOR >=20 > ** nSigOps * 50 * SIGHASH_SCALE_FACTOR. All SigOps are equal (no = witness scaling). For non-segwit txs, the sigops in output scriptPubKey = are not counted, while the sigops in input scriptPubKey are counted. >=20 > ** SigHashSize defined in the last section >=20 > Translating to new metric, the current BIP141 limit is 360,000,000. = This is equivalent to 360MB of sighashing, 2MB of serialised size, 4MB = of adjusted size, or 80000 nSigOp. >=20 > See rationales in this post: = https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/01347= 2.html >=20 >=20 > Block weight growing by time: >=20 > * Numbers for example only. Exact number to be determined. >=20 > * Block weight at HardForkTime is (5,000,000 * SIGHASH_SCALE_FACTOR) >=20 > * By every 16 seconds growth of the median-time-past, the weight is = increased by (1 * SIGHASH_SCALE_FACTOR) >=20 > * The growth stops at (16,000,000 * SIGHASH_SCALE_FACTOR) >=20 > * The growth does not dependent on the actual hardfork time. It=E2=80=99= s only based on median-time-past [using median-time-past so miners have = no incentive to use a fake timestamp] >=20 > * The limit for serialized size is 2.5 to 8MB in about 8 years. = [again, numbers for example only] >=20 >=20 > New coinbase transaction format: >=20 > * Existing coinbase format is allowed, except the new extended header = in the coinbase witness. No OP_RETURN witness commitment is needed. >=20 > * A new coinbase format is defined. The tx may have 1 or more inputs. = The outpoint of the first input MUST have an n value of 0xffffffff, and = use the previous block hash as the outpoint hash [This allows paying to = the child of a particular block by signing the block hash] >=20 > * ScriptSig of the first (coinbase) input is not executed. The size = limit increased from 100 to 252 (same for old coinbase format) >=20 > * Additional inputs MUST provide a valid scriptSig and/or witness for = spending >=20 > * Additional inputs may come from premature previous coinbase outputs = [this allows previous blocks paying subsequent blocks to encourage = confirmations] >=20 >=20 > Witness merkle root: >=20 > * If the coinbase is in old format, the witness merkle root is same as = BIP141 by setting the witness hash of the coinbase tx as 0 (without the = 32 byte witness reserved value) >=20 > * If the coinbase is in new format, the witness hash of the coinbase = tx is calculated by first removing the extended header >=20 > * The witness merkle root is put in the extended header 2, not as an = OP_RETURN output in coinbase tx. >=20 > * The witness merkle root becomes mandatory. (It was optional in = BIP141) >=20 >=20 > Other consensus changes: >=20 > * BIP9 will ignore the sign bit. [Setting the sign bit now is invalid = so this has no real consensus impact] >=20 > =3D=3D=3D=3D=3D=3D=3D=3D >=20 > An experimental implementation of the above spec could be found at = https://github.com/jl2012/bitcoin/tree/spoonnet1 >=20 > Not the same as my previous effort on the =E2=80=9Cforcenet=E2=80=9D, = the =E2=80=9Cspoonnet=E2=80=9D is a full hardfork that will get you = banned on the existing network. >=20 > Haven=E2=80=99t got the time to test the codes yet, not independently = reviewed. But it passes all existing tests in Bitcoin Core. No one = should use this in production, but I *think* it works fine on testnet = like a normal bitcoind (as long as it is not activated) >=20 > Things not implemented yet: >=20 > 1. Automated testing >=20 > 2. Post-hardfork support for old light wallets >=20 > 3. Wallet support, especially anti-tx-replay >=20 > 4. New p2p message to transmit secondary header (lower priority) >=20 > 5. Full mining and mempool support (not my priority) >=20 > =3D=3D=3D=3D=3D=3D=3D=3D >=20 > Potential second stage change: >=20 > Relative to the actual activation time, there could be a second stage = with more drastic changes to fix one or both of the following problems: >=20 > 1. SHA256 shortcut like ASICBoost. All fixes to ASICBoost are not very = elegant. But the question is, is it acceptable to have bitcoin-specific = patent in the consensus protocol? Still, I believe the best way to solve = this problem is the patent holder(s) to kindly somehow release the right = to the community.=20 >=20 > 2. Providing more nonce space in the 80-byte main header. However, = this depends on ASICBoost being a free technology. >=20 > 3. Block withholding attack. There are pros and cons, but I generally = agree with the analysis by Peter Todd at = https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/0120= 46.html . One point he didn=E2=80=99t mention is that only small really = needs pool mining, for the purpose of variance reduction. Big miners = using pools are just lazy, and they work well without pool. That means = only big solo miners are able to attack pools (i.e. small miners), while = pools cannot do any counterattack. This obviously shows why fixing this = is pro-small-miners. Also, with same hash rate, block withholding attack = is more effective against a smaller pool than a big pool. >=20 > All of these changes involve a header change and require light wallets = to upgrade. They also require firmware upgrade for all existing miners = (change 2 doesn=E2=80=99t). I think these shouldn=E2=80=99t happen at = least 2 years after the actual activation of the hardfork so people will = have enough time to upgrade. > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev