public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
* [bitcoin-dev] Reorgs on SigNet - Looking for feedback on approach and parameters
@ 2021-09-07 16:07 0xB10C
  2021-09-07 16:44 ` Jeremy
                   ` (2 more replies)
  0 siblings, 3 replies; 18+ messages in thread
From: 0xB10C @ 2021-09-07 16:07 UTC (permalink / raw)
  To: bitcoin-dev

Hello,

tl;dr: We want to make reorgs on SigNet a reality and are looking for
feedback on approach and parameters.

One of the ideas for SigNet is the possibility for it to be reliably
unreliable, for example, planned chain reorganizations. These have not
been implemented yet.

My summerofbitcoin.org mentee Nikhil Bartwal and I have been looking at
implementing support for reorgs on SigNet. We are looking for feedback
on which approach and parameters to use. Please consider answering the
questions below if you or your company is interested in chain
reorganizations on SigNet.

With feedback from AJ and Kalle Alm (thanks again!), we came up with two
scenarios that could be implemented in the current SigNet miner script
[0]. Both would trigger automatically in a fixed block interval.
Scenario 1 simulates a race scenario where two chains compete for D
blocks. Scenario 2 simulates a chain rollback where the top D blocks get
replaced by a chain that outgrows the earlier branch.

AJ proposed to allow SigNet users to opt-out of reorgs in case they
explicitly want to remain unaffected. This can be done by setting a
to-be-reorged version bit flag on the blocks that won't end up in the
most work chain. Node operators could choose not to accept to-be-reorged
SigNet blocks with this flag set via a configuration argument.

The reorg-interval X very much depends on the user's needs. One could
argue that there should be, for example, three reorgs per day, each 48
blocks apart. Such a short reorg interval allows developers in all time
zones to be awake during one or two reorgs per day. Developers don't
need to wait for, for example, a week until they can test their reorgs
next. However, too frequent reorgs could hinder other SigNet users.

We propose that the reorg depth D is deterministically random between a
minimum and a maximum based on, e.g., the block hash or the nonce of the
last block before the reorg. Compared to a local randint() based
implementation, this allows reorg-handling tests and external tools to
calculate the expected reorg depth.

# Scenario 1: Race between two chains

For this scenario, at least two nodes and miner scripts need to be
running. An always-miner A continuously produces blocks and rejects
blocks with the to-be-reorged version bit flag set. And a race-miner R
that only mines D blocks at the start of each interval and then waits X
blocks. A and R both have the same hash rate. Assuming both are well
connected to the network, it's random which miner will first mine and
propagate a block. In the end, the A miner chain will always win the race.

# Scenario 2: Chain rollback

This scenario only requires one miner and Bitcoin Core node but also
works in a multiminer setup. The miners mine D blocks with the
to-be-reorged version bit flag set at the start of the interval. After
allowing the block at height X+D to propagate, they invalidate the block
at height X+1 and start mining on block X again. This time without
setting the to-be-reorged version bit flag. Non-miner nodes will reorg
to the new tip at height X+D+1, and the first-seen branch stalls.

# Questions

    1. How do you currently test your applications reorg handling? Do
       the two discussed scenarios (race and chain rollback) cover your
       needs? Are we missing something you'd find helpful?

    2. How often should reorgs happen on the default SigNet? Should
       there be multiple reorgs a day (e.g., every 48 or 72 blocks
       assuming 144 blocks per day) as your engineers need to be awake?
       Do you favor less frequent reorgs (once per week or month)? Why?

    3. How deep should the reorgs be on average? Do you want to test
       deeper reorgs (10+ blocks) too?


# Next Steps

We will likely implement Scenario 1, the race between two chains, first.
We'll set up a public test-SigNet along with a faucet, block explorer,
and a block tree visualization. If there is interest in the second
approach, chain rollbacks can be implemented too. Future work will add
the possibility to include conflicting transactions in the two branches.
After enough testing, the default SigNet can start to do periodical
reorgs, too.

Thanks,
0xB10C

[0]: https://github.com/bitcoin/bitcoin/blob/master/contrib/signet/miner 



^ permalink raw reply	[flat|nested] 18+ messages in thread
* Re: [bitcoin-dev] Reorgs on SigNet - Looking for feedback on approach and parameters
@ 2021-09-10 13:05 Michael Folkson
  2021-09-10 18:24 ` Matt Corallo
  0 siblings, 1 reply; 18+ messages in thread
From: Michael Folkson @ 2021-09-10 13:05 UTC (permalink / raw)
  To: Matt Corallo, Bitcoin Protocol Discussion

> I see zero reason whatsoever to not simply reorg ~every block, or as often as is practical. If users opt in to wanting to test with reorgs, they should be able to test with reorgs, not wait a day to test with reorgs.

One of the goals of the default Signet was to make the default Signet
resemble mainnet as much as possible. (You can do whatever you want on
a custom signet you set up yourself including manufacturing a re-org
every block if you wish.) Hence I'm a bit wary of making the behavior
on the default Signet deviate significantly from what you might
experience on mainnet. Given re-orgs don't occur that often on mainnet
I can see the argument for making them more regular (every 8 hours
seems reasonable to me) on the default Signet but every block seems
excessive. It makes the default Signet into an environment for purely
testing whether your application can withstand various flavors of edge
case re-orgs. You may want to test whether your application can
withstand normal mainnet behavior (no re-orgs for long periods of
time) first before you concern yourself with re-orgs.

> Why bother with a version bit? This seems substantially more complicated than the original proposal that surfaced many times before signet launched to just have a different reorg signing key. Thus, users who wish to follow reorgs can use a 1-of-2 (or higher multisig) and users who wish to not follow reorgs would use a 1-of-1 (or higher multisig), simply marking the reorg blocks as invalid without touching any header bits that non-full clients will ever see.

If I understand this correctly this is introducing a need for users to
sign blocks when currently with the default Signet the user does not
need to concern themselves with signing blocks. That is entirely left
to the network block signers of the default Signet (who were AJ and
Kalle last time I checked). Again I don't think this additional
complexity is needed on the default Signet when you can set up your
own custom Signet if you want to test edge case scenarios that deviate
significantly from what you are likely to experience on mainnet. A
flag set via a configuration argument (the AJ, 0xB10C proposal) with
no-reorgs (or 8 hour re-orgs) as the default seems to me like it would
introduce no additional complexity to the casual (or alpha stage)
tester experience though of course it introduces implementation
complexity.

To move the default Signet in the direction of resembling mainnet even
closer would be to randomly generate batches of transactions to fill
up blocks and create a fee market. It would be great to be able to
test features like RBF and Lightning unhappy paths (justice
transactions, perhaps even pinning attacks etc) on the default Signet
in future.

-- 
Michael Folkson
Email: michaelfolkson@gmail.com
Keybase: michaelfolkson
PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3


^ permalink raw reply	[flat|nested] 18+ messages in thread
* Re: [bitcoin-dev] Reorgs on SigNet - Looking for feedback on approach and parameters
@ 2021-09-13 12:30 Michael Folkson
  2021-09-13 16:24 ` Matt Corallo
  0 siblings, 1 reply; 18+ messages in thread
From: Michael Folkson @ 2021-09-13 12:30 UTC (permalink / raw)
  To: Matt Corallo, Bitcoin Protocol Discussion

> Can you explain the motivation for this? From where I sit, as far as I know, I should basically be > a prime example of the target market for public signet - someone developing bitcoin applications > with regular requirements to test those applications with other developers without
> jumping through hoops to configure software the same across the globe and set up miners.
> With blocks > being slow and irregular, I’m basically not benefited at all by signet and will
> stick with testnet3/mainnet testing, which both suck.

On testnet3 you can realistically go days without blocks being found
(and conversely thousands of blocks can be found in a day), the block
discovery time variance is huge. Of course this is probabilistically
possible on mainnet too but the probability of this happening is close
to zero. Here[0] is an example of 16,000 blocks being found in a day
on testnet3.

On signet block discovery time variance mirrors mainnet.

On mainnet you are risking Bitcoin with actual monetary value. If you
don't mind doing this then you don't need testnet3, signet or anything
else. In addition proposed soft forks may be activated on signet (and
could also be on testnet3) well before they are considered for
activation on mainnet for testing and experimentation purposes.

[0] https://web.archive.org/web/20160910173004/https://blog.blocktrail.com/2015/04/in-the-darkest-depths-of-testnet3-16k-blocks-were-found-in-1-day/

-- 
Michael Folkson
Email: michaelfolkson@gmail.com
Keybase: michaelfolkson
PGP: 43ED C999 9F85 1D40 EAF4 9835 92D6 0159 214C FEE3


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2021-10-15  4:42 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-07 16:07 [bitcoin-dev] Reorgs on SigNet - Looking for feedback on approach and parameters 0xB10C
2021-09-07 16:44 ` Jeremy
2021-09-08  7:59 ` Anthony Towns
2021-09-12 14:29   ` vjudeu
2021-09-12 14:54     ` Greg Sanders
2021-09-10  0:50 ` Matt Corallo
2021-09-12  7:53   ` Anthony Towns
2021-09-13  5:33     ` Matt Corallo
2021-09-14  4:56       ` Anthony Towns
2021-09-15 15:24         ` Matt Corallo
2021-10-15  4:41           ` Anthony Towns
2021-09-10 13:05 Michael Folkson
2021-09-10 18:24 ` Matt Corallo
2021-09-10 19:00   ` Michael Folkson
2021-09-10 19:22     ` Matt Corallo
2021-09-10 20:00   ` David A. Harding
2021-09-13 12:30 Michael Folkson
2021-09-13 16:24 ` Matt Corallo

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox