[bitcoin-dev] BIP44 & BIP32 chain address look-ahead limits

[Jameson Lopp <jameson.lopp@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1788 bytes --]

I recently ran into an issue while importing a Mycelium HD wallet where it
was not finding all of my funds - upon further investigation with Mycelium
devs we realized that the wallet was following the BIP44 spec correctly,
but BIP44 may have a flaw.

The problem was a result of my creating 16 transactions in Mycelium in a
fairly short timeframe, but the first 15 transactions ended up never
confirming while the 16th was confirmed. As a result, when I later
reimported the account from the master seed, the chain derivation stopped
upon hitting this large gap of unused addresses on the internal / change
chain.

BIP44 recommends that there need not be a lookahead on internal chains
"because internal chains receive only coins that come from the associated
external chains."
https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki#Address_gap_limit

BIP32 also notes that "the look-ahead for internal chains can be very
small, as no gaps are to be expected here."
https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki#Full_wallet_sharing_m

It seems to me that there /is/ an edge case that can result in significant
gaps in internal chain address usage and as such, the recommendation should
be to look ahead on both external and internal chains when performing
account discovery. On a related note, the recommended look-ahead of 20 may
not be safe enough - perhaps it should be raised to 100 if not higher.

In addition to recommending a larger look-ahead, it may also be advisable
for BIP44 to recommend that wallets "fill in" gaps of unused chain
addresses by "looking back" from the current tip of the internal chain's
index when the wallet decides to create a new change address. This could
help mitigate the size of gaps caused by failed transactions.

- Jameson

[-- Attachment #2: Type: text/html, Size: 2166 bytes --]