Re: [Bitcoin-development] BIP 38 NFC normalisation issue

[Andreas Schildbach <andreas@schildbach.de>]

Damn, I just realized that I implement only the decoding side of BIP38.
So I cannot propose a complete test vector. Here is what I have:


Passphrase: ϓ␀𐐀💩 (\u03D2\u0301\u0000\U00010400\U0001F4A9; GREEK
UPSILON WITH HOOK, COMBINING ACUTE ACCENT, NULL, DESERET CAPITAL LETTER
LONG I, PILE OF POO)

Passphrase bytes after removing ISO control characters and NFC
normalization: 0xcf933034303066346139

Bitcoin Address: 16ktGzmfrurhbhi6JGqsMWf7TyqK9HNAeF

Unencrypted private key (WIF):
5Jajm8eQ22H3pGWLEVCXyvND8dQZhiQhoLJNKjYXk9roUFTMSZ4


Can someone calculate the encrypted key from it (using whatever
implementation) and I will verify it decodes properly in bitcoinj?



On 07/16/2014 12:46 PM, Andreas Schildbach wrote:
> I will change the bitcoinj implementation and propose a new test vector.
> 
> 
> 
> On 07/16/2014 11:29 AM, Mike Hearn wrote:
>> Yes sorry, you're right, the issue starts with the null code point.
>> Python seems to have problems starting there too. It might work if we
>> took that out.
>>
>>
>> On Wed, Jul 16, 2014 at 11:17 AM, Andreas Schildbach
>> <andreas@schildbach.de <mailto:andreas@schildbach.de>> wrote:
>>
>>     Guys, you are always talking about the Unicode astral plane, but in fact
>>     its a plain old (ASCII) control character where this problem starts and
>>     likely ends: \u0000.
>>
>>     Let's ban/filter ISO control characters and be done with it. Most
>>     control characters will never be enterable by any keyboard into a
>>     password field. Of course I assume that Character.isISOControl() works
>>     consistently across platforms.
>>
>>     http://docs.oracle.com/javase/7/docs/api/java/lang/Character.html#isISOControl%28char%29
>>
>>
>>     On 07/16/2014 12:23 AM, Aaron Voisine wrote:
>>     > If the user creates a password on an iOS device with an astral
>>     > character and then can't enter that password on a JVM wallet, that
>>     > sucks. If JVMs really can't support unicode NFC then that's a strong
>>     > case to limit the spec to the subset of unicode that all popular
>>     > platforms can support, but it sounds like it might just be a JVM
>>     > string library bug that could hopefully be reported and fixed. I get
>>     > the same result as in the test case using apple's
>>     > CFStringNormalize(passphrase, kCFStringNormalizationFormC);
>>     >
>>     > Aaron Voisine
>>     > breadwallet.com <http://breadwallet.com>
>>     >
>>     >
>>     > On Tue, Jul 15, 2014 at 11:20 AM, Mike Hearn <mike@plan99.net
>>     <mailto:mike@plan99.net>> wrote:
>>     >> Yes, we know, Andreas' code is indeed doing normalisation.
>>     >>
>>     >> However it appears the output bytes end up being different. What
>>     I get back
>>     >> is:
>>     >>
>>     >> cf930001303430300166346139
>>     >>
>>     >> vs
>>     >>
>>     >> cf9300f0909080f09f92a9
>>     >>
>>     >> from the spec.
>>     >>
>>     >> I'm not sure why. It appears this is due to the character from
>>     the astral
>>     >> planes. Java is old and uses 16 bit characters internally - it
>>     wouldn't
>>     >> surprise me if there's some weirdness that means it doesn't/won't
>>     support
>>     >> this kind of thing.
>>     >>
>>     >> I recommend instead that any implementation that wishes to be
>>     compatible
>>     >> with JVM based wallets (I suspect Android is the same) just
>>     refuse any
>>     >> passphrase that includes characters outside the BMP. At least
>>     unless someone
>>     >> can find a fix. I somehow doubt this will really hurt anyone.
>>     >>
>>     >>
>>     ------------------------------------------------------------------------------
>>     >> Want fast and easy access to all the code in your enterprise?
>>     Index and
>>     >> search up to 200,000 lines of code with a free copy of Black Duck
>>     >> Code Sight - the same software that powers the world's largest code
>>     >> search on Ohloh, the Black Duck Open Hub! Try it now.
>>     >> http://p.sf.net/sfu/bds
>>     >> _______________________________________________
>>     >> Bitcoin-development mailing list
>>     >> Bitcoin-development@lists.sourceforge.net
>>     <mailto:Bitcoin-development@lists.sourceforge.net>
>>     >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>     >>
>>     >
>>     >
>>     ------------------------------------------------------------------------------
>>     > Want fast and easy access to all the code in your enterprise?
>>     Index and
>>     > search up to 200,000 lines of code with a free copy of Black Duck
>>     > Code Sight - the same software that powers the world's largest code
>>     > search on Ohloh, the Black Duck Open Hub! Try it now.
>>     > http://p.sf.net/sfu/bds
>>     >
>>
>>
>>
>>     ------------------------------------------------------------------------------
>>     Want fast and easy access to all the code in your enterprise? Index and
>>     search up to 200,000 lines of code with a free copy of Black Duck
>>     Code Sight - the same software that powers the world's largest code
>>     search on Ohloh, the Black Duck Open Hub! Try it now.
>>     http://p.sf.net/sfu/bds
>>     _______________________________________________
>>     Bitcoin-development mailing list
>>     Bitcoin-development@lists.sourceforge.net
>>     <mailto:Bitcoin-development@lists.sourceforge.net>
>>     https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Want fast and easy access to all the code in your enterprise? Index and
>> search up to 200,000 lines of code with a free copy of Black Duck
>> Code Sight - the same software that powers the world's largest code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.net/sfu/bds
>>
>>
>>
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>
> 
> 
> 
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
>