I'm all for fixing bugs, but I know from bitter experience that outside the BMP dragons lurk. Browsers don't even expose Unicode APIs at all. You end up needing to ship an entire pure-js implementation, which can be too large for some use cases (too much time sunk on that issue in my last job).

I'm hoping BIP 38 doesn't get widely used anyway, to be frank. People moving private keys around by hand has caused quite a few problems in the past, sometimes people lost money. It's better to work at the level of a wallet and ideally ask people to move money using regular transactions. Way less potential for errors.

Regardless, I'll file a JVM bug and see what the outcome is.

On Wed, Jul 16, 2014 at 12:23 AM, Aaron Voisine <voisine@gmail.com> wrote:

If the user creates a password on an iOS device with an astral
character and then can't enter that password on a JVM wallet, that
sucks. If JVMs really can't support unicode NFC then that's a strong
case to limit the spec to the subset of unicode that all popular
platforms can support, but it sounds like it might just be a JVM
string library bug that could hopefully be reported and fixed. I get
the same result as in the test case using apple's
CFStringNormalize(passphrase, kCFStringNormalizationFormC);

Aaron Voisine
breadwallet.com

On Tue, Jul 15, 2014 at 11:20 AM, Mike Hearn <mike@plan99.net> wrote:
> Yes, we know, Andreas' code is indeed doing normalisation.
>
> However it appears the output bytes end up being different. What I get back
> is:
>
> cf930001303430300166346139
>
> vs
>
> cf9300f0909080f09f92a9
>
> from the spec.
>
> I'm not sure why. It appears this is due to the character from the astral
> planes. Java is old and uses 16 bit characters internally - it wouldn't
> surprise me if there's some weirdness that means it doesn't/won't support
> this kind of thing.
>
> I recommend instead that any implementation that wishes to be compatible
> with JVM based wallets (I suspect Android is the same) just refuse any
> passphrase that includes characters outside the BMP. At least unless someone
> can find a fix. I somehow doubt this will really hurt anyone.
>

> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>