public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: Thomas Voegtlin <thomasv1@gmx.de>
To: Bitcoin Development <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] New BIP32 structure
Date: Thu, 27 Mar 2014 14:30:18 +0100	[thread overview]
Message-ID: <533427EA.5010300@gmx.de> (raw)
In-Reply-To: <CANEZrP1v7ZCmhhoHmuXXXvKwAV1_0a02Vkf9z4nQGNfAbZBM=A@mail.gmail.com>



Le 27/03/2014 12:39, Mike Hearn a écrit :
>     One issue that I have is bandwidth: Electrum (and mycelium) cannot
>     watch as many addresses as they want, because this will create too
>     much traffic on the servers. (especially when servers send utxo merkle
>     proofs for each address, which is not the case yet, but is planned)
>
>
> This is surprising and the first time I've heard about this. Surely your
> constraint is CPU or disk seeks? Addresses are small, I find it hard to
> believe that clients uploading them is a big drain, and mostly addresses
> that are in the lookahead region won't have any hits and so won't result
> in any downloads?


To be honest, I have not carried out a comprehensive examination of 
server performance. What I can see is that Electrum servers are often 
slowed down when a wallet with a large number (thousands) of addresses 
shows up, and this is caused by disk seeks (especially on my slow VPS).

The master branch of electrum-server is also quite wasteful in terms of 
CPU, because it uses client threads. I have another branch that uses a 
socket poller, but that branch is not widely deployed yet.

I reckon that I might have been a bit too conservative, in setting the 
number of unused receiving addresses watched by Electrum clients (until 
now, the default "gap limit" has always been 5). The reason is that, if 
I increase that number, then there is no way to go back to a smaller 
value, because it needs to be compatible with all previously released 
versions. However, Electrum servers performance has improved over time, 
so I guess it could safely be raised to 20 (see previous post to slush).

In terms of bandwidth, I am referring to my Android version of Electrum. 
When it runs on a 3G connection, it sometimes takes up to 1 minute to 
synchronize (with a wallet that has hundreds of addresses). However, I 
have not checked if this was caused by addresses or block headers.



>
>     This constraint is not so important for bloom-filter clients.
>
>
> Bloom filters are a neat way to encode addresses and keys but they don't
> magically let clients save bandwidth. A smaller filter results in less
> upload bandwidth but more download (from the wallets perspective). So
> I'm worried if you think this will be an issue for your clients: I
> haven't investigated bandwidth usage deeply yet, perhaps I should.
>
> FWIW the current bitcoinj HDW alpha preview pre-gens 100 addresses on
> both receive and change branches. But I'm not sure what the right
> setting is.


Heh, may I suggest 20 in the receive branch?

For the change branch, there is no need to watch a large number of 
unused addresses, because the wallet should try to fill all the gaps in 
the sequence of change.

(Electrum does that. It also watches 3 unused addresses at the end of 
that sequence, in order to cope with possible blockchain reorgs causing 
gaps. As an extra safety, it also waits for 3 confirmations before using 
a new change address, which sometimes results in address reuse, but I 
guess a smarter strategy could avoid that).



>
> We also have to consider latency. The simplest implementation from a
> wallets POV is to step through each transaction in the block chain one
> at a time, and each time you see an address that is yours, calculate the
> next ones in the chain. But that would be fantastically slow, so we must
> instead pre-generate a larger lookahead region and request more data in
> one batch. Then you have to recover if that batch ends up using all the
> pre-genned addresses. It's just painful.



>
>     My opinion, as far as Electrum is concerned, is that merchant accounts
>     should behave differently from regular user accounts: While merchants
>     need to generate an unlimited number of receiving addresses, it is also
>     acceptable for them to have a slightly more complex wallet recovery
>     procedure
>
>
> Maybe. I dislike any distinction between users and merchants though. I
> don't think it's really safe to assume merchants are more sophisticated
> than end users.

well, it depends what we mean by "merchant". I was thinking more of a 
website running a script, rather than a brick and mortar ice cream 
seller. :)


>
>     but also because we want fully automated synchronization between
>     different
>     instances of a wallet, using only no other source of information than
>     the blockchain.
>
>
> I think such synchronization won't be possible as we keep adding
> features, because the block chain cannot sync all the relevant data. For
> instance Electrum already has a label sync feature. Other wallets need
> to compete with that, somehow, so we need to build a way to do
> cross-device wallet sync with non-chain data.

Oh, I was not referring to label sync, but only to the synchronization 
of the list of addresses in the wallet. Label sync is an Electrum plugin 
that relies on a centralized server. Using a third party server is 
acceptable in that case, IMO, because you will not lose your coins if 
the server fails.




  reply	other threads:[~2014-03-27 13:30 UTC|newest]

Thread overview: 100+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-03-26 20:49 [Bitcoin-development] New BIP32 structure Mike Hearn
2014-03-26 23:37 ` Andreas Schildbach
2014-03-27  1:01   ` Alan Reiner
2014-03-27  6:15     ` Mike Belshe
2014-03-27 10:57   ` Thomas Voegtlin
2014-03-27 11:39     ` Mike Hearn
2014-03-27 13:30       ` Thomas Voegtlin [this message]
2014-03-27 13:38         ` Mike Hearn
2014-03-27  1:13 ` Allen Piscitello
2014-03-27  5:34 ` Troy Benjegerdes
2014-03-27  7:09 ` Tamas Blummer
2014-03-27  9:42   ` Mike Hearn
2014-03-27 11:35   ` Pavol Rusnak
2014-03-27 15:43   ` Jeff Garzik
2014-03-27 15:57     ` Allen Piscitello
2014-03-27 16:06       ` Pavol Rusnak
2014-03-27 16:13         ` Allen Piscitello
2014-03-27 16:07       ` Tamas Blummer
2014-03-27 16:14       ` Pieter Wuille
2014-03-27 16:21         ` Pavol Rusnak
2014-03-27 16:28           ` Pieter Wuille
2014-03-27 17:49             ` Allen Piscitello
2014-03-28 14:59             ` slush
2014-04-08 12:43               ` slush
2014-04-08 13:18                 ` Pieter Wuille
2014-04-08 13:40                   ` slush
2014-04-08 13:43                     ` slush
2014-04-08 13:53                       ` Pieter Wuille
2014-04-08 13:59                         ` Thomas Voegtlin
2014-04-08 14:00                         ` Tamas Blummer
2014-04-08 14:35                         ` Pavol Rusnak
2014-04-08 15:41                         ` slush
2014-04-23 17:42                           ` Pieter Wuille
2014-04-23 18:01                             ` slush
2014-04-23 18:18                               ` slush
2014-04-23 18:39                                 ` Tier Nolan
2014-04-23 18:46                                   ` Pavol Rusnak
2014-04-23 19:00                                     ` Tier Nolan
2014-04-23 19:06                                       ` slush
2014-04-23 19:36                                         ` Tamas Blummer
2014-04-23 19:46                                           ` slush
2014-04-23 19:07                                       ` Tamas Blummer
2014-04-23 19:29                                       ` Pavol Rusnak
2014-04-23 19:44                                         ` Luke-Jr
2014-04-23 19:49                                           ` Pavol Rusnak
2014-04-23 19:55                                             ` Luke-Jr
2014-04-23 19:57                                               ` slush
2014-04-23 20:01                                                 ` Luke-Jr
2014-04-23 20:04                                                   ` Pavol Rusnak
2014-04-23 20:09                                                     ` Luke-Jr
2014-04-23 20:16                                                       ` Pavol Rusnak
2014-04-23 20:32                                                         ` Luke-Jr
2014-04-23 20:35                                                           ` Pavol Rusnak
2014-04-23 20:41                                                             ` Luke-Jr
2014-04-23 20:43                                                               ` Pavol Rusnak
2014-04-23 20:54                                                                 ` Pieter Wuille
2014-04-23 21:06                                                                   ` Pavol Rusnak
2014-04-23 21:18                                                                     ` Luke-Jr
2014-04-23 21:22                                                                       ` Gregory Maxwell
2014-04-23 21:33                                                                         ` Pavol Rusnak
2014-04-23 21:42                                                                           ` Pieter Wuille
2014-04-23 21:44                                                                             ` Pavol Rusnak
2014-04-23 21:48                                                                             ` Gregory Maxwell
2014-04-23 21:53                                                                           ` Luke-Jr
2014-04-23 21:24                                                                       ` Pavol Rusnak
2014-04-23 21:07                                                                   ` slush
2014-04-23 20:59                                                                 ` Luke-Jr
2014-04-23 20:17                                                       ` Tamas Blummer
2014-04-23 20:12                                                     ` Pieter Wuille
2014-04-23 20:08                                                   ` Pieter Wuille
2014-04-23 20:01                                               ` Tamas Blummer
2014-04-23 20:02                                                 ` Luke-Jr
2014-04-23 20:04                                                   ` Tamas Blummer
2014-04-24  6:54                                           ` Thomas Voegtlin
2014-04-24  7:10                                             ` Pieter Wuille
2014-04-24  7:21                                               ` Gregory Maxwell
2014-04-24  8:15                                                 ` Thomas Voegtlin
2014-04-24  7:42                                               ` Thomas Voegtlin
2014-04-24  8:09                                               ` Mike Hearn
2014-04-23 18:48                                   ` Tamas Blummer
2014-04-08 14:49                 ` Andreas Schildbach
2014-04-08 15:46                   ` slush
2014-04-08 15:58                     ` Andreas Schildbach
2014-04-08 16:24                       ` slush
2014-03-27 11:20 ` Thomas Voegtlin
     [not found]   ` <CAJna-HhmFya+3W67qQt0wMhW=B4vJvwdkr-5WnU+KEaKq7uaUA@mail.gmail.com>
2014-03-27 12:06     ` Thomas Voegtlin
2014-03-27 12:28       ` Mike Hearn
2014-03-27 12:49         ` Mike Hearn
2014-03-27 13:12           ` Thomas Kerin
2014-03-27 13:19             ` Mike Hearn
2014-03-27 13:49           ` Thomas Voegtlin
2014-03-27 13:58             ` Jim
2014-03-27 14:20               ` Mike Hearn
2014-03-27 15:17             ` Troy Benjegerdes
2014-03-27 13:07         ` Matias Alejo Garcia
2014-03-27 13:44       ` Pavol Rusnak
2014-03-27 13:53         ` Thomas Voegtlin
2014-03-27 14:03           ` Pavol Rusnak
     [not found]       ` <CAJna-HitjJbL9TnfxTY=+TvfaeNvZM6aPWmNYmHUCwmw6V8PUg@mail.gmail.com>
2014-03-27 15:04         ` Thomas Voegtlin
2014-03-27 11:36 ` Pavol Rusnak

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=533427EA.5010300@gmx.de \
    --to=thomasv1@gmx.de \
    --cc=bitcoin-development@lists.sourceforge.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox