From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1WTAO3-0006u9-FU for bitcoin-development@lists.sourceforge.net; Thu, 27 Mar 2014 13:30:27 +0000 Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmx.de designates 212.227.17.20 as permitted sender) client-ip=212.227.17.20; envelope-from=thomasv1@gmx.de; helo=mout.gmx.net; Received: from mout.gmx.net ([212.227.17.20]) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1WTAO1-0005dt-Un for bitcoin-development@lists.sourceforge.net; Thu, 27 Mar 2014 13:30:27 +0000 Received: from [192.168.1.27] ([84.101.32.222]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0LkfdE-1X1Ofv1FQ4-00aRcG for ; Thu, 27 Mar 2014 14:30:19 +0100 Message-ID: <533427EA.5010300@gmx.de> Date: Thu, 27 Mar 2014 14:30:18 +0100 From: Thomas Voegtlin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Bitcoin Development References: <53340426.4040208@gmx.de> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Provags-ID: V03:K0:EcyRG5i9Kt3mg2CzjFGAhIXVtTc/Nmu6lLO+JVZah+85Fao5ehh SLu4htDi3KVrogF3MjAg6RJ9weW78IYw20+uijiwCjljl59O7H2+aPfkkA7vjI2o/50VqYg AmSSUopI2KPv964IkVKZQ2BzkKWWgyBXapgqlqXiqHeKenbJAjfpOx1+cO4pe+xoGhkPqdB B7dxc7N/CGrSMcLcOGOXg== X-Spam-Score: -1.2 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [212.227.17.20 listed in list.dnswl.org] -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (thomasv1[at]gmx.de) -0.0 SPF_PASS SPF: sender matches SPF record 0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends in digit (thomasv1[at]gmx.de) X-Headers-End: 1WTAO1-0005dt-Un Subject: Re: [Bitcoin-development] New BIP32 structure X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 13:30:27 -0000 Le 27/03/2014 12:39, Mike Hearn a écrit : > One issue that I have is bandwidth: Electrum (and mycelium) cannot > watch as many addresses as they want, because this will create too > much traffic on the servers. (especially when servers send utxo merkle > proofs for each address, which is not the case yet, but is planned) > > > This is surprising and the first time I've heard about this. Surely your > constraint is CPU or disk seeks? Addresses are small, I find it hard to > believe that clients uploading them is a big drain, and mostly addresses > that are in the lookahead region won't have any hits and so won't result > in any downloads? To be honest, I have not carried out a comprehensive examination of server performance. What I can see is that Electrum servers are often slowed down when a wallet with a large number (thousands) of addresses shows up, and this is caused by disk seeks (especially on my slow VPS). The master branch of electrum-server is also quite wasteful in terms of CPU, because it uses client threads. I have another branch that uses a socket poller, but that branch is not widely deployed yet. I reckon that I might have been a bit too conservative, in setting the number of unused receiving addresses watched by Electrum clients (until now, the default "gap limit" has always been 5). The reason is that, if I increase that number, then there is no way to go back to a smaller value, because it needs to be compatible with all previously released versions. However, Electrum servers performance has improved over time, so I guess it could safely be raised to 20 (see previous post to slush). In terms of bandwidth, I am referring to my Android version of Electrum. When it runs on a 3G connection, it sometimes takes up to 1 minute to synchronize (with a wallet that has hundreds of addresses). However, I have not checked if this was caused by addresses or block headers. > > This constraint is not so important for bloom-filter clients. > > > Bloom filters are a neat way to encode addresses and keys but they don't > magically let clients save bandwidth. A smaller filter results in less > upload bandwidth but more download (from the wallets perspective). So > I'm worried if you think this will be an issue for your clients: I > haven't investigated bandwidth usage deeply yet, perhaps I should. > > FWIW the current bitcoinj HDW alpha preview pre-gens 100 addresses on > both receive and change branches. But I'm not sure what the right > setting is. Heh, may I suggest 20 in the receive branch? For the change branch, there is no need to watch a large number of unused addresses, because the wallet should try to fill all the gaps in the sequence of change. (Electrum does that. It also watches 3 unused addresses at the end of that sequence, in order to cope with possible blockchain reorgs causing gaps. As an extra safety, it also waits for 3 confirmations before using a new change address, which sometimes results in address reuse, but I guess a smarter strategy could avoid that). > > We also have to consider latency. The simplest implementation from a > wallets POV is to step through each transaction in the block chain one > at a time, and each time you see an address that is yours, calculate the > next ones in the chain. But that would be fantastically slow, so we must > instead pre-generate a larger lookahead region and request more data in > one batch. Then you have to recover if that batch ends up using all the > pre-genned addresses. It's just painful. > > My opinion, as far as Electrum is concerned, is that merchant accounts > should behave differently from regular user accounts: While merchants > need to generate an unlimited number of receiving addresses, it is also > acceptable for them to have a slightly more complex wallet recovery > procedure > > > Maybe. I dislike any distinction between users and merchants though. I > don't think it's really safe to assume merchants are more sophisticated > than end users. well, it depends what we mean by "merchant". I was thinking more of a website running a script, rather than a brick and mortar ice cream seller. :) > > but also because we want fully automated synchronization between > different > instances of a wallet, using only no other source of information than > the blockchain. > > > I think such synchronization won't be possible as we keep adding > features, because the block chain cannot sync all the relevant data. For > instance Electrum already has a label sync feature. Other wallets need > to compete with that, somehow, so we need to build a way to do > cross-device wallet sync with non-chain data. Oh, I was not referring to label sync, but only to the synchronization of the list of addresses in the wallet. Label sync is an Electrum plugin that relies on a centralized server. Using a third party server is acceptable in that case, IMO, because you will not lose your coins if the server fails.