From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 83D2213D9 for ; Fri, 18 Sep 2015 08:42:47 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id CA84D140 for ; Fri, 18 Sep 2015 08:42:46 +0000 (UTC) Received: by padhk3 with SMTP id hk3so45266844pad.3 for ; Fri, 18 Sep 2015 01:42:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=user-agent:in-reply-to:references:mime-version:content-type :content-transfer-encoding:subject:from:date:to:cc:message-id; bh=XJq29LxHARJ6sMG4NbZ3actfZhcspYqXpO36IARCcoc=; b=IeDQd+1HqzJ0JxnNeyzc+OrUda6WMfuLq+7GKYSLumNHrfFgZNNyBJm3uJSx29woVG EHVUyGOpp2gLT9ZZ9fzw/GO0fenyfh3l0P9UvsQNfbI1dmhyvSTTGenI/dhPkLJlNmXJ d6bBL/yd9iO290NrGEzfVojnHhOl5xfASXqbhqmB2i3+kM8RFuKKME94YhBn3mjDFgkW 9LHqWsYxFoUkAFkeONyN2MV//rsrz5v0Hc+4wiboBVXMGtpt+p+ksxcjHz2gL+fno/oV xQZeazwDug2nbC7ufHywUJAhP6GsXdAxkJDTXomwbZlzfWZzcJhyxMl7hsRIT1NqweyG MPyw== X-Received: by 10.66.253.129 with SMTP id aa1mr5917198pad.24.1442565766548; Fri, 18 Sep 2015 01:42:46 -0700 (PDT) Received: from [192.168.1.105] (cpe-76-167-237-202.san.res.rr.com. [76.167.237.202]) by smtp.gmail.com with ESMTPSA id f5sm7653278pas.23.2015.09.18.01.42.45 (version=TLSv1/SSLv3 cipher=OTHER); Fri, 18 Sep 2015 01:42:45 -0700 (PDT) User-Agent: K-9 Mail for Android In-Reply-To: <20150918000718.GA5251@amethyst.visucore.com> References: <20150916222928.GC32688@muck> <20150918000718.GA5251@amethyst.visucore.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----B7QXK5ZGLKEQ12AP1HV3N3POYETEHF" Content-Transfer-Encoding: 8bit From: Eric Lombrozo Date: Fri, 18 Sep 2015 01:42:53 -0700 To: "Wladimir J. van der Laan" , "Wladimir J. van der Laan via bitcoin-dev" , Peter Todd Message-ID: <3CADECE8-7A02-4247-99FD-380FFC332BC7@gmail.com> X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin development mailing list Subject: Re: [bitcoin-dev] libconsensus and bitcoin development process X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Sep 2015 08:42:47 -0000 ------B7QXK5ZGLKEQ12AP1HV3N3POYETEHF Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 You're aware that my entire stack was built around this model and I've even built a fully fledged desktop GUI, multisig account manager, and servers supporting pull and event subscription atop it, right? On September 17, 2015 5:07:20 PM PDT, "Wladimir J. van der Laan via bitcoin-dev" wrote: >On Wed, Sep 16, 2015 at 06:29:28PM -0400, Peter Todd via bitcoin-dev >wrote: > >> I've run into a number of cases where companies were maintaining >forks >> of Bitcoin Core unnecessarily, where a different, loosely coupled, >> architecture could do what they needed to do without including the >new >> logic in the codebase itself. > >This is the same point I have been making to Jeff privately. > >Refactors are a means to an end: a more modular, reusable and >maintainable codebase. This goal is that new functionality can be >plugged in more easily, and rebase work for e.g. functionality built on >top can go down, not up, if it just hooks into well-defined interfaces >here and there. > >Although there has been a lot of progress, bitcoind's design is still >too monolithic. To add a more involved feature, like say a new index >over the block chain data, code needs to be touched all over the place. >This change interacts with all other functionality, potentially >breaking the base node functionality - risk for users that do NOT use >the functionality. This increases risk and review time. > >- *If possible* functionality should be built without changing >bitcoind's code at all. An external process should be able to keep up >to date with the chain, notice reorgs, and process block data >accordingly. If bitcoind's interface does not allow that, or it is too >difficult, that is what should be fixed. >- *if not possible* then a change should at least touch the code in as >few places as possible, and integrate with e.g. signal notification. > >To name an example of it done right, IMO: Monero's 'simplewallet'. It >is a command-line utility wallet that communicates with the node >software, and remembers where it was in the chain, and processes >changes to the chain state since its last invocation when it >'refreshes'. >What is nice is that one can run an arbitary number of simplewallets >against one node daemon, and unlike bitcoind's wallet it doesn't need >to run as always-on daemon itself. It can be invoked when the user >wants to do something with the wallet, or see if there are new >transactions. > >An index could be implemented entirely externally in a similar way, >while still fully handling reorgs. > >What one needs for that, I think, is a library that communicate with >the node, and which offers functionality abstractly be similar to 'git >pull': give me the tree path from my current known tip to the best tip, >and supply the block hashes (and block data) along the way. > >My long-term vision of bitcoind is a P2P node with validation and >blockchain store, with a couple of data sources that can be subscribed >to or pulled from. > >Wladimir >_______________________________________________ >bitcoin-dev mailing list >bitcoin-dev@lists.linuxfoundation.org >https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev -- Sent from my Android device with K-9 Mail. Please excuse my brevity. ------B7QXK5ZGLKEQ12AP1HV3N3POYETEHF Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit You're aware that my entire stack was built around this model and I've even built a fully fledged desktop GUI, multisig account manager, and servers supporting pull and event subscription atop it, right?

On September 17, 2015 5:07:20 PM PDT, "Wladimir J. van der Laan via bitcoin-dev" <bitcoin-dev@lists.linuxfoundation.org> wrote:
On Wed, Sep 16, 2015 at 06:29:28PM -0400, Peter Todd via bitcoin-dev wrote:

 I've run into a number of cases where companies were maintaining forks
 of Bitcoin Core unnecessarily, where a different, loosely coupled,
 architecture could do what they needed to do without including the new
 logic in the codebase itself.

This is the same point I have been making to Jeff privately.

Refactors are a means to an end: a more modular, reusable and maintainable codebase. This goal is that new functionality can be plugged in more easily, and rebase work for e.g. functionality built on top can go down, not up, if it just hooks into well-defined interfaces here and there.

Although there has been a lot of progress, bitcoind's design is still too monolithic. To add a more involved feat
 ure,
like say a new index over the block chain data, code needs to be touched all over the place. This change interacts with all other functionality, potentially breaking the base node functionality - risk for users that do NOT use the functionality. This increases risk and review time.

- *If possible* functionality should be built without changing bitcoind's code at all. An external process should be able to keep up to date with the chain, notice reorgs, and process block data accordingly. If bitcoind's interface does not allow that, or it is too difficult, that is what should be fixed. 
- *if not possible* then a change should at least touch the code in as few places as possible, and integrate with e.g. signal notification.

To name an example of it done right, IMO: Monero's 'simplewallet'. It is a command-line utility wallet that communicates with the node software, and remembers where it was in the chain, and processes changes to the chain state since
  its
last invocation when it 'refreshes'. 
What is nice is that one can run an arbitary number of simplewallets against one node daemon, and unlike bitcoind's wallet it doesn't need to run as always-on daemon itself. It can be invoked when the user wants to do something with the wallet, or see if there are new transactions.

An index could be implemented entirely externally in a similar way, while still fully handling reorgs.

What one needs for that, I think, is a library that communicate with the node, and which offers functionality abstractly be similar to 'git pull': give me the tree path from my current known tip to the best tip, and supply the block hashes (and block data) along the way. 

My long-term vision of bitcoind is a P2P node with validation and blockchain store, with a couple of data sources that can be subscribed to or pulled from.

Wladimir

bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

--
Sent from my Android device with K-9 Mail. Please excuse my brevity. ------B7QXK5ZGLKEQ12AP1HV3N3POYETEHF--