From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <jtimon@jtimon.cc> Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 5040E13D9 for <bitcoin-dev@lists.linuxfoundation.org>; Wed, 23 Sep 2015 17:28:09 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 1301D161 for <bitcoin-dev@lists.linuxfoundation.org>; Wed, 23 Sep 2015 17:28:08 +0000 (UTC) Received: by wicge5 with SMTP id ge5so217784385wic.0 for <bitcoin-dev@lists.linuxfoundation.org>; Wed, 23 Sep 2015 10:28:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=YuvuykZNHvOaTKgdVLVzckYgCpFBwFZVcZrn755W61g=; b=kbdXfkbR0ZPnMAsMY2JD/LWepUiRMqhUaEuQaDF7sc8TR+SbYpdJY+sXuNN2Nv6iJV O8l/9kaF320jZh8YGpBDQk7SAFgMhc4groay+Le8Wmq2MP8nyqsJ7QOwjfyQ3lQMo3Vr ZYUjlBpqKolEJ6hcem0CKBZ+FNmCyAk3kurJUiKakpTffGhIe16iAAIpRtXhlMJZUrV4 c3arW6RBbPT/1sTi7bk+bPAhUJxM+tksoHkpgwklXgsE1qq8VGL/TIEBjFAEAEwEum5m IhiSy+yWvMCLVBy6ixZ/AAYXd70IgT2X0L50eTi0dz/16695ZS3qt5p6Xp0X7QvXAQZb vZ/w== X-Gm-Message-State: ALoCoQm49cJc9Me8Ea4vtmsmUORtb1b4mVFmh4DOUBwzGjnuHcxg8UP2fvxy5Ey7BzXOUrDxTwCF MIME-Version: 1.0 X-Received: by 10.180.107.130 with SMTP id hc2mr4880536wib.92.1443029286611; Wed, 23 Sep 2015 10:28:06 -0700 (PDT) Received: by 10.194.37.5 with HTTP; Wed, 23 Sep 2015 10:28:06 -0700 (PDT) In-Reply-To: <CAGLBAhciSUZuKTcKmKFxy5O3Pmou=_-PG9Dk5m=y9p3TSBikAA@mail.gmail.com> References: <CADm_WcY8Vy+k+5BaBS+jV6D6tmSXrok8rAxoPxxKOzUhyPWgMg@mail.gmail.com> <CABm2gDoXa9ERY7iSsouxjypq1PwV_9HuBrtFQ_jrs5pGFst=KQ@mail.gmail.com> <CAGLBAhciSUZuKTcKmKFxy5O3Pmou=_-PG9Dk5m=y9p3TSBikAA@mail.gmail.com> Date: Wed, 23 Sep 2015 19:28:06 +0200 Message-ID: <CABm2gDrPeiaC9znS+th_VrmFGk6v=yr=i0=PNDSS0AjvrPG6JQ@mail.gmail.com> From: =?UTF-8?B?Sm9yZ2UgVGltw7Nu?= <jtimon@jtimon.cc> To: Dave Scotese <dscotese@litmocracy.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin development mailing list <bitcoin-dev@lists.linuxfoundation.org> Subject: Re: [bitcoin-dev] libconsensus and bitcoin development process X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org> List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> X-List-Received-Date: Wed, 23 Sep 2015 17:28:09 -0000 On Wed, Sep 23, 2015 at 1:49 AM, Dave Scotese <dscotese@litmocracy.com> wro= te: > If I'm reading this situation correctly, Jeff is basically pointing out t= hat > developers need more links (hooks, rungs, handholds, data points, whateve= r > you want to call them) so that they can see all the things his email > insinuated are missing (a plan, order, sense, etc.). He didn't say these > things were missing, but that it kind of feels like it from the 10,000 fo= ot > view. > > If you use Google to search the list, as in <<site:lists.linuxfoundation.= org > libconsensus plan>> you DO NOT get the page Jorge gave. He wrote that pa= ge, > so he had a good idea what to search for to find it again. I just want t= o > recommend that when you describe the work you're doing on bitcoin, imagin= e > several different ways people might try to find this description in the > future and make them work. In other words, Jorge could have put "A plan = for > abstracting out libconsensus" in the email where he wrote "Here are some > things that need to happen first..." > > Likewise, if Jeff had searched for <<site:lists.linuxfoundation.org > libconsensus plan>> (maybe he did, but he didn't list any results), he ma= y > have found enough clues to see Jorge's overall plan. The "site:" keyword= on > Google fascinated me when I discovered it, so I let it inspire this email > :-) My fault: https://github.com/bitcoin/bitcoin/issues/6714 > On Tue, Sep 22, 2015 at 11:12 AM, Jorge Tim=C3=B3n > <bitcoin-dev@lists.linuxfoundation.org> wrote: >> >> On Tue, Sep 15, 2015 at 6:10 AM, Jeff Garzik via bitcoin-dev >> <bitcoin-dev@lists.linuxfoundation.org> wrote: >> > [collating a private mail and a github issue comment, moving it to a >> > better forum] >> > >> > On libconsensus >> > --------------- >> > In general there exists the reasonable goal to move consensus state >> > and code to a specific, separate lib. >> > >> > To someone not closely reviewing the seemingly endless stream of >> > libconsensus refactoring PRs, the 10,000 foot view is that there is a >> > rather random stream of refactors that proceed in fits and starts >> > without apparent plan or end other than a one sentence "isolate >> > consensus state and code" summary. >> > >> > I am hoping that >> > * There is some plan >> > * We will not see a five year stream of random consensus code movement >> > patches causing lots of downstream developer headaches. >> > >> > I read every code change in every pull request that comes into >> > github/bitcoin/bitcoin with three exceptions: >> > * consensus code movement changes - too big, too chaotic, too >> > frequent, too unfocused, laziness guarantees others will inevitably >> > ACK it without me. >> > * some non-code changes (docs) >> > * ignore 80% of the Qt changes >> > >> > As with any sort of refactoring, they are easy to prove correct, easy >> > to reason, and therefore quick and easy to ACK and merge. >> > >> > Refactors however have a very real negative impact. >> > bitcoin/bitcoin.git is not only the source tree in the universe. >> > Software engineers at home, at startups, and at major companies are >> > maintaining branches of their own. >> > >> > It is very very easy to fall into a trap where a project is merging >> > lots of cosmetic changes and not seeing the downstream ripple effects. >> > Several people complained to me at the conference about all the code >> > movement changes breaking their own work, causing them to stay on >> > older versions of bitcoin due to the effort required to rebase to each >> > new release version - and I share those complaints. >> > >> > Complex code changes with longer development cycles than simple code >> > movement patches keep breaking. It is very frustrating, and causes >> > folks to get trapped between a rock and a hard place: >> > - Trying to push non-trivial changes upstream is difficult, for normal >> > and reasonable reasons (big important changes need review etc.). >> > - Maintaining non-trivial changes out of tree is also painful, for the >> > aforementioned reasons. >> > >> > Reasonable work languishes in constant-rebase hell, and incentivizes >> > against keeping up with the latest tree. >> > >> > >> > Aside from the refactor, libconsensus appears to be engineering in the >> > dark. Where is any sort of plan? I have low standards - a photo of a >> > whiteboard or youtube clip will do. >> >> Just because you don't understand the changes proposed it doesn't mean >> that they are random. >> I may have done a poor job in communicating "my plan for libconsensus" >> but I have tried many times and in many ways. >> #bitcoin-dev logs show that I have not worked "in the dark" at all, on >> the contrary, I've been very tenacious when asking for review and >> opinions, to the point that several people (at least @laanwj and >> @theuni have complained about their github inboxes being full of >> "spam"). >> This is a relatively recent thread where I describe my plan: >> >> http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-July/009568.= html >> Not my first attempt on this list. >> >> It is very frustrating that everybody seems to agree that separating >> libconsensus is a priority to maximize the number of people that can >> safely contribute to the project, but at the same time, nobody thinks >> that reviewing the necessary refactors to do so is a priority. >> I tried creating big PRs for people to see "the big picture" #5946 but >> those were too many commits and nobody wanted to read it. Gavin asked >> for an API. >> So I tried a smaller step: exposing just VerifyHeader in libconsensus >> and leave VerifyTx and VerifyBlock for later #5995 >> Again, this was "too big" and "a moving target". In the meantime I >> always had smaller one-little-step PRs that were part of a longer >> branch: >> >> ** [8/8] MERGED Consensus >> - [X] Consensus: Decouple pow from chainparams #5812 [consensuspow] >> - [X] MOVEONLY: Move constants and globals to consensus.h #5696 >> [consensus_policy0] >> - [X] Chainparams: Refactor: Decouple IsSuperMajority from Params() >> #5968 [params_consensus] >> - [X] Remove redundant getter CChainParams::SubsidyHalvingInterval() >> #5996 [params_subsidy] >> - [X] Separate CValidationState from main #5669 [consensus] >> - [X] Consensus: Decouple ContextualCheckBlockHeader from checkpoints >> #5975 [consensus_checkpoints] >> - [X] Separate Consensus::CheckTxInputs and GetSpendHeight in >> CheckInputs #6061 [consensus_inputs] >> - [X] Bugfix: Don't check the genesis block header before accepting it >> #6299 [5975-quick-fix] >> ** [5/5] DELETED >> *** DELETED Refactor: Create CCoinsViewEfficient interface for >> CCoinsViewCache #5747 [coins] >> *** DELETED Chainparams: Explicit Consensus::Params arg in consensus >> functions #6024 [params_consensus2] >> *** DELETED MOVEONLY: Move most of consensus functions (pre-block) >> #6051 [consensus_moveonly] (depends on consensus-blocksize-0.12.99) >> *** DELETED Consensus: Refactor: Separate CheckFinalTx from >> main::IsFinalTx #6063 [consensus_finaltx] >> *** DELETED Consensus: Refactor: Turn CBlockIndex::GetMedianTimePast >> into independent function #6009 [consensus_mediantime] >> *** DELETED Consensus: Adapt declarations of most obviously consensus >> functions #6591 [consensus-params-0.12.99] >> *** DELETED Consensus: Move blocksize and related parameters to >> consensusparams ...without removing consensus/consensus.h [#6526 >> alternative] #6625 [consensus-blocksize-0.12.99] >> >> After a while I stop rebasing the longer branches and just maintained >> a few small consensus-related PRs at a time. >> >> Now I consolidated 3 of them in >> >> *** REVIEW Optimizations: Consensus: In AcceptToMemoryPool, >> ConnectBlock, and CreateNewBlock #6445 [consensus-txinputs-0.12.99] >> >> with the hope that it would be merged relatively fast. >> After that it will be much simpler to start talking about potential C >> APIs for VerifyHeader, VerifyTx and VerifyBlock; as well as separating >> the library to a subtree. >> >> I'm more than happy to answer any questions anyone may have about any >> of the PRs or commits, until everybody interested is convinced that >> there's nothing random in the proposed changes. >> I'm also more than happy to get advice on how to better communicate my >> plans and structure my PRs. >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > > > -- > I like to provide some work at no charge to prove my value. Do you need a > techie? > I own Litmocracy and Meme Racing (in alpha). > I'm the webmaster for The Voluntaryist which now accepts Bitcoin. > I also code for The Dollar Vigilante. > "He ought to find it more profitable to play by the rules" - Satoshi > Nakamoto