Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire

[Jeff Garzik <jgarzik@bitpay.com>]

On a flood-fill network, you don't want to create a storm of "I
already have this" replies.

On Fri, Jul 18, 2014 at 1:39 PM, Kaz Wesley <keziahw@gmail.com> wrote:
> Peers exchanging mempool priority policies is great; that accomplishes
> the flexibility in what txes to remember that I was going for with the
> forget-filters, but much more neatly, with less overhead and some side
> benefits.
>
> Here's what I'm picturing now:
> - exchange priority policies in peer introductions
> - assign unique sequential IDs in the order the transactions were
> inved (per peer)
> - receiving a getdata for a tx updates last-known-peer-received inv to
> all invs up to the one referenced
> - include ID-last-received, last-known-peer-received in sparse block
> - reference txes in sparse block by index in receiver's
> prioritiziation with peer's sent invs up to ID-last-received and
> sender's prior invs up to last-known-peer-received
>
> Possible new messages:
> - sparseblock
> - invack message a node can send at times when it's received a bunch
> of invs it already has, so it hasn't acked with a getdata in a while
> - gettx: getdata, but using new sequential ID to save 28 bytes per tx
>
> It seems important for ordering policies to be able to be specified in
> as much detail as possible. Parameters that should be available:
> - total inputs
> - total outputs
> - bytes
> - coin days destroyed
> - net UTXO size change
> - sigops
> - is data carrier
> - is output raw multisig
> - age in mempool
> - what else?
> This parameter set should be extensible to allow for unforeseen future factors.
>
> Ordering policies should allow arbitrary algebraic combinations of
> their parameters, as well as thresholds. Boolean combinations of
> sub-policies would also be desirable. This could be implemented with a
> tx-script-like stack-based language, in which each supported tx
> property is pushed onto the stack by a particular opcode, and
> +-*//min/max/boolean operators combine them to yield the sort key.
>
> Difficult parameters:
> * Coin-days-destroyed: changes, peers need agreement on when (if?)
> it's recalculated. Probably can just not recalculate, but peers still
> need agreement on "time seen" to get CDD.
> * Age in mempool: seems intractable in terms of time, but could be
> done easily in terms of "how many txes old is this sequential ID"
>
> One potential pitfall: this allows for an environment of completely
> heterogeneous mempool policies. I think that's a good thing, but we
> need to avoid a situation where only least-common-denominator
> transactions make it farther than a hop or two, and we don't want
> nodes to have a strong preference for connecting to like-minded peers
> since clustering reduces overall connectivity. It may be worthwhile to
> add a parallel mechanism for relay policies, to differentiate between
> what a node would keep in its mempool vs. what it wouldn't even relay
> and doesn't want to see at all. Relay policies could be specified just
> like prioritization policies, but with the final stack value evaluated
> in a boolean context.
>
> An interesting additional use of policy-scripts would be a
> standardized way for miners to include a policy script in a coinbase,
> allowing miners a mechanism to advertise things like their relative
> price of sigops vs bytes. Nodes may then choose to take this
> information into account in order to optimize their mempool policies
> for likelihood of consistency with future blocks. Since policy scripts
> provide only relative information on prices of different transaction
> properties rather than an absolute fee, this should not allow miners
> to "vote fees up", although care would need to be taken they wouldn't
> be able to drive up prices by claiming common transaction types are at
> the high end of the fee scale.
>
> ------------------------------------------------------------------------------
> Want fast and easy access to all the code in your enterprise? Index and
> search up to 200,000 lines of code with a free copy of Black Duck
> Code Sight - the same software that powers the world's largest code
> search on Ohloh, the Black Duck Open Hub! Try it now.
> http://p.sf.net/sfu/bds
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development



-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/