Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire

[Kaz Wesley <keziahw@gmail.com>]

That's true, but I think it can be balanced with the usefulness of
knowing what messages a node has received. An invack would be sent if
N invs have been received without any resulting getdata; since we're
keeping track of peer inv order, one message can cover an arbitrarily
large series of invs.

On Fri, Jul 18, 2014 at 10:48 AM, Jeff Garzik <jgarzik@bitpay.com> wrote:
> On a flood-fill network, you don't want to create a storm of "I
> already have this" replies.
>
> On Fri, Jul 18, 2014 at 1:39 PM, Kaz Wesley <keziahw@gmail.com> wrote:
>> Peers exchanging mempool priority policies is great; that accomplishes
>> the flexibility in what txes to remember that I was going for with the
>> forget-filters, but much more neatly, with less overhead and some side
>> benefits.
>>
>> Here's what I'm picturing now:
>> - exchange priority policies in peer introductions
>> - assign unique sequential IDs in the order the transactions were
>> inved (per peer)
>> - receiving a getdata for a tx updates last-known-peer-received inv to
>> all invs up to the one referenced
>> - include ID-last-received, last-known-peer-received in sparse block
>> - reference txes in sparse block by index in receiver's
>> prioritiziation with peer's sent invs up to ID-last-received and
>> sender's prior invs up to last-known-peer-received
>>
>> Possible new messages:
>> - sparseblock
>> - invack message a node can send at times when it's received a bunch
>> of invs it already has, so it hasn't acked with a getdata in a while
>> - gettx: getdata, but using new sequential ID to save 28 bytes per tx
>>
>> It seems important for ordering policies to be able to be specified in
>> as much detail as possible. Parameters that should be available:
>> - total inputs
>> - total outputs
>> - bytes
>> - coin days destroyed
>> - net UTXO size change
>> - sigops
>> - is data carrier
>> - is output raw multisig
>> - age in mempool
>> - what else?
>> This parameter set should be extensible to allow for unforeseen future factors.
>>
>> Ordering policies should allow arbitrary algebraic combinations of
>> their parameters, as well as thresholds. Boolean combinations of
>> sub-policies would also be desirable. This could be implemented with a
>> tx-script-like stack-based language, in which each supported tx
>> property is pushed onto the stack by a particular opcode, and
>> +-*//min/max/boolean operators combine them to yield the sort key.
>>
>> Difficult parameters:
>> * Coin-days-destroyed: changes, peers need agreement on when (if?)
>> it's recalculated. Probably can just not recalculate, but peers still
>> need agreement on "time seen" to get CDD.
>> * Age in mempool: seems intractable in terms of time, but could be
>> done easily in terms of "how many txes old is this sequential ID"
>>
>> One potential pitfall: this allows for an environment of completely
>> heterogeneous mempool policies. I think that's a good thing, but we
>> need to avoid a situation where only least-common-denominator
>> transactions make it farther than a hop or two, and we don't want
>> nodes to have a strong preference for connecting to like-minded peers
>> since clustering reduces overall connectivity. It may be worthwhile to
>> add a parallel mechanism for relay policies, to differentiate between
>> what a node would keep in its mempool vs. what it wouldn't even relay
>> and doesn't want to see at all. Relay policies could be specified just
>> like prioritization policies, but with the final stack value evaluated
>> in a boolean context.
>>
>> An interesting additional use of policy-scripts would be a
>> standardized way for miners to include a policy script in a coinbase,
>> allowing miners a mechanism to advertise things like their relative
>> price of sigops vs bytes. Nodes may then choose to take this
>> information into account in order to optimize their mempool policies
>> for likelihood of consistency with future blocks. Since policy scripts
>> provide only relative information on prices of different transaction
>> properties rather than an absolute fee, this should not allow miners
>> to "vote fees up", although care would need to be taken they wouldn't
>> be able to drive up prices by claiming common transaction types are at
>> the high end of the fee scale.
>>
>> ------------------------------------------------------------------------------
>> Want fast and easy access to all the code in your enterprise? Index and
>> search up to 200,000 lines of code with a free copy of Black Duck
>> Code Sight - the same software that powers the world's largest code
>> search on Ohloh, the Black Duck Open Hub! Try it now.
>> http://p.sf.net/sfu/bds
>> _______________________________________________
>> Bitcoin-development mailing list
>> Bitcoin-development@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
>
> --
> Jeff Garzik
> Bitcoin core developer and open source evangelist
> BitPay, Inc.      https://bitpay.com/