public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: Peter Tschipper <peter.tschipper@gmail.com>
To: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] request BIP number for: "Support for Datastream Compression"
Date: Tue, 10 Nov 2015 08:46:54 -0800	[thread overview]
Message-ID: <56421F7E.8070305@gmail.com> (raw)
In-Reply-To: <56421F1E.4050302@gmail.com>

[-- Attachment #1: Type: text/plain, Size: 3993 bytes --]

On 10/11/2015 8:45 AM, Peter Tschipper wrote:
> On 10/11/2015 8:30 AM, Tier Nolan via bitcoin-dev wrote:
>>
>>
>> On Tue, Nov 10, 2015 at 4:11 PM, Peter Tschipper
>> <peter.tschipper@gmail.com> wrote:
>>
>>     There are better ways of sending new blocks, that's certainly
>>     true but for sending historical blocks and seding transactions I
>>     don't think so.  This PR is really designed to save bandwidth and
>>     not intended to be a huge performance improvement in terms of
>>     time spent sending.
>>
>>
>> If the main point is for historical data, then sticking to just
>> blocks is the best plan.
>>
> at the beginning yes.
>> Since small blocks don't compress well, you could define a "cblocks"
>> message that handles multiple blocks (just concatenate the block
>> messages as payload before compression). 
>>
> Small block are rare these days (but plenty of historical block), but
> still they get a 10% compression, not bad and I think worthwhile and
> the time it takes to compress small blocks is less that a millisecond
> so no loss there in time.   But still you have a good point and
> something worthy of doing after getting compression to work.  I think
> it's wise to keep it simple at first and build on the success later.
>> The sending peer could combine blocks so that each cblock is
>> compressing at least 10kB of block data (or whatever is optimal).  It
>> is probably worth specifying a maximum size for network buffer
>> reasons (either 1MB or 1 block maximum).
> Good idea. Same answer as above.
>> Similarly, transactions could be combined together and compressed
>> "ctxs".  The inv messages could be modified so that you can request
>> groups of 10-20 transactions.  That would depend on how much of an
>> improvement compressed transactions would represent.
>>
> Good idea. Same answer as above.
>> More generally, you could define a message which is a compressed
>> message holder.  That is probably to complex to be worth the effort
>> though.
> That's actually pretty easy to do and part of the plan.  Sending a
> cmp_block rather than a block makes it all easier to implement.  It's
> just a matter of doing pnode->pushmessage("cmp_block",
> compressed_block); and handling the "cmp_block" command string at the
> other end.
>>
>>  
>>
>>>
>>>     On Tue, Nov 10, 2015 at 5:40 AM, Johnathan Corgan via
>>>     bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org
>>>     <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote:
>>>
>>>         On Mon, Nov 9, 2015 at 5:58 PM, gladoscc via bitcoin-dev
>>>         <bitcoin-dev@lists.linuxfoundation.org> wrote:
>>>          
>>>
>>>             I think 25% bandwidth savings is certainly considerable,
>>>             especially for people running full nodes in countries
>>>             like Australia where internet bandwidth is lower and
>>>             there are data caps.
>>>
>>>
>>>         ​ This reinforces the idea that such trade-off decisions
>>>         should be be local and negotiated between peers, not a
>>>         required feature of the network P2P.​
>>>          
>>>
>>>         -- 
>>>         Johnathan Corgan
>>>         Corgan Labs - SDR Training and Development Services
>>>         http://corganlabs.com
>>>
>>>         _______________________________________________
>>>         bitcoin-dev mailing list
>>>         bitcoin-dev@lists.linuxfoundation.org
>>>         <mailto:bitcoin-dev@lists.linuxfoundation.org>
>>>         https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>>
>>>
>>>
>>>
>>>     _______________________________________________
>>>     bitcoin-dev mailing list
>>>     bitcoin-dev@lists.linuxfoundation.org
>>>     <mailto:bitcoin-dev@lists.linuxfoundation.org>
>>>     https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>>
>>
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>


[-- Attachment #2: Type: text/html, Size: 12736 bytes --]

      parent reply	other threads:[~2015-11-10 16:46 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-11-09 19:18 [bitcoin-dev] request BIP number for: "Support for Datastream Compression" Peter Tschipper
2015-11-09 20:41 ` Johnathan Corgan
2015-11-09 21:04 ` Bob McElrath
2015-11-10  1:58   ` gladoscc
2015-11-10  5:40     ` Johnathan Corgan
2015-11-10  9:44       ` Tier Nolan
     [not found]         ` <5642172C.701@gmail.com>
2015-11-10 16:17           ` Peter Tschipper
2015-11-10 16:21             ` Jonathan Toomim
2015-11-10 16:30           ` Tier Nolan
2015-11-10 16:46             ` Jeff Garzik
2015-11-10 17:09               ` Peter Tschipper
2015-11-11 18:35               ` Peter Tschipper
2015-11-11 18:49                 ` Marco Pontello
2015-11-11 19:05                   ` Jonathan Toomim
2015-11-13 21:58                     ` [bitcoin-dev] Block Compression (Datastream Compression) test results using the PR#6973 compression prototype Peter Tschipper
2015-11-18 14:00                       ` [bitcoin-dev] More findings: " Peter Tschipper
2015-11-11 19:11                   ` [bitcoin-dev] request BIP number for: "Support for Datastream Compression" Peter Tschipper
2015-11-28 14:48               ` [bitcoin-dev] further test results for : "Datastream Compression of Blocks and Tx's" Peter Tschipper
2015-11-29  0:30                 ` Jonathan Toomim
2015-11-29  5:15                   ` Peter Tschipper
     [not found]             ` <56421F1E.4050302@gmail.com>
2015-11-10 16:46               ` Peter Tschipper [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=56421F7E.8070305@gmail.com \
    --to=peter.tschipper@gmail.com \
    --cc=bitcoin-dev@lists.linuxfoundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox