From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from sog-mx-1.v43.ch3.sourceforge.com ([172.29.43.191] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1XCypm-0000J2-1U for bitcoin-development@lists.sourceforge.net; Thu, 31 Jul 2014 22:28:26 +0000 Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of gmail.com designates 209.85.214.177 as permitted sender) client-ip=209.85.214.177; envelope-from=keziahw@gmail.com; helo=mail-ob0-f177.google.com; Received: from mail-ob0-f177.google.com ([209.85.214.177]) by sog-mx-1.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1XCypk-00019V-Vz for bitcoin-development@lists.sourceforge.net; Thu, 31 Jul 2014 22:28:25 +0000 Received: by mail-ob0-f177.google.com with SMTP id wp18so1987802obc.22 for ; Thu, 31 Jul 2014 15:28:19 -0700 (PDT) X-Received: by 10.60.132.203 with SMTP id ow11mr1853093oeb.47.1406845699452; Thu, 31 Jul 2014 15:28:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.61.195 with HTTP; Thu, 31 Jul 2014 15:27:59 -0700 (PDT) In-Reply-To: References: From: Kaz Wesley Date: Thu, 31 Jul 2014 15:27:59 -0700 Message-ID: To: Gregory Maxwell Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -1.6 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (keziahw[at]gmail.com) -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1XCypk-00019V-Vz Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Squashing redundant tx data in blocks on the wire X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2014 22:28:26 -0000 > the FEC still lets you fill in the missing transactions without knowing i= n advance all that will be missing. I don't see why we need to solve that problem, since the protocol already involves exchanging the information necessary to determine (with some false positives) what a peer is missing, and needs to continue doing so regardless of how blocks are transmitted. Set reconciliation does have the benefit of eliminating a subset of those false positives and offering a finer-grained mechanism for defining what a node can choose to forget from its mempool than remember-last-N, but if we implement it for block transmission I don't see why we wouldn't also use it to synchronize mempool txes, and if mempools are synchronized we don't actually need to do it as part of block-transmission to get those benefits. As far as I can tell, channel memory sparseblocks achieve most of the possible bandwidth savings, and when FEC-based mempool synchronization is implemented its benefits can be applied to the sparseblocks by resetting the channel memory to the mutual mempool state each time mempool differences are exchanged. Am I missing a benefit to doing FEC at block forwarding time that can't be realized by FEC-based mempool synchronization, implemented separately from channel-memory based index-coding? On Thu, Jul 31, 2014 at 2:51 PM, Gregory Maxwell wrote= : > On Thu, Jul 31, 2014 at 2:41 PM, Kaz Wesley wrote: >>> the need to have transmitted the transaction list [..] first >> >> 32 bits per transaction is at least double the communication overhead >> of the simple approach, and only offers a bound on the probability of >> needing a round trip. > > "(e.g. from a reconciliation step first)" the list can be communicated > in the space roughly equal to the size of the difference in sets plus > coding the permutation from the permissible orderings. If you did > have some "simple approach" that guaranteed that some transactions > would be present, then you could code those with indexes... the FEC > still lets you fill in the missing transactions without knowing in > advance all that will be missing. (Also, the suggestion on the > network block coding page of using part of a cryptographic permutation > as the key means that for unknown transactions the transmission of the > new unknown keys is always goodput=E2=80=94 doesn't add overhead) > > It's "only a bound" but you can pick whatever bound you want, > including=E2=80=94 if you send data equal to the missing amount, then it'= ll be > always successful, but no bandwidth savings. Though if the transport > is unordered (e.g. UDP or non-blocking SCTP) even sending 100% of the > missing amount can save time by eliminating a round trip that might > otherwise be needed for a retransmission.