[Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Matt Corallo]

Recently, there has been a reasonable amount of discussion about the
continued fragility of the public Bitcoin network on IRC and elsewhere
(1). To this extent, I'm organizing a system of peering between nodes in
the network by creating a system of high-speed relay nodes for miners
and merchants/exchanges. This system will a) act as a fallback in the
case that the public Bitcoin network encounters issues and b) decrease
block propagation times between miners.
It is NOT designed to in any way replace or decrease the need for the
public Bitcoin P2P network. It is NOT any kind of attempt at
centralization, and I still encourage interested parties to establish
their own private peering agreements with large miners as needed.

Currently the network consists of one specially-designed relay node, but
I hope to bring more online in the coming days.

This network is open to everyone via a few public relay nodes, but also
will have nodes which are made available only to large miners and
merchants/exchanges to mitigate the ability of malicious parties to DoS
the network.

To peer with the public relay nodes, simply select the closest region
out of us-west (West Coast US), us-east (East Coast US), eu (Western
Europe), au (Australia), or jpy (Japan) and add
public.REGION.relay.mattcorallo.com to your addnode list. Note that
since all of the relay nodes will relay between each other, you gain no
latency advantage by peering with more than the closest node to you (and
currently all the regions map to one node, so there they're redundant
anyway).

For each relay node, you can connect to either port 8334 or 8335.
Connecting on port 8334 will relay only blocks, and port 8335 will relay
both blocks and transactions. The relay nodes will request any
transactions which appear in your invs no matter which port you connect to.

Relay node details:
 * The relay nodes do some data verification to prevent DoS, but in
order to keep relay fast, they do not fully verify the data they are
relaying, thus YOU SHOULD NEVER mine a block building on top of a
relayed block without fully checking it with your own bitcoin validator
(as you would any other block relayed from the P2P network).
 * The relay nodes do not follow the standard inv-getdata-tx/block flow,
but instead relay transactions/blocks immediately after they have done
their cursory verification. They do keep some track of whether or not
your nodes claim to have seen the transactions/blocks before relaying,
but you may see transactions/blocks being sent which you already have
and have not requested, if this is a problem for you due to bandwith
issues, you should reconsider your bandwith constraints and/or are
peering with too many nodes.
 * The relay nodes will all relay among themselves very quickly, so
there is no advantage to peering with as many relay nodes as you can
find, in fact, the increased incoming bandwidth during block relay
spikes may result in higher latency for your nodes.
 * The relay nodes are NOT designed to ensure that you never miss data,
and may fail to relay some transactions. Additionally, because the relay
nodes do not respond to standard getdata requests, if you miss a relay
and then reconnect, that data will not be sent again by the relay nodes.
The relay nodes are NOT a replacement for having peers on the standard
P2P network, they are only there to augment the existing P2P network.

If you are a merchant/exchange/large miner/other important node operator
and wish to gain access to additional domain names which map to relay
nodes with fewer peers, please fill out the form at
https://docs.google.com/forms/d/1UL82QdcXXEhZwSHJAK04Sk_cWg4zLOu8a216nO7Mt8c/viewform

You can find the source for the relay nodes at
https://github.com/TheBlueMatt/RelayNode

If you have any comments/concerns/suggestions, please do not hesitate to
email bitcoin-peering@mattcorallo.com

Thanks,
Matt


(1) There has been extended discussion on #bitcoin-wizards as well as
#bitcoin-dev of the very small number of active, listening nodes.
Additionally, because many of those nodes are versions prior to 0.8.4,
it seems very likely that maliciously creating network splits or at
least drastically reducing the number of peers for most nodes would not
be particularly challenging in the current network. Also,
http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf
noted that they were able to single-handledly decrease the network-wide
orphan rate by around 50% by improving network peering. Finally, you've
all seen the recent discussion on malicious mining algorithms. Though
those are not entirely prevented by reducing block propagation times,
they can be significantly limited compared to the current, rather
disjoint, network.

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 6002 bytes --]

Very cool, thanks Matt.

I was actually thinking this morning, maybe we should require all nodes to
go through the inv/getdata dance. Otherwise it's possible to improve your
chances at racing a block by mining a block, waiting to see a block inv
from another node, then blasting out your block while other nodes are still
waiting on their getdatas.


On Wed, Nov 6, 2013 at 6:50 AM, Matt Corallo <bitcoin-list@bluematt.me>wrote:

> Recently, there has been a reasonable amount of discussion about the
> continued fragility of the public Bitcoin network on IRC and elsewhere
> (1). To this extent, I'm organizing a system of peering between nodes in
> the network by creating a system of high-speed relay nodes for miners
> and merchants/exchanges. This system will a) act as a fallback in the
> case that the public Bitcoin network encounters issues and b) decrease
> block propagation times between miners.
> It is NOT designed to in any way replace or decrease the need for the
> public Bitcoin P2P network. It is NOT any kind of attempt at
> centralization, and I still encourage interested parties to establish
> their own private peering agreements with large miners as needed.
>
> Currently the network consists of one specially-designed relay node, but
> I hope to bring more online in the coming days.
>
> This network is open to everyone via a few public relay nodes, but also
> will have nodes which are made available only to large miners and
> merchants/exchanges to mitigate the ability of malicious parties to DoS
> the network.
>
> To peer with the public relay nodes, simply select the closest region
> out of us-west (West Coast US), us-east (East Coast US), eu (Western
> Europe), au (Australia), or jpy (Japan) and add
> public.REGION.relay.mattcorallo.com to your addnode list. Note that
> since all of the relay nodes will relay between each other, you gain no
> latency advantage by peering with more than the closest node to you (and
> currently all the regions map to one node, so there they're redundant
> anyway).
>
> For each relay node, you can connect to either port 8334 or 8335.
> Connecting on port 8334 will relay only blocks, and port 8335 will relay
> both blocks and transactions. The relay nodes will request any
> transactions which appear in your invs no matter which port you connect to.
>
> Relay node details:
>  * The relay nodes do some data verification to prevent DoS, but in
> order to keep relay fast, they do not fully verify the data they are
> relaying, thus YOU SHOULD NEVER mine a block building on top of a
> relayed block without fully checking it with your own bitcoin validator
> (as you would any other block relayed from the P2P network).
>  * The relay nodes do not follow the standard inv-getdata-tx/block flow,
> but instead relay transactions/blocks immediately after they have done
> their cursory verification. They do keep some track of whether or not
> your nodes claim to have seen the transactions/blocks before relaying,
> but you may see transactions/blocks being sent which you already have
> and have not requested, if this is a problem for you due to bandwith
> issues, you should reconsider your bandwith constraints and/or are
> peering with too many nodes.
>  * The relay nodes will all relay among themselves very quickly, so
> there is no advantage to peering with as many relay nodes as you can
> find, in fact, the increased incoming bandwidth during block relay
> spikes may result in higher latency for your nodes.
>  * The relay nodes are NOT designed to ensure that you never miss data,
> and may fail to relay some transactions. Additionally, because the relay
> nodes do not respond to standard getdata requests, if you miss a relay
> and then reconnect, that data will not be sent again by the relay nodes.
> The relay nodes are NOT a replacement for having peers on the standard
> P2P network, they are only there to augment the existing P2P network.
>
> If you are a merchant/exchange/large miner/other important node operator
> and wish to gain access to additional domain names which map to relay
> nodes with fewer peers, please fill out the form at
>
> https://docs.google.com/forms/d/1UL82QdcXXEhZwSHJAK04Sk_cWg4zLOu8a216nO7Mt8c/viewform
>
> You can find the source for the relay nodes at
> https://github.com/TheBlueMatt/RelayNode
>
> If you have any comments/concerns/suggestions, please do not hesitate to
> email bitcoin-peering@mattcorallo.com
>
> Thanks,
> Matt
>
>
> (1) There has been extended discussion on #bitcoin-wizards as well as
> #bitcoin-dev of the very small number of active, listening nodes.
> Additionally, because many of those nodes are versions prior to 0.8.4,
> it seems very likely that maliciously creating network splits or at
> least drastically reducing the number of peers for most nodes would not
> be particularly challenging in the current network. Also,
>
> http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf
> noted that they were able to single-handledly decrease the network-wide
> orphan rate by around 50% by improving network peering. Finally, you've
> all seen the recent discussion on malicious mining algorithms. Though
> those are not entirely prevented by reducing block propagation times,
> they can be significantly limited compared to the current, rather
> disjoint, network.
>
>
> ------------------------------------------------------------------------------
> November Webinars for C, C++, Fortran Developers
> Accelerate application performance with scalable programming models.
> Explore
> techniques for threading, error checking, porting, and tuning. Get the most
> from the latest Intel processors and coprocessors. See abstracts and
> register
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 7342 bytes --]

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Jeff Garzik]

[-- Attachment #1: Type: text/plain, Size: 5973 bytes --]

Good stuff.  I have been pushing for private peering agreeents and a
"backbone" for years. Even had a paltry effort going with exmulti.net + a
few manually connected parties.

I hope parties in the bitcoin space take it upon themselves to network with
major sites - miners, payment processors, exchanges, popular sites, etc.
 On Nov 6, 2013 12:50 AM, "Matt Corallo" <bitcoin-list@bluematt.me> wrote:

> Recently, there has been a reasonable amount of discussion about the
> continued fragility of the public Bitcoin network on IRC and elsewhere
> (1). To this extent, I'm organizing a system of peering between nodes in
> the network by creating a system of high-speed relay nodes for miners
> and merchants/exchanges. This system will a) act as a fallback in the
> case that the public Bitcoin network encounters issues and b) decrease
> block propagation times between miners.
> It is NOT designed to in any way replace or decrease the need for the
> public Bitcoin P2P network. It is NOT any kind of attempt at
> centralization, and I still encourage interested parties to establish
> their own private peering agreements with large miners as needed.
>
> Currently the network consists of one specially-designed relay node, but
> I hope to bring more online in the coming days.
>
> This network is open to everyone via a few public relay nodes, but also
> will have nodes which are made available only to large miners and
> merchants/exchanges to mitigate the ability of malicious parties to DoS
> the network.
>
> To peer with the public relay nodes, simply select the closest region
> out of us-west (West Coast US), us-east (East Coast US), eu (Western
> Europe), au (Australia), or jpy (Japan) and add
> public.REGION.relay.mattcorallo.com to your addnode list. Note that
> since all of the relay nodes will relay between each other, you gain no
> latency advantage by peering with more than the closest node to you (and
> currently all the regions map to one node, so there they're redundant
> anyway).
>
> For each relay node, you can connect to either port 8334 or 8335.
> Connecting on port 8334 will relay only blocks, and port 8335 will relay
> both blocks and transactions. The relay nodes will request any
> transactions which appear in your invs no matter which port you connect to.
>
> Relay node details:
>  * The relay nodes do some data verification to prevent DoS, but in
> order to keep relay fast, they do not fully verify the data they are
> relaying, thus YOU SHOULD NEVER mine a block building on top of a
> relayed block without fully checking it with your own bitcoin validator
> (as you would any other block relayed from the P2P network).
>  * The relay nodes do not follow the standard inv-getdata-tx/block flow,
> but instead relay transactions/blocks immediately after they have done
> their cursory verification. They do keep some track of whether or not
> your nodes claim to have seen the transactions/blocks before relaying,
> but you may see transactions/blocks being sent which you already have
> and have not requested, if this is a problem for you due to bandwith
> issues, you should reconsider your bandwith constraints and/or are
> peering with too many nodes.
>  * The relay nodes will all relay among themselves very quickly, so
> there is no advantage to peering with as many relay nodes as you can
> find, in fact, the increased incoming bandwidth during block relay
> spikes may result in higher latency for your nodes.
>  * The relay nodes are NOT designed to ensure that you never miss data,
> and may fail to relay some transactions. Additionally, because the relay
> nodes do not respond to standard getdata requests, if you miss a relay
> and then reconnect, that data will not be sent again by the relay nodes.
> The relay nodes are NOT a replacement for having peers on the standard
> P2P network, they are only there to augment the existing P2P network.
>
> If you are a merchant/exchange/large miner/other important node operator
> and wish to gain access to additional domain names which map to relay
> nodes with fewer peers, please fill out the form at
>
> https://docs.google.com/forms/d/1UL82QdcXXEhZwSHJAK04Sk_cWg4zLOu8a216nO7Mt8c/viewform
>
> You can find the source for the relay nodes at
> https://github.com/TheBlueMatt/RelayNode
>
> If you have any comments/concerns/suggestions, please do not hesitate to
> email bitcoin-peering@mattcorallo.com
>
> Thanks,
> Matt
>
>
> (1) There has been extended discussion on #bitcoin-wizards as well as
> #bitcoin-dev of the very small number of active, listening nodes.
> Additionally, because many of those nodes are versions prior to 0.8.4,
> it seems very likely that maliciously creating network splits or at
> least drastically reducing the number of peers for most nodes would not
> be particularly challenging in the current network. Also,
>
> http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf
> noted that they were able to single-handledly decrease the network-wide
> orphan rate by around 50% by improving network peering. Finally, you've
> all seen the recent discussion on malicious mining algorithms. Though
> those are not entirely prevented by reducing block propagation times,
> they can be significantly limited compared to the current, rather
> disjoint, network.
>
>
> ------------------------------------------------------------------------------
> November Webinars for C, C++, Fortran Developers
> Accelerate application performance with scalable programming models.
> Explore
> techniques for threading, error checking, porting, and tuning. Get the most
> from the latest Intel processors and coprocessors. See abstracts and
> register
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 7273 bytes --]

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Tier Nolan]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

On Wed, Nov 6, 2013 at 5:50 AM, Matt Corallo <bitcoin-list@bluematt.me>wrote:

> Relay node details:
>  * The relay nodes do some data verification to prevent DoS, but in
> order to keep relay fast, they do not fully verify the data they are
> relaying, thus YOU SHOULD NEVER mine a block building on top of a
> relayed block without fully checking it with your own bitcoin validator
> (as you would any other block relayed from the P2P network).
>

Wouldn't this cause disconnects due to misbehavior?

A standard node connecting to a relay node would receive
blocks/transactions that are not valid in some way and then disconnect.

Have you looked though the official client to find what things are
considered signs that a peer is hostile?  I assume things like double
spending checks count as misbehavior and can't be quickly checked by a
relay node.

Maybe another bit could be assigned in the services field as "relay".  This
means that the node doesn't do any checking.

Connects to relay nodes could be command line/config file only.  Peers
wouldn't connect to them.

[-- Attachment #2: Type: text/html, Size: 1518 bytes --]

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Matt Corallo]

No, the transactions relayed are piped through a bitcoind first (ie
fully verified by a bitcoind). For blocks, for which the timing needs to
be tighter, bitcoinj does SPV-validation. Though it is possible to
create a block which passes SPV validation but causes a DoS score, doing
so would cost a miner a full block's worth of profits, which they are
fairly unlikely to do. In any case, if it every becomes a problem, its
not hard to adapt addnode to allow higher DoS scores for individual nodes.

Matt

On 11/06/13 07:25, Tier Nolan wrote:
> 
> 
> 
> On Wed, Nov 6, 2013 at 5:50 AM, Matt Corallo <bitcoin-list@bluematt.me
> <mailto:bitcoin-list@bluematt.me>> wrote:
> 
>     Relay node details:
>      * The relay nodes do some data verification to prevent DoS, but in
>     order to keep relay fast, they do not fully verify the data they are
>     relaying, thus YOU SHOULD NEVER mine a block building on top of a
>     relayed block without fully checking it with your own bitcoin validator
>     (as you would any other block relayed from the P2P network).
> 
> 
> Wouldn't this cause disconnects due to misbehavior? 
> 
> A standard node connecting to a relay node would receive
> blocks/transactions that are not valid in some way and then disconnect.
> 
> Have you looked though the official client to find what things are
> considered signs that a peer is hostile?  I assume things like double
> spending checks count as misbehavior and can't be quickly checked by a
> relay node.
> 
> Maybe another bit could be assigned in the services field as "relay". 
> This means that the node doesn't do any checking. 
> 
> Connects to relay nodes could be command line/config file only.  Peers
> wouldn't connect to them.

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Mike Hearn]

[-- Attachment #1: Type: text/plain, Size: 3155 bytes --]

I took a brief look at the code - it's looking very reasonable. You can
replace any construct like

try {
  Thread.sleep(1000);
} catch (InterruptedException e) {
  throw new RuntimeException(e);
}

which is quite verbose, just with
Uninterruptibles.sleepUninterruptably(1000, TimeUnit.MILLISECONDS); (and of
course static imports help too)

I think for this concept to take off, you'd need a website and to recruit
someone to help you market it. Pool operators won't reach out to you.

I still find it perhaps more elegant to just boost the connectivity of the
existing network with bitcoind changes, but this can help for now.



On Thu, Nov 7, 2013 at 12:35 AM, Matt Corallo <bitcoin-list@bluematt.me>wrote:

> No, the transactions relayed are piped through a bitcoind first (ie
> fully verified by a bitcoind). For blocks, for which the timing needs to
> be tighter, bitcoinj does SPV-validation. Though it is possible to
> create a block which passes SPV validation but causes a DoS score, doing
> so would cost a miner a full block's worth of profits, which they are
> fairly unlikely to do. In any case, if it every becomes a problem, its
> not hard to adapt addnode to allow higher DoS scores for individual nodes.
>
> Matt
>
> On 11/06/13 07:25, Tier Nolan wrote:
> >
> >
> >
> > On Wed, Nov 6, 2013 at 5:50 AM, Matt Corallo <bitcoin-list@bluematt.me
> > <mailto:bitcoin-list@bluematt.me>> wrote:
> >
> >     Relay node details:
> >      * The relay nodes do some data verification to prevent DoS, but in
> >     order to keep relay fast, they do not fully verify the data they are
> >     relaying, thus YOU SHOULD NEVER mine a block building on top of a
> >     relayed block without fully checking it with your own bitcoin
> validator
> >     (as you would any other block relayed from the P2P network).
> >
> >
> > Wouldn't this cause disconnects due to misbehavior?
> >
> > A standard node connecting to a relay node would receive
> > blocks/transactions that are not valid in some way and then disconnect.
> >
> > Have you looked though the official client to find what things are
> > considered signs that a peer is hostile?  I assume things like double
> > spending checks count as misbehavior and can't be quickly checked by a
> > relay node.
> >
> > Maybe another bit could be assigned in the services field as "relay".
> > This means that the node doesn't do any checking.
> >
> > Connects to relay nodes could be command line/config file only.  Peers
> > wouldn't connect to them.
>
>
> ------------------------------------------------------------------------------
> November Webinars for C, C++, Fortran Developers
> Accelerate application performance with scalable programming models.
> Explore
> techniques for threading, error checking, porting, and tuning. Get the most
> from the latest Intel processors and coprocessors. See abstracts and
> register
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 4466 bytes --]

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[John Dillon]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

> To peer with the public relay nodes, simply select the closest region
> out of us-west (West Coast US), us-east (East Coast US), eu (Western
> Europe), au (Australia), or jpy (Japan) and add
> public.REGION.relay.mattcorallo.com to your addnode list. Note that
> since all of the relay nodes will relay between each other, you gain no
> latency advantage by peering with more than the closest node to you (and
> currently all the regions map to one node, so there they're redundant
> anyway).
>
> For each relay node, you can connect to either port 8334 or 8335.
> Connecting on port 8334 will relay only blocks, and port 8335 will relay
> both blocks and transactions. The relay nodes will request any
> transactions which appear in your invs no matter which port you connect to.

You should split the block-only and block+tx not only by port number, but also
by DNS address. DoS attack by flooding blocks is fundamentally more difficult
than DoS attack by flooding transctions, so doing the split by IP address
ensures that in the event of an attack the more important block relaying
functionality is less likely to be damaged. In the meantime point both DNS
addresses to the same IP until it becomes an issue.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBCAAGBQJSg91YAAoJEEWCsU4mNhiPq2oH/03kVqfHsXJ1l10qHhYaBPMy
Le26Cp30Jt9BELiPVQISWjPeuOLsB0m7Say52GWHxBCfoNx3NYag6p8G3woSdWqv
guc5U2lTwfhXS5R7y0B5diaGJ+Jaq70me4DYGdEnmkBf0F38wcgOtK92V2esLyVx
TmCsRGxjAE8Ary0YHJOlb7sU4CNvQ8k1PDX6Hd+GCZVMvRtisILunGV4UDgSS62u
yddZfrOs0yWZr2bwwI4koB2Sc0cFjK6/gMhr/d19ikQj2i2uqxYtwZIxuaAvYNdA
hSmeouR4EFtVHTEQybF82VcfGcTcU11HncyKHU6FOAZQLZUgc3A/M3QgXc0mQrI=
=l6GI
-----END PGP SIGNATURE-----

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Matt Corallo]

On 11/08/13 06:46, Mike Hearn wrote:
> I took a brief look at the code - it's looking very reasonable. You can
> replace any construct like
> 
> try {
>   Thread.sleep(1000);
> } catch (InterruptedException e) {
>   throw new RuntimeException(e);
> }
> 
> which is quite verbose, just with
> Uninterruptibles.sleepUninterruptably(1000, TimeUnit.MILLISECONDS); (and
> of course static imports help too)

Thanks, fixed.


> 
> I think for this concept to take off, you'd need a website and to
> recruit someone to help you market it. Pool operators won't reach out to
> you.

Yes, I've done some initial outreach and plan on doing another major
round now that the initial network is up and Im working on running some
relay time benchmarks. Finding someone to help push peering would be
nice, if you have any suggestions, Im all ears.

> 
> I still find it perhaps more elegant to just boost the connectivity of
> the existing network with bitcoind changes, but this can help for now.

Agreed, improving relay times across the regular P2P network would be
nice, however I really dont see this as a part of the P2P network. Its
more of a backup relay network that just happens to follow the P2P
protocol (mostly, it doesnt do full block verification, so technically
it breaks spec). In this model, this is really a nice augment to the P2P
network no matter what improvements are made. Having more protocols/ways
blocks are relayed is always nice (anyone wanna launch a satellite?)

Matt

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Matt Corallo]

In the short-term, maybe. Keep in mind that the code for tx relay is
fairly different and the bandwidth for transaction relay on these
nodes is already lower than it is for blocks (by design). That said,
I'd like to look into doing tx-less block relays for transactions that
peers already have to limit block relay times even for large blocks,
in which case tx relay is very much required.

Matt

On 11/13/13 15:13, John Dillon wrote:
> You should split the block-only and block+tx not only by port
> number, but also by DNS address. DoS attack by flooding blocks is
> fundamentally more difficult than DoS attack by flooding
> transctions, so doing the split by IP address ensures that in the
> event of an attack the more important block relaying functionality
> is less likely to be damaged. In the meantime point both DNS 
> addresses to the same IP until it becomes an issue.
> 
> 

Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network

[Matt Corallo]

For those who have been using this to get faster relays to/from the
network, you may have noticed some instability recently. This is because
the nodes were all being upgraded to use some new relaying code which
should cut down on duplicate transaction relaying in blocks, improving
relay speed within the network and to nodes which run new clients which
use the same relaying technique. Essentially instead of relaying entire
blocks, nodes keep a rolling window of recently-seen transactions and
skip those when relaying blocks.

You can find a simple client which connects to a local bitcoind and a
relay node at http://bitcoin.ninja/RelayNodeClient.jar and the source
for the whole thing at https://github.com/TheBlueMatt/RelayNode.

Matt

On 11/06/13 05:50, Matt Corallo wrote:
> Recently, there has been a reasonable amount of discussion about the
> continued fragility of the public Bitcoin network on IRC and elsewhere
> (1). To this extent, I'm organizing a system of peering between nodes in
> the network by creating a system of high-speed relay nodes for miners
> and merchants/exchanges. This system will a) act as a fallback in the
> case that the public Bitcoin network encounters issues and b) decrease
> block propagation times between miners.
> It is NOT designed to in any way replace or decrease the need for the
> public Bitcoin P2P network. It is NOT any kind of attempt at
> centralization, and I still encourage interested parties to establish
> their own private peering agreements with large miners as needed.
> 
> Currently the network consists of one specially-designed relay node, but
> I hope to bring more online in the coming days.
> 
> This network is open to everyone via a few public relay nodes, but also
> will have nodes which are made available only to large miners and
> merchants/exchanges to mitigate the ability of malicious parties to DoS
> the network.
> 
> To peer with the public relay nodes, simply select the closest region
> out of us-west (West Coast US), us-east (East Coast US), eu (Western
> Europe), au (Australia), or jpy (Japan) and add
> public.REGION.relay.mattcorallo.com to your addnode list. Note that
> since all of the relay nodes will relay between each other, you gain no
> latency advantage by peering with more than the closest node to you (and
> currently all the regions map to one node, so there they're redundant
> anyway).
> 
> For each relay node, you can connect to either port 8334 or 8335.
> Connecting on port 8334 will relay only blocks, and port 8335 will relay
> both blocks and transactions. The relay nodes will request any
> transactions which appear in your invs no matter which port you connect to.
> 
> Relay node details:
>  * The relay nodes do some data verification to prevent DoS, but in
> order to keep relay fast, they do not fully verify the data they are
> relaying, thus YOU SHOULD NEVER mine a block building on top of a
> relayed block without fully checking it with your own bitcoin validator
> (as you would any other block relayed from the P2P network).
>  * The relay nodes do not follow the standard inv-getdata-tx/block flow,
> but instead relay transactions/blocks immediately after they have done
> their cursory verification. They do keep some track of whether or not
> your nodes claim to have seen the transactions/blocks before relaying,
> but you may see transactions/blocks being sent which you already have
> and have not requested, if this is a problem for you due to bandwith
> issues, you should reconsider your bandwith constraints and/or are
> peering with too many nodes.
>  * The relay nodes will all relay among themselves very quickly, so
> there is no advantage to peering with as many relay nodes as you can
> find, in fact, the increased incoming bandwidth during block relay
> spikes may result in higher latency for your nodes.
>  * The relay nodes are NOT designed to ensure that you never miss data,
> and may fail to relay some transactions. Additionally, because the relay
> nodes do not respond to standard getdata requests, if you miss a relay
> and then reconnect, that data will not be sent again by the relay nodes.
> The relay nodes are NOT a replacement for having peers on the standard
> P2P network, they are only there to augment the existing P2P network.
> 
> If you are a merchant/exchange/large miner/other important node operator
> and wish to gain access to additional domain names which map to relay
> nodes with fewer peers, please fill out the form at
> https://docs.google.com/forms/d/1UL82QdcXXEhZwSHJAK04Sk_cWg4zLOu8a216nO7Mt8c/viewform
> 
> You can find the source for the relay nodes at
> https://github.com/TheBlueMatt/RelayNode
> 
> If you have any comments/concerns/suggestions, please do not hesitate to
> email bitcoin-peering@mattcorallo.com
> 
> Thanks,
> Matt
> 
> 
> (1) There has been extended discussion on #bitcoin-wizards as well as
> #bitcoin-dev of the very small number of active, listening nodes.
> Additionally, because many of those nodes are versions prior to 0.8.4,
> it seems very likely that maliciously creating network splits or at
> least drastically reducing the number of peers for most nodes would not
> be particularly challenging in the current network. Also,
> http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf
> noted that they were able to single-handledly decrease the network-wide
> orphan rate by around 50% by improving network peering. Finally, you've
> all seen the recent discussion on malicious mining algorithms. Though
> those are not entirely prevented by reducing block propagation times,
> they can be significantly limited compared to the current, rather
> disjoint, network.
> 
> ------------------------------------------------------------------------------
> November Webinars for C, C++, Fortran Developers
> Accelerate application performance with scalable programming models. Explore
> techniques for threading, error checking, porting, and tuning. Get the most 
> from the latest Intel processors and coprocessors. See abstracts and register
> http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>