[Bitcoin-development] Bootstrapping via BitTorrent trackers

[Bitcoin-development] Bootstrapping via BitTorrent trackers

[Christian Decker]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

Hi all.

Just wanted to carry the discussion from the Forum over to the dev-list.

We have quite a few bootstrapping mechanisms, starting with the overly
complex (IMHO) IRC bootstrapping, which is often suspected as bot-activity.
Then we have a few hardcoded nodes and some fallback nodes. I was wondering
why we didn't adopt BitTorrent tracker bootstrapping until now? It's
basically all it does. Given a hash (SHA1 hash of the genesis bloc would be
nice ^^) it gives you a list of other nodes with the same hash.

Given that there are quite a few open trackers (accepting and tracking any
hash you throw at them) we could just decide to use 2-3 of those to
bootstrap.

The downside would be that they return bencoded data, which has to be
interpreted first, but it's easier than implementing the IRC stuff, I think.

Any comments?

Regards,
Chris

[-- Attachment #2: Type: text/html, Size: 913 bytes --]

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Jeff Garzik]

On Mon, Jun 13, 2011 at 4:55 AM, Christian Decker
<decker.christian@gmail.com> wrote:
> We have quite a few bootstrapping mechanisms, starting with the overly
> complex (IMHO) IRC bootstrapping, which is often suspected as bot-activity.
> Then we have a few hardcoded nodes and some fallback nodes. I was wondering
> why we didn't adopt BitTorrent tracker bootstrapping until now? It's
> basically all it does. Given a hash (SHA1 hash of the genesis bloc would be
> nice ^^) it gives you a list of other nodes with the same hash.

It seems to offer few benefits over DNS seeding, while potentially
potentially creating a vulnerable hot spot in the DHT.  Sybil attacks
on DHTs are well documented.

-- 
Jeff Garzik
exMULTI, Inc.
jgarzik@exmulti.com

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Christian Decker]

[-- Attachment #1: Type: text/plain, Size: 1593 bytes --]

Don't get me wrong, DNS Seeding is an excellent way to bootstrap via trusted
nodes, I'm not trying to replace it.
What I'm trying to get rid of is the IRC bootstrapping and the hardcoded
nodes in the client, they're easy targets.

BitTorrent trackers are used to handle several thousands of requests, so
they would probably scale well enough. I'm not even talking about using the
DHT trackers, but using old fashioned HTTP based trackers. The fact that
each bitcoin client would contact the tracker would make it very hard for an
attacker to get bootstrapping clients to exclusively connect to his
compromised clients. I would say that using a tracker such as OpenBittorrent
provides the same advantages as using an IRC channel.

On Mon, Jun 13, 2011 at 11:09 AM, Jeff Garzik <jgarzik@exmulti.com> wrote:

> On Mon, Jun 13, 2011 at 4:55 AM, Christian Decker
> <decker.christian@gmail.com> wrote:
> > We have quite a few bootstrapping mechanisms, starting with the overly
> > complex (IMHO) IRC bootstrapping, which is often suspected as
> bot-activity.
> > Then we have a few hardcoded nodes and some fallback nodes. I was
> wondering
> > why we didn't adopt BitTorrent tracker bootstrapping until now? It's
> > basically all it does. Given a hash (SHA1 hash of the genesis bloc would
> be
> > nice ^^) it gives you a list of other nodes with the same hash.
>
> It seems to offer few benefits over DNS seeding, while potentially
> potentially creating a vulnerable hot spot in the DHT.  Sybil attacks
> on DHTs are well documented.
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti.com
>

[-- Attachment #2: Type: text/html, Size: 2082 bytes --]

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Jeff Garzik]

On Mon, Jun 13, 2011 at 5:38 AM, Christian Decker
<decker.christian@gmail.com> wrote:
> BitTorrent trackers are used to handle several thousands of requests, so
> they would probably scale well enough. I'm not even talking about using the
> DHT trackers, but using old fashioned HTTP based trackers. The fact that
> each bitcoin client would contact the tracker would make it very hard for an
> attacker to get bootstrapping clients to exclusively connect to his
> compromised clients. I would say that using a tracker such as OpenBittorrent
> provides the same advantages as using an IRC channel.

And how does the client discover HTTP trackers?  You're either
hardcoding -those- into the client, or adding an additional bootstrap
step to discover them.  Either way, it has the same problems as other
current methods.

The history and experience of gnutella's web caches vs. UDP host
caches seems highly relevant here.

-- 
Jeff Garzik
exMULTI, Inc.
jgarzik@exmulti.com

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Vladimir Marchenko]

one possible bootstrap method of last resort,

1. create a convention of bitcoind listening on a specific last octest
of IPv4 address, let's say, .14 when possible. Those of us who have
access to IP space would use .14's.

2. if no other bootstrap method works, client could start scanning
x.x.x.14 addresses, perhaps in some semi-intelligent order (starting
from more pobable /8's and /16's), if enough people place bitcoind on
x.x.x.14 than after a 10-100 thousand checks it bound to find a
bitcoind peer.

It's messy, with all the excessive scanning etc... but it does not
depend on anything except a bunch of bitcoind by convention preferring
listening on x.x.x.14's.

Given that this is a method of last resort in bootrap chain it whould
hopefully not lead to DDOS on those unlucky to own *.14 and not
running bitcoind there. Also the more people are running bitcoind on
.14, the quicker it would find a peer, the less scanning to do. It is
kind of self-regualting.

For whatever it worth...


On 13 June 2011 10:56, Jeff Garzik <jgarzik@exmulti.com> wrote:
> On Mon, Jun 13, 2011 at 5:38 AM, Christian Decker
> <decker.christian@gmail.com> wrote:
>> BitTorrent trackers are used to handle several thousands of requests, so
>> they would probably scale well enough. I'm not even talking about using the
>> DHT trackers, but using old fashioned HTTP based trackers. The fact that
>> each bitcoin client would contact the tracker would make it very hard for an
>> attacker to get bootstrapping clients to exclusively connect to his
>> compromised clients. I would say that using a tracker such as OpenBittorrent
>> provides the same advantages as using an IRC channel.
>
> And how does the client discover HTTP trackers?  You're either
> hardcoding -those- into the client, or adding an additional bootstrap
> step to discover them.  Either way, it has the same problems as other
> current methods.
>
> The history and experience of gnutella's web caches vs. UDP host
> caches seems highly relevant here.
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti.com
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Christian Decker]

[-- Attachment #1: Type: text/plain, Size: 4245 bytes --]

Yes, those trackers would be hard coded, just like the IRC servers and
channels are hardcoded right now.

The advantages over IRC and DNS Seeds are:
 - sporadic HTTP requests to a tracker, as opposed to keeping an IRC
connection open at all times
 - no virus/botnet like behaviour (automatically join IRC channel with
cryptic name), ISPs tend to bother network admins (like myself) with alerts
when they see this...
 - adapts faster than DNS Seeds which require configuration changes on seed
should the nodes become unreachable
 - we already use HTTP to determine our external IP, so it would be a
consolidation of transports
 - more peers than DNS Seeds (better load balancing)

As for Vladimirs proposal, seems like an extreme measure, that is not really
practical. Also it leads to network partitions since nodes will prefer their
own /8 and /16 networks. IPv6 will also soon be a problem for this method.

On Mon, Jun 13, 2011 at 12:54 PM, Vladimir Marchenko <
vladimir@marchenko.co.uk> wrote:

> one possible bootstrap method of last resort,
>
> 1. create a convention of bitcoind listening on a specific last octest
> of IPv4 address, let's say, .14 when possible. Those of us who have
> access to IP space would use .14's.
>
> 2. if no other bootstrap method works, client could start scanning
> x.x.x.14 addresses, perhaps in some semi-intelligent order (starting
> from more pobable /8's and /16's), if enough people place bitcoind on
> x.x.x.14 than after a 10-100 thousand checks it bound to find a
> bitcoind peer.
>
> It's messy, with all the excessive scanning etc... but it does not
> depend on anything except a bunch of bitcoind by convention preferring
> listening on x.x.x.14's.
>
> Given that this is a method of last resort in bootrap chain it whould
> hopefully not lead to DDOS on those unlucky to own *.14 and not
> running bitcoind there. Also the more people are running bitcoind on
> .14, the quicker it would find a peer, the less scanning to do. It is
> kind of self-regualting.
>
> For whatever it worth...
>
>
> On 13 June 2011 10:56, Jeff Garzik <jgarzik@exmulti.com> wrote:
> > On Mon, Jun 13, 2011 at 5:38 AM, Christian Decker
> > <decker.christian@gmail.com> wrote:
> >> BitTorrent trackers are used to handle several thousands of requests, so
> >> they would probably scale well enough. I'm not even talking about using
> the
> >> DHT trackers, but using old fashioned HTTP based trackers. The fact that
> >> each bitcoin client would contact the tracker would make it very hard
> for an
> >> attacker to get bootstrapping clients to exclusively connect to his
> >> compromised clients. I would say that using a tracker such as
> OpenBittorrent
> >> provides the same advantages as using an IRC channel.
> >
> > And how does the client discover HTTP trackers?  You're either
> > hardcoding -those- into the client, or adding an additional bootstrap
> > step to discover them.  Either way, it has the same problems as other
> > current methods.
> >
> > The history and experience of gnutella's web caches vs. UDP host
> > caches seems highly relevant here.
> >
> > --
> > Jeff Garzik
> > exMULTI, Inc.
> > jgarzik@exmulti.com
> >
> >
> ------------------------------------------------------------------------------
> > EditLive Enterprise is the world's most technically advanced content
> > authoring tool. Experience the power of Track Changes, Inline Image
> > Editing and ensure content is compliant with Accessibility Checking.
> > http://p.sf.net/sfu/ephox-dev2dev
> > _______________________________________________
> > Bitcoin-development mailing list
> > Bitcoin-development@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> >
>
>
> ------------------------------------------------------------------------------
> EditLive Enterprise is the world's most technically advanced content
> authoring tool. Experience the power of Track Changes, Inline Image
> Editing and ensure content is compliant with Accessibility Checking.
> http://p.sf.net/sfu/ephox-dev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>

[-- Attachment #2: Type: text/html, Size: 5522 bytes --]

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Jeff Garzik]

On Mon, Jun 13, 2011 at 7:48 AM, Christian Decker
<decker.christian@gmail.com> wrote:
> The advantages over IRC and DNS Seeds are:
>  - sporadic HTTP requests to a tracker, as opposed to keeping an IRC
> connection open at all times

No advantage over DNS (you're talking about IRC)

>  - no virus/botnet like behaviour (automatically join IRC channel with
> cryptic name), ISPs tend to bother network admins (like myself) with alerts
> when they see this...

No advantage over DNS (you're talking about IRC)

>  - adapts faster than DNS Seeds which require configuration changes on seed
> should the nodes become unreachable

False.  A proper DNS seed actively samples the bitcoin network for live nodes.

>  - we already use HTTP to determine our external IP, so it would be a
> consolidation of transports

Irrelevant -- it's still a bunch of additional code.  We "already use"
DNS lookups as well, so I could apply similar logic to DNS seeding.

>  - more peers than DNS Seeds (better load balancing)

This is an invalid assumption, based on an overly simplistic sampling.
 DNS seeds are perfectly capable of returning new and useful peers on
each query.

-- 
Jeff Garzik
exMULTI, Inc.
jgarzik@exmulti.com

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Vladimir Marchenko]

Jeff is absolutely correct, stating that DNS bootstrapping can
potentially be very robust.

Consider, for example, that seeds can be hosted by services like
zerigo, who provide decent management API's, as well as by thousands
of small VPS operated by people all over the world. Moreover, if
namecoin lives up to it's promises than some seeds can be hosted using
it's tlds, and as such we would get around centralised ICANN/registrar
weakness.

The most practical way I see how to improve DNS bootstrapping is
publishing build instructions for a simple DNS server build and a
script which feeds it with node data. Something simple like djbdns plus
a perl script, or similar zerigo API script, would do. Than anyone can
host it. All left to do than is to hardcode a bunch of such DNS seeds
into the client.

Hardcoding seeds is not good enough? Get a convention that anyone
owning a bitcoin.* domain should point seed.bitcoin.* to a decent DNS
seed. Than clients simply scan 100-200 tld's to find working
seed.bitcoin.*

I am not quite sure, though, is if this would enable some attacks by
poisoned seeds.

Vladimir.

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Gavin]

> The most practical way I see how to improve DNS bootstrapping is
> publishing build instructions for a simple DNS server build and a
> script which feeds it with node data. Something simple like djbdns plus
> a perl script, or similar zerigo API script, would do. Than anyone can
> host it. All left to do than is to hardcode a bunch of such DNS seeds
> into the client.

Adding that code to bitcoin in a contrib/dns folder seems like a good idea.  I'd be happy to run a dns seed.

Re: [Bitcoin-development] Bootstrapping via BitTorrent trackers

[Jeff Garzik]

On Mon, Jun 13, 2011 at 2:41 PM, Gavin <gavinandresen@gmail.com> wrote:
>> The most practical way I see how to improve DNS bootstrapping is
>> publishing build instructions for a simple DNS server build and a
>> script which feeds it with node data. Something simple like djbdns plus
>> a perl script, or similar zerigo API script, would do. Than anyone can
>> host it. All left to do than is to hardcode a bunch of such DNS seeds
>> into the client.
>
> Adding that code to bitcoin in a contrib/dns folder seems like a good idea.

Agreed.  I'll see if I can put together a couple different solutions,
and stick them in there.  ETA 3-4 weeks though, given current bugfix
workload + personal issues (moving to a new house).

I had a thought related to this, but readily admit I haven't thought
this through:  create an RPC in bitcoin that returns a random
selection of fresh P2P node addresses, from addr.dat.  One problem I
was running into was accessing this data from outside bitcoin.

Using ArtForz' half-a-node as a base was a possibility, but creating
"getnodes" RPC seemed far easier.

djbdns (tinydns) is great for this:  it is highly scalable and stable,
and its input is a -very- simple text format, easier to generate than
BIND zone files.

> I'd be happy to run a dns seed.

Great!  A big missing piece is simply trusted people running DNS seeds
on stable connectivity, once the DNS software is there.

-- 
Jeff Garzik
exMULTI, Inc.
jgarzik@exmulti.com