public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: Mike Hearn <mike@plan99.net>
To: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: [Bitcoin-development] Cartographer
Date: Sun, 28 Dec 2014 18:25:29 +0000	[thread overview]
Message-ID: <CANEZrP1tGYOsShk5Z91_JwH7E0pX8WcyT_-ZBkrWBqZAyrcd1A@mail.gmail.com> (raw)

[-- Attachment #1: Type: text/plain, Size: 2615 bytes --]

Hi there!

Lately we have been bumping up against the limitations of DNS as a protocol
for learning about the p2p network. As a proposal for how to address this,
I have written a new network crawler and seed:

https://github.com/mikehearn/httpseed

It implements a standard DNS seed with a minimal embedded DNS server (you
can find one running at dnsseed.vinumeris.com) and also has the following
extra features:


   - Can serve seed data using gzipped, timestamped digitally signed
   protocol buffers over HTTP. This fixes authentication, auditability,
   malware false positives and extensibility. The signature uses secp256k1.
   SSL is *not* used, to simplify deployment and to allow ISPs to cache the
   results transparently when a future version sets cache control headers.
   - Can additionally serve data in JSON, XML and HTML (examples for json
   <http://vinumeris.com:8081/peers.json> xml
   <http://vinumeris.com:8081/peers.xml> html
   <http://vinumeris.com:8081/peers.html>) for ease of use with other
   tools, like web browsers.
   - Results can be restricted using query parameters, e.g. for a service
   flags bit mask. Cartographer tests nodes that set service bit 2 to see if
   they really support BIP 64, and this requirement can also be specified as
   an argument to the query.
   - Crawl speed can be specified in terms of successful connects per
   second, rather than the number-of-threads approach used by other crawlers.
   - Can export statistics and controls using JMX, so you can reconfigure
   it at runtime and view charts of things like connects/sec or CPU usage
   using any JMX console, like Mission Control.
   - A client for it is in bitcoinj master branch.

To provide all these features Cartographer relies heavily on libraries and
is written in a concise new language called Kotlin <http://kotlinlang.org/>,
so it fits in about 650 lines of code. Kotlin is easy to learn for anyone
who knows Scala or Java, so it should be straightforward to hack on and
there is no chance of any buffer/heap exploits in the DNS, HTTP or Bitcoin
protocol stacks.

In the new year I will probably write a BIP describing the protocol. For
now you can see the definition here
<https://github.com/mikehearn/httpseed/blob/master/src/main/peerseeds.proto>
or just read the textual forms from the links above. It's pretty self
explanatory. I hope that in future other DNS seeds will start supporting
this protocol too, as it has many advantages.

Future versions might include data like how long the peer has been around,
node keys if we add auth/encrypt support to the p2p protocol and so on.

[-- Attachment #2: Type: text/html, Size: 3372 bytes --]

             reply	other threads:[~2014-12-29  7:30 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-12-28 18:25 Mike Hearn [this message]
2014-12-29  8:47 ` [Bitcoin-development] Cartographer Thomas Zander
2014-12-29 10:39   ` Peter Todd
2014-12-29 11:30   ` Mike Hearn
2014-12-29 11:49     ` Thomas Zander
2014-12-29 11:59       ` Peter Todd
2014-12-29 12:13     ` Btc Drak
2014-12-29 13:08       ` Mistr Bigs

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CANEZrP1tGYOsShk5Z91_JwH7E0pX8WcyT_-ZBkrWBqZAyrcd1A@mail.gmail.com \
    --to=mike@plan99.net \
    --cc=bitcoin-development@lists.sourceforge.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox