Re: [Bitcoin-development] Introducing BitcoinKit.framework

[Mike Hearn <mike@plan99.net>]

[-- Attachment #1: Type: text/plain, Size: 3181 bytes --]

Let's re-add the list as this is a topic of general interest.

On Tue, Jul 16, 2013 at 11:36 AM, Jonas Schnelli <jonas.schnelli@include7.ch
> wrote:

> What do you think about extending the BitcoinKit.framework (based on
> bitcoind) so that the kit could handle the very basic SPV concept
> (getHeaders aka fast catchup, wallet-birthday, bloom filters)?
> Maybe it would be possible to only port some of the bitcoinj work to c++
> (and use it for extending BitcoinKit or bitcoind)?
> Maybe then it could also be a starting point for someone who likes to
> create a SPV mode for bitcoind?
>

Making bitcoind/Bitcoin-Qt support SPV mode was the original plan some
years ago, Satoshi even sent me some code he wrote that did the first
parts, but it was incomplete.

At the time, I decided to do a separate implementation for a few different
reasons. One is that my understanding of his code wasn't so good back then
and I lacked confidence to change it. Especially as there were no unit
tests back then (and still aren't any for most of it), making invasive
changes to the core validation code was and is highly risky. A separate
code base seemed to reduce the risk a lot.

Another reason is that Satoshi encouraged me to write a simple
re-implementation that people could learn from. And I wanted a documented,
object oriented API that people could use to build a variety of apps.

Yet another reason was bitcoind is security critical code that scrapes
complex data structures from untrusted sources on the internet, and it's
written in an unmanaged language. Ordinarily this would be a recipe for
disaster as a single overflow or memory management error could lead to
hacking and theft on a massive scale. It's like taking a chainsaw and using
it to carve an ice sculpture. Satoshi, incredibly, pulled it off, mostly by
using advanced C++ features that made his code hard to read for many people
and by being very, very careful. I was not convinced I could do such a good
job and was worried about accidentally introducing vulnerabilities.

A final reason is that it was clear that the bitcoind codebase would need
serious changes for mobiles, beyond that required for ordinary SPV support.
For example, Satoshi's code assumes it has access to block headers via a
std::map and that assumption is made in a lot of places. On Android phones,
you can't fit all the block headers in RAM. bitcoinj uses a circular ring
buffer of the last N thousand headers for this reason. It's quite different
to how bitcoind works.

All that said, it was a ton of work and it's still unclear that it was the
right call.

Anyway, your situation is a little different. Firstly you don't care about
mobiles, your app is intended for desktops. So the changes required are
less invasive. Also, there are more unit tests and more people with a good
understanding of the code these days, so perhaps the risk of introducing
bugs is lower. And these days we have some nice APIs for building apps so
that need is already met.

If you wanted to implement SPV mode in bitcoind, Gavin or I could send you
Satoshi's old patch although of course it is no longer usable. It would
indicate the basic cut lines though.

[-- Attachment #2: Type: text/html, Size: 3826 bytes --]