Re: [Bitcoin-development] Scalability issues

[Pieter Wuille <pieter.wuille@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 1828 bytes --]

And now to the list...

On Jul 27, 2012 6:21 AM, "grarpamp" <grarpamp@gmail.com> wrote:
>
> Update: this class of machine just became useless for bitcoin.
> When blk0002.dat was created to store more blocks, all forward
> progress processing blocks turned into losing ground by 20 or so
> a day. Guessing both datfiles were being accessed at once resulting
> in disk based overload. I've not seen any other mentions of crypto
> in this thread so I'm not sure how well new hardware would perform.
> Going shopping I guess.

I doubt the encryption is the problem. I have a much more recent machine
(i7 with 8 GiB RAM), and my blockchain is on a (userspace!) encrypted
filesystem. However, I do not notice any measurable slowdown from doing so.

You are however using a filesystem (ZFS) that is uses its own filesystem
caching implementation to reach some performance, and is known to be very
memory-hungry at that. Furthermore, I believe it is known to have
performance issues on 32-bit architectures. The bdb backend Bitcoin uses
does many I/O operations, and writes them synchronously to disk, killing
whatever caching your filesystem provides. For those who run a large
database on ZFS, I believe it is advised to put ZFS's intent log on a
separate SSD-backed device, to get fast synchronous writes.

That said, some improvememts may be coming. Mike has been working on
changing the backend from bdb to leveldb, which may (or may not) result in
a very different performance profile on your machine. Also, I've been
working on switching the bitcoin block verifier to use a different style
database layout ("ultraprune"), which is smaller, faster, and will support
pruning. A recent test on my own machine synced the blockchain up to the
latest checkpoint in 7 minutes (6 minutes when stored in RAM instead of
disk).

-- 
Pieter

[-- Attachment #2: Type: text/html, Size: 2034 bytes --]