public inbox for bitcoindev@googlegroups.com
 help / color / mirror / Atom feed
From: grarpamp <grarpamp@gmail•com>
To: bitcoin-development@lists•sourceforge.net
Subject: Re: [Bitcoin-development] Scalability issues
Date: Mon, 23 Jul 2012 18:33:24 -0400	[thread overview]
Message-ID: <CAD2Ti2_Pz9-SsHP49+6MKnM0er9zdAaFKDQaOgqDpju1_igd_g@mail.gmail.com> (raw)
In-Reply-To: <CAAS2fgREzk_dU0ie+YvDdRwKcTk6tk_i=a2Bb74w9uF=EwYhGA@mail.gmail.com>

> You're seriously suggesting that I'm using a system
> which is 720x (one month vs one hour) faster than your
> P4 1.8GHz?

Don't know what you're using since you've not stated it.

> I find this doubtful, especially since bitcoin's sync is effectively
> single threaded.

Extra cores help with disk, crypto, net, etc...

> month

I've spent about two weeks crunching about the last month's
worth of new blocks.

> Your results are not expected and are not believed to be
> representative.

The config is reproducible, and not believed to be uncommon.

> try to isolate it

Mostly disk and crypto.
Shall everyone instead run in bitrot and no-privacy mode?
What do we do when we've got 10k trans a day coming in?
50k? 100k, 1M? When the chain gets 1M, 50M, 500M, 1B long?

Forget my swamped box, these numbers are coming to others.

> try to sync from a local node into tmpfs

I'd bet some people using 'tmpfs' probably have it unknowingly
[fall]backed to swap instead of core.

Bitcoin already takes up 3GiB of disk, how many have that much
free RAM? How many have 30GiB, 300GiB?

> If you're building against BDB later than the recommended 4.8
> be aware that there have been performance regressions with later
> versions

Yes, I left out that bit of platform so here are the remaining
bits... db4830, boost149, vm.kmem_size=650000000


I'm not bashing anything or anybody, just detailing a stock config
that is underwater. Anybody wishing to verify can get the hardware
from their junk pile and the software from freebsd.org. I'll certainly
be looking at both it and different setups too. If anyone's using
say Linux/BSD, BTRFS/ZFS, crypto, on i386/amd64, they could
chime in with their times too.

Disk is the cheapest, easiest thing for Joe to get. Think about
indexing and checkpointing into said disk. I don't know what
bitcoin's doing, but if it's verifying every transaction back to
the root, that would seem a bit ridiculous.

Joe probably won't be happy buying TiB's for bitcoind, so after that's
filled (assuming there's CPU to do it), the trust model has to change.
These scales are coming...



  parent reply	other threads:[~2012-07-23 22:33 UTC|newest]

Thread overview: 23+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-07-22 22:37 grarpamp
2012-07-23  7:23 ` Raphael NICOLLE
2012-07-23  7:35 ` Gregory Maxwell
2012-07-23  9:00   ` Michael Grønager
2012-07-23 15:11     ` grarpamp
     [not found]       ` <CAAS2fgREzk_dU0ie+YvDdRwKcTk6tk_i=a2Bb74w9uF=EwYhGA@mail.gmail.com>
2012-07-23 22:33         ` grarpamp [this message]
2012-07-27  4:20           ` grarpamp
2012-07-27  4:59             ` Gregory Maxwell
2012-07-27  5:59               ` grarpamp
2012-07-27  6:03                 ` Luke-Jr
2012-07-27  6:28                   ` grarpamp
2012-07-27  6:56                 ` Gregory Maxwell
2012-07-27 19:37                   ` grarpamp
2012-07-27  8:53                 ` Andreas Petersson
     [not found]             ` <CAPg+sBjpqqxL_GxYc+8Ry7DcXMkouO9bYi5VqOyEjj_5s0x06Q@mail.gmail.com>
2012-07-27  9:59               ` Pieter Wuille
2012-07-27 19:26               ` grarpamp
2012-07-23 15:54     ` steve
2012-07-24  8:25       ` Michael Grønager
2012-07-24  9:18         ` Mike Hearn
2012-07-24 19:56         ` steve
2012-07-25  9:45           ` Michael Grønager
2012-07-25 14:24             ` steve
2012-07-23 22:50 ` Aidan Thornton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAD2Ti2_Pz9-SsHP49+6MKnM0er9zdAaFKDQaOgqDpju1_igd_g@mail.gmail.com \
    --to=grarpamp@gmail$(echo .)com \
    --cc=bitcoin-development@lists$(echo .)sourceforge.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox