From: Gregory Maxwell <gmaxwell@gmail•com>
To: telemaco <telemaco@neomailbox•net>
Cc: Bitcoin Dev <bitcoin-dev@lists•linuxfoundation.org>
Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db
Date: Thu, 29 Oct 2015 08:17:27 +0000 [thread overview]
Message-ID: <CAAS2fgQ=+5b4tht9SeGVJPXDbqKan=Y8x7Q5F=uT5iiq+BnHJA@mail.gmail.com> (raw)
In-Reply-To: <5631C363.5060705@neomailbox.net>
On Thu, Oct 29, 2015 at 6:57 AM, telemaco via bitcoin-dev
<bitcoin-dev@lists•linuxfoundation.org> wrote:
> Why not "outsource" totally that data management part to the already
> existing with decades of experience database world. People would be able to
> create incredibly easy bitcoin statistics/graphs/analisys with existing
> software packages (hey even excel or libreoffice like) or connect bitcoin
> data to their own sources and if so they chose analyze bitcoin data on a
> datawarehouse or any imaginable approach. Of course every transaction would
> be have to do through the bitcoin node and only the data management would be
> on rdbms side.
The word "database" is likely confusing people here. This is not a
database in an ordinary sense.
The bitcoin core consensus engine requires a highly optimized ultra
compact data structure to perform the lookups for coin existence. The
data stored is highly compressed and very specialized, it would not be
useful to other applications. Right now, on boring laptop hardware,
during network synchronization updates to this database run at over
10,000 records per second, while the system is also busy doing the
other validation chores of a node. This is backended by a high
performance transactional key value store. The need for performance
here is essential to even keeping up with the network, it's not about
enabling any kind of fancy querying (bitcoin core does not offer fancy
querying), it's about the base load that every node must handle to
usably sync up and keep up with the Bitcoin network.
The backend can be swapped out for something else that provides the
same properties, but doing so does not give you any of the
inspection/analytics that you're looking for. Systems that do that
exist, and they require databases taking hundreds of gigabytes of
storage and take days to weeks to import the network data. They're
great for what they're for, but they're not suitable for consensus use
in the system for space efficiency, performance, and consensus
consistency reasons.
next prev parent reply other threads:[~2015-10-29 8:17 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-29 6:57 telemaco
2015-10-29 8:03 ` Luke Dashjr
2015-10-30 3:04 ` Simon Liu
2015-10-30 3:35 ` Gregory Maxwell
2015-10-30 4:04 ` Peter R
2015-10-30 4:28 ` Gregory Maxwell
2015-11-15 1:02 ` Peter R
2015-11-15 1:08 ` Gregory Maxwell
2015-11-15 1:45 ` Peter R
2015-11-15 2:10 ` Gregory Maxwell
2015-11-15 2:58 ` Peter R
2015-11-15 3:30 ` Gregory Maxwell
2015-11-15 4:10 ` Peter R
2015-11-15 10:12 ` Jorge Timón
2015-11-15 11:28 ` Jorge Timón
2015-11-15 15:48 ` Peter R
2015-11-15 17:06 ` Peter R
2015-11-17 13:54 ` Tamas Blummer
2015-11-17 15:24 ` Tom Harding
2015-11-17 22:17 ` telemaco
2015-11-20 14:15 ` Jorge Timón
2015-11-16 1:52 ` Rusty Russell
2015-11-15 3:04 ` Luke Dashjr
2015-11-15 3:17 ` Peter R
2015-10-29 8:17 ` Gregory Maxwell [this message]
-- strict thread matches above, loose matches on Subject: below --
2015-10-22 21:26 Jeff Garzik
2015-10-22 21:54 ` Patrick Strateman
2015-10-22 21:56 ` Joseph Gleason ⑈
2015-10-23 6:53 ` Jonas Schnelli
2015-10-23 7:45 ` Lucas Betschart
2015-10-28 20:28 ` Sean Lynch
2015-10-28 21:11 ` Jeff Garzik
2015-10-23 10:30 ` Tom Zander
2015-10-26 18:06 ` Douglas Roark
2015-10-28 15:52 ` Tom Zander
2015-11-18 0:06 ` Jonathan Wilkins
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAAS2fgQ=+5b4tht9SeGVJPXDbqKan=Y8x7Q5F=uT5iiq+BnHJA@mail.gmail.com' \
--to=gmaxwell@gmail$(echo .)com \
--cc=bitcoin-dev@lists$(echo .)linuxfoundation.org \
--cc=telemaco@neomailbox$(echo .)net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox