From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 25DAE405 for ; Thu, 22 Oct 2015 21:26:45 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f171.google.com (mail-wi0-f171.google.com [209.85.212.171]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 8505615C for ; Thu, 22 Oct 2015 21:26:43 +0000 (UTC) Received: by wijp11 with SMTP id p11so51237714wij.0 for ; Thu, 22 Oct 2015 14:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=/0/3BlSj9GULR8HhNGXUll7h0vpy8fICkdEoR4xq7jE=; b=gDSvnhYE/PL11U7yaGPxCAZpqOeXjjWqsg5XEurvxhfvcUR49m+LBtxgDui9PycT/a vaMNx0kdz82IiP2CANMNF1ZRYJfaYO9s139MJsHOgwcCW4xtdhat4E766uKH7JKT+Sg/ tiQvVhl2aS+FEpGIHANY384c/8zVCahCMannrqjhpMi9Qi+ktiCY6+Ularo6xkG+9D8T r2Uc2ypVBkqaiYwP4+bX8c7CRHG03oKnUG7j0xUwm7vbRUf8oHbnvAfiUQmQmxdZnnX9 TC71KlppgMSBw4fitAuSxpkc0WxJvS2p7zByhKGhbTOonC1gHsF5lyvkNZQ2k8VssreE R3Yg== MIME-Version: 1.0 X-Received: by 10.180.75.36 with SMTP id z4mr606071wiv.68.1445549202085; Thu, 22 Oct 2015 14:26:42 -0700 (PDT) Received: by 10.28.100.130 with HTTP; Thu, 22 Oct 2015 14:26:42 -0700 (PDT) Date: Thu, 22 Oct 2015 17:26:42 -0400 Message-ID: From: Jeff Garzik To: Bitcoin development mailing list Content-Type: multipart/alternative; boundary=f46d043c7cdab21f780522b826bf X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Thu, 22 Oct 2015 21:37:45 +0000 Subject: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 21:26:45 -0000 --f46d043c7cdab21f780522b826bf Content-Type: text/plain; charset=UTF-8 Here is the beginnings of an implementation to replace leveldb with sqlite: https://github.com/jgarzik/bitcoin/tree/2015_sqlite It builds, but still needs work before passing tests. It was noted that leveldb is unmaintained, and this is part of researching alternatives that are maintained and reliable. --f46d043c7cdab21f780522b826bf Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Here is the beginnings of an implementation to replace lev= eldb with sqlite:=C2=A0https://github.com/jgarzik/bitcoin/tree/2015_sqlite

It builds, but still needs work before passing tests.

It was noted that leveldb is unmaintained, and this is part= of researching alternatives that are maintained and reliable.

--f46d043c7cdab21f780522b826bf-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 80C3A89F for ; Thu, 22 Oct 2015 21:56:30 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f178.google.com (mail-wi0-f178.google.com [209.85.212.178]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id C7F11166 for ; Thu, 22 Oct 2015 21:56:29 +0000 (UTC) Received: by wicfx6 with SMTP id fx6so7383957wic.1 for ; Thu, 22 Oct 2015 14:56:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :content-type; bh=C3f/CP2OC18Lftjfrp/nBTWodIxkFLvm8qPpEouaTys=; b=izUUkp0d2+p21riR+xGtmSbmgckcm22ibx4IN6HaNvV5+4mDurO7bLwWn70fMkK3Z5 DoE2PBhPvBxOtEKvtx61sNIpEvIb7I5DS+vT9+To5XQ6jy+6oEo6JgG8g98PMlEswrpX aks73oDADgcd+ydak/QXoLyAsLEvAdDAYbdj1Lkg8oCJxH3hIehCXNPXusvlJ8QDX2FU jyCrtoGlB8zWAbAXCBgXnzPHTSQca8dVYOYInxaItmQmCVr6zKMbvwuGBOS0ev9I0Xfk X4MjSS5PAyLnSK69/Lk6qtn60pXw1txhaMsrx9++IuMXjXQGZ6boAh309StZ4wiULWpX JkDw== X-Received: by 10.28.131.84 with SMTP id f81mr36719wmd.57.1445550988436; Thu, 22 Oct 2015 14:56:28 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: =?UTF-8?B?Sm9zZXBoIEdsZWFzb24g4pGI?= Date: Thu, 22 Oct 2015 21:56:18 +0000 Message-ID: To: Jeff Garzik , Bitcoin development mailing list Content-Type: multipart/alternative; boundary=001a11443d1c2bacbe0522b89116 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Thu, 22 Oct 2015 22:41:32 +0000 Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 21:56:30 -0000 --001a11443d1c2bacbe0522b89116 Content-Type: text/plain; charset=UTF-8 I have done a lot of recent work on local key value stores, mostly for a java electrum server I am working on. I'd suggest considering LMDB. One downside is that it is memory mapped so 32-bit systems that need over 2gb of storage are right out. Other than that, it is quite fast and seems reliable in my testing. On Thu, Oct 22, 2015 at 2:37 PM Jeff Garzik via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Here is the beginnings of an implementation to replace leveldb with > sqlite: https://github.com/jgarzik/bitcoin/tree/2015_sqlite > > It builds, but still needs work before passing tests. > > It was noted that leveldb is unmaintained, and this is part of researching > alternatives that are maintained and reliable. > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > --001a11443d1c2bacbe0522b89116 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

I have done a lot of recent work on local key value stores= , mostly for a java electrum server I am working on.

I&#= 39;d suggest considering LMDB.=C2=A0 One downside is that it is memory mapp= ed so 32-bit systems that need over 2gb of storage are right out.=C2=A0 Oth= er than that, it is quite fast and seems reliable in my testing.

On Thu, Oct= 22, 2015 at 2:37 PM Jeff Garzik via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

Here = is the beginnings of an implementation to replace leveldb with sqlite:=C2= =A0 https://github.com/jgarzik/bitcoin/tree/2015_sqlite
It builds, but still needs work before passing tests.

It was noted that leveldb is unmaintained, and this is part= of researching alternatives that are maintained and reliable.

_______________________________________________
bitcoin-dev mailing list
= bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev
--001a11443d1c2bacbe0522b89116-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id BA290892 for ; Thu, 22 Oct 2015 21:54:07 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pa0-f51.google.com (mail-pa0-f51.google.com [209.85.220.51]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id AE376E5 for ; Thu, 22 Oct 2015 21:54:06 +0000 (UTC) Received: by padhk11 with SMTP id hk11so97561554pad.1 for ; Thu, 22 Oct 2015 14:54:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:disposition-notification-to:date:from:user-agent :mime-version:to:subject:references:in-reply-to:content-type; bh=9O8LvKPPXiUKB/5IRPiQvJfcd/OfwbPq6cMsuywNm20=; b=F0UHvTP/sIz0//NPQXIbsG/scUdHv2FuF/tb7vGdZqAy/mBhMhlFmtJqpp+1aV0j2f N3HiCJ1aP+7gP5UE8eFyPKwR4jZE9BlxsmCAp8KKr07S3XEiEnKX0QanUxdg7Roh9TyQ XQQPss7uDGrHDaDkIKzG5qpCwscBdRoRaekAe5i4rWmOHCa9c4RNQeZagD7r22LPg2RG wwini1TLv/WNQOhsUEvriKXouyWk5z7ZFCA3/QL6CyHTsBTMQ4pQLKtTUqk1GgsCRE9X +Pydi4JwFDDY5RXWyvZCpgwW6WsystvUKQlii/OBq8jXW00XXgbDjqtmOGKm0kZyiqm6 9WPQ== X-Received: by 10.66.62.198 with SMTP id a6mr604400pas.68.1445550846471; Thu, 22 Oct 2015 14:54:06 -0700 (PDT) Received: from [10.45.134.131] (strateman.ninja. [66.175.221.254]) by smtp.googlemail.com with ESMTPSA id yp5sm13800835pac.38.2015.10.22.14.54.05 for (version=TLSv1/SSLv3 cipher=OTHER); Thu, 22 Oct 2015 14:54:06 -0700 (PDT) Message-ID: <56295B06.8010200@gmail.com> Date: Thu, 22 Oct 2015 14:54:14 -0700 From: Patrick Strateman User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Icedove/31.8.0 MIME-Version: 1.0 To: bitcoin-dev@lists.linuxfoundation.org References: In-Reply-To: Content-Type: multipart/alternative; boundary="------------060403080001090407050205" X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Thu, 22 Oct 2015 22:43:22 +0000 Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2015 21:54:07 -0000 This is a multi-part message in MIME format. --------------060403080001090407050205 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Benchmarks? I cant imagine that's very fast. On 10/22/2015 02:26 PM, Jeff Garzik via bitcoin-dev wrote: > Here is the beginnings of an implementation to replace leveldb with > sqlite: https://github.com/jgarzik/bitcoin/tree/2015_sqlite > > It builds, but still needs work before passing tests. > > It was noted that leveldb is unmaintained, and this is part of > researching alternatives that are maintained and reliable. > > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev --------------060403080001090407050205 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit Benchmarks?

I cant imagine that's very fast.

On 10/22/2015 02:26 PM, Jeff Garzik via bitcoin-dev wrote:

Here is the beginnings of an implementation to replace leveldb with sqlite: https://github.com/jgarzik/bitcoin/tree/2015_sqlite

It builds, but still needs work before passing tests.

It was noted that leveldb is unmaintained, and this is part of researching alternatives that are maintained and reliable.
_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

--------------060403080001090407050205-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 0B42367 for ; Fri, 23 Oct 2015 06:53:28 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from server3 (server3.include7.ch [144.76.194.38]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 5B185EB for ; Fri, 23 Oct 2015 06:53:27 +0000 (UTC) Received: by server3 (Postfix, from userid 115) id 20ECC2E601C1; Fri, 23 Oct 2015 08:53:26 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00, FSL_HELO_NON_FQDN_1 autolearn=ham version=3.3.1 Received: from Jonass-MacBook-Pro.local (cable-static-140-182.teleport.ch [87.102.140.182]) by server3 (Postfix) with ESMTPSA id 60D462D002B3; Fri, 23 Oct 2015 08:53:24 +0200 (CEST) To: Jeff Garzik , Bitcoin development mailing list References: From: Jonas Schnelli X-Enigmail-Draft-Status: N1110 Message-ID: <5629D960.8060109@jonasschnelli.ch> Date: Fri, 23 Oct 2015 08:53:20 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Fri, 23 Oct 2015 08:22:10 +0000 Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 06:53:28 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 > Here is the beginnings of an implementation to replace leveldb > with sqlite: https://github.com/jgarzik/bitcoin/tree/2015_sqlite > > It builds, but still needs work before passing tests. Nice work! Although not sure if we should focus directly on sqlite4 (could be optional with a configure flag and a subtree [until stable], sqlite3 supported over depends). Also i personally would recommend to not implement it as a replacement, instead, support multiple backends (wrapper header / different wrapper implementations [leveldb / sqlite3 / sqlite4]). But – agreed – should not be the focus, but a nice additional flexibility if it not require much more work. And – in theory – multiple database back-ends would allow migration. Before investigating to deep, i think we need a dbwrapper bench tool that represents our needs and our style how we hit the database. Gavins recently added bench target / bench environment could be used for that. /jonas -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIcBAEBAgAGBQJWKdlgAAoJECnUvLZBb1Ps7J8P/2L6215fd0rWGv5uvbLSnQvm hy1T2AaOfH5HXd2m95icaKYk+ugvQAL80Q/67YwZbPLsT4fErgegw8n75Z6nh/Pc OJ1EtAvD+Kc/Vm0Wcvt421HXwnm4f+j+8eoEvpG6DdC8gfIG+efhM7DljeNbPFrA ieNe7XrQ1EZ29lMzpQv0nx3bi6tUHOWuazk82B6vnK49MjH7nrUFipcc18nXbSpM ZKjQakXmfqIBG8QP9ZUYUlW4aoG0oaOoZnGjQA2LeXBJpIPLpE/WPg0XaXubC+No 232CJNtNyUOmjkb2qbep6vSYgqGJNy1HbCU5y3qooxJhFnKdo63CQsyJKSpL/ssi 0lWraNxjbacxsBn+63In3wEkj02orwm2zTO4I77wCrZmJgpvBFb9bZWTeL8DCYSG DTkZoKWEK74xvm+dNpEWXK9Lm1ltfyhPdaFeMoRDub4w2uuYlk3KuD8Vdy81HYJb sak8FbkiWw9xx2OP9+G56Arf9W6mnJ3YHJGrY4SXeRAfuNYwGFHIGt6I1JobuHy4 VWmcHuooz/Q9JLUu3Rr3HsEdXYgCmgmWuzTgWG+Hx92Y6XLWV5pOX+vFRO6J5fSm aTYPD4GsTM3FROXw5ezlXJ+2y1+ITzmrfm03fEDQvoIyH0TwBqBc5sMBBkma5DDR 0HUthPHWCD+AxbBPbRVa =CUMX -----END PGP SIGNATURE----- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id DBB4D1BB for ; Fri, 23 Oct 2015 07:45:11 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f172.google.com (mail-wi0-f172.google.com [209.85.212.172]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 0EA0CE1 for ; Fri, 23 Oct 2015 07:45:11 +0000 (UTC) Received: by wicll6 with SMTP id ll6so19554894wic.1 for ; Fri, 23 Oct 2015 00:45:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=MNtiAld5AKxinvni5zydqG/WsgeYtQ2rkWEeiJThtlA=; b=SsE6/3jbvru0zgalBVTXINB1Lehy2Fk5riF97M47J6yw9s8OLKQmhR6dH32KMIKaDo KI+1kXIsWZMhXVxCVJjWzwwHeYxsBWyIVloxhIuyhQWFwPDFmdNmtk1WcuuQthZdJxaj +P+VmkBUFQ6UibjDMfg4HluGBrmJ+PzCPK1Lz+Uv6wHMfOWPDf0kmo5OPkFBpuqoL0Oy plHNMgWKWxeB84d2o1H8Tj2zW1afQVOnHck+lRpaOkF9SSzQHVol2RiMwWETCMOJ9Cl1 eJSdFw21agRc654bCxejxLi9z2ZGzMC2ry3HNqBEbojb3yQJ6bGKgYV7Ebs+AZTz/p05 cmlg== MIME-Version: 1.0 X-Received: by 10.180.210.210 with SMTP id mw18mr2625641wic.18.1445586309478; Fri, 23 Oct 2015 00:45:09 -0700 (PDT) Received: by 10.28.212.141 with HTTP; Fri, 23 Oct 2015 00:45:09 -0700 (PDT) In-Reply-To: References: Date: Fri, 23 Oct 2015 09:45:09 +0200 Message-ID: From: Lucas Betschart To: Jeff Garzik Content-Type: multipart/alternative; boundary=001a11c25bd078046e0522c0caf6 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Fri, 23 Oct 2015 08:22:46 +0000 Cc: Bitcoin development mailing list Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 07:45:12 -0000 --001a11c25bd078046e0522c0caf6 Content-Type: text/plain; charset=UTF-8 Facebook has a LevelDB fork which is maintained. It's called RocksDB and the API seems to be nearly the same as for LevelDB, thus maybe easy to replace: http://rocksdb.org/ https://github.com/facebook/rocksdb Although I don't know if we might have some negative effects for our use-case since RocksDB was optimized for big databases running on multiple cores. 2015-10-22 23:26 GMT+02:00 Jeff Garzik via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org>: > Here is the beginnings of an implementation to replace leveldb with > sqlite: https://github.com/jgarzik/bitcoin/tree/2015_sqlite > > It builds, but still needs work before passing tests. > > It was noted that leveldb is unmaintained, and this is part of researching > alternatives that are maintained and reliable. > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > --001a11c25bd078046e0522c0caf6 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Facebook has a LevelDB fork which is maintained.

It= 9;s called RocksDB and the API seems to be nearly the same as for LevelDB, = thus maybe easy to replace:=C2=A0http://roc= ksdb.org/ =C2=A0=C2=A0h= ttps://github.com/facebook/rocksdb

Although I = don't know if we might have some negative effects for our use-case sinc= e RocksDB was optimized for big databases running on multiple cores.

<= div>

2015-10-22 23:26 GMT+02:00 Jeff Garzik via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>:

Here is the beginnings of an implementat= ion to replace leveldb with sqlite:=C2=A0https://github.com/jgarzik/= bitcoin/tree/2015_sqlite

It builds, but still needs = work before passing tests.

It was noted that level= db is unmaintained, and this is part of researching alternatives that are m= aintained and reliable.

_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.= linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev

--001a11c25bd078046e0522c0caf6-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 2608F9C for ; Fri, 23 Oct 2015 10:30:48 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mx-out01.mykolab.com (mx01.mykolab.com [95.128.36.1]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6F6A2128 for ; Fri, 23 Oct 2015 10:30:47 +0000 (UTC) X-Virus-Scanned: amavisd-new at kolabnow.com X-Spam-Score: -2.9 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 Received: from mx03.mykolab.com (mx03.mykolab.com [10.20.7.101]) by mx-out01.mykolab.com (Postfix) with ESMTPS id E8ED2601B3 for ; Fri, 23 Oct 2015 12:30:38 +0200 (CEST) From: Tom Zander To: bitcoin-dev@lists.linuxfoundation.org Date: Fri, 23 Oct 2015 11:30:37 +0100 Message-ID: <3162730.lzR74nC3xW@garp> In-Reply-To: References: Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Fri, 23 Oct 2015 10:42:36 +0000 Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2015 10:30:48 -0000 On Thursday 22 Oct 2015 17:26:42 Jeff Garzik via bitcoin-dev wrote: > It was noted that leveldb is unmaintained, and this is part of researching > alternatives that are maintained and reliable. Apart from it being unmaintained, any links to what are problems with levelDB? From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id DFBAF7AE for ; Mon, 26 Oct 2015 19:26:25 +0000 (UTC) X-Greylist: delayed 01:19:17 by SQLgrey-1.7.6 Received: from omr1.cc.vt.edu (outbound.smtp.vt.edu [198.82.183.121]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 40658A9 for ; Mon, 26 Oct 2015 19:26:25 +0000 (UTC) Received: from mr6.cc.vt.edu (mr6.cc.ipv6.vt.edu [IPv6:2001:468:c80:2105:0:250:3bcb:5b17]) by omr1.cc.vt.edu (8.14.4/8.14.4) with ESMTP id t9QI76Vp014134 for ; Mon, 26 Oct 2015 14:07:07 -0400 Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) by mr6.cc.vt.edu (8.14.4/8.14.4) with ESMTP id t9QI71uW020977 for ; Mon, 26 Oct 2015 14:07:06 -0400 Received: by padhk11 with SMTP id hk11so195203881pad.1 for ; Mon, 26 Oct 2015 11:07:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vt_edu.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-type:content-transfer-encoding; bh=/1tV/cy7Sd7D4cGKAvNXtltyRj9s1P2xZQ747gcjAPU=; b=vWoBmS7hF0N3mDrBC7HYU1/voop9UzZfF/n2WiRLlbGgxrLIH+mRbK9fne/UfSgE9U gCQzZs1o8VleAYPxuGrkTPtRwaoLWASh4sc4E1JplSeSjp0YmNtt1eqttgWMs2m9Jw6W xvG6mTgE+OB1tAk+ulN1dfKVpqx9Evobb+za1XzSOuc5ANyIwRxssXaSM8j3qEVIAsqh 4Zz1Uoltp4vr+9Sh/PsmoXQUKG7t9z/REEwGWXbzJ9EVAgcwnWGifSAHVmmzCRqLUmRn B+7LKghM9OzvA5fJ5mQYaLb2/GcpdngWn8K+g2HlGS7s4USYK3QmkyzGgO68RKqAVHb3 OR0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=/1tV/cy7Sd7D4cGKAvNXtltyRj9s1P2xZQ747gcjAPU=; b=KD2Ga07knOct+5rmmp3L4yJrCCKrcLBDdFepD7TzqP0+puer+G1Sj00vaa5581n9ym LbKadUp9Lk4aivRWHlxNRbZ5O5GB4CT6D9+tsb8PhtzWtVs824JXJWWGIygRsKyBgDTO HVlgZZBX8gu8BKUcdwyFDSPrHnW4pYRvxWF4RW7BU4PMqSEltckWpR/8ofmNjIY/lZXX zzjteWFUm2C8AA18YlHhM4/xEjERHUsc06r/D3QTVHjkB2WnBtiI2wGeLXiLDLWaxJZb 0wQ4WMUKSX+rC+nnDKQYwiwyOnvbkLNwklob+fEx6UDz1sHx6Kt9ek0NTvdJjagbsMdZ VSJQ== X-Gm-Message-State: ALoCoQkB68DDWXomT4OeqzmYaJOSfjdVs9gZiBxOTkJfUKeP+N13+UmbaBYyqtWfYpOiDYtCQIhn0/xo3BK0IFQ31nx0SSAQ5A/M7eV6PlgXIsYSbZZ1nj224XBp39fmEkB//fcCpyoGYf7bUo7t9dg08ZBG7sB5RLVVolPJa80IzkYfoXpFNa6IPIB9RWjKcnqWLA/XYa7bW21VRGKvbcsHUuHJOpjLDg== X-Received: by 10.68.203.2 with SMTP id km2mr23056689pbc.155.1445882821121; Mon, 26 Oct 2015 11:07:01 -0700 (PDT) X-Received: by 10.68.203.2 with SMTP id km2mr23056680pbc.155.1445882820999; Mon, 26 Oct 2015 11:07:00 -0700 (PDT) Received: from [192.168.1.230] (c-24-22-36-12.hsd1.or.comcast.net. [24.22.36.12]) by smtp.googlemail.com with ESMTPSA id g12sm35308427pat.36.2015.10.26.11.07.00 for (version=TLSv1/SSLv3 cipher=OTHER); Mon, 26 Oct 2015 11:07:00 -0700 (PDT) To: bitcoin-dev@lists.linuxfoundation.org References: <3162730.lzR74nC3xW@garp> From: Douglas Roark X-Enigmail-Draft-Status: N1110 Message-ID: <562E6BC0.7010002@vt.edu> Date: Mon, 26 Oct 2015 11:06:56 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <3162730.lzR74nC3xW@garp> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_MED, RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Tue, 27 Oct 2015 00:10:24 +0000 Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Oct 2015 19:26:26 -0000 On 2015/10/23 03:30, Tom Zander via bitcoin-dev wrote: > On Thursday 22 Oct 2015 17:26:42 Jeff Garzik via bitcoin-dev wrote: >> It was noted that leveldb is unmaintained, and this is part of researching >> alternatives that are maintained and reliable. > > Apart from it being unmaintained, any links to what are problems with levelDB? While not exactly the most rigorous link, https://en.wikipedia.org/wiki/LevelDB#Bugs_and_Reliability seems like an okay place to start. One thing I can attest to is that, when Armory used LevelDB (0.8 - 0.92, IIRC), quite a few users had DB corruption issues, particularly on Windows. Even when a switch to LMDB occurred for 0.93, loads of complaints would come in from users whose LevelDB-based Core DBs would fail. I know that the guy who moved Armory over to LMDB would love to have more time in the day so that he could write a Core patch that does the same. It's a very sore spot for him. (FWIW, LMDB seems to work quite nicely, at least once you patch up the source a little bit. The latest version is also compatible with Core's cross-compiling scheme. I'd love to see it added to Core one day.) Doug From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 7FF17884 for ; Wed, 28 Oct 2015 15:53:01 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from mx-out01.mykolab.com (mx01.mykolab.com [95.128.36.1]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9A1931D2 for ; Wed, 28 Oct 2015 15:53:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at kolabnow.com X-Spam-Score: -2.9 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 Received: from mx03.mykolab.com (mx03.mykolab.com [10.20.7.101]) by mx-out01.mykolab.com (Postfix) with ESMTPS id 7D3686161C; Wed, 28 Oct 2015 16:52:57 +0100 (CET) From: Tom Zander To: bitcoin-dev@lists.linuxfoundation.org, Douglas Roark Date: Wed, 28 Oct 2015 15:52:53 +0000 Message-ID: <2265352.UXJLTKs9q7@garp> In-Reply-To: <562E6BC0.7010002@vt.edu> References: <3162730.lzR74nC3xW@garp> <562E6BC0.7010002@vt.edu> Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 15:53:01 -0000 On Monday 26 Oct 2015 11:06:56 Douglas Roark via bitcoin-dev wrote: > While not exactly the most rigorous link, > https://en.wikipedia.org/wiki/LevelDB#Bugs_and_Reliability seems like an > okay place to start. Thanks for that link! Another Google open source product I'll avoid like the plague ;) From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 9A05767 for ; Wed, 28 Oct 2015 20:28:10 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ig0-f169.google.com (mail-ig0-f169.google.com [209.85.213.169]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 0E271FE for ; Wed, 28 Oct 2015 20:28:09 +0000 (UTC) Received: by igbhv6 with SMTP id hv6so16188073igb.0 for ; Wed, 28 Oct 2015 13:28:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=literati_org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-type; bh=KxXAhgwwJaDh27Tksfq2ylBbJWtNyOn2cn83j7pIAng=; b=tL4C8IY6O5qnmA7dNPEAp/Q+5uS2VfDgjdA+YtlDpuzfPFWmQuF20SgVnsDTofsDfT XWNOnpb42A1evd7NJmaq99liXzqJMayYkpdv7R2vYS2hKULbZe2Xb4BBRCJgYEqjKjOA pghWu2ZN+H5Wf2Twx6KJ6hxVJGj2Z1D7v/j6qXVGP3nzh6tj+9RpOwSfvWtTn7FRR6sv pQGIbIHdfraULBptjQNlXqH87U5EQD1rG3x4GZsWFKNnxChxca31RKXmeFvRrXXqYMP+ xv+hzrkfk5ayIariZxaHBkwrogLa0/I2G2zrDEaIEt4EcJrVlj7ctfaXc0StDB+nsrbv p8jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-type; bh=KxXAhgwwJaDh27Tksfq2ylBbJWtNyOn2cn83j7pIAng=; b=fektGNpYJk5neP57P00fx/SEeK9iXtgF+s0rQCm0+CXZ7+z79mIgT2klchuauRo/LD O/iAZWdTkEl9bd72mblNj4aQWykc+yNQskju3i7FyeC66uMfEQZwYZZi211dcMEb1jZ4 LDiDfGpHJJtC2ng1PlzN7/dVAZYglsv4sJQ70GncwvTTGX7hP/271vfkhjUiFDW0jZnc QICAtkueoCWNipBh5sOWAw7pEeQJoP6Yw+NqMrRRLKd2+FnAT5vi/EGixI6gTxrqARu1 xU4o6D/pT88grYHMyZrNlg2vqClJWTnw0xJckVSPT8Hx+R4fzPdLaoEpIb5JBX0T8gy5 7mVg== X-Gm-Message-State: ALoCoQn9qxBqCf6ADWSqXjPu9xpaaQsaML+/t6fos2QRPl3+ub0rDhXLlangHvtY5WVa7UAxkmfi X-Received: by 10.50.72.108 with SMTP id c12mr5250708igv.63.1446064089381; Wed, 28 Oct 2015 13:28:09 -0700 (PDT) MIME-Version: 1.0 References:

In-Reply-To: From: Sean Lynch Date: Wed, 28 Oct 2015 20:28:00 +0000 Message-ID: To: Lucas Betschart , Jeff Garzik Content-Type: multipart/alternative; boundary=047d7bdc123c5efabd0523300877 X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Wed, 28 Oct 2015 20:47:39 +0000 Cc: Bitcoin development mailing list Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 20:28:10 -0000 --047d7bdc123c5efabd0523300877 Content-Type: text/plain; charset=UTF-8 On Fri, Oct 23, 2015 at 1:23 AM Lucas Betschart via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > Facebook has a LevelDB fork which is maintained. > It's called RocksDB and the API seems to be nearly the same as for > LevelDB, thus maybe easy to replace: http://rocksdb.org/ > https://github.com/facebook/rocksdb > > Although I don't know if we might have some negative effects for our > use-case since RocksDB was optimized for big databases running on multiple > cores. > While RocksDB is pretty decent, note that it's optimized for flash. Not sure how well it will work on spinning disks. --047d7bdc123c5efabd0523300877 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Fri, Oct 23= , 2015 at 1:23 AM Lucas Betschart via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:

Face= book has a LevelDB fork which is maintained.
It's called RocksDB an= d the API seems to be nearly the same as for LevelDB, thus maybe easy to re= place:=C2=A0 http://rocksd= b.org/ =C2=A0=C2=A0https://github.com/facebook/rocksdb

<= div>Although I don't know if we might have some negative effects for ou= r use-case since RocksDB was optimized for big databases running on multipl= e cores.

While RocksDB is prett= y decent, note that it's optimized for flash. Not sure how well it will= work on spinning disks. =C2=A0

--047d7bdc123c5efabd0523300877-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 9B3063EE for ; Wed, 28 Oct 2015 21:11:49 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f181.google.com (mail-wi0-f181.google.com [209.85.212.181]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 993A0157 for ; Wed, 28 Oct 2015 21:11:48 +0000 (UTC) Received: by wicll6 with SMTP id ll6so212445798wic.0 for ; Wed, 28 Oct 2015 14:11:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ljE//ilyMs7J7Mru6voZBDz7HeaOrjoP+KeQG1ja+rM=; b=0f5E5H2261yTnkdCr8lOwqmmAd+ppV+kKyVXrWHT34to2Mu3st/Xy4uHI+83onWeWF 6qB5SaRSB+uiDeEh3Y0N8kTlRUaH2vWbVf3Z3b55Ev9cHW/7kbCeMu/D1fSj6jdbt8Zz 7vJkg4ynoRQcO4oHYYhdhiWmY8Z0rfwI+ehYim8u4omLlUYcnGXhuhaD150Y29ukGkcc wimeqwDEvrhiqE4jIasu7MShjWn17D0NwKKOrJMDQGSnWBRU98SY10o026QnKBI0cCL4 dUoo0s03LJppnOU/uQtZaCxN28nghtXHAKjfdUGGu4qrW+04JYZ1GRkAT4pIKat3up2M Eutg== MIME-Version: 1.0 X-Received: by 10.28.88.135 with SMTP id m129mr2870347wmb.67.1446066707261; Wed, 28 Oct 2015 14:11:47 -0700 (PDT) Received: by 10.28.100.130 with HTTP; Wed, 28 Oct 2015 14:11:47 -0700 (PDT) In-Reply-To: References:

Date: Wed, 28 Oct 2015 17:11:47 -0400 Message-ID: From: Jeff Garzik To: Sean Lynch Content-Type: multipart/alternative; boundary=001a11443518686dcc052330a43e X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin development mailing list Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 21:11:49 -0000 --001a11443518686dcc052330a43e Content-Type: text/plain; charset=UTF-8 On Wed, Oct 28, 2015 at 4:28 PM, Sean Lynch wrote: > On Fri, Oct 23, 2015 at 1:23 AM Lucas Betschart via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > >> Facebook has a LevelDB fork which is maintained. >> It's called RocksDB and the API seems to be nearly the same as for >> LevelDB, thus maybe easy to replace: http://rocksdb.org/ >> https://github.com/facebook/rocksdb >> >> Although I don't know if we might have some negative effects for our >> use-case since RocksDB was optimized for big databases running on multiple >> cores. >> > > While RocksDB is pretty decent, note that it's optimized for flash. Not > sure how well it will work on spinning disks. > That's OK for our purposes. We have a huge database which already incentivized having zero seek time. --001a11443518686dcc052330a43e Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

On Wed, Oct 28, 2015 at 4:28 PM, Sean Lynch <seanl@liter= ati.org> wrote:

On Fri, Oct 23, 2015 at 1:23 = AM Lucas Betschart via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.or= g> wrote:
F= acebook has a LevelDB fork which is maintained.
It's called RocksDB= and the API seems to be nearly the same as for LevelDB, thus maybe easy to= replace:=C2=A0http://roc= ksdb.org/ =C2=A0=C2=A0https://github.com/facebook/rocksdb

Although I don't know if we might have some negative effects for= our use-case since RocksDB was optimized for big databases running on mult= iple cores.

While RocksD= B is pretty decent, note that it's optimized for flash. Not sure how we= ll it will work on spinning disks. =C2=A0

That's OK for o= ur purposes.=C2=A0 We have a huge database which already incentivized havin= g zero seek time.

--001a11443518686dcc052330a43e-- From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id E33C190 for ; Thu, 29 Oct 2015 06:57:38 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from s1.neomailbox.net (s1.neomailbox.net [5.148.176.57]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 7C17C137 for ; Thu, 29 Oct 2015 06:57:38 +0000 (UTC) To: bitcoin-dev@lists.linuxfoundation.org From: telemaco Message-ID: <5631C363.5060705@neomailbox.net> Date: Thu, 29 Oct 2015 07:57:39 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_LOW, RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Thu, 29 Oct 2015 07:37:44 +0000 Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Oct 2015 06:57:39 -0000 Why not allow two options: 1/ a default RocksDB/SQLite/LevelDB (whatever is decided) 2/ alternative provide instructions for connection to any other rdbms using odbc or jdbc. Why not allowing async disk writes or incredibly fast database systems if someone wants to have a node in a very fast datacenter or connected with their existing leveraged dataservers. It is the traditional approach to just use the open standard for database connectivity. Any person or any organization would just need to have one machine with their bitcoin node with a rdbms client installed (SAP Sybase client, or oracle client, or microsoft). The bitcoin node would just store their data using the odbc/jdbc protocol on ANY rdbms installed anywhere in their organization (other machine or the same). They would just need to issue a "create table" with a very simple table structure and they would benefit from async and indexes and using their already licensed, and configured system of their choosing, with bitcoin information being available to thousands of software packages and available aswell to thousands of programmers that work with rdbms and not just "RocksDB" or some obscure database system. Why not "outsource" totally that data management part to the already existing with decades of experience database world. People would be able to create incredibly easy bitcoin statistics/graphs/analisys with existing software packages (hey even excel or libreoffice like) or connect bitcoin data to their own sources and if so they chose analyze bitcoin data on a datawarehouse or any imaginable approach. Of course every transaction would be have to do through the bitcoin node and only the data management would be on rdbms side. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 02006258 for ; Thu, 29 Oct 2015 08:05:18 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from zinan.dashjr.org (zinan.dashjr.org [192.3.11.21]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 8315C79 for ; Thu, 29 Oct 2015 08:05:18 +0000 (UTC) Received: from ishibashi.localnet (unknown [IPv6:2001:470:5:265:61b6:56a6:b03d:28d6]) (Authenticated sender: luke-jr) by zinan.dashjr.org (Postfix) with ESMTPSA id 0558038A58D9; Thu, 29 Oct 2015 08:03:54 +0000 (UTC) X-Hashcash: 1:25:151029:telemaco@neomailbox.net::YAQ/04VX5ZOqEkFw:fDarR X-Hashcash: 1:25:151029:bitcoin-dev@lists.linuxfoundation.org::T=sqWz=WOjPyl6GU:0shO From: Luke Dashjr To: telemaco Date: Thu, 29 Oct 2015 08:03:50 +0000 User-Agent: KMail/1.13.7 (Linux/4.1.9-gentoo-r1; KDE/4.14.8; x86_64; ; ) References: <5631C363.5060705@neomailbox.net> In-Reply-To: <5631C363.5060705@neomailbox.net> X-PGP-Key-Fingerprint: E463 A93F 5F31 17EE DE6C 7316 BD02 9424 21F4 889F X-PGP-Key-ID: BD02942421F4889F X-PGP-Keyserver: hkp://pgp.mit.edu MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201510290803.52734.luke@dashjr.org> X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: bitcoin-dev@lists.linuxfoundation.org Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Oct 2015 08:05:19 -0000 On Thursday, October 29, 2015 6:57:39 AM telemaco via bitcoin-dev wrote: > Why not allow two options: > > 1/ a default RocksDB/SQLite/LevelDB (whatever is decided) > 2/ alternative provide instructions for connection to any other rdbms > using odbc or jdbc. I predict this would be a disaster. UTXO storage is CONSENSUS-CRITICAL code. Any divergence in implementation behaviour, including bugs AND bugfixes, may cause consensus failure. For this to have a reasonable *hope* of working, we need to choose one storage engine, and *will* need to maintain consensus- compatibility of it ourselves (since nobody else cares). Fixing LevelDB frankly seems like an easier task than switching to anything SQL-based, which would require a *lot* more *difficult-to-get-consensus- compatible* code that we are all (or at least mostly) very unfamiliar with. Research is fine, but let's be realistic about deployment. Luke From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 14E53258 for ; Thu, 29 Oct 2015 08:17:28 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-io0-f174.google.com (mail-io0-f174.google.com [209.85.223.174]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9FD35DF for ; Thu, 29 Oct 2015 08:17:27 +0000 (UTC) Received: by iofz202 with SMTP id z202so37513295iof.2 for ; Thu, 29 Oct 2015 01:17:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=vBj9vLBozmBy0zz4eVHJ3cVSnj1AfJe+O0G+yHbCGaM=; b=qQMhUNpL6ancR52ihOB55vH5NpQNRw63bhLxLpnjo0lmLkdYAmrV113Zc0VGWjdgat Y5fdNUJiIS6/viQvwpDQRXYMEEvEOdLalFWX+NupkMbujKTZ9rcKO7vlUa1my/MSDMWk i3bW7s5Cq5GdQ4BhaHZy1+VEgBppsfkh+7IX4Q4OvLFfVqFmFrIaBKfKIrar1hr3mIFX Hw1eY8LZnuz4NL6DiIfMYsaLWEspowrs74d3PHUQaGlNGRNT6V3ImVg2jF6zhFKIhRKe 6qCqpefXE62CQmUaDaUtfiMG5ZAsEbN58vr0MW5rlY9hNrn29S7DngGiifTdmpyHTBtR l2Yg== MIME-Version: 1.0 X-Received: by 10.107.25.199 with SMTP id 190mr1891580ioz.134.1446106647103; Thu, 29 Oct 2015 01:17:27 -0700 (PDT) Received: by 10.107.192.199 with HTTP; Thu, 29 Oct 2015 01:17:27 -0700 (PDT) In-Reply-To: <5631C363.5060705@neomailbox.net> References: <5631C363.5060705@neomailbox.net> Date: Thu, 29 Oct 2015 08:17:27 +0000 Message-ID: From: Gregory Maxwell To: telemaco Content-Type: text/plain; charset=UTF-8 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Oct 2015 08:17:28 -0000 On Thu, Oct 29, 2015 at 6:57 AM, telemaco via bitcoin-dev wrote: > Why not "outsource" totally that data management part to the already > existing with decades of experience database world. People would be able to > create incredibly easy bitcoin statistics/graphs/analisys with existing > software packages (hey even excel or libreoffice like) or connect bitcoin > data to their own sources and if so they chose analyze bitcoin data on a > datawarehouse or any imaginable approach. Of course every transaction would > be have to do through the bitcoin node and only the data management would be > on rdbms side. The word "database" is likely confusing people here. This is not a database in an ordinary sense. The bitcoin core consensus engine requires a highly optimized ultra compact data structure to perform the lookups for coin existence. The data stored is highly compressed and very specialized, it would not be useful to other applications. Right now, on boring laptop hardware, during network synchronization updates to this database run at over 10,000 records per second, while the system is also busy doing the other validation chores of a node. This is backended by a high performance transactional key value store. The need for performance here is essential to even keeping up with the network, it's not about enabling any kind of fancy querying (bitcoin core does not offer fancy querying), it's about the base load that every node must handle to usably sync up and keep up with the Bitcoin network. The backend can be swapped out for something else that provides the same properties, but doing so does not give you any of the inspection/analytics that you're looking for. Systems that do that exist, and they require databases taking hundreds of gigabytes of storage and take days to weeks to import the network data. They're great for what they're for, but they're not suitable for consensus use in the system for space efficiency, performance, and consensus consistency reasons. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 73E4967 for ; Fri, 30 Oct 2015 03:04:49 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pa0-f47.google.com (mail-pa0-f47.google.com [209.85.220.47]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 02CC9109 for ; Fri, 30 Oct 2015 03:04:48 +0000 (UTC) Received: by pacfv9 with SMTP id fv9so61794072pac.3 for ; Thu, 29 Oct 2015 20:04:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bitcartel_com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=svWVhKcpm9c5UmI7UFLkTppIOysK3orEmrO6+ObIQK0=; b=EJKXAQHcWzwQpr9eFc9Fa57hVKcV1L+Jva/JDv8+DtbOSYzCGtR83pNCZRrOohKN0a l7M/sdmHeuGlEqNLeztI6TZj7ZfY0RyL8eqJelFR7U5dBj2bhO17W45Z1H2Y9zFNgUcB ouLy6d409DjLHmxAQw0mq3nyE60HEK30Z2/c2vfaZUiyxgjdS970E8gitsdLinjW0sQj 2svLMJhtCHLX1kwfNp3QrojIg/ETC+VWmlFP59/8hSX4Rs+jUvSVYEfvObVgxwkZ6OLr oJdCm/5acHcRz2i65ra0nfCvdNzyCGZVxxsp242gahGBHva5GTHpTUXGmZ1vgLppPGDS uXPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-type :content-transfer-encoding; bh=svWVhKcpm9c5UmI7UFLkTppIOysK3orEmrO6+ObIQK0=; b=RSbnYqNM5JvI2qk2RkLY7x4yCj5A1IIrcEZv/BNRb9enjRJdkJJO0+wkCddBiw4dmz vDEXK234bFVUvpJbdRih1EGKqzrh0fEgUMorZ87Of/l9Krq4vVHppb9Hp0W1Bz9eEMj5 d5yz1UvU6T1AfLgHCWIFBV0WKkPXL51J6lNJTOFxA0rV2hXs6K6KWL3Y/Hy9oHhH0wXb Zct+cdQwM3Beh9AmKwbiHoafvk+N+/vkjZHjv4Ehmpg1piNEZ1oBC3ZjoB7AjMidKVDG 7HBn5a7ptIswMor1e+q0XjrYU4xz707P3JSvQWefE+4MibUelsbRF4fP8TjnlOggadq8 69IQ== X-Gm-Message-State: ALoCoQkSyOM/nEPMdGfM2ADRIAO9uFsLuWE0CnefMCuOnK2MW6GyfYPLxdbg88FdfDRh2scj6lS0 X-Received: by 10.68.254.137 with SMTP id ai9mr5751515pbd.68.1446174288680; Thu, 29 Oct 2015 20:04:48 -0700 (PDT) Received: from [192.168.2.8] (c-50-161-101-12.hsd1.ca.comcast.net. [50.161.101.12]) by smtp.googlemail.com with ESMTPSA id xm9sm4925176pbc.32.2015.10.29.20.04.20 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 29 Oct 2015 20:04:27 -0700 (PDT) To: Luke Dashjr , telemaco References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> From: Simon Liu Message-ID: <5632DE33.7030600@bitcartel.com> Date: Thu, 29 Oct 2015 20:04:19 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <201510290803.52734.luke@dashjr.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Fri, 30 Oct 2015 03:29:33 +0000 Cc: bitcoin-dev@lists.linuxfoundation.org Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Oct 2015 03:04:49 -0000 Storage of UTXO data looks like an implementation detail and thus one would have thought that the choice of database would not increase the odds of consensus protocol failure. Btcd, a full node implementation written in Go, already provides a database interface which supports different backends: https://github.com/btcsuite/btcd/tree/master/database Given that UTXO storage is considered critical, it seems reasonable to let a node operator decide for themselves if they want data stored in LevelDB (which is not fully ACID compliant) or a database like Sqlite, Oracle, DB2 etc. If the storage requirements for UTXO data are fairly simple, consisting mainly of puts and gets, there is a decent argument that using a dedicated key-value store provides superior performance over a traditional SQL database. However, from a practical perspective, given that nodes operate on a range of different hardware and even a little Raspberry Pi can run a full node and keep up with the network, why not let those users with the resources to operate big iron databases do so? It would be a good feature to have. On 10/29/2015 01:03 AM, Luke Dashjr via bitcoin-dev wrote: > I predict this would be a disaster. UTXO storage is CONSENSUS-CRITICAL code. > Any divergence in implementation behaviour, including bugs AND bugfixes, may > cause consensus failure. For this to have a reasonable *hope* of working, we > need to choose one storage engine, and *will* need to maintain consensus- > compatibility of it ourselves (since nobody else cares). > > Fixing LevelDB frankly seems like an easier task than switching to anything > SQL-based, which would require a *lot* more *difficult-to-get-consensus- > compatible* code that we are all (or at least mostly) very unfamiliar with. > > Research is fine, but let's be realistic about deployment. > > Luke > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id EB33B721 for ; Fri, 30 Oct 2015 03:35:34 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ig0-f182.google.com (mail-ig0-f182.google.com [209.85.213.182]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id E4D4F128 for ; Fri, 30 Oct 2015 03:35:33 +0000 (UTC) Received: by igbdj2 with SMTP id dj2so2989085igb.1 for ; Thu, 29 Oct 2015 20:35:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=iNL04aM5c2HDk65vKgm/f6p6ajqszT9keLJqHG4T6ho=; b=1Jx2N+7WaHFDvNnHrVcBsI4LRusscfckpMYVtkEGua2y/XZ9IAlZBfHMHwv+SWkmmw bGif54nZn+LxZdZNK/bXNaLREicTkeiP7iI9+kfrGleGxWHeBcXXECa59b6mv6bXT/pl JRUTJ9aXsVn2x78Obw/jhSwWlDtj6dxpa/haec4x9PQWkmKCdxvbuWaABOCJr0GF4V31 +lAU06iMnHyO9qN8JnDVCMbtlcDeBgqcym2wbH8KhIu9xQ4bF/rjQQWbMGf557gRAKW/ mqnAuCtZULvzFGHsq2nphmtIxK3NKGiwKuVKKFElKKLFJ5Eiv0TZCow1wa9S4Qmqzqn1 bRxA== MIME-Version: 1.0 X-Received: by 10.50.27.10 with SMTP id p10mr715575igg.66.1446176133409; Thu, 29 Oct 2015 20:35:33 -0700 (PDT) Received: by 10.107.192.199 with HTTP; Thu, 29 Oct 2015 20:35:33 -0700 (PDT) In-Reply-To: <5632DE33.7030600@bitcartel.com> References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> <5632DE33.7030600@bitcartel.com> Date: Fri, 30 Oct 2015 03:35:33 +0000 Message-ID: From: Gregory Maxwell To: Simon Liu Content-Type: text/plain; charset=UTF-8 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev , telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Oct 2015 03:35:35 -0000 On Fri, Oct 30, 2015 at 3:04 AM, Simon Liu via bitcoin-dev wrote: > Given that UTXO storage is considered critical, it seems reasonable to This sounds like a misunderstanding of what consensus criticial means. It does not mean that it must be right (though obviously that is preferable) but that it must be _consistent_, between all nodes. > full node and keep up with the network, why not let those users with the > resources to operate big iron databases do so? It would be a good > feature to have. Because it provides no value, the data is opaque and propritarily encoded with a compression function which we may change from version to version, and because many of these alternatives are enormously slow; enough that they present problems with falling behind the network even on high performance hardware. Moreover, additional functional which will not be sufficiently used will not adequately maintained and result in increased maintains costs and more bugs. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 4EBDD25A for ; Fri, 30 Oct 2015 04:28:48 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ig0-f173.google.com (mail-ig0-f173.google.com [209.85.213.173]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 9EC1212A for ; Fri, 30 Oct 2015 04:28:47 +0000 (UTC) Received: by igbhv6 with SMTP id hv6so3604513igb.0 for ; Thu, 29 Oct 2015 21:28:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=72FH0ykTMrej2HRR1BUp919wGQ23XrwJdF9+7PWz7Lw=; b=lVS8LTNCteUOrialmITRJKZbnzHW99lJ6ZJNhou0rDl5p+2dlgkBrBBRCld4BaKUdI 86/JMg0tw7WW9cwffVD0PzrpmKt9BuNhzJsFrGyUYSHeh+azc3SOzrKjc/gDO6BTEiVm GwFNv4VaY3FcAfHnVF1b4icE6ULojnfK4ujK+nYB7RTyBRRnDrQ77ysyqjIalcHmpmE6 kK1AavIMsMb4fzGArPHi7V0iPUAegMDMON+wl8Hw3fP04atEr+OFAa5tC13rsK79u25y tBkQ3j193dCXIc4jh732recbwmhju/8g18IbCivwSOdYqvs1JTfLlSlL1bg06+sMXff4 xhrQ== MIME-Version: 1.0 X-Received: by 10.50.43.200 with SMTP id y8mr819646igl.48.1446179327142; Thu, 29 Oct 2015 21:28:47 -0700 (PDT) Received: by 10.107.192.199 with HTTP; Thu, 29 Oct 2015 21:28:47 -0700 (PDT) In-Reply-To: <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com> References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> <5632DE33.7030600@bitcartel.com> <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com> Date: Fri, 30 Oct 2015 04:28:47 +0000 Message-ID: From: Gregory Maxwell To: Peter R Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev , telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Oct 2015 04:28:48 -0000 On Fri, Oct 30, 2015 at 4:04 AM, Peter R wrote: > Can you give a specific example of how nodes that used different database= technologies might determine different answers to whether a given transact= ion is valid or invalid? I=E2=80=99m not a database expert, but to me it w= ould seem that if all the unspent outputs can be found in the database, and= if the relevant information about each output can be retrieved without cor= ruption, then that=E2=80=99s all that really matters as far as the database= is concerned. If you add to those set of assumptions the handling of write ordering is the same (e.g. multiple updates in an change end up with the same entry surviving) and read/write interleave returning the same results then it wouldn't. But databases sometimes have errors which cause them to fail to return records, or to return stale data. And if those exist consistency must be maintained; and "fixing" the bug can cause a divergence in consensus state that could open users up to theft. Case in point, prior to leveldb's use in Bitcoin Core it had a bug that, under rare conditions, could cause it to consistently return not found on records that were really there (I'm running from memory so I don't recall the specific cause). Leveldb fixed this serious bug in a minor update. But deploying a fix like this in an uncontrolled manner in the bitcoin network would potentially cause a fork in the consensus state; so any such fix would need to be rolled out in an orderly manner. > I=E2=80=99d like a concrete example to help me understand why more than o= ne implementation of something like the UTXO database would be unreasonable= . It's not unreasonable, but great care is required around the specifics. Bitcoin consensus implements a mathematical function that defines the operation of the system and above all else all systems must agree (or else the state can diverge and permit double-spends); if you could prove that a component behaves identically under all inputs to another function then it can be replaced without concern but this is something that cannot be done generally for all software, and proving equivalence even in special cases it is an open area of research. The case where the software itself is identical or nearly so is much easier to gain confidence in the equivalence of a change through testing and review. With that cost in mind one must then consider the other side of the equation-- utxo database is an opaque compressed representation, several of the posts here have been about desirability of blockchain analysis interfaces, and I agree they're sometimes desirable but access to the consensus utxo database is not helpful for that. Similarly, other things suggested are so phenomenally slow that it's unlikely that a node would catch up and stay synced even on powerful hardware. Regardless, in Bitcoin core the storage engine for this is fully internally abstracted and so it is relatively straight forward for someone to drop something else in to experiment with; whatever the motivation. I think people are falling into a trap of thinking "It's a , I know a for that!"; but the application and needs are very specialized here; no less than, say-- the table of pre-computed EC points used for signing in the ECDSA application. It just so happens that on the back of the very bitcoin specific cryptographic consensus algorithim there was a slot where a pre-existing high performance key-value store fit; and so we're using one and saving ourselves some effort. If, in the future, Bitcoin Core adopts a merkelized commitment for the UTXO it would probably need to stop using any off-the-shelf key value store entirely, in order to avoid a 20+ fold write inflation from updating hash tree paths (And Bram Cohen has been working on just such a thing, in fact). From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id C9FA58A5 for ; Fri, 30 Oct 2015 04:04:37 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 59711E4 for ; Fri, 30 Oct 2015 04:04:37 +0000 (UTC) Received: from [192.168.1.71] ([50.92.106.24]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0Lrw2c-1acVgC3I7C-013fek; Fri, 30 Oct 2015 05:04:27 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) From: Peter R In-Reply-To: Date: Thu, 29 Oct 2015 21:04:22 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com> References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> <5632DE33.7030600@bitcartel.com> To: Gregory Maxwell X-Mailer: Apple Mail (2.2104) X-Provags-ID: V03:K0:bL0pNKynBfzMu1ZYMXOvxXWyKzeGX2iyENRRDDN9HbCBkzT9ZIY UGM4IaFSVioIYa25fr9amaIG8vkkdQOjJyljhSgosL2qlrYfG7se8lOgV6r05h+IJR3Oh6T i8uH8ckVEsjm/iFs0V+XV5XkVU7lhkKVQ4RDbuMPvxaIH3/pyWHOZ3OJQOoFKU8Ixv133kS hdgQyQ7Tvq78zQK8SbTlg== X-UI-Out-Filterresults: notjunk:1;V01:K0:UxJ/NYiPa3c=:DudwNrF/0rmaLKtyCAv57q +ROqPIkfy4/PIZBnaVR1VkV/FfjKj6l/T1/7KAwgMOWgmdvnABw+RAoKh9Cf1dgSHvl1CATea 5rE7hAcB9kpiIs/fgYvlI5WLfrGNBgtKRZ1MqOvDA9CF5x5xSRtTq27BozwkaNddZZXvqLpFH EBz+x5pmz3ZGHnmY662xpFuSkHTuLpP1bOvLKA9Ualf8g5WQ8ktVYPmmUUy46OpQupPpJZ17H uHbWZU3l9v2CpyLQmSe40KyZ7QsktXLuqEYZxRnahTXy1/QP6rGrsGS4Rrv10RBPgMs0ZdTKJ h3l0sw2ZoSroM7BnamDLH56EByQHjsuvSujXEbkQTGG+eRtO7Z+wpFpHYuY/mu9LlPuUe4tEu kpAReyEqkU0tWmFXhqPQren83U2zQ/rZYOPO9E54Jd2n2I8kFfXqXudJVistqAUH5AdOPXWoo lkCjoZKDtfY5I8wqFvZbv6/YbCEKT0JfaBC3KQ1KhKLb3Uwpq+JsG+O/BEsqMzSUHU9efJDGM nfNtttG9vjYNiup8NkRlf1m3+tajahcbAhkHkPmn/8V0AA8Fy0P+5ndB74YSDjfNdMe87d+bE yXV+eIfQHNFYuo3gh8wE582oeR6N5XABFK8kQ9yXPfGHRnZzNHn/qsDN3VTWrn0hMcTkszh6z 5Tz+xkfRSF0xt9f2KSet7VE1vejFA1xZrzu9F13iafz0mJ5Fauz6Vv8afRf0CFPiRXhJh8qlL rOWxZrJ/OzXHToWt+NzwK+h2KLvKFIxoVHLoW7sctOMW2vTOU5Nkz5h+TQrlh4CwKllAvBnM7 TwLTbsz X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Fri, 30 Oct 2015 05:30:24 +0000 Cc: Bitcoin Dev , telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Oct 2015 04:04:37 -0000 > On Oct 29, 2015, at 8:35 PM, Gregory Maxwell via bitcoin-dev = wrote: >=20 > On Fri, Oct 30, 2015 at 3:04 AM, Simon Liu via bitcoin-dev > wrote: >> Given that UTXO storage is considered critical, it seems reasonable = to >=20 > This sounds like a misunderstanding of what consensus criticial means. > It does not mean that it must be right (though obviously that is > preferable) but that it must be _consistent_, between all nodes. Can you give a specific example of how nodes that used different = database technologies might determine different answers to whether a = given transaction is valid or invalid? I=E2=80=99m not a database = expert, but to me it would seem that if all the unspent outputs can be = found in the database, and if the relevant information about each output = can be retrieved without corruption, then that=E2=80=99s all that really = matters as far as the database is concerned. Let=E2=80=99s use an unspent pay-to-pubkey-hash output as an example: = Alice spends this to Bob (she signs it properly), the TX propagates = across the network and=E2=80=A6then what? Do some nodes disagree on = whether or not the TX is valid? What exactly would they disagree on? = Are you suggesting that a database bug would cause some nodes to think = the output was actually already spent, while others can correctly see = that it=E2=80=99s unspent? Or maybe some nodes think the output = doesn=E2=80=99t exist while others do? Or are you suggesting that the = details about this output might be retrieved with errors from certain = databases but correctly from others? =20 I=E2=80=99d like a concrete example to help me understand why more than = one implementation of something like the UTXO database would be = unreasonable. =20 Peter From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 35BAB8EE for ; Sun, 15 Nov 2015 01:02:46 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mout.gmx.net (mout.gmx.net [212.227.17.22]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 8D12390 for ; Sun, 15 Nov 2015 01:02:44 +0000 (UTC) Received: from [192.168.50.29] ([69.50.179.106]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0LqQTl-1aaw1N3P5F-00e3Sr; Sun, 15 Nov 2015 02:02:37 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) From: Peter R In-Reply-To: Date: Sat, 14 Nov 2015 17:02:33 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <571D9B7F-077D-4B80-B577-1C18FF2ECF31@gmx.com> References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> <5632DE33.7030600@bitcartel.com> <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com> To: Gregory Maxwell X-Mailer: Apple Mail (2.2104) Sender: Peter_R@gmx.com X-Provags-ID: V03:K0:hJOP4A6k42YNTcRTX5IZ6HaS2pH4pUtL7PvuQOYXJ5bYf41cauH 3cslW3aN40fJ1WoNxrA1C4NqcTr35xZfxvPfLHZxtm3hI7hPYjvh2fprk6TGOXGvFlalQ6V x58hKz0Bjg9Q37n+ZEhVPTxXqwpW044vrZAiUshw6HQYzF669+zvq4Pn+16WhoqVhZdvDvF 0bKxupFj0yLNpGS8ZZyxw== X-UI-Out-Filterresults: notjunk:1;V01:K0:m5oPBZ0c7f0=:4gwraTYQLajA2hJDobBcFt jLc3g2CnW9uBNW11Ie/7+cu9IcVuBF3tIGcNVyd6y7M/1PIPQOKUIerdnIkuuZZx+yPHBH57I 1m6XTJd7Zfnw/0lUigS+XI4CVcrxNlKPXVwGzb7FmkqM8A9NlWDT3p4UpiKGLMvvcZxia7baa pZB7c5lQ5d1vN/hTHysOaHsoh1lBDLCw77Egkjlc/VP5eVyqG9KASN+XyNjNIRgWkziBJfNfE CwZUmD7+8iFZlTFjC71furR7mcGvRciYGSqOeIFO1yh3BKDGZZTnBhuEWuz4D32UaAvd7iQLI 4BfmwpBqzI6qHRk2xyEdx2p691p9fexZPUzzyuM2Eg4C2qxTN1iVdoApBWttCtJ6bJ9Ni5UAy /jk71wWk8lR6ZK9s9c0T5lOyt38ZxVWFm7y0WGprzxF02Pck/9erhlCg3vLiD+evwtLOir49i y9U1eGwTLaM7FuB6hDoO+2OOYZUwbz7UOqd8SW6lDNs6RJla8dMt4im9Y1QzuAESMa4JFULaX EFeEajmweZ6wCrqedduRVCdV2pvQJ12/bdpuGUNNvpWFJoTPpuVxLHsb/kryD8l5LClulzXt5 ouaq0VPUFeUnytN9StdKSR/JtT2leY51q5p3xdMahq2k1X6251MuPe2Vyrysu3sbMBKQQM8zV NLytqbpTxTRydZ0XH5/foTvofbESVY4Ir6ihviNxjnB/ZnHABV9J+X0oxvitLxvwp63QtWWEW h/dXyYLv/Sj6vGR4gIycgtyh3NMz7zEdK+FpfRDa8IZBJwkCz8GztXOuDO3jg5do6auYvXJXh GbaFlMg X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev , telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2015 01:02:46 -0000 Hi Greg, Like you said, the issue with using more than one database technology is = not that one node would prove that Block X is valid while the another = node proves that Block X is NOT valid. Instead, the problem is that one = node might say =E2=80=9Cvalid=E2=80=9D while the other node says =E2=80=9C= I don=E2=80=99t know.=E2=80=9D It is often said that what caused the Level DB fork was that the old = version determined that the triggering block was invalid while the new = version determined the opposite. This interpretation of the fork event = has lead to the =E2=80=9Cbug-for-bug=E2=80=9D-compatibility fear = regarding multiple implementations of the protocol (or multiple database = technologies). In reality, this fear is largely unfounded. =20 The old version ran out of LDB locks and couldn=E2=80=99t execute = properly. If the software was written with the philosophy that tracking = consensus was more important than bug-for-bug compatibility, it would = have returned an exception rather than =E2=80=9Cinvalid.=E2=80=9D At = that point, the software would have checked what decision the rest of = the network came to about the block in question. My node would have = forked to the longest chain, automatically assuming that the = questionable block was valid; your node may have made a different = decision (it=E2=80=99s a question of ideology at that point). =20 A group of us have been exploring this =E2=80=9Cmeta-cognition=E2=80=9D = idea with Bitcoin Unlimited. For example, Bitcoin Unlimited can be = (optionally) made to automatically fork to the longest chain if it = =E2=80=9Cgets stuck=E2=80=9D and can neither prove that a block is valid = nor that the block is invalid. Similarly, Bitcoin Unlimited can be = (optionally) made to automatically fork to a chain that contains a block = marginally bigger than its block size limit=E2=80=94once that block is = buried sufficiently in what has emerged as the longest persistent chain.=20= Thinking from this perspective might go along way towards decentralizing = development, the emergence of multiple competing implementations of the = protocol, and true =E2=80=9Cbottom up=E2=80=9D governance. =20 Best regards, Peter > On Oct 29, 2015, at 9:28 PM, Gregory Maxwell = wrote: >=20 > On Fri, Oct 30, 2015 at 4:04 AM, Peter R wrote: >> Can you give a specific example of how nodes that used different = database technologies might determine different answers to whether a = given transaction is valid or invalid? I=E2=80=99m not a database = expert, but to me it would seem that if all the unspent outputs can be = found in the database, and if the relevant information about each output = can be retrieved without corruption, then that=E2=80=99s all that really = matters as far as the database is concerned. >=20 > If you add to those set of assumptions the handling of write ordering > is the same (e.g. multiple updates in an change end up with the same > entry surviving) and read/write interleave returning the same results > then it wouldn't. >=20 > But databases sometimes have errors which cause them to fail to return > records, or to return stale data. And if those exist consistency must > be maintained; and "fixing" the bug can cause a divergence in > consensus state that could open users up to theft. >=20 > Case in point, prior to leveldb's use in Bitcoin Core it had a bug > that, under rare conditions, could cause it to consistently return not > found on records that were really there (I'm running from memory so I > don't recall the specific cause). Leveldb fixed this serious bug in a > minor update. But deploying a fix like this in an uncontrolled manner > in the bitcoin network would potentially cause a fork in the consensus > state; so any such fix would need to be rolled out in an orderly > manner. >=20 >> I=E2=80=99d like a concrete example to help me understand why more = than one implementation of something like the UTXO database would be = unreasonable. >=20 > It's not unreasonable, but great care is required around the = specifics. >=20 > Bitcoin consensus implements a mathematical function that defines the > operation of the system and above all else all systems must agree (or > else the state can diverge and permit double-spends); if you could > prove that a component behaves identically under all inputs to another > function then it can be replaced without concern but this is something > that cannot be done generally for all software, and proving > equivalence even in special cases it is an open area of research. The > case where the software itself is identical or nearly so is much > easier to gain confidence in the equivalence of a change through > testing and review. >=20 > With that cost in mind one must then consider the other side of the > equation-- utxo database is an opaque compressed representation, > several of the posts here have been about desirability of blockchain > analysis interfaces, and I agree they're sometimes desirable but > access to the consensus utxo database is not helpful for that. > Similarly, other things suggested are so phenomenally slow that it's > unlikely that a node would catch up and stay synced even on powerful > hardware. Regardless, in Bitcoin core the storage engine for this is > fully internally abstracted and so it is relatively straight forward > for someone to drop something else in to experiment with; whatever the > motivation. >=20 > I think people are falling into a trap of thinking "It's a , > I know a for that!"; but the application and needs are > very specialized here; no less than, say-- the table of pre-computed > EC points used for signing in the ECDSA application. It just so > happens that on the back of the very bitcoin specific cryptographic > consensus algorithim there was a slot where a pre-existing high > performance key-value store fit; and so we're using one and saving > ourselves some effort. If, in the future, Bitcoin Core adopts a > merkelized commitment for the UTXO it would probably need to stop > using any off-the-shelf key value store entirely, in order to avoid a > 20+ fold write inflation from updating hash tree paths (And Bram Cohen > has been working on just such a thing, in fact). From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 24DCC8F5 for ; Sun, 15 Nov 2015 01:08:18 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-ig0-f173.google.com (mail-ig0-f173.google.com [209.85.213.173]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 22C2EA8 for ; Sun, 15 Nov 2015 01:08:17 +0000 (UTC) Received: by igvi2 with SMTP id i2so57246162igv.0 for ; Sat, 14 Nov 2015 17:08:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=50hNF9bDJ0dN5tuQj2AJYDIx6Sqm3DO7Ief8oGeZC1c=; b=H8aSQ9nTWa0wjSRewRo0cMYovPIE//F2IORehsSXd9HwHgHogDnkPJlDrgVo8Xd1U7 9X/u6djLYsNK2vGqgS6GbYBilocS8yWUoqSwxs7a378IJ6Va2IQK7Ij/q4MB1FwbouRC Hyr2MXW4k2DW/k1q+9WI0IZkn8G+0Z316D3iCphbG7zCcehuTBxolcQflAY1wRUzAKfM 2PiN2d/lA56tq/np8w4P5hZ4+UIzcdYbbE93yAsiDiEz5I82+U5hHdlmscix760ccFTy ujvzB9St6Kbl8w4xlxfITawc0cr0lRjaFD9JuUJuW6TypkdDI9y0sfIcD7VtfJLgmLDS xAlw== MIME-Version: 1.0 X-Received: by 10.50.12.42 with SMTP id v10mr667985igb.48.1447549696604; Sat, 14 Nov 2015 17:08:16 -0800 (PST) Received: by 10.107.192.199 with HTTP; Sat, 14 Nov 2015 17:08:16 -0800 (PST) In-Reply-To: <571D9B7F-077D-4B80-B577-1C18FF2ECF31@gmx.com> References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> <5632DE33.7030600@bitcartel.com> <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com> <571D9B7F-077D-4B80-B577-1C18FF2ECF31@gmx.com> Date: Sun, 15 Nov 2015 01:08:16 +0000 Message-ID: From: Gregory Maxwell To: Peter R Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev , telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2015 01:08:18 -0000 On Sun, Nov 15, 2015 at 1:02 AM, Peter R wrote: > Hi Greg, > > Like you said, the issue with using more than one database technology is = not that one node would prove that Block X is valid while the another node = proves that Block X is NOT valid. Instead, the problem is that one node mi= ght say =E2=80=9Cvalid=E2=80=9D while the other node says =E2=80=9CI don=E2= =80=99t know.=E2=80=9D Sometimes errors are such that you can catch them (if you're super vigilant and know an error is even possible in that case)-- and indeed, in that case you can get a "I don't know, something is wrong.", other times errors are undetectable. > In reality, this fear is largely unfounded. I cited an issue in leveldb (before we used it) where it would silently fail to return records. > If the software was written with the philosophy that tracking consensus w= as more important than bug-for-bug compatibility, it would have returned an= exception rather than =E2=80=9Cinvalid.=E2=80=9D That invariant requires the software to be completely free of errors that would violate it. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B9C968FE for ; Sun, 15 Nov 2015 01:45:14 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mout.gmx.net (mout.gmx.net [212.227.15.15]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id F312AF5 for ; Sun, 15 Nov 2015 01:45:12 +0000 (UTC) Received: from [192.168.50.29] ([69.50.179.106]) by mail.gmx.com (mrgmx001) with ESMTPSA (Nemesis) id 0Lb5nF-1ahXJb0Hgv-00kce5; Sun, 15 Nov 2015 02:45:06 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) From: Peter R In-Reply-To: Date: Sat, 14 Nov 2015 17:45:03 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <6DAD1D38-A156-4507-B506-BF66F26E6594@gmx.com> References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> <5632DE33.7030600@bitcartel.com> <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com> <571D9B7F-077D-4B80-B577-1C18FF2ECF31@gmx.com> To: Gregory Maxwell X-Mailer: Apple Mail (2.2104) Sender: Peter_R@gmx.com X-Provags-ID: V03:K0:OK7BbcnKkct5SXvG7/nvqFVg/+Ix4JdECyfYebh2NQH3phohCHR mvLgDbw+YiiPwLZKhTgbuC7+mX+F87fa5LKtWKj1kkiIi0EeZNQ4ezMNinxygWhSt1HZBba hlhBXDBfsguUpibNyIPompHf3yOmFqOWO7XJ9CTUjPdLG9dIn4Lil4MUmU8bUikLGaNgu4v tZdAbG5DfB3DMUsy6Ml6w== X-UI-Out-Filterresults: notjunk:1;V01:K0:eqsFxPntEZI=:5PQbrCPjPGcYg1pFSIfA+2 xLI7U33lgLU2ZeZIdy6lCfwk0pdEA6BhLgYTBne+Z/N+U8Ohv8DMY6PeAUSaRYjis8IXcnSEb 1kR4kZR4gEa0rDgsSr2AtF5kLD6cjYNQRUejPfdIZjpFlrwg6CJiDySvbbTrFY5TiliIvZ9ub TiXZOulLd7xB4xX4l4TySQur7JtoKYxAoFn5Dsgh4uA+SHM2jYyBkJFHdPQBkJd7XEhG1QmvJ HPepZiIPffFEurCL9eUA6mUwNRZj7Y4dwLQhNZD3T1kckjt1MDjl99LfGmanZD97HG9aej95Q 7X8PjJum3gTGuzIy6KpzDK9z2LIU5GuHeKsAVQZaRwuWLTAbGP23dlqRWjManBVkipB16wU3v rCnSCP0CGbqOC72d0voE4Emep7TRtsTA8FVQm93e2CekosAq5cGXF12g/9EojRMzCsEh2kf3b Rbf/AjaNrbO8hDimTjAUTv5xLLfYKpCdFouN4mTgbUY9jxs1wJa2kMo1kzo5h9gWhw7osHHyf LoX7slhpvccPaL24/WC+wSs4l9mQBP1UvUmDHKtc74aaJ6i4c0Ji+UvvOmy/u3kP/06IMw4J6 K9uJLAksRWuhLNyzmO5LEj7RR/UUhtfLGG2Otyr4mjbVDSt8HJ0Gx/bGN/UPMvlxOUriK1Tz7 SZMsnnqkNnPLhCPyOWgCZyH1a/Pp1VRlYigP2QHf6tfMNgvdgVaguQiQm3hhJ4sQ6itmcxglW c0lY1ck3QvyK0Gls/tpXmwMRK4NT7/gK3k5pduJuptpMmNkBQB/uUxN/UjHOKD7XG3+PIQdcs HNanwOc X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, MIME_QP_LONG_LINE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev , telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2015 01:45:14 -0000 > On Nov 14, 2015, at 5:08 PM, Gregory Maxwell = wrote: >=20 > On Sun, Nov 15, 2015 at 1:02 AM, Peter R wrote: >> Hi Greg, >>=20 >> Like you said, the issue with using more than one database technology = is not that one node would prove that Block X is valid while the another = node proves that Block X is NOT valid. Instead, the problem is that one = node might say =E2=80=9Cvalid=E2=80=9D while the other node says =E2=80=9C= I don=E2=80=99t know.=E2=80=9D >=20 > Sometimes errors are such that you can catch them (if you're super > vigilant and know an error is even possible in that case)-- and > indeed, in that case you can get a "I don't know, something is > wrong.", other times errors are undetectable. Agreed. There are two cases to consider: Type 1. One implementation says =E2=80=9Cyes=E2=80=9D or =E2=80=9Cno,=E2=80= =9D while the other says =E2=80=9CI don=E2=80=99t know=E2=80=9D, and Type 2. One implementation says =E2=80=9Cyes=E2=80=9D and the other = says =E2=80=9Cno,=E2=80=9D because of a bug. =20 My previous email described how Type 1 consensus failures can be safely = dealt with. These include many kinds of database exceptions (e.g., the = LevelDB fork at block #225,430), or consensus mismatches regarding the = max size of a block. =20 Type 2 consensus failures are more severe but also less likely (I=E2=80=99= m not aware of a Type 2 consensus failure besides the 92 million bitcoin = bug from August 2010). If Core was to accept a rogue TX that created = another 92 million bitcoins, I think it would be a good thing if the = other implementations forked away from it (we don=E2=80=99t want = bug-for-bug compatibility here). =20 This once again reveals the benefits of multiple competing = implementations. =20 Sincerely, Peter=20 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id EC35494A for ; Sun, 15 Nov 2015 02:10:44 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-io0-f179.google.com (mail-io0-f179.google.com [209.85.223.179]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 53DDCCB for ; Sun, 15 Nov 2015 02:10:44 +0000 (UTC) Received: by iouu10 with SMTP id u10so124070333iou.0 for ; Sat, 14 Nov 2015 18:10:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=2BbPlE1Z44QoqqLQ5dCubzn1V3wPabvEFsB88bQeTCM=; b=BAC57n+U1x5QzBMZ6uGmcHbSE66RpMPmD7istVVh2iKLYp4aA3AMlRD/HNg+Pu8Wbf FUgUjlwhwHVFIiMWinN0ndG5F71iUIjKPjwMmfaOZvt6C/wsIEX6b1S3iFDDloDtMbue jLBxOUxC6zwHnCTiKvpXekrAzeul9A4CNBxdyNewOjKbjauwAcBP8ID4CN7dEvyA/YnL 3ISIbDF9G9v+6KMRZ1ACXHytFX17nMjtSSmNK1qZViWrA1B+iO3cHB7UZ0RZz12bjRwq tGaatCxiXj6/FuHuBU7eHX8Fts/zZvaRQchsaN7TGAEvJnoKi4As7oEQ6tVwHgFF7qBy BQWw== MIME-Version: 1.0 X-Received: by 10.107.4.83 with SMTP id 80mr28143033ioe.150.1447553443710; Sat, 14 Nov 2015 18:10:43 -0800 (PST) Received: by 10.107.192.199 with HTTP; Sat, 14 Nov 2015 18:10:43 -0800 (PST) In-Reply-To: <6DAD1D38-A156-4507-B506-BF66F26E6594@gmx.com> References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> <5632DE33.7030600@bitcartel.com> <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com> <571D9B7F-077D-4B80-B577-1C18FF2ECF31@gmx.com> <6DAD1D38-A156-4507-B506-BF66F26E6594@gmx.com> Date: Sun, 15 Nov 2015 02:10:43 +0000 Message-ID: From: Gregory Maxwell To: Peter R Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev , telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2015 02:10:45 -0000 On Sun, Nov 15, 2015 at 1:45 AM, Peter R wrote: > My previous email described how Type 1 consensus failures can be safely d= ealt with. These include many kinds of database exceptions (e.g., the Leve= lDB fork at block #225,430), or consensus mismatches regarding the max size= of a block. The size of a block is not a type 1 failure. There is a clear, known, unambiguous, and easily measured rule in the system that a block is not permitted to have a size over a threshold. A block that violates this threshold is invalid. The only way I can see to classify that as a type 1 failure is to call the violation of any known system rule a type 1 failure. "Spends a coin that was already spent" "provides a signature that doesn't verify with the pubkey" "creates bitcoins out of thin air" -- these are not logically different than "if size>x return false". > Type 2 consensus failures are more severe but also less likely (I=E2=80= =99m not aware of a Type 2 consensus failure besides the 92 million bitcoin= bug from August 2010). If Core was to accept a rogue TX that created anot= her 92 million bitcoins, I think it would be a good thing if the other impl= ementations forked away from it (we don=E2=80=99t want bug-for-bug compatib= ility here). The only consensus consistency error we've ever that I would have classified as potentially type 1 had has been the BDB locks issue. Every other one, including the most recently fixed one (eliminated by BIP66) was a type 2, by your description. They are _much_ more common; because if the author understood the possible cases well enough to create an "I don't know" case, they could have gone one step further and fully defined it in a consistent way. The fact that an inconsistency was possible was due to a lack of complete knowledge. Even in the BDB locks case, I don't know that the type 1 distinction completely applies, a lower layer piece of software threw an error that higher layer software didn't know was possible, and so it interpreted "I don't know" as "no"-- it's not that it chose to treat I don't know as no, its that it wasn't authored with the possibility of I don't know in mind, and that exceptions were used to handle general failures (which should have been treated as no). Once you step back to the boundary of the calling code (much less the whole application) the "I don't know" doesn't exist anymore; there. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D873D941 for ; Sun, 15 Nov 2015 02:59:01 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 6513111C for ; Sun, 15 Nov 2015 02:59:00 +0000 (UTC) Received: from [192.168.50.29] ([69.50.179.106]) by mail.gmx.com (mrgmx103) with ESMTPSA (Nemesis) id 0MMCSP-1a30j21Lmn-00810Q; Sun, 15 Nov 2015 03:58:53 +0100 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) From: Peter R In-Reply-To: Date: Sat, 14 Nov 2015 18:58:50 -0800 Content-Transfer-Encoding: quoted-printable Message-Id: <13D7C936-4D2E-4BAC-AC61-3DA80581C946@gmx.com> References: <5631C363.5060705@neomailbox.net> <201510290803.52734.luke@dashjr.org> <5632DE33.7030600@bitcartel.com> <3CB90C47-293E-4C18-A381-E5203483D68F@gmx.com> <571D9B7F-077D-4B80-B577-1C18FF2ECF31@gmx.com> <6DAD1D38-A156-4507-B506-BF66F26E6594@gmx.com> To: Gregory Maxwell X-Mailer: Apple Mail (2.2104) X-Provags-ID: V03:K0:D5UCaQW2otFM6YpDOcn5g8G7Ev4WajM1z1nhJH2HQI9TXQptgLb +c1uMdBmnz4xhwlMZPY0CiAwITexV+a9aAxJty8dQVXvvgTL4xflYJteX+LiYkSgxIXcQHR OkOwtQ9NUIXXAkBAwhshCug6Yhu5gravAGXZjIkScN4ZXmAOdebTlQg1veA6WYpNkM0ZGEo gfheEBfsZrS11tcTbF+kQ== X-UI-Out-Filterresults: notjunk:1;V01:K0:0UYMxGbv7No=:B7Iax/8OEd+4TBG4A6Tfzz XGnaVq8uIOpm8zyqSJ4NErlq9mrcXDzEVztIqrLFHn5zbHIMO5Nd+xslJ3pnDbgeBZ7fMAhem WV5SCKN0cL0sJaFgQYlMT4l//puajnmPa9mkuBSIrv38G0GL5SVcLVYP6fkPCZNM264T9E16R KPdtYcphXiQ/MWA8RJFPZdqg6SOOwHEIyaGG8f3pPL8AdFFDybcHV+EzwisMFP/QQGYLzV+he kF+VafS7OhID4xDRL34yRY7/Qmc/rmBuQTxBbsdD06aVPg/eQVs37EOXq+940YEIuFrnJtzrI ZNXVztAoD9ET+NYvbTE1H9OuOx9PTM8X5V2PEgYTxguYKLJIQ98F5IPhSGV132N1OE6rV93RI jCjYPUvzTyrvz8oj0QBVXOHWtIGDX+x4MMwJJvXv2ZmR3pQ2ZMH11A5pt3UXItQcxZxYiCDxU 7lZL2NSbiqpn5ZGxybQe4ENzsVOFK10M2V8z2NHZaSr9IwakDZrzt6EaEnpnJo4pKbvtpPcNq RkZyfgTTAAEPJoyQZcmqDftH/RBHQ6WqVrDlt7g6lBgE2S3gDqJziKEhmb1udAuDNUF9FRKZw AYPSANaJ/2gxWmg2JskLIdlWS+x9jWDRv/gwWFbS+cOEWUEzXG/l6r7yVYrGa2k3XD+GvWWPe x8GU1HiBN1gSDF38JqeZiuqbc3kwFh4+PshKcJ3EE5+RESm/I8J1C1+mk6TtnbW+LcwlzZmOR y35/JnrtMWckMKPihu3QDdx4oS5YW8wrJKxl5FQy8QUmm/02s9cEWWGzVjQ9OOqYQN3IqkNqk K71GvXA X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev , telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Nov 2015 02:59:02 -0000 > On Nov 14, 2015, at 6:10 PM, Gregory Maxwell = wrote: >=20 > On Sun, Nov 15, 2015 at 1:45 AM, Peter R wrote: >> My previous email described how Type 1 consensus failures can be = safely dealt with. These include many kinds of database exceptions = (e.g., the LevelDB fork at block #225,430), or consensus mismatches = regarding the max size of a block. >=20 > The size of a block is not a type 1 failure. There is a clear, known, > unambiguous, and easily measured rule in the system that a block is > not permitted to have a size over a threshold. A block that violates > this threshold is invalid. =20 You are looking at the problem from a =E2=80=9Ctop down=E2=80=9D = governance perspective assuming you know what code is actually being run = and what rules the market wants. The reality is that you can only make = estimates about these two things. As Bitcoin evolves away from the = current development monoculture, rules such as the max block size may no = longer be perfectly clear. However, we can prove that the following = results will continue to hold: 1. A node with a block size limit greater than the hash-power weighted = median will always follow the longest chain. 2. An excessive (e.g., greater than 1 MB) block will be accepted into = the longest chain if it is smaller than the hash-power weighted median = block size limit. Already today, different nodes have different block size limits. There = are even some miners today that will attempt to build upon blocks larger = than 1 MB if anyone dares to create one. At some point, the majority of = the hash power will support something larger than 1 MB. When that first = bigger block comes and gets buried in the longest chain, hold-out nodes = (e.g., MP says he will never increase his limit) will have to make a = choice: fight consensus or track consensus! I know that I would want my = node to give up on its block size limit and track consensus. You may = decide to make a different choice. =20 You said that "a block that violates this [block size limit] threshold = is invalid.=E2=80=9D I agree. If the nodes and miners rejected the = block as invalid then it would not persist as the longest chain. If the = nodes and miners accepted the block and continued to build on top of it, = then that chain would be Bitcoin (whether you personally agree of not). =20= Bitcoin is ultimately a creature of the market, governed by the code = people freely choose to run. Consensus is then an emergent property, = objectively represented by the longest persistent chain. Proof-of-work = both enforces and defines the rules of the network. =20 > The only way I can see to classify that > as a type 1 failure is to call the violation of any known system rule > a type 1 failure. "Spends a coin that was already spent" "provides a > signature that doesn't verify with the pubkey" "creates bitcoins out > of thin air" -- these are not logically different than "if size>x > return false=E2=80=9D. I think you=E2=80=99re being intentionally obtuse here: accepting a = block composed entirely of valid transactions that is 1.1 MB is entirely = different than accepting a TX that creates a ten thousand bitcoins out = of thin air. The market would love the former but abhor the later. I = believe you can recognize the difference. =20 >> Type 2 consensus failures are more severe but also less likely (I=E2=80= =99m not aware of a Type 2 consensus failure besides the 92 million = bitcoin bug from August 2010). If Core was to accept a rogue TX that = created another 92 million bitcoins, I think it would be a good thing if = the other implementations forked away from it (we don=E2=80=99t want = bug-for-bug compatibility here). >=20 > The only consensus consistency error we've ever that I would have > classified as potentially type 1 had has been the BDB locks issue. Thank you for conceding on that point.=20 > Every other one, including the most recently fixed one (eliminated by > BIP66) was a type 2, by your description. They are _much_ more common; > because if the author understood the possible cases well enough to > create an "I don't know" case, they could have gone one step further > and fully defined it in a consistent way. The fact that an > inconsistency was possible was due to a lack of complete knowledge. >=20 > Even in the BDB locks case, I don't know that the type 1 distinction > completely applies, a lower layer piece of software threw an error > that higher layer software didn't know was possible, and so it > interpreted "I don't know" as "no"-- it's not that it chose to treat I > don't know as no, its that it wasn't authored with the possibility of > I don't know in mind, and that exceptions were used to handle general > failures (which should have been treated as no). Once you step back to > the boundary of the calling code (much less the whole application) the > "I don't know" doesn't exist anymore; there. Please don=E2=80=99t take my comments and observations as criticisms. I = think the Core Dev team has done excellent work! What I am saying = instead is that as we move forward=E2=80=94as development becomes = decentralized and multiple protocol implementations emerge=E2=80=94develop= ment philosophies will change. Tracking consensus and the will of the = market will be most important. Personally, I hope to see design = philosophies that support =E2=80=9Cbottom up=E2=80=9D governance instead = of the current =E2=80=9Ctop down=E2=80=9D model. =20 Best regards, Peter From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id A2531941 for ; Sun, 15 Nov 2015 03:04:47 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from zinan.dashjr.org (zinan.dashjr.org [192.3.11.21]) by smtp1.linuxfoundation.org (Postfix) with ESMTP id 4465911C for ; Sun, 15 Nov 2015 03:04:47 +0000 (UTC) Received: from ishibashi.localnet (unknown [IPv6:2001:470:5:265:61b6:56a6:b03d:28d6]) (Authenticated sender: luke-jr) by zinan.dashjr.org (Postfix) with ESMTPSA id D665038A6FC8; Sun, 15 Nov 2015 03:04:41 +0000 (UTC) X-Hashcash: 1:25:151115:bitcoin-dev@lists.linuxfoundation.org::ie8bK=lfPwUgUokG:cwTH X-Hashcash: 1:25:151115:peter_r@gmx.com::PzFege2V=1Y/kMyl:ber7 X-Hashcash: 1:25:151115:gmaxwell@gmail.com::D24FUIIYLHNvMiEM:DSbx X-Hashcash: 1:25:151115:telemaco@neomailbox.net::6pqpKRZYIsQkk3EP:d52T From: Luke Dashjr To: bitcoin-dev@lists.linuxfoundation.org, Peter R Date: Sun, 15 Nov 2015 03:04:33 +0000 User-Agent: KMail/1.13.7 (Linux/4.1.9-gentoo-r1; KDE/4.14.8; x86_64; ; ) References: <5631C363.5060705@neomailbox.net> <571D9B7F-077D-4B80-B577-1C18FF2ECF31@gmx.com> In-Reply-To: <571D9B7F-077D-4B80-B577-1C18FF2ECF31@gmx.com> X-PGP-Key-Fingerprint: E463 A93F 5F31 17EE DE6C 7316 BD02 9424 21F4 889F X-PGP-Key-ID: BD02942421F4889F X-PGP-Keyserver: hkp://pgp.mit.edu MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201511150304.41003.luke@dashjr.org> X-Spam-Status: No, score=-2.3 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: telemaco Subject: Re: [bitcoin-dev] [patch] Switching Bitcoin Core to sqlite db X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: ,