* [bitcoin-dev] Do we really need a mempool? (for relay nodes) @ 2015-07-18 18:52 Peter Todd 2015-07-18 19:46 ` Patrick Strateman 0 siblings, 1 reply; 4+ messages in thread From: Peter Todd @ 2015-07-18 18:52 UTC (permalink / raw) To: bitcoin-dev [-- Attachment #1: Type: text/plain, Size: 1616 bytes --] As in, do relay nodes need to keep a record of the transactions they've relayed? Strictly speaking, the answer is no: one a tx is relayed modulo DoS concerns the entire thing can be discarded by the node. (unconfirmed txs spending other unconfirmed txs can be handled by creating packages of transactions, evaluated as a whole) To mitigate DoS concerns, we of course have to have some per-UTXO limit on bandwidth relayed, but that task can be accomplished by simply maintaining some kind of per-UTXO record of bandwidth used. For instance if the weighted fee and fee/KB were recorded, and forced to - say - double for each additional tx relayed that spent a given UTXO you would have a clear and simple upper limit of lifetime bandwidth. Equally it's easy to limit bandwidth moment to moment by asking peers for highest fee/KB transactions they advertise first, stopping when our bandwidth limit is reached. You probably could even remove IsStandard() pretty much entirely with the right increasingly expensive "replacement" policy, relying on it alone to provide anti-DoS. Obviously this would simplify some of the debates around mining policy! This could even be re-used for scalable a general-purpose messaging network paid by coin ownership if the UTXO set is split up, and some kind of expiration over time policy is implemented. Miners of course would still want to have a mempool, but that codebase may prove simpler if it doesn't have to work double-duty for relaying as well. -- 'peter'[:-1]@petertodd.org 00000000000000000b675c4d825a10c278b8d63ee4df90a19393f3b6498fd073 [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 646 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [bitcoin-dev] Do we really need a mempool? (for relay nodes) 2015-07-18 18:52 [bitcoin-dev] Do we really need a mempool? (for relay nodes) Peter Todd @ 2015-07-18 19:46 ` Patrick Strateman 2015-07-19 8:59 ` odinn 2015-07-20 22:14 ` Jorge Timón 0 siblings, 2 replies; 4+ messages in thread From: Patrick Strateman @ 2015-07-18 19:46 UTC (permalink / raw) To: bitcoin-dev [-- Attachment #1: Type: text/plain, Size: 1964 bytes --] Relay nodes do not need a mempool, but do need some mechanism to avoid DoS issues. Wallet nodes can use the mempool for fee estimation (in addition to looking at past blocks). On 07/18/2015 11:52 AM, Peter Todd via bitcoin-dev wrote: > As in, do relay nodes need to keep a record of the transactions they've > relayed? Strictly speaking, the answer is no: one a tx is relayed modulo > DoS concerns the entire thing can be discarded by the node. (unconfirmed > txs spending other unconfirmed txs can be handled by creating packages > of transactions, evaluated as a whole) > > To mitigate DoS concerns, we of course have to have some per-UTXO limit > on bandwidth relayed, but that task can be accomplished by simply > maintaining some kind of per-UTXO record of bandwidth used. For instance > if the weighted fee and fee/KB were recorded, and forced to - say - > double for each additional tx relayed that spent a given UTXO you would > have a clear and simple upper limit of lifetime bandwidth. Equally it's > easy to limit bandwidth moment to moment by asking peers for highest > fee/KB transactions they advertise first, stopping when our bandwidth > limit is reached. > > You probably could even remove IsStandard() pretty much entirely with > the right increasingly expensive "replacement" policy, relying on it > alone to provide anti-DoS. Obviously this would simplify some of the > debates around mining policy! This could even be re-used for scalable a > general-purpose messaging network paid by coin ownership if the UTXO set > is split up, and some kind of expiration over time policy is > implemented. > > Miners of course would still want to have a mempool, but that codebase > may prove simpler if it doesn't have to work double-duty for relaying as > well. > > > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [-- Attachment #2: Type: text/html, Size: 2588 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [bitcoin-dev] Do we really need a mempool? (for relay nodes) 2015-07-18 19:46 ` Patrick Strateman @ 2015-07-19 8:59 ` odinn 2015-07-20 22:14 ` Jorge Timón 1 sibling, 0 replies; 4+ messages in thread From: odinn @ 2015-07-19 8:59 UTC (permalink / raw) To: bitcoin-dev -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Some brief thoughts, adding here a suggestion for a dynamic approach to the issue. (e.g. each additional tx relayed has some thing associated with it, that is, a "doubling" for each additional tx relayed that spends a given UTXO, doesn't sound like it would be the most dynamic approach to the issue; considering that full nodes use the (UTXOs) to establish if transactions are valid – all inputs to a transaction must be in the UTXO database for it to be valid, but rather, would end up ratcheting upward the fee/kB for each additional tx relayed, as proposed). A more dynamic fee approach would be a better one, imho, but how is this to occur? Quoting from Gavin Andresen's http://gavinandresen.ninja/utxo-uhoh, "The entire UTXO set doesn’t have to be in RAM; it can be stored on an SSD or spinning hard disk. The access pattern to the UTXO is not random; outputs that were spent recently are more likely to be re-spent than outputs that have not been spent in a long time. Bitcoin Core already has a multi-level UTXO cache, thanks to the hard work of Pieter Wuille." The relay nodes would, in this scenario that is proposed here in this message, be confirming and discarding; the wallet nodes, if I understand properly, in this scenario, as proposed, should be retaining (keeping a record of the transactions they've relayed and using a mempool). There are some questions here: - - How is the mempool to be limited? - - What is the mechanism by which the UTXO set is stored (or proposed to be stored)? - - How would dynamic fee determinations be calculated? - - Please describe more the general purpose messaging network? Thank you On 07/18/2015 12:46 PM, Patrick Strateman via bitcoin-dev wrote: > Relay nodes do not need a mempool, but do need some mechanism to > avoid DoS issues. > > Wallet nodes can use the mempool for fee estimation (in addition > to looking at past blocks). > > On 07/18/2015 11:52 AM, Peter Todd via bitcoin-dev wrote: >> As in, do relay nodes need to keep a record of the transactions >> they've relayed? Strictly speaking, the answer is no: one a tx is >> relayed modulo DoS concerns the entire thing can be discarded by >> the node. (unconfirmed txs spending other unconfirmed txs can be >> handled by creating packages of transactions, evaluated as a >> whole) >> >> To mitigate DoS concerns, we of course have to have some per-UTXO >> limit on bandwidth relayed, but that task can be accomplished by >> simply maintaining some kind of per-UTXO record of bandwidth >> used. For instance if the weighted fee and fee/KB were recorded, >> and forced to - say - double for each additional tx relayed that >> spent a given UTXO you would have a clear and simple upper limit >> of lifetime bandwidth. Equally it's easy to limit bandwidth >> moment to moment by asking peers for highest fee/KB transactions >> they advertise first, stopping when our bandwidth limit is >> reached. >> >> You probably could even remove IsStandard() pretty much entirely >> with the right increasingly expensive "replacement" policy, >> relying on it alone to provide anti-DoS. Obviously this would >> simplify some of the debates around mining policy! This could >> even be re-used for scalable a general-purpose messaging network >> paid by coin ownership if the UTXO set is split up, and some kind >> of expiration over time policy is implemented. >> >> Miners of course would still want to have a mempool, but that >> codebase may prove simpler if it doesn't have to work double-duty >> for relaying as well. >> >> >> >> _______________________________________________ bitcoin-dev >> mailing list bitcoin-dev@lists•linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > > > _______________________________________________ bitcoin-dev mailing > list bitcoin-dev@lists•linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > - -- http://abis.io ~ "a protocol concept to enable decentralization and expansion of a giving economy, and a new social good" https://keybase.io/odinn -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVq2cFAAoJEGxwq/inSG8CIo4IAJAZ97NvW6Qdjd6HLN8q2074 sEUGdDeonARiQZXLfGyTJVg43Yb6LKPqkjWPQEgl9LLNni8t99iUqu3BJxedRDjd 8x+/F8n5VJrUrn1pXUcbC1aWss1y8JPTO2KpF/WL254IFc8iE8MJf4YF8PDSgy5j 9uW8NvWvdT4dz+rXu9vqfcplz8x7NGQ+CWN2N2JlChhKLMFprXPIx8a7NQwaKdrY lTpgAJWGMyPGNCmYQprBjIjOfp8tdTLQFlsLUAsXDmEisJX9goRVGMsHOWLTREoA l3kTgM0WMv6MIG7NOQQcWLD7cZdwWYR9e49hc27VcHt2R/FTepvnwPqo2GtE0tM= =eRbR -----END PGP SIGNATURE----- ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [bitcoin-dev] Do we really need a mempool? (for relay nodes) 2015-07-18 19:46 ` Patrick Strateman 2015-07-19 8:59 ` odinn @ 2015-07-20 22:14 ` Jorge Timón 1 sibling, 0 replies; 4+ messages in thread From: Jorge Timón @ 2015-07-20 22:14 UTC (permalink / raw) To: Patrick Strateman; +Cc: bitcoin-dev On Sat, Jul 18, 2015 at 9:46 PM, Patrick Strateman via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote: > Relay nodes do not need a mempool, but do need some mechanism to avoid DoS > issues. > > Wallet nodes can use the mempool for fee estimation (in addition to looking > at past blocks). Exactly, so an anti-DoS mechanism that would be sufficient for a non-mempool node would be also useful for small values in -maxmempool. I think a simple cache for transaction validations should be enough. Please, review a draft for that in the newest #6448. https://github.com/bitcoin/bitcoin/pull/6448 I would be happy to rebase it back to 0.11 and even 0.10. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2015-07-20 22:14 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-07-18 18:52 [bitcoin-dev] Do we really need a mempool? (for relay nodes) Peter Todd 2015-07-18 19:46 ` Patrick Strateman 2015-07-19 8:59 ` odinn 2015-07-20 22:14 ` Jorge Timón
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox