* [bitcoin-dev] Bitcoin Transaction Relay over Nostr
@ 2023-05-23 7:19 Joost Jager
2023-05-23 13:25 ` alicexbt
2023-05-28 2:37 ` David A. Harding
0 siblings, 2 replies; 6+ messages in thread
From: Joost Jager @ 2023-05-23 7:19 UTC (permalink / raw)
To: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 2865 bytes --]
Hi,
I write to get your thoughts on an alternative approach for Bitcoin
transaction relay, addressing some of the limitations in the current
peer-to-peer transaction relay system. To the best of my knowledge, the
credit for the original concept goes to Ben Carman. I felt it would be
beneficial to share the idea on this list to garner wider perspectives and
feedback.
The existing peer-to-peer (P2P) transaction relay system comes with a set
of limitations that may negatively impact applications, notably those like
Lightning that make extensive use of pre-signed transactions. A key
limitation lies in the system's inability to relay transaction packages.
This constraint can lead to HTLCs expiring before being swept, thereby
risking fund losses. In addition, the P2P system falls short in supporting
non-standard transactions, despite an established demand for such
transactions in the marketplace.
Nostr, an open and decentralized network of relays for public and ephemeral
messages between pseudonymous entities, could help address these
shortcomings. With the standards defined in NIP-89 [1], it becomes possible
to broadcast arbitrary Bitcoin transaction packages, overcoming one of the
key hurdles in the current relay system.
In this proposed alternative relay mechanism, miners would listen for these
broadcasted transaction packages and insert the packages into their local
mempool. They can take advantage of the `submitpackage` RPC, limited to
safe topologies only - specifically child and direct parents, tree only
[2]. This feature could serve as an interim solution for package relay
until it becomes available through the traditional P2P method.
A notable advantage of this approach is that it delegates the
responsibility of dealing with Denial-of-Service (DoS) threats to the
relays themselves. They could, for example, require a payment to mitigate
such concerns. There are in fact paid nostr relays already in operation.
This partitioning would result in a clear separation between the Bitcoin
transaction layer and DoS protection, introducing more flexibility in the
system and potentially boosting its resilience.
Implementing Nostr as a relay mechanism also has the potential to
democratize access to miner mempools, thus leveling the playing field in
the Bitcoin network. In the current state, those with direct connections or
certain privileges can more readily submit transactions to miners, perhaps
even through means as informal as email.
I have been working on a prototype of this concept (based on [3]) and have
captured its workings in a demonstration video [4].
Joost
[1] https://github.com/nostr-protocol/nips/pull/476
[2] https://github.com/bitcoin/bitcoin/pull/27609#issuecomment-1544414801
[3] https://github.com/benthecarman/nostr-tx-broadcast
[4] https://twitter.com/joostjgr/status/1658487013237211155
[-- Attachment #2: Type: text/html, Size: 9533 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bitcoin-dev] Bitcoin Transaction Relay over Nostr
2023-05-23 7:19 [bitcoin-dev] Bitcoin Transaction Relay over Nostr Joost Jager
@ 2023-05-23 13:25 ` alicexbt
2023-05-23 15:26 ` Joost Jager
2023-05-28 2:37 ` David A. Harding
1 sibling, 1 reply; 6+ messages in thread
From: alicexbt @ 2023-05-23 13:25 UTC (permalink / raw)
To: Joost Jager; +Cc: bitcoin-dev
Hi Joost,
Transaction relay over nostr sounds interesting. I have 2 suggestions:
- Transactions could be encrypted when published as nostr events initially except size, fee rate and offer. This can be used by different clients to show them as external mempool with transactions sorted by fee rate without affecting privacy of users.
- Mining pools will be incentivized to include these transaction in their blocks if they are using a higher fee rate compared to transactions in normal mempool used by bitcoin nodes or there is a mechanism to accept published offers, NIP4 is used to privately coordinate everything between user and pool. User can lock some sats in a 2of2 multisig and release it to mining pool on confirmation.
/dev/fd0
floppy disk guy
Sent with Proton Mail secure email.
------- Original Message -------
On Tuesday, May 23rd, 2023 at 12:49 PM, Joost Jager via bitcoin-dev <bitcoin-dev@lists•linuxfoundation.org> wrote:
> Hi,
>
>
>
> I write to get your thoughts on an alternative approach for Bitcoin transaction relay, addressing some of the limitations in the current peer-to-peer transaction relay system. To the best of my knowledge, the credit for the original concept goes to Ben Carman. I felt it would be beneficial to share the idea on this list to garner wider perspectives and feedback.
>
>
>
> The existing peer-to-peer (P2P) transaction relay system comes with a set of limitations that may negatively impact applications, notably those like Lightning that make extensive use of pre-signed transactions. A key limitation lies in the system's inability to relay transaction packages. This constraint can lead to HTLCs expiring before being swept, thereby risking fund losses. In addition, the P2P system falls short in supporting non-standard transactions, despite an established demand for such transactions in the marketplace.
>
>
>
> Nostr, an open and decentralized network of relays for public and ephemeral messages between pseudonymous entities, could help address these shortcomings. With the standards defined in NIP-89 [1], it becomes possible to broadcast arbitrary Bitcoin transaction packages, overcoming one of the key hurdles in the current relay system.
>
>
>
> In this proposed alternative relay mechanism, miners would listen for these broadcasted transaction packages and insert the packages into their local mempool. They can take advantage of the `submitpackage` RPC, limited to safe topologies only - specifically child and direct parents, tree only [2]. This feature could serve as an interim solution for package relay until it becomes available through the traditional P2P method.
>
>
>
> A notable advantage of this approach is that it delegates the responsibility of dealing with Denial-of-Service (DoS) threats to the relays themselves. They could, for example, require a payment to mitigate such concerns. There are in fact paid nostr relays already in operation. This partitioning would result in a clear separation between the Bitcoin transaction layer and DoS protection, introducing more flexibility in the system and potentially boosting its resilience.
>
>
>
> Implementing Nostr as a relay mechanism also has the potential to democratize access to miner mempools, thus leveling the playing field in the Bitcoin network. In the current state, those with direct connections or certain privileges can more readily submit transactions to miners, perhaps even through means as informal as email.
>
>
>
> I have been working on a prototype of this concept (based on [3]) and have captured its workings in a demonstration video [4].
>
>
>
> Joost
>
>
>
> [1] https://github.com/nostr-protocol/nips/pull/476
>
> [2] https://github.com/bitcoin/bitcoin/pull/27609#issuecomment-1544414801
>
> [3] https://github.com/benthecarman/nostr-tx-broadcast
>
> [4] https://twitter.com/joostjgr/status/1658487013237211155
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bitcoin-dev] Bitcoin Transaction Relay over Nostr
2023-05-23 13:25 ` alicexbt
@ 2023-05-23 15:26 ` Joost Jager
0 siblings, 0 replies; 6+ messages in thread
From: Joost Jager @ 2023-05-23 15:26 UTC (permalink / raw)
To: alicexbt; +Cc: bitcoin-dev
[-- Attachment #1: Type: text/plain, Size: 1179 bytes --]
Hi fd0,
> - Transactions could be encrypted when published as nostr events initially
> except size, fee rate and offer. This can be used by different clients to
> show them as external mempool with transactions sorted by fee rate without
> affecting privacy of users.
>
I don't think this will work because those encrypted transactions could all
be fake and distort the view clients have on this 'mempool'?
> - Mining pools will be incentivized to include these transaction in their
> blocks if they are using a higher fee rate compared to transactions in
> normal mempool used by bitcoin nodes or there is a mechanism to accept
> published offers, NIP4 is used to privately coordinate everything between
> user and pool. User can lock some sats in a 2of2 multisig and release it to
> mining pool on confirmation.
>
I believe you are suggesting out-of-band payment in case the fee included
in the transaction itself is insufficient and CPFP/RBF is impossible or
impractical? The question is why would the miner trust you to indeed
release after confirmation? The 2-of-2 presumably has a clawback clause
with a timeout that could be used by the user to avoid paying.
Joost
[-- Attachment #2: Type: text/html, Size: 1610 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bitcoin-dev] Bitcoin Transaction Relay over Nostr
2023-05-23 7:19 [bitcoin-dev] Bitcoin Transaction Relay over Nostr Joost Jager
2023-05-23 13:25 ` alicexbt
@ 2023-05-28 2:37 ` David A. Harding
2023-05-30 12:30 ` Joost Jager
1 sibling, 1 reply; 6+ messages in thread
From: David A. Harding @ 2023-05-28 2:37 UTC (permalink / raw)
To: Joost Jager, Bitcoin Protocol Discussion
On 2023-05-22 21:19, Joost Jager via bitcoin-dev wrote:
> A notable advantage of this approach is that it delegates the
> responsibility of dealing with Denial-of-Service (DoS) threats to the
> relays themselves. They could, for example, require a payment to
> mitigate such concerns.
Hi Joost,
Thanks for working on this! One quick thought I had was that a possibly
interesting avenue for exploration would be that, in addition to
relaying individual transactions or packages, it might be worth relaying
block templates and weak blocks as both of those provide inherent DoS
resistance and can offer useful features.
A block template is an ordered list of raw transactions that can all be
included in the next block (with some space reserved for a coinbase
transaction). A full node can validate those transactions and calculate
how much fee they pay. A Nostr relay can simply relay almost[1] any
template that pays more fees than the previous best template it saw for
the next block. That can be more flexible than the current
implementation of submitblock with package relay which still enforces a
lot of the rules that helps keep a regular relay node safe from DoS and
a miner node able to select mineable transactions quickly.
A weak block is a block whose header doesn't quite hash to low enough of
a value to be included on the chain. It still takes an extraordinary
amount of hashrate to produce, so it's inherently DoS resistant. If
miners are producing block that include transactions not seen by typical
relay nodes, that can reduce the efficiency and effectiveness of BIP152
compact block relay, which hurts the profitability of miners of custom
blocks. To compensate, miners could relay weak blocks through Nostr to
full nodes and other miners so that they could quickly relay and accept
complete blocks that later included the same custom transactions. This
would also help fee estimation and provide valuable insights to those
trying to get their transactions included into the next block.
Regarding size, the block template and weak block could both be sent in
BIP152 compact block format as a diff against the expected contents of a
typical node, allowing Alice to send just a small amount of additional
data for relay over what she'd have to send anyway for each transaction
in a package. (Although it's quite possible that BetterHash or Stratum
v2 have even better solutions, possibly already implemented.)
If nothing else, I think Nostr could provide an interesting playground
for experimenting with various relay and mining ideas we've talked about
for years, so thanks again for working on this!
-Dave
[1] In addition to validating transactions, a relay would probably want
to reject templates that contained transactions that took
excessively long to validate (which could cause a block including
them to become stale) or that included features reserved for
upgrades (as a soft fork that happened before the relay's node was
upgraded might make that block invalid).
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bitcoin-dev] Bitcoin Transaction Relay over Nostr
2023-05-28 2:37 ` David A. Harding
@ 2023-05-30 12:30 ` Joost Jager
2023-05-30 13:30 ` Greg Sanders
0 siblings, 1 reply; 6+ messages in thread
From: Joost Jager @ 2023-05-30 12:30 UTC (permalink / raw)
To: David A. Harding; +Cc: Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 3171 bytes --]
Hi David,
> A block template is an ordered list of raw transactions that can all be
> included in the next block (with some space reserved for a coinbase
> transaction). A full node can validate those transactions and calculate
> how much fee they pay. A Nostr relay can simply relay almost[1] any
> template that pays more fees than the previous best template it saw for
> the next block. That can be more flexible than the current
> implementation of submitblock with package relay which still enforces a
> lot of the rules that helps keep a regular relay node safe from DoS and
> a miner node able to select mineable transactions quickly.
>
Interesting idea! This would also make it easy for external services to try
to do the best possible block building using advanced algorithms. Miners
would just select the best template available from various sources
including nostr.
> A weak block is a block whose header doesn't quite hash to low enough of
> a value to be included on the chain. It still takes an extraordinary
> amount of hashrate to produce, so it's inherently DoS resistant. If
> miners are producing block that include transactions not seen by typical
> relay nodes, that can reduce the efficiency and effectiveness of BIP152
> compact block relay, which hurts the profitability of miners of custom
> blocks. To compensate, miners could relay weak blocks through Nostr to
> full nodes and other miners so that they could quickly relay and accept
> complete blocks that later included the same custom transactions. This
> would also help fee estimation and provide valuable insights to those
> trying to get their transactions included into the next block.
>
I believe this would be useful right away, wouldn't it? Looking at
mempool.space's block audit, there are definitely blocks that have a
"surprising" content and might take long to download.
The anti-dos measures that you describe for both weak blocks and block
templates seem very robust, but they would require a more intelligent nostr
relay to enforce. Not sure if it is still allowed to call it nostr at that
point. Perhaps it becomes more of a specialised bitcoin relay. btcstr -
"bitcoin stuff transmitted by relays".
Regarding size, the block template and weak block could both be sent in
> BIP152 compact block format as a diff against the expected contents of a
> typical node, allowing Alice to send just a small amount of additional
> data for relay over what she'd have to send anyway for each transaction
> in a package. (Although it's quite possible that BetterHash or Stratum
> v2 have even better solutions, possibly already implemented.)
>
Sounds like a great way to repurpose what already exists to reduce resource
usage for these additional message types.
> If nothing else, I think Nostr could provide an interesting playground
> for experimenting with various relay and mining ideas we've talked about
> for years, so thanks again for working on this!
>
I think so too! The main question on my mind though is how to actually make
this work. There is a bit of a chicken-egg problem here with users and
miners possibly waiting for each other to adopt.
Joost
[-- Attachment #2: Type: text/html, Size: 4131 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [bitcoin-dev] Bitcoin Transaction Relay over Nostr
2023-05-30 12:30 ` Joost Jager
@ 2023-05-30 13:30 ` Greg Sanders
0 siblings, 0 replies; 6+ messages in thread
From: Greg Sanders @ 2023-05-30 13:30 UTC (permalink / raw)
To: Joost Jager, Bitcoin Protocol Discussion
[-- Attachment #1: Type: text/plain, Size: 4505 bytes --]
Hi Joost, David,
In my mind, weak blocks' main benefit would be that it improves block relay
by giving PoW-hints on what are in miner's mempools. Non-standard
transactions could even be cached(even if not validated until block
inclusion), which would tolerate more heterogeneity in policies without
drastically increasing relay times. Of course, it can also have the side
effect of gossiping better transaction packages, though I think this would
be a ton of work to really take advantage of. Perhaps we might be able to
do better in a post-cluster-mempool world, gossiping chunks.
At present I think energy would be best spent writing a weak blocks BIP
proposal, since one has never been written before(?), and it would be
fairly trivial to swap out p2p things with RPC calls if so desired for fast
experimentation over alternative relays.
Cheers,
Greg
On Tue, May 30, 2023 at 8:58 AM Joost Jager via bitcoin-dev <
bitcoin-dev@lists•linuxfoundation.org> wrote:
> Hi David,
>
>
>> A block template is an ordered list of raw transactions that can all be
>> included in the next block (with some space reserved for a coinbase
>> transaction). A full node can validate those transactions and calculate
>> how much fee they pay. A Nostr relay can simply relay almost[1] any
>> template that pays more fees than the previous best template it saw for
>> the next block. That can be more flexible than the current
>> implementation of submitblock with package relay which still enforces a
>> lot of the rules that helps keep a regular relay node safe from DoS and
>> a miner node able to select mineable transactions quickly.
>>
>
> Interesting idea! This would also make it easy for external services to
> try to do the best possible block building using advanced algorithms.
> Miners would just select the best template available from various sources
> including nostr.
>
>
>> A weak block is a block whose header doesn't quite hash to low enough of
>> a value to be included on the chain. It still takes an extraordinary
>> amount of hashrate to produce, so it's inherently DoS resistant. If
>> miners are producing block that include transactions not seen by typical
>> relay nodes, that can reduce the efficiency and effectiveness of BIP152
>> compact block relay, which hurts the profitability of miners of custom
>> blocks. To compensate, miners could relay weak blocks through Nostr to
>> full nodes and other miners so that they could quickly relay and accept
>> complete blocks that later included the same custom transactions. This
>> would also help fee estimation and provide valuable insights to those
>> trying to get their transactions included into the next block.
>>
>
> I believe this would be useful right away, wouldn't it? Looking at
> mempool.space's block audit, there are definitely blocks that have a
> "surprising" content and might take long to download.
>
> The anti-dos measures that you describe for both weak blocks and block
> templates seem very robust, but they would require a more intelligent nostr
> relay to enforce. Not sure if it is still allowed to call it nostr at that
> point. Perhaps it becomes more of a specialised bitcoin relay. btcstr -
> "bitcoin stuff transmitted by relays".
>
> Regarding size, the block template and weak block could both be sent in
>> BIP152 compact block format as a diff against the expected contents of a
>> typical node, allowing Alice to send just a small amount of additional
>> data for relay over what she'd have to send anyway for each transaction
>> in a package. (Although it's quite possible that BetterHash or Stratum
>> v2 have even better solutions, possibly already implemented.)
>>
>
> Sounds like a great way to repurpose what already exists to reduce
> resource usage for these additional message types.
>
>
>> If nothing else, I think Nostr could provide an interesting playground
>> for experimenting with various relay and mining ideas we've talked about
>> for years, so thanks again for working on this!
>>
>
> I think so too! The main question on my mind though is how to actually
> make this work. There is a bit of a chicken-egg problem here with users and
> miners possibly waiting for each other to adopt.
>
> Joost
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists•linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
[-- Attachment #2: Type: text/html, Size: 5912 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-05-30 13:30 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-05-23 7:19 [bitcoin-dev] Bitcoin Transaction Relay over Nostr Joost Jager
2023-05-23 13:25 ` alicexbt
2023-05-23 15:26 ` Joost Jager
2023-05-28 2:37 ` David A. Harding
2023-05-30 12:30 ` Joost Jager
2023-05-30 13:30 ` Greg Sanders
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox