* [Bitcoin-development] BIP to improve the availability of blocks @ 2012-04-30 16:40 Rebroad (sourceforge) 2012-04-30 18:26 ` Wladimir 0 siblings, 1 reply; 5+ messages in thread From: Rebroad (sourceforge) @ 2012-04-30 16:40 UTC (permalink / raw) To: bitcoin-development Dear Bitcoin developers, In brief, the proposal I have is to extend the protocol to allow partial block download and upload. This is for people with intermittent connectivity or restricted connectivity. e.g. my own internet connection is quite slow, and my ISP routinely sends RSTs to both sides of connections to severe them. This often happens during block download and upload. I also often encounter the reception of blocks I have already received, further wasting bandwidth. This happens as quite often it can be far more than 2 minutes before block reception occurs following the getdata request, by which time my node has already sent a new getdata to another node requesting the same block. My proposal is that in addition to the size (which is advertised in the header), the hash is also advertised in the header (of a block). This would help nodes to determine whether they wanted to reject the download. (e.g. if it already had a block matching that hash). This of course wouldn't prevent a rogue node from sending an incorrect hash, but this would aid in saving bandwidth amongst behaving nodes. The other part of the proposal is to allow nodes to request upload and download blocks that have already been partially downloaded. This could be done by modifying the existing methods of upload, download, or by adding a new method, perhaps even using HTTP/HTTPS or something similar. This would also help nodes to obtain the blockchain who have restrictive ISPs, especially if they are being served on port 80 or 443. This could perhaps also allow web caches to keep caches of the blockchain, thereby making it also more available also. Currently, without this functionality, nodes with restrictive (or slow) internet have some options, such as going via a tor proxy, but due to the latency, the problem with multiple receptions of the same block still occur. Hopefully, not too clueless a post for my first post to this mailing list. Regards, Ed ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] BIP to improve the availability of blocks 2012-04-30 16:40 [Bitcoin-development] BIP to improve the availability of blocks Rebroad (sourceforge) @ 2012-04-30 18:26 ` Wladimir 2012-04-30 19:11 ` Amir Taaki 0 siblings, 1 reply; 5+ messages in thread From: Wladimir @ 2012-04-30 18:26 UTC (permalink / raw) To: Rebroad (sourceforge); +Cc: bitcoin-development [-- Attachment #1: Type: text/plain, Size: 2086 bytes --] On Mon, Apr 30, 2012 at 6:40 PM, Rebroad (sourceforge) < rebroad+sourceforge.net@gmail•com> wrote: > > My proposal is that in addition to the size (which is advertised in > the header), the hash is also advertised in the header (of a block). > This would help nodes to determine whether they wanted to reject the > download. (e.g. if it already had a block matching that hash). This of > course wouldn't prevent a rogue node from sending an incorrect hash, > but this would aid in saving bandwidth amongst behaving nodes. > I suppose it would make sense for clients to be able to reject blocks that they already have, if that's not currently possible. The other part of the proposal is to allow nodes to request upload and > download blocks that have already been partially downloaded. > > This could be done by modifying the existing methods of upload, > download, or by adding a new method, perhaps even using HTTP/HTTPS or > something similar. This would also help nodes to obtain the blockchain > who have restrictive ISPs, especially if they are being served on port > 80 or 443. This could perhaps also allow web caches to keep caches of > the blockchain, thereby making it also more available also. > You don't need a BIP if you want to somehow fetch the (initial) block chain outside the bitcoin protocol. You could download it from some http server or even pass it along on an USB stick. Then with a simple client change you can import it: https://github.com/bitcoin/bitcoin/pull/883 . Currently, without this functionality, nodes with restrictive (or > slow) internet have some options, such as going via a tor proxy, but > due to the latency, the problem with multiple receptions of the same > block still occur. > If you're behind such a slow internet connection, and concerned about every bit of bandwidth, it is better to run a lightweight node. For example, Electrum. Even if you could reduce the wasted bandwidth a bit by puzzling around with partial blocks, the download will still be substantial (and that's going to get worse before it gets better). Wladimir [-- Attachment #2: Type: text/html, Size: 2854 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] BIP to improve the availability of blocks 2012-04-30 18:26 ` Wladimir @ 2012-04-30 19:11 ` Amir Taaki 2012-04-30 20:02 ` Zell Faze 0 siblings, 1 reply; 5+ messages in thread From: Amir Taaki @ 2012-04-30 19:11 UTC (permalink / raw) To: bitcoin-development This is optimisation where it isn't needed. Bandwidth is not the bottleneck of the Bitcoin system. It is the immense time needed to validate the blockchain. And clients should never send blocks first. They always send an inv packet, then you request the block. It is a disruptive change and brings little. We don't need to optimise every aspect of Bitcoin :) Just focus on the big bits that matter, while keeping everything working with minimal changes. For instance say we did this and to maintain backwards compatible, we introduced a new message called "hash+block". Now there are 2 code branches that must be maintained - urgh. ________________________________ From: Wladimir <laanwj@gmail•com> To: Rebroad (sourceforge) <rebroad+sourceforge.net@gmail•com> Cc: bitcoin-development@lists•sourceforge.net Sent: Monday, April 30, 2012 7:26 PM Subject: Re: [Bitcoin-development] BIP to improve the availability of blocks On Mon, Apr 30, 2012 at 6:40 PM, Rebroad (sourceforge) <rebroad+sourceforge.net@gmail•com> wrote: >My proposal is that in addition to the size (which is advertised in >the header), the hash is also advertised in the header (of a block). >This would help nodes to determine whether they wanted to reject the >download. (e.g. if it already had a block matching that hash). This of >course wouldn't prevent a rogue node from sending an incorrect hash, >but this would aid in saving bandwidth amongst behaving nodes. > I suppose it would make sense for clients to be able to reject blocks that they already have, if that's not currently possible. The other part of the proposal is to allow nodes to request upload and >download blocks that have already been partially downloaded. > >This could be done by modifying the existing methods of upload, >download, or by adding a new method, perhaps even using HTTP/HTTPS or >something similar. This would also help nodes to obtain the blockchain >who have restrictive ISPs, especially if they are being served on port >80 or 443. This could perhaps also allow web caches to keep caches of >the blockchain, thereby making it also more available also. > You don't need a BIP if you want to somehow fetch the (initial) block chain outside the bitcoin protocol. You could download it from some http server or even pass it along on an USB stick. Then with a simple client change you can import it: https://github.com/bitcoin/bitcoin/pull/883 . Currently, without this functionality, nodes with restrictive (or >slow) internet have some options, such as going via a tor proxy, but >due to the latency, the problem with multiple receptions of the same >block still occur. > If you're behind such a slow internet connection, and concerned about every bit of bandwidth, it is better to run a lightweight node. For example, Electrum. Even if you could reduce the wasted bandwidth a bit by puzzling around with partial blocks, the download will still be substantial (and that's going to get worse before it gets better). Wladimir ------------------------------------------------------------------------------ Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ _______________________________________________ Bitcoin-development mailing list Bitcoin-development@lists•sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] BIP to improve the availability of blocks 2012-04-30 19:11 ` Amir Taaki @ 2012-04-30 20:02 ` Zell Faze 2012-04-30 20:54 ` Peter Vessenes 0 siblings, 1 reply; 5+ messages in thread From: Zell Faze @ 2012-04-30 20:02 UTC (permalink / raw) To: bitcoin-development Although quite true, I actually agree though that there should be some sort of checksum for the blocks. Bandwidth may not be a bottleneck now (or ever), but it may be at some point. This change will help Bitcoin scale. ------------------------ "It stopped being just a website a long time ago. For many of us, most of us, Wikipedia has become an indispensable part of our daily lives." — Jimmy Wales, Founder of Wikipedia Help protect it now. Please make a donation today: http://www.wikimediafoundation.org/wiki/Donate --- On Mon, 4/30/12, Amir Taaki <zgenjix@yahoo•com> wrote: > From: Amir Taaki <zgenjix@yahoo•com> > Subject: Re: [Bitcoin-development] BIP to improve the availability of blocks > To: "bitcoin-development@lists•sourceforge.net" <bitcoin-development@lists•sourceforge.net> > Date: Monday, April 30, 2012, 3:11 PM > This is optimisation where it isn't > needed. Bandwidth is not the bottleneck of the Bitcoin > system. It is the immense time needed to validate the > blockchain. > > And clients should never send blocks first. They always send > an inv packet, then you request the block. It is a > disruptive change and brings little. > > We don't need to optimise every aspect of Bitcoin :) Just > focus on the big bits that matter, while keeping everything > working with minimal changes. > > For instance say we did this and to maintain backwards > compatible, we introduced a new message called "hash+block". > Now there are 2 code branches that must be maintained - > urgh. > > > ________________________________ > From: Wladimir <laanwj@gmail•com> > To: Rebroad (sourceforge) <rebroad+sourceforge.net@gmail•com> > > Cc: bitcoin-development@lists•sourceforge.net > > Sent: Monday, April 30, 2012 7:26 PM > Subject: Re: [Bitcoin-development] BIP to improve the > availability of blocks > > > On Mon, Apr 30, 2012 at 6:40 PM, Rebroad (sourceforge) > <rebroad+sourceforge.net@gmail•com> > wrote: > > > >My proposal is that in addition to the size (which is > advertised in > >the header), the hash is also advertised in the header > (of a block). > >This would help nodes to determine whether they wanted > to reject the > >download. (e.g. if it already had a block matching that > hash). This of > >course wouldn't prevent a rogue node from sending an > incorrect hash, > >but this would aid in saving bandwidth amongst behaving > nodes. > > > > I suppose it would make sense for clients to be able to > reject blocks that they already have, if that's not > currently possible. > > > The other part of the proposal is to allow nodes to request > upload and > >download blocks that have already been partially > downloaded. > > > >This could be done by modifying the existing methods of > upload, > >download, or by adding a new method, perhaps even using > HTTP/HTTPS or > >something similar. This would also help nodes to obtain > the blockchain > >who have restrictive ISPs, especially if they are being > served on port > >80 or 443. This could perhaps also allow web caches to > keep caches of > >the blockchain, thereby making it also more available > also. > > > > You don't need a BIP if you want to somehow fetch the > (initial) block chain > outside the bitcoin protocol. You could download it from > some http > server or even pass it along on an USB stick. Then with a > simple client change you can import it: https://github.com/bitcoin/bitcoin/pull/883 . > > > Currently, without this functionality, nodes with > restrictive (or > >slow) internet have some options, such as going via a > tor proxy, but > >due to the latency, the problem with multiple receptions > of the same > >block still occur. > > > > If you're behind such a slow internet connection, and > concerned about > every bit of bandwidth, it is better to run a lightweight > node. For example, Electrum. > > Even if you could reduce the wasted bandwidth a bit by > puzzling > around with partial blocks, the download will still be > substantial (and that's going to get worse before it gets > better). > > Wladimir > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's > security and > threat landscape has changed and how IT managers can > respond. Discussions > will include endpoint security, mobile security and the > latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's > security and > threat landscape has changed and how IT managers can > respond. Discussions > will include endpoint security, mobile security and the > latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] BIP to improve the availability of blocks 2012-04-30 20:02 ` Zell Faze @ 2012-04-30 20:54 ` Peter Vessenes 0 siblings, 0 replies; 5+ messages in thread From: Peter Vessenes @ 2012-04-30 20:54 UTC (permalink / raw) To: Zell Faze; +Cc: bitcoin-development [-- Attachment #1: Type: text/plain, Size: 6800 bytes --] Blocks already checksum; they hash to a low number. Also inre: block headers, you are furnished with a previous hash in the first 80 bytes of the block. You can always cut the connection at that moment if you've already seen the block headers. Peter On Mon, Apr 30, 2012 at 1:02 PM, Zell Faze <zellfaze@yahoo•com> wrote: > Although quite true, I actually agree though that there should be some > sort of checksum for the blocks. Bandwidth may not be a bottleneck now (or > ever), but it may be at some point. This change will help Bitcoin scale. > > ------------------------ > "It stopped being just a website a long time ago. For many of us, most of > us, Wikipedia has become an indispensable part of our daily lives." > — Jimmy Wales, Founder of Wikipedia > Help protect it now. Please make a donation today: > http://www.wikimediafoundation.org/wiki/Donate > > > > --- On Mon, 4/30/12, Amir Taaki <zgenjix@yahoo•com> wrote: > > > From: Amir Taaki <zgenjix@yahoo•com> > > Subject: Re: [Bitcoin-development] BIP to improve the availability of > blocks > > To: "bitcoin-development@lists•sourceforge.net" < > bitcoin-development@lists•sourceforge.net> > > Date: Monday, April 30, 2012, 3:11 PM > > This is optimisation where it isn't > > needed. Bandwidth is not the bottleneck of the Bitcoin > > system. It is the immense time needed to validate the > > blockchain. > > > > And clients should never send blocks first. They always send > > an inv packet, then you request the block. It is a > > disruptive change and brings little. > > > > We don't need to optimise every aspect of Bitcoin :) Just > > focus on the big bits that matter, while keeping everything > > working with minimal changes. > > > > For instance say we did this and to maintain backwards > > compatible, we introduced a new message called "hash+block". > > Now there are 2 code branches that must be maintained - > > urgh. > > > > > > ________________________________ > > From: Wladimir <laanwj@gmail•com> > > To: Rebroad (sourceforge) <rebroad+sourceforge.net@gmail•com> > > > > Cc: bitcoin-development@lists•sourceforge.net > > > > Sent: Monday, April 30, 2012 7:26 PM > > Subject: Re: [Bitcoin-development] BIP to improve the > > availability of blocks > > > > > > On Mon, Apr 30, 2012 at 6:40 PM, Rebroad (sourceforge) > > <rebroad+sourceforge.net@gmail•com> > > wrote: > > > > > > >My proposal is that in addition to the size (which is > > advertised in > > >the header), the hash is also advertised in the header > > (of a block). > > >This would help nodes to determine whether they wanted > > to reject the > > >download. (e.g. if it already had a block matching that > > hash). This of > > >course wouldn't prevent a rogue node from sending an > > incorrect hash, > > >but this would aid in saving bandwidth amongst behaving > > nodes. > > > > > > > I suppose it would make sense for clients to be able to > > reject blocks that they already have, if that's not > > currently possible. > > > > > > The other part of the proposal is to allow nodes to request > > upload and > > >download blocks that have already been partially > > downloaded. > > > > > >This could be done by modifying the existing methods of > > upload, > > >download, or by adding a new method, perhaps even using > > HTTP/HTTPS or > > >something similar. This would also help nodes to obtain > > the blockchain > > >who have restrictive ISPs, especially if they are being > > served on port > > >80 or 443. This could perhaps also allow web caches to > > keep caches of > > >the blockchain, thereby making it also more available > > also. > > > > > > > You don't need a BIP if you want to somehow fetch the > > (initial) block chain > > outside the bitcoin protocol. You could download it from > > some http > > server or even pass it along on an USB stick. Then with a > > simple client change you can import it: > https://github.com/bitcoin/bitcoin/pull/883 . > > > > > > Currently, without this functionality, nodes with > > restrictive (or > > >slow) internet have some options, such as going via a > > tor proxy, but > > >due to the latency, the problem with multiple receptions > > of the same > > >block still occur. > > > > > > > If you're behind such a slow internet connection, and > > concerned about > > every bit of bandwidth, it is better to run a lightweight > > node. For example, Electrum. > > > > Even if you could reduce the wasted bandwidth a bit by > > puzzling > > around with partial blocks, the download will still be > > substantial (and that's going to get worse before it gets > > better). > > > > Wladimir > > > > > > > ------------------------------------------------------------------------------ > > Live Security Virtual Conference > > Exclusive live event will cover all the ways today's > > security and > > threat landscape has changed and how IT managers can > > respond. Discussions > > will include endpoint security, mobile security and the > > latest in malware > > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > > _______________________________________________ > > Bitcoin-development mailing list > > Bitcoin-development@lists•sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > ------------------------------------------------------------------------------ > > Live Security Virtual Conference > > Exclusive live event will cover all the ways today's > > security and > > threat landscape has changed and how IT managers can > > respond. Discussions > > will include endpoint security, mobile security and the > > latest in malware > > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > > _______________________________________________ > > Bitcoin-development mailing list > > Bitcoin-development@lists•sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists•sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > -- Peter J. Vessenes CEO, CoinLab M: 206.595.9839 Skype: vessenes Google+ <https://plus.google.com/112885659993091300749> [-- Attachment #2: Type: text/html, Size: 9724 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-04-30 21:25 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-04-30 16:40 [Bitcoin-development] BIP to improve the availability of blocks Rebroad (sourceforge) 2012-04-30 18:26 ` Wladimir 2012-04-30 19:11 ` Amir Taaki 2012-04-30 20:02 ` Zell Faze 2012-04-30 20:54 ` Peter Vessenes
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox