* [Bitcoin-development] BIP to improve the availability of blocks
@ 2012-04-30 16:40 Rebroad (sourceforge)
2012-04-30 18:26 ` Wladimir
0 siblings, 1 reply; 5+ messages in thread
From: Rebroad (sourceforge) @ 2012-04-30 16:40 UTC (permalink / raw)
To: bitcoin-development
Dear Bitcoin developers,
In brief, the proposal I have is to extend the protocol to allow
partial block download and upload. This is for people with
intermittent connectivity or restricted connectivity. e.g. my own
internet connection is quite slow, and my ISP routinely sends RSTs to
both sides of connections to severe them. This often happens during
block download and upload. I also often encounter the reception of
blocks I have already received, further wasting bandwidth. This
happens as quite often it can be far more than 2 minutes before block
reception occurs following the getdata request, by which time my node
has already sent a new getdata to another node requesting the same
block.
My proposal is that in addition to the size (which is advertised in
the header), the hash is also advertised in the header (of a block).
This would help nodes to determine whether they wanted to reject the
download. (e.g. if it already had a block matching that hash). This of
course wouldn't prevent a rogue node from sending an incorrect hash,
but this would aid in saving bandwidth amongst behaving nodes.
The other part of the proposal is to allow nodes to request upload and
download blocks that have already been partially downloaded.
This could be done by modifying the existing methods of upload,
download, or by adding a new method, perhaps even using HTTP/HTTPS or
something similar. This would also help nodes to obtain the blockchain
who have restrictive ISPs, especially if they are being served on port
80 or 443. This could perhaps also allow web caches to keep caches of
the blockchain, thereby making it also more available also.
Currently, without this functionality, nodes with restrictive (or
slow) internet have some options, such as going via a tor proxy, but
due to the latency, the problem with multiple receptions of the same
block still occur.
Hopefully, not too clueless a post for my first post to this mailing list.
Regards,
Ed
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] BIP to improve the availability of blocks
2012-04-30 16:40 [Bitcoin-development] BIP to improve the availability of blocks Rebroad (sourceforge)
@ 2012-04-30 18:26 ` Wladimir
2012-04-30 19:11 ` Amir Taaki
0 siblings, 1 reply; 5+ messages in thread
From: Wladimir @ 2012-04-30 18:26 UTC (permalink / raw)
To: Rebroad (sourceforge); +Cc: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 2086 bytes --]
On Mon, Apr 30, 2012 at 6:40 PM, Rebroad (sourceforge) <
rebroad+sourceforge.net@gmail•com> wrote:
>
> My proposal is that in addition to the size (which is advertised in
> the header), the hash is also advertised in the header (of a block).
> This would help nodes to determine whether they wanted to reject the
> download. (e.g. if it already had a block matching that hash). This of
> course wouldn't prevent a rogue node from sending an incorrect hash,
> but this would aid in saving bandwidth amongst behaving nodes.
>
I suppose it would make sense for clients to be able to reject blocks that
they already have, if that's not currently possible.
The other part of the proposal is to allow nodes to request upload and
> download blocks that have already been partially downloaded.
>
> This could be done by modifying the existing methods of upload,
> download, or by adding a new method, perhaps even using HTTP/HTTPS or
> something similar. This would also help nodes to obtain the blockchain
> who have restrictive ISPs, especially if they are being served on port
> 80 or 443. This could perhaps also allow web caches to keep caches of
> the blockchain, thereby making it also more available also.
>
You don't need a BIP if you want to somehow fetch the (initial) block chain
outside the bitcoin protocol. You could download it from some http server
or even pass it along on an USB stick. Then with a simple client change you
can import it: https://github.com/bitcoin/bitcoin/pull/883 .
Currently, without this functionality, nodes with restrictive (or
> slow) internet have some options, such as going via a tor proxy, but
> due to the latency, the problem with multiple receptions of the same
> block still occur.
>
If you're behind such a slow internet connection, and concerned about every
bit of bandwidth, it is better to run a lightweight node. For example,
Electrum.
Even if you could reduce the wasted bandwidth a bit by puzzling around with
partial blocks, the download will still be substantial (and that's going to
get worse before it gets better).
Wladimir
[-- Attachment #2: Type: text/html, Size: 2854 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] BIP to improve the availability of blocks
2012-04-30 18:26 ` Wladimir
@ 2012-04-30 19:11 ` Amir Taaki
2012-04-30 20:02 ` Zell Faze
0 siblings, 1 reply; 5+ messages in thread
From: Amir Taaki @ 2012-04-30 19:11 UTC (permalink / raw)
To: bitcoin-development
This is optimisation where it isn't needed. Bandwidth is not the bottleneck of the Bitcoin system. It is the immense time needed to validate the blockchain.
And clients should never send blocks first. They always send an inv packet, then you request the block. It is a disruptive change and brings little.
We don't need to optimise every aspect of Bitcoin :) Just focus on the big bits that matter, while keeping everything working with minimal changes.
For instance say we did this and to maintain backwards compatible, we introduced a new message called "hash+block". Now there are 2 code branches that must be maintained - urgh.
________________________________
From: Wladimir <laanwj@gmail•com>
To: Rebroad (sourceforge) <rebroad+sourceforge.net@gmail•com>
Cc: bitcoin-development@lists•sourceforge.net
Sent: Monday, April 30, 2012 7:26 PM
Subject: Re: [Bitcoin-development] BIP to improve the availability of blocks
On Mon, Apr 30, 2012 at 6:40 PM, Rebroad (sourceforge) <rebroad+sourceforge.net@gmail•com> wrote:
>My proposal is that in addition to the size (which is advertised in
>the header), the hash is also advertised in the header (of a block).
>This would help nodes to determine whether they wanted to reject the
>download. (e.g. if it already had a block matching that hash). This of
>course wouldn't prevent a rogue node from sending an incorrect hash,
>but this would aid in saving bandwidth amongst behaving nodes.
>
I suppose it would make sense for clients to be able to reject blocks that they already have, if that's not currently possible.
The other part of the proposal is to allow nodes to request upload and
>download blocks that have already been partially downloaded.
>
>This could be done by modifying the existing methods of upload,
>download, or by adding a new method, perhaps even using HTTP/HTTPS or
>something similar. This would also help nodes to obtain the blockchain
>who have restrictive ISPs, especially if they are being served on port
>80 or 443. This could perhaps also allow web caches to keep caches of
>the blockchain, thereby making it also more available also.
>
You don't need a BIP if you want to somehow fetch the (initial) block chain
outside the bitcoin protocol. You could download it from some http
server or even pass it along on an USB stick. Then with a simple client change you can import it: https://github.com/bitcoin/bitcoin/pull/883 .
Currently, without this functionality, nodes with restrictive (or
>slow) internet have some options, such as going via a tor proxy, but
>due to the latency, the problem with multiple receptions of the same
>block still occur.
>
If you're behind such a slow internet connection, and concerned about
every bit of bandwidth, it is better to run a lightweight node. For example, Electrum.
Even if you could reduce the wasted bandwidth a bit by puzzling
around with partial blocks, the download will still be substantial (and that's going to get worse before it gets better).
Wladimir
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists•sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] BIP to improve the availability of blocks
2012-04-30 19:11 ` Amir Taaki
@ 2012-04-30 20:02 ` Zell Faze
2012-04-30 20:54 ` Peter Vessenes
0 siblings, 1 reply; 5+ messages in thread
From: Zell Faze @ 2012-04-30 20:02 UTC (permalink / raw)
To: bitcoin-development
Although quite true, I actually agree though that there should be some sort of checksum for the blocks. Bandwidth may not be a bottleneck now (or ever), but it may be at some point. This change will help Bitcoin scale.
------------------------
"It stopped being just a website a long time ago. For many of us, most of us, Wikipedia has become an indispensable part of our daily lives."
— Jimmy Wales, Founder of Wikipedia
Help protect it now. Please make a donation today: http://www.wikimediafoundation.org/wiki/Donate
--- On Mon, 4/30/12, Amir Taaki <zgenjix@yahoo•com> wrote:
> From: Amir Taaki <zgenjix@yahoo•com>
> Subject: Re: [Bitcoin-development] BIP to improve the availability of blocks
> To: "bitcoin-development@lists•sourceforge.net" <bitcoin-development@lists•sourceforge.net>
> Date: Monday, April 30, 2012, 3:11 PM
> This is optimisation where it isn't
> needed. Bandwidth is not the bottleneck of the Bitcoin
> system. It is the immense time needed to validate the
> blockchain.
>
> And clients should never send blocks first. They always send
> an inv packet, then you request the block. It is a
> disruptive change and brings little.
>
> We don't need to optimise every aspect of Bitcoin :) Just
> focus on the big bits that matter, while keeping everything
> working with minimal changes.
>
> For instance say we did this and to maintain backwards
> compatible, we introduced a new message called "hash+block".
> Now there are 2 code branches that must be maintained -
> urgh.
>
>
> ________________________________
> From: Wladimir <laanwj@gmail•com>
> To: Rebroad (sourceforge) <rebroad+sourceforge.net@gmail•com>
>
> Cc: bitcoin-development@lists•sourceforge.net
>
> Sent: Monday, April 30, 2012 7:26 PM
> Subject: Re: [Bitcoin-development] BIP to improve the
> availability of blocks
>
>
> On Mon, Apr 30, 2012 at 6:40 PM, Rebroad (sourceforge)
> <rebroad+sourceforge.net@gmail•com>
> wrote:
>
>
> >My proposal is that in addition to the size (which is
> advertised in
> >the header), the hash is also advertised in the header
> (of a block).
> >This would help nodes to determine whether they wanted
> to reject the
> >download. (e.g. if it already had a block matching that
> hash). This of
> >course wouldn't prevent a rogue node from sending an
> incorrect hash,
> >but this would aid in saving bandwidth amongst behaving
> nodes.
> >
>
> I suppose it would make sense for clients to be able to
> reject blocks that they already have, if that's not
> currently possible.
>
>
> The other part of the proposal is to allow nodes to request
> upload and
> >download blocks that have already been partially
> downloaded.
> >
> >This could be done by modifying the existing methods of
> upload,
> >download, or by adding a new method, perhaps even using
> HTTP/HTTPS or
> >something similar. This would also help nodes to obtain
> the blockchain
> >who have restrictive ISPs, especially if they are being
> served on port
> >80 or 443. This could perhaps also allow web caches to
> keep caches of
> >the blockchain, thereby making it also more available
> also.
> >
>
> You don't need a BIP if you want to somehow fetch the
> (initial) block chain
> outside the bitcoin protocol. You could download it from
> some http
> server or even pass it along on an USB stick. Then with a
> simple client change you can import it: https://github.com/bitcoin/bitcoin/pull/883 .
>
>
> Currently, without this functionality, nodes with
> restrictive (or
> >slow) internet have some options, such as going via a
> tor proxy, but
> >due to the latency, the problem with multiple receptions
> of the same
> >block still occur.
> >
>
> If you're behind such a slow internet connection, and
> concerned about
> every bit of bandwidth, it is better to run a lightweight
> node. For example, Electrum.
>
> Even if you could reduce the wasted bandwidth a bit by
> puzzling
> around with partial blocks, the download will still be
> substantial (and that's going to get worse before it gets
> better).
>
> Wladimir
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's
> security and
> threat landscape has changed and how IT managers can
> respond. Discussions
> will include endpoint security, mobile security and the
> latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's
> security and
> threat landscape has changed and how IT managers can
> respond. Discussions
> will include endpoint security, mobile security and the
> latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Bitcoin-development] BIP to improve the availability of blocks
2012-04-30 20:02 ` Zell Faze
@ 2012-04-30 20:54 ` Peter Vessenes
0 siblings, 0 replies; 5+ messages in thread
From: Peter Vessenes @ 2012-04-30 20:54 UTC (permalink / raw)
To: Zell Faze; +Cc: bitcoin-development
[-- Attachment #1: Type: text/plain, Size: 6800 bytes --]
Blocks already checksum; they hash to a low number.
Also inre: block headers, you are furnished with a previous hash in the
first 80 bytes of the block. You can always cut the connection at that
moment if you've already seen the block headers.
Peter
On Mon, Apr 30, 2012 at 1:02 PM, Zell Faze <zellfaze@yahoo•com> wrote:
> Although quite true, I actually agree though that there should be some
> sort of checksum for the blocks. Bandwidth may not be a bottleneck now (or
> ever), but it may be at some point. This change will help Bitcoin scale.
>
> ------------------------
> "It stopped being just a website a long time ago. For many of us, most of
> us, Wikipedia has become an indispensable part of our daily lives."
> — Jimmy Wales, Founder of Wikipedia
> Help protect it now. Please make a donation today:
> http://www.wikimediafoundation.org/wiki/Donate
>
>
>
> --- On Mon, 4/30/12, Amir Taaki <zgenjix@yahoo•com> wrote:
>
> > From: Amir Taaki <zgenjix@yahoo•com>
> > Subject: Re: [Bitcoin-development] BIP to improve the availability of
> blocks
> > To: "bitcoin-development@lists•sourceforge.net" <
> bitcoin-development@lists•sourceforge.net>
> > Date: Monday, April 30, 2012, 3:11 PM
> > This is optimisation where it isn't
> > needed. Bandwidth is not the bottleneck of the Bitcoin
> > system. It is the immense time needed to validate the
> > blockchain.
> >
> > And clients should never send blocks first. They always send
> > an inv packet, then you request the block. It is a
> > disruptive change and brings little.
> >
> > We don't need to optimise every aspect of Bitcoin :) Just
> > focus on the big bits that matter, while keeping everything
> > working with minimal changes.
> >
> > For instance say we did this and to maintain backwards
> > compatible, we introduced a new message called "hash+block".
> > Now there are 2 code branches that must be maintained -
> > urgh.
> >
> >
> > ________________________________
> > From: Wladimir <laanwj@gmail•com>
> > To: Rebroad (sourceforge) <rebroad+sourceforge.net@gmail•com>
> >
> > Cc: bitcoin-development@lists•sourceforge.net
> >
> > Sent: Monday, April 30, 2012 7:26 PM
> > Subject: Re: [Bitcoin-development] BIP to improve the
> > availability of blocks
> >
> >
> > On Mon, Apr 30, 2012 at 6:40 PM, Rebroad (sourceforge)
> > <rebroad+sourceforge.net@gmail•com>
> > wrote:
> >
> >
> > >My proposal is that in addition to the size (which is
> > advertised in
> > >the header), the hash is also advertised in the header
> > (of a block).
> > >This would help nodes to determine whether they wanted
> > to reject the
> > >download. (e.g. if it already had a block matching that
> > hash). This of
> > >course wouldn't prevent a rogue node from sending an
> > incorrect hash,
> > >but this would aid in saving bandwidth amongst behaving
> > nodes.
> > >
> >
> > I suppose it would make sense for clients to be able to
> > reject blocks that they already have, if that's not
> > currently possible.
> >
> >
> > The other part of the proposal is to allow nodes to request
> > upload and
> > >download blocks that have already been partially
> > downloaded.
> > >
> > >This could be done by modifying the existing methods of
> > upload,
> > >download, or by adding a new method, perhaps even using
> > HTTP/HTTPS or
> > >something similar. This would also help nodes to obtain
> > the blockchain
> > >who have restrictive ISPs, especially if they are being
> > served on port
> > >80 or 443. This could perhaps also allow web caches to
> > keep caches of
> > >the blockchain, thereby making it also more available
> > also.
> > >
> >
> > You don't need a BIP if you want to somehow fetch the
> > (initial) block chain
> > outside the bitcoin protocol. You could download it from
> > some http
> > server or even pass it along on an USB stick. Then with a
> > simple client change you can import it:
> https://github.com/bitcoin/bitcoin/pull/883 .
> >
> >
> > Currently, without this functionality, nodes with
> > restrictive (or
> > >slow) internet have some options, such as going via a
> > tor proxy, but
> > >due to the latency, the problem with multiple receptions
> > of the same
> > >block still occur.
> > >
> >
> > If you're behind such a slow internet connection, and
> > concerned about
> > every bit of bandwidth, it is better to run a lightweight
> > node. For example, Electrum.
> >
> > Even if you could reduce the wasted bandwidth a bit by
> > puzzling
> > around with partial blocks, the download will still be
> > substantial (and that's going to get worse before it gets
> > better).
> >
> > Wladimir
> >
> >
> >
> ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's
> > security and
> > threat landscape has changed and how IT managers can
> > respond. Discussions
> > will include endpoint security, mobile security and the
> > latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Bitcoin-development mailing list
> > Bitcoin-development@lists•sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> >
> >
> ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's
> > security and
> > threat landscape has changed and how IT managers can
> > respond. Discussions
> > will include endpoint security, mobile security and the
> > latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Bitcoin-development mailing list
> > Bitcoin-development@lists•sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/bitcoin-development
> >
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists•sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
--
Peter J. Vessenes
CEO, CoinLab
M: 206.595.9839
Skype: vessenes
Google+ <https://plus.google.com/112885659993091300749>
[-- Attachment #2: Type: text/html, Size: 9724 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-04-30 21:25 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-04-30 16:40 [Bitcoin-development] BIP to improve the availability of blocks Rebroad (sourceforge)
2012-04-30 18:26 ` Wladimir
2012-04-30 19:11 ` Amir Taaki
2012-04-30 20:02 ` Zell Faze
2012-04-30 20:54 ` Peter Vessenes
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox