--- Log opened Mon Dec 23 00:00:46 2019
01:07 -!- belcher [~belcher@unaffiliated/belcher] has joined #rust-bitcoin
01:15 -!- andytoshi [~apoelstra@unaffiliated/andytoshi] has quit [Ping timeout: 258 seconds]
01:40 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Ping timeout: 260 seconds]
02:16 -!- jonatack [~jon@54.76.13.109.rev.sfr.net] has joined #rust-bitcoin
03:31 < stevenroose> andytoshi very much should fix his IRC machine ...
04:11 -!- jonatack [~jon@54.76.13.109.rev.sfr.net] has quit [Quit: jonatack]
05:10 -!- psyche16 [67d0e0a2@103.208.224.162] has joined #rust-bitcoin
05:24 -!- psyche16 [67d0e0a2@103.208.224.162] has quit [Remote host closed the connection]
05:43 < elichai2> stevenroose: hehe, I think if you fly to canada for him he'll be glad :P
05:43 < stevenroose> Installing ZNC on a machine somewhere is really no effort. I think he just wants to keep his more exotic setup :D
05:45 < stevenroose> elichai2: did you see my suggestion to not use the possibilities?
05:45 < stevenroose> You can just have a Rng for NetworkMessage (an enum) and for the Rng for RawNetworkMessage, you can rng the payload and use payload.command() as the CommandString.
05:45 < elichai2> I actually just gave up and paid IRCCloud. feel non hacker now but one thing off my mind :)
05:46 < elichai2> hmm i'll look at it again, I did it because it failed my test
05:46 < elichai2> right. I remember now. `RawNetworkMessage` is basically some int and a NetworkMessage
05:46 < stevenroose> elichai2: I feel that ZNC is really trivial. The only mistake I made is that I used a port for the web admin that is blocked by all browsers (an IRC port IIRC). So I always have to ssh tunnel because I'm too lazy to change the port and I always think it will be the last time I will have to change some setting :)
05:47 < elichai2> I don't want to randomize the whole `RawNetworkMessage`. I want to test `NetworkMessage` encoding. but that doesn't impl Encodable/Decodable. only `RawNetworkMessage` does
05:47 < elichai2> stevenroose: well what about connecting through the phone?
05:47 < elichai2> and you need to make sure you have some always online server
05:50 < stevenroose> elichai2: yeah having a server is the bare min requirement, sure. But that is the case for any more exotic setup as well :) And through the phone, ZNC would probably work just as well. It's just a bouncer, not a client. So your phone and your laptop can connect to the same bouncer.
05:52 < stevenroose> I just notice that the message serialization code allocates the data and then writes it into the writer.
05:52 < stevenroose> That's not nice. We should be able to do that with a HashWriter of some kind.
05:52 < stevenroose> Let me look at that.
06:00 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin
06:09 < elichai2> I'm not sure what you're trying to say about the use-serse
06:15 < stevenroose> That what we mean with cfg(feature = serde) is not "is the feature serde is enabled" but "if the serde crate is built"
06:15 < stevenroose> elichai2: but there is just no other way to say the latter
06:15 < stevenroose> I just realized that using a hashwriter doesn't work because Bitcoin's stupid checked data thing requires the length of the payload be hashes as well
06:16 < elichai2> hmmm. but we have a feature name
06:16 < elichai2> you can still calculate the length into some local array/vec without allocating the rest of the data i think
06:17 < stevenroose> well basically it requires the payload be length-prefixed*
06:18 < stevenroose> elichai2: heh yeah the length is actually not hashes
06:19 < stevenroose> so we could write zeroes for the length, then hash-write the payload and then fill in the length lateron. That's a but hacky though.
06:21 < stevenroose> elichai2:
06:21 < stevenroose> shit. that doesn't work because lengths are VarInts
06:21 < stevenroose> So that would only work if we reserve the max varint size for that length.
06:24 < stevenroose> BlueMatt: are non-minimal varints illegal in Core? In rust-bitcoin we error if we get one.
06:25 < stevenroose> ah nvm, the payload length is u32, not varint
06:28 -!- idurando [bec12591@190.193.37.145] has joined #rust-bitcoin
06:29 < stevenroose> elichai2: yeah that also doesn't work :/ because you can't rewind an &mut dyn io::Write
06:30 < stevenroose> how annoying
06:30 -!- idurando [bec12591@190.193.37.145] has quit [Remote host closed the connection]
07:22 < elichai2> could you link to the code you're talking about?
07:24 < stevenroose> elichai2: I was talking about code is master
07:24 < stevenroose> RawNetworkMessage serializes payload into a vector and then serializes CheckedData(&payload), which serializes the length, payload and checksum
07:25 < stevenroose> The length prefix is there as DoS protection, I guess.
07:26 < elichai2> I'm looking at the code now
07:26 < stevenroose> I was trying to get rid of that allocation
07:27 < stevenroose> But it's not possible.
07:27 < elichai2> You're talking about the `serialize(dat)` and CheckedData right?
07:27 < stevenroose> uhu
07:27 < elichai2> I think the problem is CheckedData
07:27 < stevenroose> The problem is the payload length that needs to be serialized.
07:29 < elichai2> where is it serialized? I see `serialize(dat)` which just plainly serializes it. and then again consensus_encode for the checked data ontop of the serialized message
07:56 < stevenroose> > RawNetworkMessage serializes payload into a vector and then serializes CheckedData(&payload), which serializes the length, payload and checksum
07:56 < stevenroose> elichai2: ^
07:57 < elichai2> Right. So the problem is the checksumming, that we can't read through twice
07:58 < elichai2> Oh do we also need the length *before* the payload?
07:58 < elichai2> (ie in the hash)
08:54 < stevenroose> yes, that's why I'm saying
08:54 < stevenroose> > The problem is the payload length that needs to be serialized.
08:54 < stevenroose> :)
08:55 < stevenroose> elichai2: I have a branch now that uses a HashWriter to get the checksum while serializing, but you still need to allocate because of the length prefix :(
08:55 < stevenroose> It's a good DoS measure, though.
08:57 < stevenroose> We could have an optional size_hint method that can return something if the size can be known without serializing, but I'm not sure if the CPU overhead to do that is worth the allocation. I guess Andrew would say yes :D
08:58 < stevenroose> If we really want to, we can serialize twice, first into the void just to get the length :D
08:58 < elichai2> You should be able to use a stack array for that I think
08:59 < elichai2> Like the length shouldn't be more than ~4-8 bytes
08:59 < elichai2> I want to try and minimize allocations in bitcoin core serialization code. Because that takes so much time in IBD
09:00 < elichai2> But the templating and operator overloads make this so damn hard
09:46 -!- dongcarl[m] [dongcarlca@gateway/shell/matrix.org/x-fcykwvmbaqfkvzmv] has quit [Quit: killed]
09:46 -!- dpc [dpcmatrixo@gateway/shell/matrix.org/x-iilvzzsgjancbebs] has quit [Quit: killed]
09:53 -!- rjected [~rjected@natp-128-119-202-34.wireless.umass.edu] has quit [Quit: WeeChat 2.6]
09:56 -!- andytoshi [~apoelstra@wpsoftware.net] has joined #rust-bitcoin
09:56 -!- andytoshi [~apoelstra@wpsoftware.net] has quit [Changing host]
09:56 -!- andytoshi [~apoelstra@unaffiliated/andytoshi] has joined #rust-bitcoin
10:02 -!- dpc [dpcmatrixo@gateway/shell/matrix.org/x-mwwcxztqjhxkzexm] has joined #rust-bitcoin
10:41 -!- dongcarl[m] [dongcarlca@gateway/shell/matrix.org/x-qhskxpsifvkcwuvu] has joined #rust-bitcoin
11:15 < stevenroose> elichai2: templating and operator overloads?
11:16 < stevenroose> I think the amount of allocations is already quite low. Whenever I see things that can be optimized, I PR them. I haven't seen everything, though.
11:16 < stevenroose> Ah you're talking Bitcoin Core.
11:17 < elichai2> Yeah. It's ~8% of IBD
11:19 < elichai2> (the new operator)
11:25 < stevenroose> andytoshi: doesn't rust-miniscript do script verification? Ah no only miniscript verification.
11:26 < andytoshi> only miniscript verification, and not efficiently
11:26 < andytoshi> the point is to determine which spending conditions were used
11:26 < andytoshi> not to get a binary pass/fail
11:26 < stevenroose> We do have bitcoinconsensus, though
11:37 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Remote host closed the connection]
11:37 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin
13:15 -!- andytoshi [~apoelstra@unaffiliated/andytoshi] has quit [Ping timeout: 260 seconds]
13:52 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has quit [Quit: jonatack]
14:30 -!- Kiminuo [~mix@141.98.103.100] has quit [Quit: Leaving]
14:30 -!- andytoshi [~apoelstra@unaffiliated/andytoshi] has joined #rust-bitcoin
14:31 < andytoshi> stevenroose: do you feel strongly about sha256d::Hash vs MerkleNode?
14:31 < andytoshi> i could really go either way on this
14:31 < andytoshi> but i would really like to get the PR in :)
14:34 < stevenroose> yeah I'm fine
14:35 < stevenroose> I feel more strongly about the root being named differently. And I wasn't sure what I understood was the conclusion on that.
14:35 < stevenroose> But I would also prefer to just merge the damn thing already :D
14:38 < andytoshi> yeah, i'd prefer it be called root
14:38 < andytoshi> but w/e, node also works
14:38 < andytoshi> users can always rename the type themselves when they import it :P
14:38 -!- jonatack [~jon@2a01:e0a:53c:a200:bb54:3be5:c3d0:9ce5] has joined #rust-bitcoin
15:11 < andytoshi> several of the commits in #349 fail to compile
18:33 -!- andytoshi [~apoelstra@unaffiliated/andytoshi] has quit [Ping timeout: 258 seconds]
18:49 -!- andytoshi [~apoelstra@unaffiliated/andytoshi] has joined #rust-bitcoin
21:56 -!- vindard [~vindard@190.83.165.233] has joined #rust-bitcoin
22:49 -!- vindard [~vindard@190.83.165.233] has quit [Remote host closed the connection]
22:51 -!- vindard [~vindard@190.83.165.233] has joined #rust-bitcoin
--- Log closed Tue Dec 24 00:00:46 2019