--- Day changed Wed Jul 25 2018
00:02 -!- savil [~savil@c-98-207-86-82.hsd1.ca.comcast.net] has quit [Read error: Connection reset by peer]
00:11 -!- savil [~savil@c-98-207-86-82.hsd1.ca.comcast.net] has joined #rust-bitcoin
00:42 < BlueMatt> for those who missed it: rust-lightning can: (a) make and accept connections, (b) make and accept channels, confirm the funding and do the close dance, at least in the case you have no pending HTLCs (c) send payments both single-hop and multi-hop!
00:43 < BlueMatt> there's a simple example client at https://github.com/TheBlueMatt/rust-lightning-bitcoinrpc that can do all of the above via a simple interactive shell, though it doesn't persist most state to disk yet
00:43 < BlueMatt> and I havent pushed it all yet
00:44 < BlueMatt> or, havent pushed all the stuff to rust-lightning-bitcoinrpc, all the work on rust-lightning is there
00:52 < BlueMatt> anyone seen sgeisler recently?
00:59 < ariard42> Really cool for the client!
00:59 < ariard42> I'll have a look after I'm done with logging shit
02:48 -!- savil [~savil@c-98-207-86-82.hsd1.ca.comcast.net] has quit [Ping timeout: 248 seconds]
02:58 -!- savil [~savil@73.93.152.136] has joined #rust-bitcoin
03:05 -!- savil [~savil@73.93.152.136] has quit [Ping timeout: 240 seconds]
03:56 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has joined #rust-bitcoin
04:16 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
04:30 < dongcarl> BlueMatt: what can't rust-lightning do yet?
14:00 < andytoshi> dongcarl: in https://github.com/rust-bitcoin/rust-secp256k1/pull/38/files i implement serde 1.0 support for rust-secp, in which all types are serialized/deserialized as byte arrays
14:00 < andytoshi> no hex anywhere
14:02 < andytoshi> though it might be more ergonomic to check `Serializer::is_human_readable` and use hex for human-readable formats like json
14:04 < andytoshi> there isn't a nice way to unit test that though, without adding a dev-dependency on a json lib or something, so i'd rather not
15:32 < BlueMatt> dongcarl: see https://github.com/rust-bitcoin/rust-lightning/issues
15:32 < BlueMatt> there's still some basic things like persisting to disk
15:32 < BlueMatt> and after that its filling out all the error handling
15:33 < BlueMatt> the structures are there for it, just gotta actually figure out which of several error handling options we should do for each error site
15:36 < BlueMatt> dongcarl: also there are some limitations in https://github.com/rust-bitcoin/rust-lightning-invoice/issues that prevent invoice generation in rust-lightning-bitcoinrpc
15:38 < BlueMatt> andytoshi: it looks like sgeisler may be mia (he's a student right, so, dunno, summer break or something?)
15:40 < andytoshi> BlueMatt: yeah, i've noticed that (though also, as you've noticed i tend to disappear too for weeks at a time, just other projects..)
15:40 < BlueMatt> yea, not mad at him, just pointing it out
15:40 < BlueMatt> obviously thats also why its all in rust-bitcoin org
15:40 < andytoshi> cool. yeah
15:40 < BlueMatt> so we could theoretically merge shit if its blocking
15:40 < BlueMatt> though i guess not publish to crates, but you can switch the dep to git
15:40 < andytoshi> right, that's kinda frustrating..
15:41 < andytoshi> i would like https://github.com/rust-bitcoin/rust-bech32-bitcoin/pulls merged and published but i think i need clark moody to do it
15:41 < BlueMatt> I mean you could take it and switch the dep to a git commitid
15:41 < BlueMatt> has clark been mia too?
15:41 < andytoshi> he's never been super active, hard to tell
15:41 < BlueMatt> oh, wait, huh @sgeisler sgeisler changed the title from WIP: Use bech32 v0.6.1 to WIP: Use bech32 v0.7.0 23 minutes ago
15:41 < BlueMatt> i guess he's around
15:41 < BlueMatt> does he irc?
15:41 < andytoshi> heh
15:42 < andytoshi> yeah, i've talked to him here
15:44 < andytoshi> going from rust-bech32-bitcoin 0.6.1 to 0.7.0 i think is insufficient, we need my PR as well (which bumps to 0.8.0)
16:03 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has joined #rust-bitcoin
16:13 -!- TamasBlummer [~Thunderbi@p200300E38F029A99E4CF8178E7CF33BC.dip0.t-ipconnect.de] has joined #rust-bitcoin
17:14 -!- savil_ [~savil@2604:a880:2:d0::2119:6001] has joined #rust-bitcoin
17:16 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has quit [Quit: Textual IRC Client: www.textualapp.com]
17:20 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has joined #rust-bitcoin
17:25 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has quit [Quit: Textual IRC Client: www.textualapp.com]
17:26 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has joined #rust-bitcoin
17:29 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has left #rust-bitcoin []
18:13 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has joined #rust-bitcoin
18:18 -!- elichai2 [uid212594@gateway/web/irccloud.com/x-fakuekoyqqpcfyhv] has quit [Quit: Connection closed for inactivity]
18:24 -!- savil_ [~savil@2604:a880:2:d0::2119:6001] has left #rust-bitcoin ["WeeChat 1.4"]
19:09 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has quit [Quit: Textual IRC Client: www.textualapp.com]
19:12 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has joined #rust-bitcoin
19:13 -!- savil_ [~savil@2604:a880:2:d0::2119:6001] has joined #rust-bitcoin
19:14 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has left #rust-bitcoin []
19:35 < BlueMatt> rust-bitcoin: Artisinal Bitcoin: Opening a Lightning Channel with your own Lightning implementation, mining the opening transaction with your own mining software stack with your own hashpower
19:36 -!- mode/#rust-bitcoin [+o BlueMatt] by ChanServ
19:37 -!- BlueMatt changed the topic of #rust-bitcoin to: Rust-Bitcoin Libraries Discussion. Artisinal Bitcoin for the rest of us. See https://github.com/rust-bitcoin If you created a repo, and I forgot to make you admin on it (stupid github defaults), ping BlueMatt
19:37 -!- mode/#rust-bitcoin [-o BlueMatt] by BlueMatt
20:14 < andytoshi> awesome, the new version of rust-bitcoin #100 no longer crashes on the test case that the fuzzer found
20:14 < andytoshi> i'll fuzz for 20 or 30 minutes then ack it
20:40 < andytoshi> i see something that looks like a fuzzer error, but i can't reproduce the crash
20:40 < andytoshi> SIGNAL: SIGABRT (6)
20:40 < andytoshi> FAULT ADDRESS: (nil)
20:40 < BlueMatt> whats it look like?
20:40 < BlueMatt> SIGABRT is usually rust panic
20:40 < andytoshi> does that (nil) suggest that it's just complaining about being terminated?
20:40 < andytoshi> one sec i'll 0bin
20:40 < BlueMatt> yea
20:40 < BlueMatt> is there nothing in the crashes folder
20:40 < BlueMatt> that starts with SIGABRT....
20:41 < andytoshi> there is one thing, it is TB1d412TB5d440
20:41 < andytoshi> and your shell script outputs a hex-ified version of that
20:41 < andytoshi> but when i stick it into the unit test, it passes
20:41 < BlueMatt> ugh
20:41 < BlueMatt> weird
20:42 < andytoshi> one sec, vim messed up my terminal scrollback, i gotta redo it ... but i will 0bin in a couple mins
20:42 < BlueMatt> I've only ever had that when I confused myself as to what branch/file I was looking at
20:42 < BlueMatt> which I have done many times
20:42 < andytoshi> like, maybe the fuzzer is running different code than `cargo test` is?
20:43 < BlueMatt> no, I mean the scripts are written to literally call the same function
20:43 < BlueMatt> so I'd be surprised
20:43 < andytoshi> https://0bin.net/paste/TYRhIXDvduFtcCnM#h4cIfKRE0PiPaH1FbEyGh1yGN4ZhvWQqqwDGTNea5jd
20:44 < BlueMatt> and that hex string is whats in the file it told you?
20:44 < BlueMatt> FUZZ_FNAME: hfuzz_workspace/deserialize_address/SIGABRT.PC.7ffff721086b.STACK.193758a7c1.CODE.-6.ADDR.(nil).INSTR.mov____0x108(%rsp),%rcx.fuzz
20:44 < andytoshi> yes
20:45 < BlueMatt> and cd fuzz && cargo test passes when you put that string in the deserialize_address.rs file in place of the dummy "00"?
20:45 < andytoshi> yes
20:45 < BlueMatt> you didnt miss the "cd fuzz" part, right?
20:45 < andytoshi> nope
20:45 < BlueMatt> ugh, hmm, welp, I'm out of ideas :/
20:45 < andytoshi> it's clearly running the `duplicate_crash` test of the deserialize_address thing
20:46 < andytoshi> heh
20:46 < BlueMatt> cargo test --release?
20:46 < andytoshi> just tried that, also passed
20:46 < BlueMatt> ohoh, out of memory?
20:46 < andytoshi> hmmm that could be it
20:46 < BlueMatt> try running the test binary with ulimit set low
20:46 < BlueMatt> like cargo test --release and then find the binary it generated and run just the one test
20:47 < andytoshi> i'm fuzzing in top and the memory usage seems constant.. i have about 1.5Gb free, that's not changing
20:47 < andytoshi> aaand it crashed
20:47 < andytoshi> so i don't think that's it
20:47 < BlueMatt> nono, the fuzzer sets the limit low
20:47 < BlueMatt> on purpose
20:47 < andytoshi> ah
20:47 < BlueMatt> deserializing a small thing like that shouldnt eat 100MB unless you have a bug :p
20:48 < andytoshi> wait, is that what -N does?
20:48 < BlueMatt> no -N is "number of iterations"
20:49 < andytoshi> ok, cool, i'm not crazy :P
20:49 < BlueMatt> it calls the program that many times then gives up and assumes its fine
20:49 < BlueMatt> there is a separate option for memory limit, though i dont recall what it is
20:49 < BlueMatt> and it defaults to something low-ish
20:50 < andytoshi> heh, even with a ulimit of 2, the test binary works
20:50 < andytoshi> which makes me suspicious of ulimit more than anything..
20:50 < BlueMatt> ulimit -m 2?
20:51 < andytoshi> lol thank you
20:51 < andytoshi> why the hell does it default to file size
20:51 < BlueMatt> that should break everything, -m is in kbytes lol
20:51 < andytoshi> hm, it still works even with `ulimit -m 0
20:52 < andytoshi> and jst `ulimit -m` confirms that it set to 0
20:52 < BlueMatt> i think 0 means "unlimited"
20:52 < andytoshi> ok. it also works with 1, 10, 100, 1000
20:52 < BlueMatt> or try ulimit -v?
20:52 < andytoshi> what's -v?
20:52 < andytoshi> my manpage doesn't have any of these flags
20:52 < BlueMatt> virtual memory
20:52 < andytoshi> even though ulimit clearly supports them
20:53 < BlueMatt> ulimit -a will tell you useful things
20:53 < andytoshi> nice, thanks!
20:53 < andytoshi> there we go, -v 1000 causes it to crash
20:54 < BlueMatt> ok, phew
20:54 < BlueMatt> now good luck debugging *where* it crashes
20:54 < andytoshi> lol
20:54 < BlueMatt> try like 100_000 and make sure it still crashes
20:54 < BlueMatt> cause i think thats honggfuzz's default limit
20:55 < andytoshi> hmm, no crash at 100000
20:56 < andytoshi> crashes at 25000 but not at 50000
20:56 < andytoshi> oh, though i'm running the release test binary
20:56 < andytoshi> i'll bet the debug one crashes higher
20:56 < BlueMatt> the fuzzer runs release
20:56 < BlueMatt> oh, huh, your log says  ASLimit         : 0 (MiB)
20:56 < BlueMatt>  RSSLimit        : 0 (MiB)
20:56 < BlueMatt>  DATALimit       : 0 (MiB)
20:57 < BlueMatt> I'm still left somewhat confused here
20:57 < BlueMatt> see if it crashes in an obvious place (like a ::with_capacity(big_number))
20:57 < BlueMatt> and if it doesnt I'll dig into it
20:57 < BlueMatt> more
20:57 < andytoshi> like, use ulimit to trigger a crash then gdb it?
20:58 < BlueMatt> well I usually use ulimit to trigger a crash and then println!() it, but, sure lol
20:58 < andytoshi> lol fuck gdb does not work with so little memory
21:00 < andytoshi> the crash seems to happen before i even enter the unit test
21:00 < andytoshi> i think this is a red herring
21:00 < BlueMatt> quite possibly
21:03 < dongcarl> andytoshi: yeah... I'd like the more ergonomic way but what you have works now...
21:03 < dongcarl> what are we using serde for anyways? persisting to disk?
21:03 < andytoshi> yes
21:03 < andytoshi> (in liquid)
21:03 < dongcarl> Okay makes sense
21:04 < andytoshi> it would be nice though if this stuff also worked with core's jsonrpc format, which outputs pubkeys as hex strings
21:04 < andytoshi> and secret keys as base58ck strings, i guess :(
21:05 < dongcarl> I'm implementing more test for psbt... wondering if there's a way for me to recognize which pubkeys are used in a script? Like for an updater he can just hand the API a bunch of pubkeys and they'll get dumped in the right structs
21:05 < andytoshi> it's a real PITA to find pubkeys in scripts
21:05 < andytoshi> well, depending how complex you think the scripts might be
21:06 < dongcarl> Here: https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki
21:06 < dongcarl> Grep for " Given the above PSBT, an updater with only the following: "
21:07 < dongcarl> I guess for this example you'd also need to be able to recognize which script goes with which input
21:07 < andytoshi> looks like the public keys are listed explicitly
21:08 < dongcarl> What I mean is, given a script and a list of pubkeys, how can I match up the scripts with relevant pubkeys
21:08 < andytoshi> i'd just search the scripts for the keys
21:09 < dongcarl> At least that's what the core API does, they add the keys and scripts to a wallet, and there's a function to automatically fill up the PSBT
21:09 < andytoshi> as byte vectors
21:09 < dongcarl> Right
21:09 < dongcarl> I guess it won't be possible to accidentally match a pubkey on something that's not it
21:09 < andytoshi> well, no, it will be possible if you're treating scripts as byte vectors
21:09 < andytoshi> because you won't be respecting push boundaries and stuff
21:10 < andytoshi> the correct thing is to use `Instructions` iterator in rust-bitcoin script module
21:10 < dongcarl> So I should make it into an iter of Instructions?
21:10 < andytoshi> yeah
21:10 < dongcarl> okay cool
21:10 < dongcarl> cool
21:12 < andytoshi> BlueMatt: lol wtf, so i learned about `hfuzz run-debug` which will re-run on a crashfile and drop you into gdb
21:12 < andytoshi> which tells me "This crashfile didn't trigger any panics..."
21:13 < andytoshi> drops me into a gdb session with no program running, since it exited successfully
21:13 < andytoshi> and then when i quit gdb, i get the crash output similar to in the 0bin
21:14 < andytoshi> https://0bin.net/paste/ZC0HeTbU9TzoTfYT#f4iBSKa-zbgpXWv9bEzrSxxDlbznHvoIeNhhDEUMvt9
21:14 < andytoshi> i'm gonna update my nightly rustc and see if that fixes it..
21:14 < andytoshi> but i'm pretty confident now this is a problem with my system/toolchain, not rust-bitcoin
21:15 < dongcarl> andytoshi: naive question but doesn't Rust use LLVM?
21:15 < andytoshi> yeah, it's lldb, not gdb
21:17 < andytoshi> waait a minute
21:18 < andytoshi> `hfuzz_workspace/$FILE/HONGGFUZZ.REPORT.TXT` is not deleted
21:18 < andytoshi> travis-fuzz.sh looks for this file and outputs it if it's there ... but it was last updated 6 hours ago
21:18 < andytoshi> dammit
21:18 < BlueMatt> no it is not
21:18 < BlueMatt> oh lolol
21:18 < BlueMatt> sorry, that script wasnt really written for people to use themselves
21:18 < BlueMatt> I have my own fuzzing scripts locally lol
21:18 < andytoshi> no worries :P
21:18 < andytoshi> i should write some myself
21:19 < andytoshi> i like that travis-fuzz.sh is so short and simple, it's easy to learn from
21:20 < BlueMatt> yea, this stuff is pretty easy to run
21:20 < BlueMatt> just run like cargo hfuzz run yourself lol
21:20 < BlueMatt> I have a ton of scripts that manage like honggfuzz and afl runs on the same file sets and then also across hosts, etc
21:21 < BlueMatt> but even those are pretty short
21:22 < andytoshi> TamasBlummer: i think you are blocking https://github.com/rust-bitcoin/rust-bitcoin/pull/100 because you "requested changes" through the github api
21:22 < andytoshi> oh, no, i can dismiss his review maybe
21:23 < andytoshi> ok, unblocked, never mind
21:27 -!- TamasBlummer [~Thunderbi@p200300E38F029A99E4CF8178E7CF33BC.dip0.t-ipconnect.de] has quit [Remote host closed the connection]
21:36 < andytoshi> dongcarl: do you have a couple minutes to do a cursory review of https://github.com/rust-bitcoin/rust-secp256k1/pull/38 ?
22:18 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has joined #rust-bitcoin
22:38 -!- savil [~savil@c-71-202-1-72.hsd1.ca.comcast.net] has left #rust-bitcoin ["Textual IRC Client: www.textualapp.com"]
23:42 < dongcarl> andytoshi: sorry didn't see, doing so now
23:47 < dongcarl> andytoshi: very simple. I like it
23:49 < andytoshi> great, thanks
23:49 < andytoshi> i'll merge it and have a "bump version  to 0.10.0" pr in the next 90 seconds
23:50 < andytoshi> https://github.com/rust-bitcoin/rust-secp256k1/pull/39
23:51 < andytoshi> now that rust-secp has serde 1.0 support my plan is to similarly update rust-bitcoin to use (feature-gated) serde 1.0, using similar binary encoding for byte-string types
23:52 < andytoshi> then the only hex encoding left should be in Sha256dHash's LowerHex/UpperHex impls, which are fine (they basically just pass through to the same impls for u8)
23:52 < andytoshi> and then maybe `hex` will be a dev-dependency but it won't need to be a main dependency
23:53 < andytoshi> and we should be set for 0.14.0
23:53 < dongcarl> I have a general design question about the rust-bitcoin repos, I see a lot of `fn is_valid` for a lot of the structs that are defined. Why is that? If we make sure that 1. fields are private 2.  methods of that struct only do valid state changes 3. deserializing returns a Result, wouldn't we eliminate the need for `is_valid`?
23:54 < dongcarl> andytoshi: the above sounds fantastic, lmk if I can be of service
23:55 < andytoshi> dongcarl: we should take those on a case-by-case basis. there are some things like `Transaction` where it's easiest if all members are public
23:55 < andytoshi> but in general you're probably right that i overdo this .. a lot of rust-bitcoin was designed alongside rust 0.6 and never really evolved
23:56 < andytoshi> now that we got rid of PublicKey::is_valid, there are no more `is_valid` methods in rust-secp for example, nor is there any way for a context object to have the wrong capabilities for a function
23:57 < dongcarl> andytoshi: easy to access? or to initialize?
23:59 < dongcarl> sidenote: https://docs.rs/bitcoin/0.13.2/bitcoin/blockdata/script/struct.Script.html#method.to_v0_p2wsh works for witnessscripts too no?