--- Day changed Thu Nov 15 2018
00:13 < stevenroose> Is there a place in rust-bitcoin where cumulative proof-of-work is calculated?
00:14 < stevenroose> Aha, rust-bitcoin-chain does it
00:27 < sgeisler> I think my bitcoin_constants PR is complete enough now that it could replace the custom network enums used in rust-bech32-bitcoin and rust-bitcoin
00:27 < sgeisler> if anyone wants to review: https://github.com/rust-bitcoin/bitcoin-constants/pull/1
00:28 < sgeisler> ^^^ andytoshi, dongcarl, stevenroose
01:03 < stevenroose> sgeisler: If I'm honest, I'm not the biggest fan of (1) trying to keep non-Bitcoin data there. I think any project that does more than one network will also have a direct dependency to more than one network-specific crate. And (2) adding the complexity of multiple traits instead of just a simple global struct :(
01:13 < sgeisler> I'm personally not investid in any of the altcoins (that's also why only partial data is provided for them) but what I want to achieve is (1) interoperability between rust-bitcoin crates and (2) avoiding redundancy. the bech32-bitcoin crate for example supports litecoin and vertcoin (and lightning will probably want to be able to support LTC too for cross chain experitments) but andytoshi doesn't want to support them in rust-bitcoin.
01:13 < sgeisler>  The current version of the PR tries to avoid havingto make a decision on a blessed set of supported networks.
01:14 < sgeisler> That's why there is a trait that everyone can implement for an enum containing his shitcoin.
01:15 < sgeisler> And in theory he should be able to use rust-bech32-bitcoin, rust-bitcoin, rust-lightning-invoice without any problems as long as his shitcoin is just a clone of a up to date version of BTC.
01:19 < sgeisler> I would find it quite sad if someone had to clone all the libraries, rename them rust-litecoin-* and just rebase some patches on top of them from time to time just to support a 1-to-1 clone of BTC.
01:21 < sgeisler> With monomorphization there isn't even an increase in code size or runtime vs. the current enum approach
01:35 < dongcarl> Sure
10:30 < stevenroose> sgeisler: hmm, so if it's trivial for altcoins to define their own implementations of the traits, we can in fact drop them?
10:30 < stevenroose> And they just need rust-vertcoin-constants to implement the trait?
10:31 < stevenroose> sgeisler: how would the "network_from_property" thing work for those externally defined coins?
16:07 < dongcarl> andytoshi: w/re API for serialization in secp, I am also uncomfortable with returning `Vec`s for `Signature`... Do you think there's any good solution other than waiting on https://github.com/rust-lang/rust/issues/48055?
16:08 < dongcarl> Perhaps I'll implement `write_into` for `Signature` as well?
16:14 < BlueMatt> TamasBlummer: oops, should document that better, but you should return Err(ChainError::NotSupported)
16:14 < BlueMatt> https://docs.rs/lightning/0.0.6/lightning/chain/chaininterface/enum.ChainError.html
16:14 < BlueMatt> TamasBlummer: you can freely not support that function at all, or support it only partially, as you see fit
16:28 <+andytoshi> dongcarl: i think `Signature` should have a `write_der` function that writes into an io::Write
16:28 <+andytoshi> and returns the serialized lencgth
16:28 <+andytoshi> tbh i don't see the purpose of having such functions for public keys
16:29 <+andytoshi> i guess you could make one that took a boolean indicated compressedness, and that'd be more ergonomic for some usescases
16:29 < dongcarl> > tbh i don't see the purpose of having such functions for public keys
16:29 < dongcarl> At least not in secp, I will implement it in `rust-bitcoin` as it'll know about the compressedness
16:30 <+andytoshi> yeah that makes sense
16:43 < stevenroose> I'm having a silly rust issue.  I have trezor-api crate that depends on bitcoin, but needs a [patch] for PSBT.  Then I have another crate depending on trezor-api (using path=) and on bitcion with the same [patch].  Now somehow trezor's [patch] is not used and building gives errors on psbt types in the trezor crate not existing :/
16:44 < stevenroose> I also tried copying the path= line of trezor into the top crate's [patch] section.
16:44 < stevenroose> So basically I cannot use a patched dependency.
16:55 <+andytoshi> it should work if you have explicit paths everywhere
17:05 < BlueMatt> andytoshi: ugh, I really hate that tripplet guy's responses on that thread
17:05 < BlueMatt> he appears to completely and deliberately ignore the complaints about rust stability being stated there
17:06 < BlueMatt> like, uh, can rust seriously not commit to the compiler codebase being compileable with at least a few previous versions? what about supporting a rust compiler for more than the duration of until the next compiler is released
17:07 < BlueMatt> like, uh, literally every other software project ever does?
17:14 < stevenroose> andytoshi: I can't have explicit paths to bitcoin (outside of [patch]) because then it will conflict with other deps that depend on the bitcoin crate and I receive those message like "expected Address, but got Address" :D
17:15 < stevenroose> Because cargo thinks a crates-io dep is a different crate than a path-dep
17:17 <+andytoshi> BlueMatt: yeah, he really bugged me too
17:17 <+andytoshi> stevenroose: make them all explicit paths :P
17:18 < BlueMatt> andytoshi: is there a rust dev irc channel I can go make obnoxiously passive aggressive comments in?
17:19 <+andytoshi> BlueMatt: there is #rust-internals on irc.mozilla.org but irritatingly most of the chat has moved to some website called "zulip"
17:19 <+andytoshi> which has no irc bridge and is generally a pile of shit
17:19 < BlueMatt> ffs
17:19 < BlueMatt> man the rust ecosystem bugs the shit out of me sometimes
17:19 <+andytoshi> yeah :/
17:19 < BlueMatt> rust is a great language, but its ecosystem almost makes me want to stop using rust
17:20 < BlueMatt> the "nah, we do it our own way, fuck the way literally everyone else does it, clearly ours is better, cause we said so" approach is obnoxious af
17:20 <+andytoshi> agreed
17:27 < stevenroose> I think "fuck the way literally everyone else does it" is double.  I think they are trying to do it the "more modern way", where it's normal to assume users update the second things are released, don't care about huge dependency trees, don't verify each others' code as long as it works, communication is on some proprietary web-based thingy, ...
17:28 < stevenroose> Basically the trend "modern" javascript started
17:29 < stevenroose> I think most big open sourceprojects since the last 3 years or so do it that way. gitter/slack/discourse + centralized package manager + barely community RFC, just public for the face but actually decided by the team owning the project, ...
17:31 < stevenroose> I was quite surprised how much Rust looks like javascript when you start to use all those futures-based frameworks like tokio, hyper, and the like
17:33 < BlueMatt> stevenroose: fair, I guess its how modern shit is built, but that has never been true for systems programming, and introducing that in systems programming is obnoxious af
17:33 < BlueMatt> I found tokio to be a bit more comfortable in terms of dep tree, at least its like only one or two nursery projects, but, indeed, nursery projects appear to have exactly 0 code review
17:33 < BlueMatt> which i find distasteful
17:34 < esotericnonsense> heh
17:34 < esotericnonsense> stevenroose: the "crates are everything" mindset I find very very odd, yes.
17:34 < esotericnonsense> it feels like key features are arbitrary things pulled in from non-vetted repositories.
17:35 < BlueMatt> the though-must-install-rust-via-curl-piped-to-shell thing that all the rust people espouse is honestly massively offputting
17:36 < esotericnonsense> don't
17:36 < esotericnonsense> my god
17:36 < BlueMatt> I dont, but its essentially the only supported way to use rust
17:36 < BlueMatt> or, recommended at least
17:36 < esotericnonsense> just even mentioning it makes me squirm
17:36 < esotericnonsense> heh
17:36 < stevenroose> Actually, that's one thing I liked about Go when it came out.  While all other languages coming out came with a default package manager (npm, pip, gems, lein, pub, ...), Go didn't. It's still mostly Git-based.
17:36 < esotericnonsense> yeah, it's honestly utterly baffling
17:37 < BlueMatt> I do really like the cargo/crates package management stuff, but the ecosystem approach of using dependencies freely without audit is....
17:37 < stevenroose> It should be more possible to use cargo without crates. F.e. I should be able to specify a dependency using Git and tag v1.0.0 and use it in a compatible way with deps that return cratio-io-v1.0.0 types
17:38 < BlueMatt> it is?
17:38 < stevenroose> It's not possible.
17:38 < BlueMatt> you can depend on git trees with commitish'es
17:38 < BlueMatt> maybe I'm missing what you're asking?
17:38 < stevenroose> I made a crate that had all dependencies as git. But then the types of those deps are not identical to the types of the crates.io types.
17:39 <+andytoshi> BlueMatt: you can't publish to crates.io if you do that
17:39 < stevenroose> so bitcoin(git=v1.0.0)::Address != bitcoin(crates=1.0.0)::Address
17:39 < stevenroose> So you can't interoperate between them.
17:39 < BlueMatt> ah, the dep type is broken indeed
17:39 <+andytoshi> yeah, that's stupid, cargo has some sort of hash of every project, surely it can tell that a local copy is the same as a crates copy
17:39 < stevenroose> Should be possible to have cross-source versioning or something.
17:39 < BlueMatt> yea, the no-publish rule is weird, I mean I guess that makes sense so you don't depend on something arbitrary that goes away (leftpad...)
17:40 <+andytoshi> yeah, it's just annoying
17:40 < BlueMatt> but you should be able to say "git.cargo.rs, vX"
17:40 < stevenroose> BlueMatt: that exists??
17:40 < BlueMatt> no?
17:40 < BlueMatt> I'm saying it should :p
17:40 < stevenroose> ah
17:41 < stevenroose> k, true :) Is it actually possible / does it happen, to have rust-dependencies via your OS package manager?
17:41 < BlueMatt> tbh I was hoping most of this was language maturity stuff, and to some extent some of it will go away as rust runs into the real world (ala debian...)
17:41 <+andytoshi> yeah, it's actually quite difficult to compare crates.io's versions of packages to the corresponding git tags (because crates does not track that anywhere, it basically considers published crates to be blobs from nowhere)
17:41 < stevenroose> With python it's quite common actually and i like it.
17:41 <+andytoshi> so you've gotta find the code in your ~/.cargo, copy it into a git repo, and run `git diff` to see if it differs
17:41 < stevenroose> I always use pacman for python deps instead of pip.
17:42 < BlueMatt> but a lot of this appears to be The Way Rust Should Do It (tm)
17:42 <+andytoshi> BlueMatt: i think it will improve yes, because the debian folks aren't going to change their attitude (nor will opensuse, or rhel, etc)
17:42 < BlueMatt> which is just fucking stupid
17:42 <+andytoshi> it will probably wind up with there being a parallel rustc distribution mechanism
17:42 <+andytoshi> that distributions use, and which depends on mrustc or something instead of bootstrapping from a binary rustc
17:42 < BlueMatt> I'm curious what will happen when a build system fails to connect to cargo and fails to build a package
17:42 < esotericnonsense> yeah that's pretty much how I expect it to go
17:42 < BlueMatt> will debian cry bloody murder and demand to be able to build sans-crates
17:43 < esotericnonsense> (similar to python, basically)
17:43 <+andytoshi> BlueMatt: so, fwiw `cargo vendor` has been a thing for a long time .. we use it at blockstream actually
17:43 <+andytoshi> so we can download all our deps once and forget about crates.io after that
17:43 < stevenroose> haha, I just installed it to try and fix the issue I had. without success
17:43 < BlueMatt> yea, but thats just persisted to ~/.cargo, no?
17:44 < BlueMatt> I mean you can also cargo build --frozen or something like that iirc
17:44 <+andytoshi> BlueMatt: no, we have a vendor/ directory in our source control that we can distribute
17:44 <+andytoshi> actually i forget where we put it.. but it's not just "in .cargo somewhere"
17:45 < stevenroose> Go also does that now.  I did that for a long time.  Get's nasty when you have the whole Ethereum history in your vendor directory because the vendoring tools don't prune it :D
17:45 < BlueMatt> ahhhh, thats nice...so whats gonna happen is debian et al are gonna start shipping a /usr/share/cargo/vendor and putting all the packages in there and hack up cargo to pull from that before crates
17:45 <+andytoshi> BlueMatt: that's what i think. and hopefully that workflow will be maintained because right now it's pretty poorly documented and has weird edge cases
17:45 < BlueMatt> yea
17:46 < BlueMatt> well that will be nice
17:46 <+andytoshi> though it is a million times better than it was before `cargo vendor`..
17:46 < BlueMatt> heh, yea
17:46 <+andytoshi> anyway, overall i'm optimistic. but folks like josh triplett and the webyolo people really piss me off
17:47 < esotericnonsense> vaguely related to rust (though still horrendous backend as I want to get an MVP before cleaning up)
17:47 < stevenroose> Let me crate an issue for the bitcoin(git=v1.0.0)::Address != bitcoin(crates=1.0.0)::Address thing :) see how they react
17:47 < esotericnonsense> https://files.esotericnonsense.com/public/bitcoin-top-wallet-send.jpg that box at left is a QR scanner, works on firefox android and desktop
17:48 < esotericnonsense> so i now have a sort of working bitcoin core android wallet :>
17:52 < stevenroose> esotericnonsense: I was thinking of trying to port bitcoin-top to use rust-bitcoincore-rpc, would that be an improvement?
17:54 < esotericnonsense> stevenroose: it certainly would! I think it would make sense to not do it just yet though because i'm probably going to blow away the backend entirely
17:56 < esotericnonsense> i wanted to get something actually working to prove that it's "possible" if that makes sense, to basically make a full web UI for bitcoind that includes everything you would reasonably want
17:56 < dongcarl> Is our channel logged?
17:56 < esotericnonsense> so spending tons of time on the backend not knowing if there's like, no reasonable javascript QR scanner or something, seemed like a bad use of time
17:57 <+andytoshi> dongcarl: no :/
17:57 <+andytoshi> it should be
17:58 < dongcarl> Yeah, would like to be able to link back to convos somtimes
17:59 < dongcarl> Lol botbot is down
18:00 < ariard> dongcarl: hey I have most of them since july if you want
18:00 < dongcarl> ariard: Oh that's great... Nah I'm thinking more like a public place where I can link to convos from github issues
18:00 < dongcarl> If anyone's got good logging services they use...
18:00 < dongcarl> lmk
18:06 < stevenroose> BlueMatt: irc.bitcoin.ninja would be a great place to host one ;)
18:09 < stevenroose> botbotme source code seems to be public, backend is in Go, web server in Python :)
18:14 < BlueMatt> can we not re-use the bitcoin-core-dev/lightning-dev botbot instance?
18:15 < BlueMatt> instead of running paralell infrastructure
18:15 < BlueMatt> I mean I'm obviously happy to put up a cname or something, but I dont really want to maintain more servers :p
18:15 < BlueMatt> last count I already had like 20 under my care
18:16 < BlueMatt> closer to 25
18:17 < BlueMatt> ariard: oh, ugh, fuck, sorry, didnt mean to send you down a rabiit hole
18:17 < BlueMatt> but just generating the staticoutputdescriptor once on closing_signed isnt enough either - our counterparty could get an older cooperative close tx confirmed by signing a previous copy we sent them
18:17 < dongcarl> BlueMatt: who's maintaining that?
18:18 < BlueMatt> dongcarl: uhhhhh? no idea?
18:18 < BlueMatt> decker ran one on bitcoinstats.com like 4 years ago but then we went to something else
18:18 < BlueMatt> i dont recall what now....
18:18 < dongcarl> K I'll ask on bitcoin-core-dev
18:23 < BlueMatt> both core-dev and lightning-dev link to gnusha.org in their topic
18:25  * BlueMatt -> holyfucknaptime (just got off 11 hour flight, now on another one)
18:31 < ariard> BlueMatt: hmm that's right will think on it tonight, but on the principle do you find the staticoutputdescriptor thing useful if it can be made reliable ?
18:31 < BlueMatt> ariard: I mean in principal yes, I guess, but I dont think we should bend over backwards to make it happen
18:31 < BlueMatt> ariard: it should be low-ish work for clients to deal without it (as long as the outputs are all directly to pubkeys that they provided)
18:32 < BlueMatt> if we can do it easily, great, if its a lot of work (it seems to already be turning into that) oh well, we can revisit later if we get bored :p
18:33 < ariard> low-ish work clients seems good to me, well I try to find a solution tonight and if no one came easily will put a TODO
22:05 < dongcarl> andytoshi: Did we talk about what to do for this: https://github.com/rust-bitcoin/rust-bitcoin/pull/183#discussion_r230574518
22:08 <+andytoshi> dongcarl: i don't understand what you're saying there
22:14 < dongcarl> What I wanted to say was: Because the Address::p2* methods all take ownership of the util::key::PublicKey, we need to retain the util::key::PublicKey somehow even in the case of p2shwpkh and p2wpkh. This means that if we were to move the public_key field from Address to variants of Payload, we would have to add the public_key field to ALL variants of Payload, and perhaps it'd be better if the public_key field stayed in
22:14 < dongcarl> Address.
22:16 <+andytoshi> why do we need to add the public_key field to all variants of Payload?
22:16 <+andytoshi> that doesn't make any sense for half of them
22:16 <+andytoshi> what is the "ec public key" of a p2sh address?
22:24 < dongcarl> I believe in our original conversation, you proposed that the public_key field of Address be an `Option<key::PublicKey>`, and have it only be `Some` when it is the result of constructing it from one of the p2*k* methods on Address
22:26 <+andytoshi> maybe. i don't recall liking that idea, but if i did, i don't like it now.. i think the Option<key::PublicKey> should live inside the variants of Payload where it makes sense
22:26 <+andytoshi> which i think are only p2pk, p2pkh, and 32-byte v0 witness programs
22:29 < dongcarl> andytoshi: what do you think about not having the `Option<key::PublicKey>` fields at all inside Payloads, meaning that we only borrow them, and just cloning the `key::PublicKey` for `Payload::Pubkey`, which is only used by `p2pk(pk: key::PublicKey, network: Network) -> Address `?
22:32 <+andytoshi> why would we borrow and then clone?
22:35 < dongcarl> andytoshi: well, if we don't have the `Option<key::PublicKey>` fields at all inside Payloads or Addresses, it'd be nice to have all fo the p2*k* methods be somewhat the same in terms of ownership, i.e. `p2*k*(pk: &key::PublicKey, network: Network) -> Address`.
22:35 < dongcarl> Perhaps because p2pk is sooooo deprecated we can make the exception that only it takes ownership of its `key::PublicKey` to stick inside its `Payload::Pubkey`?
22:36 <+andytoshi> i'm sorry, you really need to spell this out for me
22:36 <+andytoshi> is there any function that would take a &key::PublicKey and not clone it?
22:36 <+andytoshi> which one? what is it doing with that public key?
22:38 < sgeisler> instead of putting an `Option<PublicKey>` into all payload variants that need them for PSBT you could also make Address & Payload generic over a liftime 'f and just include an `Option<&'f PublicKey>` if I undersstood it correctly
22:39 < sgeisler> but that sounds like pain to work with
22:39 <+andytoshi> i think that would be the worst of all worlds
22:40 <+andytoshi> the user would have to keep a PublicKey alive alongside the address
22:40 <+andytoshi> users who don't care about the underlying public key would need to deal with the lifetime
22:40 < sgeisler> but I don't know if that's what dongcarl meant, it's just my understanding of it
22:40 < dongcarl> I'm sorry if I'm not making myself clear. I'm trying to propose that we do not put any `PublicKey` ownership inside `p2*k*` methods, and asking if that is acceptable. Here's my thought: aside from `p2pk`, all the other `p2*k*` methods only use the `write_into` method I'll provide to get a serialization, and therefore only have to borrow the `key::PublicKey`. For `p2pk` however, its payload `Payload::Pubkey` embeds a
22:40 < dongcarl> `key::PublicKey` inside it, therefore it has to take ownership.
22:40 <+andytoshi> i don't think he's suggesting storing a reference inside the Address structure anywhere
22:41 <+andytoshi> dongcarl: p2pkh needs to store the actual public key
22:42 <+andytoshi> can you please list explicitly all the p2*k* methods you're thinking of, i don't have the list of legacy addresses in my head right now
22:42 < dongcarl> andytoshi: when would p2pkh need to store the actual public key?
22:42 < dongcarl> p2pkh, p2pk, p2wpkh, p2shwpkh
22:44 <+andytoshi> dongcarl: in all the same contexts that p2pk needs to
22:44 <+andytoshi> i don't understand the difference here
22:44 <+andytoshi> it's needed in p2bt to serialize and identify which keys are needed; it's used in output descriptors to map keys to owners and vice-versa
22:44 <+andytoshi> it's used in getaddressinfo RPC-like contexts
22:49 < dongcarl> andytoshi: Okay, but wouldn't we also need this for p2wpkh and p2shwpkh?
22:50 < dongcarl> this as in storing the actual public key
22:50 <+andytoshi> yeah
22:51 < dongcarl> Okay, and we use Payload::WitnessProgram for p2wpkh and Payload::ScriptHash for p2shwpkh
22:52 <+andytoshi> ah i see
22:52 < dongcarl> and if we choose to have the public key field inside Payloads, both these variants would have to have a public key field?
22:52 < dongcarl> Right.
22:52 < dongcarl> I'm trying to see if we can decouple it
22:52 <+andytoshi> what a shitty situation
22:52 < dongcarl> as the Pubkey that's inside Payload::Pubkey is only temporary
22:52 < dongcarl> It's used later to Hash160
22:52 <+andytoshi> maybe we should replace the Address type with output descriptors
22:52 <+andytoshi> like, implement all the simple things that Core has
22:53 <+andytoshi> and then relegate Address to purely a serialization newtype around a descriptor
22:53 < dongcarl> Okay that's actually kinda sexy
22:53 <+andytoshi> heh sorry to let you spend weeks on a design that i just now decided to throw out  :P
22:54 < dongcarl> Haha no worries, as long as good design ends up in the repo I'm cool with it
22:55 <+andytoshi> so, here is the descriptor type from miniscript .. https://github.com/apoelstra/rust-miniscript/blob/master/src/descriptor/mod.rs#L39 .. you can see it's missing p2shwpkh which i think was an oversight on my part
22:55 <+andytoshi> and the Miniscript<P> types i guess should be replaced with Script
22:55 <+andytoshi> for rust-bitcoin
22:56 <+andytoshi> oh, it's also missing pk
22:58 <+andytoshi> added both
22:59 <+andytoshi> https://github.com/bitcoin/bitcoin/blob/v0.17.0/doc/descriptors.md is the official Core descriptor doc .. i'm unsure if my Display and FromStr implementations match that
23:03 < dongcarl> Hmmmm
23:03 < dongcarl> I'm guessing I shouldn't add it as a dep, but instead copy the code?
23:26 < sgeisler> andytoshi: the script descriptors idea looks neat, but with your current implementation (without addr) we will only be able to serialize and not to deserialize addresses. I'd argue that you want two types for these two scenarios anyway since the AddressWithPubKeys type has more capabilities than the ParsedAddress type. Maybe they could both implement a trait that lets you construct txouts from them but only the AddressWithPubKeys has
23:26 < sgeisler>  all the additional functionality that you currently want to have in Address?