--- Log opened Mon Jun 01 00:00:38 2020 01:26 -!- dr-orlovsky [~dr-orlovs@77-58-192-184.dclient.hispeed.ch] has joined #rust-bitcoin 01:49 -!- dr-orlovsky [~dr-orlovs@77-58-192-184.dclient.hispeed.ch] has quit [Quit: My MacBook has gone to sleep. ZZZzzz…] 02:50 -!- dr-orlov1ky [~user@31.14.40.19] has joined #rust-bitcoin 03:01 -!- Kiminuo [~mix@141.98.103.180] has joined #rust-bitcoin 03:04 -!- Asa11Dach [~Asa11Dach@static.57.1.216.95.clients.your-server.de] has joined #rust-bitcoin 03:14 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 03:18 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Ping timeout: 256 seconds] 05:01 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 05:05 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Ping timeout: 256 seconds] 05:15 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 06:08 < sgeisler> 06:08 < sgeisler> I opened a PR to fix the 1.22 issue 06:09 < sgeisler> I think it's probably easiest to merge it first, then re-run your PR's tests and merge it afterwards 06:19 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Ping timeout: 265 seconds] 06:38 < Kiminuo> andytoshi, hi, would you mind have a look at https://github.com/apoelstra/rust-miniscript/pull/90? 06:41 -!- Asa11Dach [~Asa11Dach@static.57.1.216.95.clients.your-server.de] has quit [Ping timeout: 256 seconds] 07:02 < stevenroose> sgeisler: yeah so personally I have always been using the hex crate, just in a fixed version so that it couldn't silently raise MSRV. One other thing is the FromHex trait I think that is quite nice and hex might not have. But we could add it. And I think trying to add some MSRV measures (README, Travis, requires ACKs before signing) in their repository would be reasonable 07:05 < sgeisler> Yeah, getting them to maintain a MSRV would be optimal. We can try. I wonder how one can get to the topic in a diplomatic manner, I read that rust-base64 thread, oh my … 07:08 < cloudhead> sgeisler: great, thanks! 07:22 < elichai2> sgeisler: enjoy https://github.com/serde-rs/serde/issues/1602 😅 07:23 -!- Kiminuo [~mix@141.98.103.180] has quit [Ping timeout: 260 seconds] 07:35 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection] 07:35 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #rust-bitcoin 07:36 < andytoshi> sgeisler: stevenroose: yeah, ack to either forking the hex crate or extracting it from bitcoin_hashes 07:36 < andytoshi> it feels a bit weird to fork it, afaik it has never caused us problems 07:36 < andytoshi> but BlueMatt wanted to remove it at some point to reduce the total dependency footprint 07:37 < andytoshi> which is a bit weird, we didn't do the same for base64 ... i guess the difference is that you can write a hex crate in like 20 LOC 07:37 < andytoshi> really it's ridiculous that the stdlib doesn't have this 07:41 < stevenroose> true, perhaps we should just PR it to stdlib :D :D 07:41 < andytoshi> i did this in 2014 07:41 < andytoshi> they said that hex encoding was 'niche' 07:41 < andytoshi> and i nearly gave up on the language entirely because they were clearly morons in charge of stdlib 07:41 < andytoshi> but ... it was like 20 LOC for me to just work around it :) 07:43 < elichai2> hmm if you'll find a natural place to put it in libstd, and an implementation that will satisfy everyone I believe it might be accepted these days 07:44 < elichai2> I added things that are way more niche :P 07:44 < andytoshi> yeah that's true, things are much more mature now 07:44 < andytoshi> in 2014 they were in the process of removing basic collection data structures 07:44 < elichai2> maybe `str::from_hex(&[u8])` 07:44 < andytoshi> because gankro was doing design iteration in his own crates 07:44 < elichai2> ops opposite 07:44 < andytoshi> lol yeah opposite 07:45 < elichai2> yeah I think they had a bunch more *modules* in 2014 07:49 < elichai2> actually the right thing for toHex is implementing `LowerHex` and `UpperHex` for `[u8]` 07:49 < stevenroose> elichai2: well you'd like to avoid the allocation in many cases 07:50 < elichai2> stevenroose: yeah that's what I'm trying to think, how it can fit in libstd but in libcore 07:50 < elichai2> the FromHex is a little more tricky 07:51 < stevenroose> I think a hex crate is fine, though 07:51 < elichai2> although trait implementation are insta-stable so it won't be as easy to get into libstd 07:51 < stevenroose> but it should just never change 07:51 < elichai2> stevenroose: I'm thinking long term :P 07:52 < elichai2> and yeah It's probably fine, altough in the past I've used this: https://docs.rs/rustc-hex/2.1.0/rustc_hex/ 07:52 < stevenroose> Cargo should really do a better job to have dependencies using git and hashes etc. in Go there was this movement (gx I think) of an IPFS based package manager, where the version was the hash and it would fetch it over IPFS. quite cool conceptually 07:53 < elichai2> personally I will take cargo every day over go's "package management" 07:53 < andytoshi> sure, but it is true that cargo really discourages git hashes 07:54 < andytoshi> and also makes it really hard to check what version of software you're using when you give version numbers 07:54 < elichai2> yeah, and semver somewhat reassembles dynamic linking 07:54 < andytoshi> i think they'd accept patches to improve this situation but the design work is hard because they imagine themselves to be independent of git as a source control transport 07:55 < elichai2> while git versions is a "no updates" scheme for dependencies 07:55 < andytoshi> also, git uses sha1 hashes everywhere and i don't trust those either :) 07:55 < elichai2> lol yeah 07:55 < elichai2> andytoshi: well there's a bunch of problems with using git for this 07:55 < elichai2> I'll look for that post but I remember github saying some language crashed their servers because of package managers 07:57 < andytoshi> lol interesting. i believe that 07:57 < andytoshi> but github kinda brought that on themselves 07:57 < andytoshi> by trying to be the One True Git Host 07:57 < andytoshi> instead of doing a distributed design 07:59 < elichai2> Found it :) https://github.com/CocoaPods/CocoaPods/issues/4989#issuecomment-193772935 08:01 < stevenroose> yeah that could be Go. Go's packages are git-native, so the package name is the repo and the versin is the tag, so whenever someone builds a new go project, they fetch a bunch of github repos 08:02 < stevenroose> well, rust-bitcoin could be hosting out of github, I'd encourage that as well 08:02 < stevenroose> like some nixos-based gitea installation or so 08:04 < stevenroose> but well, back to hex, elichai2 I'd be open to `hex` or `rustc_hex` either way, as long as we like the implementation and it's good and stable. I like the ease of use of hex::encode a lot for testing etc 08:08 < elichai2> rustc_hex was a copy of the hex impl from inside rustc, but that impl was apparently removed last month https://github.com/rust-lang/rust/pull/71236 08:09 < elichai2> https://github.com/rust-lang/rust/blob/4ca5fd2d7b6b1d75b6cb8f679e8523fb3e7b19e2/src/libserialize/hex.rs 08:10 < elichai2> andytoshi: there was even rand in libstd back then hehe https://doc.rust-lang.org/0.8/std/rand/index.html 08:12 < elichai2> and don't forget md4 loool https://doc.rust-lang.org/0.8/extra/md4/index.html 08:44 -!- moneyball [sid299869@gateway/web/irccloud.com/x-idfrgkxmsbhwxvay] has joined #rust-bitcoin 09:12 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection] 09:13 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #rust-bitcoin 09:46 < andytoshi> hahaha 10:02 < BlueMatt> lol, well I stand by just yanking all non-test depedencies now :) 10:20 -!- ghost43 [~daer@gateway/tor-sasl/daer] has quit [Remote host closed the connection] 10:20 -!- ghost43 [~daer@gateway/tor-sasl/daer] has joined #rust-bitcoin 10:44 < sgeisler> BlueMatt: that's not really a scalable solution for software development imo. It _currently_ makes some sense for high assurance libraries like rust-bitcoin because trust management is virtually non-existent. But with the adoption of tools like crev and aggressive pinning we could achieve security while avoiding the overhead of reinventing the wheel over and over again. Rust's stdlib is just too small for that 10:44 < sgeisler> approach to really work. 10:47 < BlueMatt> I significantly disagree. sure, for some things you take a dependency, but virtually the entire ecosystem of C/C++ programs is proof-positive that absolutely-minimal-dependencies is a perfectly scalable way to write software. 10:48 < BlueMatt> and virtually the entire python/javascript is proof that relying on dependencies for everything is *not* scalable 10:49 < BlueMatt> many of those C/C++ projects take a few hundred lines of code here and there from other applications to incorporate into their own build, but maintain it themselves or only take 3/4 dependencies in total, which may be reasonable. 10:50 < BlueMatt> not to mention nvm is good evidence for dependencies being a massive, gaping security vulnerability 10:52 < sgeisler> It is indeed a gaping security hole if unmanaged. But so is unmanaged redundancy to a certain degree. If you copy code you loose out on upstream bugfixes if you don't have a process for managing that redundancy. 10:53 < sgeisler> Also: the more people use a library as a dependency the larger is the amount of people actually taking a critical look at it. 10:57 < sgeisler> And it "working" for C/C++ is more a sign of a lack of sane dependency management (I heard it got better tho). I can only imagine the developer years wasted on redundant implementations of simple things like base64 encoding. Sure, it's not complex, but also thoroughly testing the code is. 10:59 < sgeisler> We should view NPM as a warning, but not to abandon dependencies, but to better manage them and their respective "supply chain trust" 11:00 < BlueMatt> always a question of degrees. taking a dependency for something like hex encoding or base64 is just asking for trouble (ala npm), whereas maybe for something like tls, well you arent gonna do much better. 11:00 < andytoshi> agreed sgeisler . but bear in mind that cargo is antogonistic to this in a couple ways, and we need to find a way around this 11:00 < andytoshi> i see no reason that using a hex/base64/endianness dep should be asking for trouble ala npm 11:00 < BlueMatt> but, more to the point, specifically in the rust ecosystem, there isn't a lot we can do *except* not take any dependencies 11:00 < andytoshi> if we're able to read the code and pin it 11:00 < andytoshi> and read the diff when we update the pin 11:01 < andytoshi> we do this for liquid stuff at blockstream BTW - we have vendored every dependency and source controlled the vendored deps 11:01 < BlueMatt> i mean to start with you're probably better off not taking the dependency just cause rustc doesnt do cross-crate optimization lol 11:01 < andytoshi> unfortunately in the public crates.io ecosystem (a) cargo makes you a second-class citizen if you use a git hash; (b) there are no tools for using cargo's hashes; (c) everything auto-updates out from under you 11:01 < BlueMatt> right, and by the time you've done that, you've put in about 10x more work than just writing out a hex encoder/decoder... 11:02 < BlueMatt> for something big, sure, but doing it "right" takes real work, and so for anything small enough to do in 100 lines or less, just fucking do it. 11:02 < andytoshi> not really, reading a hex crate (and skimming through the test suite) is way faster than reimplementing, and then it's one line to add to Cargo.toml, then you rerun contrib/revendor.sh.. 11:02 < andytoshi> +1 for the specific case of hex 11:02 < andytoshi> although ... we did that, and the result is that we have hex code in bitcoin_hashes 11:02 < andytoshi> and everything else has a bitcoin_hashes dep anyway, so we wind up using bitcoin_hashes as "the hex dependency" 11:03 < andytoshi> which is a bit silly, it seems strictly more flexible and useful to split it out 11:04 < BlueMatt> i mean or just copy one file. like, seriously, its a hex encoder....... 11:04 < BlueMatt> but, sure, or take it from bitcoin_hashes 11:04 < BlueMatt> its about 3x less work to write code than to audit it 11:05 < BlueMatt> especially if you have to audit code every 6-12 months when the dependency updates 11:05 < BlueMatt> and, exactly, cargo makes it hard, what we do by taking a dependency is make downstream do the work. 11:05 < BlueMatt> cause now downstream projects get auto-updated stuff and we cant control that 11:06 < BlueMatt> or we can pin it, but that results in more issues and crap to deal with 11:06 < andytoshi> i don't understand how writing code is easier than auditing it. if you write it it still has to go through code review, and you have to do all the testing yourself 11:06 < BlueMatt> sure, but at least if you write it I can consider it as requiring one less review :) 11:06 < andytoshi> and if you've copy-pasted the resulting code everywhere you have no hope of updating it if you find bugs 11:06 < andytoshi> ok, sure 11:06 < sgeisler> what are the drawbacks of going full-git with ref pinning btw? (except that one might not trust sha1 anymore) 11:07 < andytoshi> sgeisler: cargo won't let you upload to crates.io 11:07 < andytoshi> so we'd be parting ways with the crates.io ecosystem completely 11:08 < andytoshi> i'm curious BTW if there's a way around it ... like can you keep the crates.io source and version, and -also- pin a ref somehow? i don't think so because crates.io doesn't keep track of git commits 11:10 < BlueMatt> and, more importantly, most of the changes you see in something like hex is some crazy optimization for xyz, which takes effort to review and almost certainly doesnt improve things for our use-cases 11:10 < sgeisler> Thinking about it more strict lib-level pinning isn't that good of an idea either. That way you can end up with weird incompatibilities between libraries because cargo can't find a common denominator anymore iiuc. 11:13 -!- Kiminuo [~mix@141.98.103.196] has joined #rust-bitcoin 11:19 < andytoshi> yes, if you have traits or APIs or something that need to be compatible 11:19 < andytoshi> in this case trying to get by without deps is even worse! 11:20 < andytoshi> well i guess if you bypass cargo, then cargo won't get confused 11:20 < BlueMatt> hah, right, i wish there were a rust ecosystem for people who avoided cargo completely 11:20 < BlueMatt> rustc is actually really nice to work with 11:41 -!- icota[m] [icotamatri@gateway/shell/matrix.org/x-sctvyrivogphjxke] has quit [Ping timeout: 252 seconds] 11:42 -!- dpc [dpcmatrixo@gateway/shell/matrix.org/x-mapquevltlkthhzu] has quit [Ping timeout: 246 seconds] 12:18 -!- dpc [dpcmatrixo@gateway/shell/matrix.org/x-hamewajlrgkctuqc] has joined #rust-bitcoin 12:34 -!- dr-orlov1ky [~user@31.14.40.19] has quit [Remote host closed the connection] 12:40 -!- icota[m] [icotamatri@gateway/shell/matrix.org/x-wfhuaxymifzteebb] has joined #rust-bitcoin 13:22 -!- Kiminuo [~mix@141.98.103.196] has quit [Ping timeout: 246 seconds] 14:00 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 14:01 -!- Alvina3Fay [~Alvina3Fa@static.57.1.216.95.clients.your-server.de] has joined #rust-bitcoin 14:45 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Remote host closed the connection] 14:49 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 14:54 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Ping timeout: 260 seconds] 15:17 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 15:20 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Remote host closed the connection] 15:25 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 15:29 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Ping timeout: 256 seconds] 17:09 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 17:16 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Remote host closed the connection] 17:16 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has joined #rust-bitcoin 17:48 -!- Alvina3Fay [~Alvina3Fa@static.57.1.216.95.clients.your-server.de] has quit [Ping timeout: 265 seconds] 18:13 -!- Nash84Glover [~Nash84Glo@static.57.1.216.95.clients.your-server.de] has joined #rust-bitcoin 20:04 -!- surja795 [~surja795@c-24-62-248-154.hsd1.ma.comcast.net] has quit [Remote host closed the connection] 22:56 -!- Kiminuo [~mix@141.98.103.196] has joined #rust-bitcoin --- Log closed Tue Jun 02 00:00:40 2020