--- Log opened Tue Aug 15 00:00:50 2017 00:02 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Ping timeout: 248 seconds] 00:13 -!- emeraldgreen [~user@92.222.68.248] has quit [Quit: Leaving.] 00:24 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has quit [Ping timeout: 240 seconds] 00:29 -!- augur [~augur@noisebridge130.static.monkeybrains.net] has quit [Remote host closed the connection] 00:29 -!- augur [~augur@noisebridge130.static.monkeybrains.net] has joined ##hplusroadmap 00:33 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 00:34 -!- augur [~augur@noisebridge130.static.monkeybrains.net] has quit [Ping timeout: 240 seconds] 00:53 -!- emeraldgreen [~user@92.222.68.248] has joined ##hplusroadmap 01:18 -!- emeraldgreen [~user@92.222.68.248] has quit [Ping timeout: 240 seconds] 01:29 -!- CheckDavid [uid14990@gateway/web/irccloud.com/x-eszwbucdxddlfdry] has quit [Quit: Connection closed for inactivity] 01:30 -!- emeraldgreen [~user@92.222.68.248] has joined ##hplusroadmap 01:41 -!- emeraldgreen [~user@92.222.68.248] has quit [Quit: Leaving.] 01:42 -!- emeraldgreen [~user@92.222.68.248] has joined ##hplusroadmap 01:49 -!- emeraldgreen [~user@92.222.68.248] has quit [Ping timeout: 240 seconds] 01:57 -!- emeraldgreen [~user@92.222.68.248] has joined ##hplusroadmap 02:04 -!- jtimon [~quassel@143.29.134.37.dynamic.jazztel.es] has joined ##hplusroadmap 02:22 -!- augur [~augur@2601:645:4000:46b5:b881:cbe9:9c94:24ee] has joined ##hplusroadmap 02:26 -!- augur [~augur@2601:645:4000:46b5:b881:cbe9:9c94:24ee] has quit [Ping timeout: 240 seconds] 02:33 -!- Jenda` [~bablbam@dekatron.hrach.eu] has joined ##hplusroadmap 02:43 -!- midnightmagic [~midnightm@unaffiliated/midnightmagic] has joined ##hplusroadmap 03:18 < archels> AGI takeover: imminent https://i.imgur.com/led15Z7.gifv 03:51 -!- preview [~quassel@2407:7000:842d:4000::3] has quit [Read error: No route to host] 03:52 -!- preview [~quassel@2407:7000:842d:4000::3] has joined ##hplusroadmap 03:56 -!- dustinm [~dustinm@68.ip-149-56-14.net] has quit [Quit: Leaving] 04:08 -!- eidolonn9 [~eidolonn9@h57.89.213.151.dynamic.ip.windstream.net] has joined ##hplusroadmap 04:13 -!- esmerelda [~mabel@unaffiliated/jacco] has quit [Ping timeout: 240 seconds] 04:19 -!- dustinm [~dustinm@68.ip-149-56-14.net] has joined ##hplusroadmap 04:26 -!- jtimon [~quassel@143.29.134.37.dynamic.jazztel.es] has quit [Remote host closed the connection] 04:40 < kanzure> jelmer has posted some thoughts about enzymatic error correction and dna synthesis: https://groups.google.com/d/msg/enzymaticsynthesis/WvDidIldm7c/o7LYEwYgAgAJ 04:51 -!- emeraldgreen [~user@92.222.68.248] has quit [Ping timeout: 240 seconds] 04:52 < kanzure> "Advances in design of protein folds and assemblies" http://diyhpl.us/~bryan/papers2/bio/protein-engineering/Advances%20in%20design%20of%20protein%20folds%20and%20assemblies%20-%202017.pdf 04:59 < kanzure> "Enzymatic synthesis of psilocybin" http://onlinelibrary.wiley.com/doi/10.1002/anie.201705489/abstract https://news.ycombinator.com/item?id=15015880 05:05 < kanzure> http://www.lowtechmagazine.com/2010/12/hand-powered-drilling-tools-and-machines.html https://news.ycombinator.com/item?id=15013563 05:14 < kanzure> rusty russell removing himself from some maintainer files https://lkml.org/lkml/2017/8/14/892 05:20 -!- emeraldgreen [~user@92.222.68.248] has joined ##hplusroadmap 05:23 < kanzure> https://jeremykun.com/2014/02/24/elliptic-curves-as-python-objects/ 05:29 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Remote host closed the connection] 05:30 -!- augur [~augur@2601:645:4000:46b5:dca:8358:45e:79f4] has joined ##hplusroadmap 05:34 -!- augur [~augur@2601:645:4000:46b5:dca:8358:45e:79f4] has quit [Ping timeout: 240 seconds] 05:36 -!- Gurkenglas_ [~Gurkengla@dslb-178-000-218-087.178.000.pools.vodafone-ip.de] has joined ##hplusroadmap 05:56 -!- Storyteller [~Storytell@unaffiliated/storyteller] has joined ##hplusroadmap 06:02 -!- Storyteller [~Storytell@unaffiliated/storyteller] has quit [Max SendQ exceeded] 06:09 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 06:10 -!- Jenda` [~bablbam@dekatron.hrach.eu] has quit [Quit: leaving] 06:10 -!- Jenda` [~bablbam@dekatron.hrach.eu] has joined ##hplusroadmap 06:11 -!- Jenda` [~bablbam@dekatron.hrach.eu] has quit [Client Quit] 06:14 < kanzure> https://johncarlosbaez.wordpress.com/2017/08/15/norbert-blum-on-p-versus-np/ 06:15 < kanzure> i spent some time in 2009 reading everything baez ever posted. sort of disappointed that there is more content now. 06:47 < kanzure> .tw https://twitter.com/jimhaseloff/status/897447703919026177 06:47 < yoleaux> Extreme handheld macro with Mk II camera, example with wild Marchantia, more details at (http://data.plantsci.cam.ac.uk/Haseloff/imaging/macrophotography/index.html) https://pbs.twimg.com/media/DHRf7AlXYAAV8vN.jpg (@jimhaseloff) 06:47 < kanzure> ( http://data.plantsci.cam.ac.uk/Haseloff/imaging/macrophotography/index.html ) 06:47 < kanzure> "Label-free optical detection of action potential in mammalian neurons" https://www.osapublishing.org/boe/abstract.cfm?uri=boe-8-8-3700 06:48 < kanzure> Imaging action potential in single mammalian neurons by tracking the accompanying sub-nanometer mechanical motion" http://www.biorxiv.org/content/biorxiv/early/2017/07/25/168054.full.pdf 06:49 < kanzure> "On cell surface deformation during an action potential" https://arxiv.org/pdf/1703.04608 06:49 < kanzure> cc superkuh 06:51 < kanzure> ".. in squid eyes, globular proteins of different sizes form a gel of varying density, thereby building up a refractive index gradient." http://science.sciencemag.org/content/357/6351/546 https://twitter.com/sciencemagazine/status/897444017499234304 ("Patchy proteins form a perfect lens in squid") 06:52 < emeraldgreen> kanzure could this be used to interface with nerves outside of brain, hmm 06:52 < emeraldgreen> А solid state interferomter, a good camera and a decoder asic 06:58 < kanzure> "Conformational landscape of a virus by single-particle X-ray scattering" https://www.nature.com/nmeth/journal/vaop/ncurrent/full/nmeth.4395.html https://twitter.com/naturemethods/status/897205923416027137 07:00 < kanzure> "Large-scale mapping of cortical synaptic projections with extracellular electrode arrays" https://www.nature.com/nmeth/journal/vaop/ncurrent/full/nmeth.4393.html https://twitter.com/naturemethods/status/897198691387416576 07:02 < kanzure> "Engineering reproducible neural tissue from pluripotent stem cells" https://www.youtube.com/watch?v=hTILq3mXX1g 07:03 < archels> This Springer Nature service is currently down for maintenance. 07:03 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Ping timeout: 260 seconds] 07:06 < archels> emeraldgreen: the myelination of peripheral nerve fibres might be an obstacle 07:08 < archels> so the plant cell moves by micrometres, but the mammalian cells only nanometres 07:10 < archels> the heartbeat artefact is probably much larger already 07:11 < kanzure> most neurons probably don't fire due to heartbeat artifact, right? 07:15 -!- jaboja [~jaboja@jaboja.pl] has joined ##hplusroadmap 07:20 < kanzure> https://github.com/vahidk/EffectiveTensorflow 07:21 < kanzure> various bookmarks collections about penetration testing and reverse engineering and stuff https://github.com/Hack-with-Github/Awesome-Hacking 07:21 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 07:34 < pasky> > Luca Trevisan Andreev's function, which is claimed to have superpolynomial circuit complexity (abstract, then section 7), is just univariate polynomial interpolation in a finite field, which, if I am not missing something, is solvable by Gaussian elimination 07:37 -!- Storyteller [~Storytell@unaffiliated/storyteller] has joined ##hplusroadmap 07:46 -!- augur [~augur@2601:645:4000:46b5:6544:c097:e26d:765] has joined ##hplusroadmap 07:50 -!- augur [~augur@2601:645:4000:46b5:6544:c097:e26d:765] has quit [Ping timeout: 240 seconds] 07:56 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Ping timeout: 248 seconds] 08:00 < kanzure> https://github.com/blockstream/satellite 08:07 < fenn> cool 08:07 < fenn> is there an uplink capability in the future? 08:08 < pasky> https://blockstream.com/2017/08/15/announcing-blockstream-satellite.html for context 08:09 < pasky> I sorta miss the point though, you can't _send_ transactions anyway so the usecase seems pretty narrow? more like a publicity stunt? 08:13 < kanzure> transactions can be sent on the terrestrial network using very slow sneakernet. 08:14 < kanzure> i agree that it's more like a publicity stunt. (i made the same criticism when the idea was proposed at another company.) 08:14 < kanzure> however, the point is that this is another layer of protection against network partitions 08:15 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 08:15 < pasky> hmm that makes sense, also for miners i guess 08:47 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Ping timeout: 240 seconds] 08:52 < fltrz> kanzure: woah thanks for those action potential imaging links 08:54 < fltrz> emeraldgreen: I believe these effects are at the basis of the FOS "fast optical signal" gratton et al 08:54 -!- Storyteller [~Storytell@unaffiliated/storyteller] has quit [Read error: Connection timed out] 08:55 < fltrz> modulating a light source at around 100Mhz and then demodulated imaging should suffice 08:56 < fltrz> (I am actually trying to do this with a kinect v2, but the driver stack is opaque binary, so I desoldered the flash chip, and intend to read it out one of these days) 08:57 < fltrz> (alternatively, I need to identify the control lines for the MIPI CSI2 interface, and deduce the protocols) 09:02 < superkuh> Thanks, kanzure. 09:03 < superkuh> It's crazy how this has been known since Tasaki reported it in the 1950s but everyone just keeps ignoring it despite re-'finding' every decade. 09:04 < fltrz> where can I learn more about Tasaki 's report? 09:06 < superkuh> He used to have everything on his NIH page (dead now). But just search for Ichiji Tasaki (the guy that discovered myelination). 09:07 < superkuh> https://science.nichd.nih.gov/confluence/display/sqits/Ichiji+Tasaki%2C+MD 09:08 < superkuh> http://web.archive.org/web/20061002012409/http://eclipse.nichd.nih.gov/nichd/stbb/tasaki.html 09:08 < superkuh> Also, http://erewhon.superkuh.com/library/Neuroscience/Lipid%20Membrane/ 09:10 < superkuh> http://erewhon.superkuh.com/library/Neuroscience/Lipid%20Membrane/Volume%20expansion%20of%20nonmyelinated%20nerve%20fibers%20during%20impulse%20conduction_%20Tasaki%20I_%20Byrne%20PM_%201990.pdf 09:16 < kanzure> well at least the third link (the arxiv one) references tasaki. 09:16 < superkuh> Oh. So it does. 09:17 < superkuh> Saying neither Tasaki nor Heimburg "it neither contains nor predicts non-electrical manifestations of the AP." is false. It's the natural consequence of the monovalent versus divalent cation exchange. 09:19 < fltrz> superkuh: thanks! 09:29 < nmz787> fltrz: any links on this MIPI CSI? 09:35 < fltrz> on the sensor used in the kinect v2: http://ieeexplore.ieee.org/document/6964815/ 09:36 < fltrz> nmz787: it has 2 MIPI DPhy's, and according to the paper the ToF sensor can deliver 600 fields per second, it is underutilized in the Kinect v2 at 300 fields per second 09:37 < fltrz> so approach 2 has the potential for better better performance (higher bandwidth spectrum per pixel) 09:38 < fltrz> but would require designing a board with FPGA 09:38 < fltrz> to which to attach the ToF module in the kinect v2 09:39 < fltrz> the mipi csi2 specification can be found on the interwebs 09:39 < fltrz> (for MIPI CSI2 in general, without register description of the ToF sensor of the v2) 09:40 -!- emeraldgreen [~user@92.222.68.248] has quit [Quit: Leaving.] 09:44 < fltrz> i.e. if we can modify the firmware of the kinect, it may be possible to have every delivered field be taken at the highest frequency => 300/2 = 150Hz nyquist limit 09:45 < fltrz> if we can deduce the protocol/registers for the ToF sensor, and design appropriate interface board, we could have 600/2=300Hz nyquist limit 09:45 < fltrz> (for the brain waves) 09:46 < fltrz> that would be 512*424 independent brain wave signals 09:46 < kanzure> the brain what nows? 09:47 < fltrz> kanzure: I don't understand the question? 09:47 < kanzure> eh nevermind. 09:50 < fltrz> the problem with using kinect as is, is that of the repeating sequence of 10 fields, 7 fields are useless 09:52 < nmz787> fltrz: hmm, I have RTL for a MIPI CSI IP... will checkout the public protocol stuff 09:52 < fltrz> I can imagine the parameters for each field (frequency, phase, ...) to be initialized at startup and stored on sensor once, and then during operation a simple command with an index indicating which kind of field is requested next 10:11 -!- Solgriffin [~Sol@c-69-141-24-242.hsd1.nj.comcast.net] has quit [] 10:20 < fltrz> nmz787: the control lines for MIPI CSI are I2C 10:22 < fltrz> I have 3 kinect v2's of which I disassembled one, I have the ToF module (the sensor chip and lens are mounted to a pcb, the pcb also has some other components, and a connector to connect to the main board of the kinect 10:22 < fltrz> I have not yet identified which pins are which 10:24 < fltrz> (I have an 1Ghz oscilloscope TDS5104, and an FPGA board, but no experience reverse engineering hardware, not sure what the best approach is for identifying the i2c lines) 10:25 < fltrz> my FPGA experience is very minimal, but I know how to use the oscilloscope (no experience with logic analyzers though, and I don't think my scope has protocol recognition like logic analyzers 10:28 -!- jaboja [~jaboja@jaboja.pl] has quit [Remote host closed the connection] 10:31 < fltrz> I believe this method is how facebook is doing this https://techcrunch.com/2017/04/19/facebook-brain-interface/ 10:32 < fltrz> Now I want a BCI, but I don't want third-wheel peeping tom hidden in the closet Zuckerberg between me and my friends, let alone between my brain and my computer 10:40 < nmz787> fltrz: http://hackaday.com/2017/02/13/dummies-guide-to-reverse-engineering/ 10:42 < fltrz> I'll check it right now :) 10:42 -!- jaboja [~jaboja@jaboja.pl] has joined ##hplusroadmap 11:17 -!- eidolonn9 [~eidolonn9@h57.89.213.151.dynamic.ip.windstream.net] has quit [] 11:17 -!- eidolonn9 [~eidolonn9@h57.89.213.151.dynamic.ip.windstream.net] has joined ##hplusroadmap 11:17 -!- eidolonn9 [~eidolonn9@h57.89.213.151.dynamic.ip.windstream.net] has quit [Client Quit] 11:27 -!- Storyteller [~Storytell@unaffiliated/storyteller] has joined ##hplusroadmap 11:58 -!- Storyteller [~Storytell@unaffiliated/storyteller] has quit [Ping timeout: 240 seconds] 12:03 -!- Netsplit *.net <-> *.split quits: dustinm, preview 12:03 -!- Netsplit over, joins: dustinm 12:04 -!- preview [~quassel@2407:7000:842d:4000::3] has joined ##hplusroadmap 12:07 -!- danfox [~danfox@vulpinedesigns.co.uk] has quit [Ping timeout: 260 seconds] 12:08 -!- danfox [~danfox@vulpinedesigns.co.uk] has joined ##hplusroadmap 12:09 -!- jnpl [~jnplx@h57.89.213.151.dynamic.ip.windstream.net] has joined ##hplusroadmap 12:17 -!- augur [~augur@noisebridge130.static.monkeybrains.net] has joined ##hplusroadmap 12:18 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 12:20 -!- jnpl [~jnplx@h57.89.213.151.dynamic.ip.windstream.net] has quit [Quit: Leaving] 12:21 -!- jnplx [~jnplx@h57.89.213.151.dynamic.ip.windstream.net] has joined ##hplusroadmap 12:23 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Ping timeout: 276 seconds] 12:24 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 12:32 -!- jnplx [~jnplx@h57.89.213.151.dynamic.ip.windstream.net] has quit [Quit: Leaving] 12:34 -!- jnplx [~jnplx@h57.89.213.151.dynamic.ip.windstream.net] has joined ##hplusroadmap 12:36 < kanzure> "Rocket launching: A universal and efficient framework for training well-performing light net" https://arxiv.org/pdf/1708.04106.pdf 12:51 < kanzure> .tw https://twitter.com/GeneticJen/status/897156594936537088 12:51 < yoleaux> Get this, the dung beetle can't see an individual star but it CAN see the Milky Way. Our spiral galaxy. It orients itself by our galaxy https://pbs.twimg.com/media/DHNXHCIXgAAJva6.jpg (@GeneticJen, in reply to tw:897156294595014656) 12:52 < kanzure> ("you pass butter.") 12:53 < poppingtonic> http://dx.doi.org.sci-hub.cc/10.1016/j.cub.2012.12.034 12:54 < kanzure> ultimately, ain't we all just moving our own individual piles of crap thorugh the galaxy? 13:02 -!- Storyteller [~Storytell@unaffiliated/storyteller] has joined ##hplusroadmap 13:11 -!- CheckDavid [uid14990@gateway/web/irccloud.com/x-auuprvkrjcbejaik] has joined ##hplusroadmap 13:20 -!- Storyteller [~Storytell@unaffiliated/storyteller] has quit [Ping timeout: 255 seconds] 13:21 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Ping timeout: 240 seconds] 13:37 < maaku> pasky: there are many options for uplink to send transactions, and we're waiting on seeing how people want to use it before deciding on how to do that 13:37 < maaku> major point is that you would not use the same system for uplink (sending transactions) as you would for downlink 13:37 < maaku> in fact most usage modes we've encountered would be perfectly fine using 3G for sending transactions 13:38 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 13:43 < maaku> kanzure: #blockstream-satellite is a good way to distribute time-lock encryption keys. maybe you can imagine some real use cases for that ;) 13:48 < kanzure> well, libgen would have to actually distribute the data first. but perhaps they could be more public about the nature of the encrypted data. 13:48 < kanzure> e.g. upload the encrypted data to more US accessible servers, and just be open and frank about the nature of the data. 13:49 < kanzure> i'm not sure it's illegal to distribute encrypted copies of "illegal data". 13:50 < kanzure> i think that DMCA safe harbor should protect the host in this scenario.... you need to provide evidence that the material is owned by whoever requests the takedown, right? 13:51 < kanzure> huh did i just completely break the DMCA? 13:51 < kanzure> that can't be right. 13:52 < kanzure> "Circumvention of access controls by one who has lawfully obtained the encrypted copy is permitted if the circumvention is done in the course of "an act of good faith encryption research." The researcher must first have made a good faith effort to obtain authorization before the circumvention, and the circumvention itself must not constitute infringement. The researcher may also develop and ... 13:52 < kanzure> ...employ tools to circumvent the access controls for the sole purpose of carrying out the research, and may share those tools with collaborative researchers." 14:03 < fenn> good job, it was a piece of crap anyway 14:04 < kanzure> no no i'm surely missing something. 14:06 < fenn> here is some mysterious data, it's either 1) a cure for cancer 2) a malevolent ai 3) all scientific knowledge 14:07 < fenn> one could hardly be faulted for sharing it, especially if there are no encryption keys for it 14:07 < kanzure> (no public decryption keys... yet.) 14:07 < fenn> well you don't know what will happen in the future 14:07 < kanzure> anything can happy with those there crazy computers and internets! 14:07 < fenn> timelock requires someone to work on it too 14:08 < fenn> with enough timelock puzzles you can swamp their ability to solve all the puzzles 14:08 < kanzure> swamping the public's ability? 14:09 < fenn> sure why not 14:09 < kanzure> you only need to swamp long enough for the copies to go far and wide. 14:09 < fenn> like say you underestimate the cost of future computing hardware, and it turns out nobody can afford to solve timelock puzzles 14:09 < kanzure> so the total timelock duration could be calculated based on some worst case scenario bandwidth assumptions and replication rate estimates. 14:12 < kanzure> i don't know if you commented on swamping intentionally but that's essentially how ralph merkle came up with public key encryption. 14:12 < fenn> it sounds more like diffie helman 14:12 < fenn> n 14:13 < fenn> hell 14:13 < fenn> man 14:13 < fenn> fuck it 14:14 * kanzure nods 14:15 < kanzure> so why do you need more than one puzzle? 14:16 < kanzure> if you are using more than one puzzle then your threat model is that your adversary is buying up hardware and trying to crack your single puzzle, so therefore you need to use many puzzles instead? 14:17 < fenn> no, you can't solve a single timelock puzzle by buying more hardware 14:17 < kanzure> and if they buy enough hardware to solve all of the puzzles, then they can go ahead and get the data blacklisted. 14:17 < fenn> well, yes 14:18 < fenn> i don't know. i was just saying you can't be perfectly sure of what will happen 14:20 < kanzure> 14:15 <+jcowan> It would be easy to prove what the encrypted text was, precisely because there is so much of it, unless you use a cipher resistant to chosen-plaintext attacks. 14:20 < kanzure> 14:20 <+kanzure> alright well... let's assume we are not bad at cryptography. 14:21 < kanzure> 14:21 <+kanzure> i mean. outside of review of a specific proposal, let's not assume we are bad at it. 14:21 < kanzure> https://people.csail.mit.edu/rivest/lcs35-puzzle-description.txt 14:22 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Ping timeout: 248 seconds] 14:26 < kanzure> Taek: what do you think about the above proposal. also, would it be possible to fund a network like sia to autonomously copy a bunch of encrypted data, or do you need to decrypt it first before buying more redundancy? and are you compatible with timelock puzzles? 14:26 < fenn> compatible? 14:27 < kanzure> siacoin has its own weird stuff going on under the hood. 14:27 < kanzure> https://diyhpl.us/wiki/transcripts/mit-bitcoin-expo-2016/siacoin/ 14:27 < fenn> brute forcing random crypto stuff seems outside of the scope of siacoin 14:29 < kanzure> oh definitely. but i meant the part about copying around the data to get high redundancy of the data. 14:29 < fenn> i guess you could set up your contract so that they have to have the puzzle key to get paid 14:30 < kanzure> 14:29 <+lysobit> kanzure, you might want to ask kim dotcom about that with his mega.co.nz.. 14:30 < kanzure> 14:30 <+kanzure> lysobit: he did not use timelock encryption; he literally ran infrastructure that served up immediately-decryptable data. 14:30 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 14:35 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Remote host closed the connection] 14:40 < kanzure> 14:40 <+lysobit> well yes... that's like saying a crime that happens in a locked room could be a crime or not, it only counts if you can prove it 14:41 < kanzure> 14:40 <+lysobit> it's still a crime... but can't discover that crime 14:41 < kanzure> 14:41 <+lysobit> so for all intents and purposes that crime didn't occur 14:42 < fenn> this sounds fun. you could get all sorts of people to commit various crimes and not even know it 14:42 < kanzure> inject some encrypted data into random cat pics? 14:42 < fenn> "share this important wikileaks document!!!one" -> cat porn 14:43 < fenn> bestiality is a crime in many states 14:44 < fenn> with steganography it's just too easy 14:47 < kanzure> it would be helpful if the encrypted data could be scrambled and re-crypted by any user, where the scrambling user can keep a proof around that it's the same as the original data, but the hosting provider wont be able to tell. and then the scrambler just tells the downloaders some extra information to say how to de-scramble back to the original encrypted format. and ideally this technique ... 14:47 < kanzure> ...would not have high overhead. 14:47 < fenn> this reminds me of the old email twisters 14:48 < fenn> for anonymity 14:48 < kanzure> bram cohen's private information retrieval stuff? 14:48 < kanzure> or the mixmaster mixmailer mixbadnaming things? 14:49 < fenn> yeah something like that 14:51 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has joined ##hplusroadmap 14:55 < kanzure> without the rescrambling, H(data) by the hosting provider would show that they are actually hosting something that is alleged to be libgen's data. and that's where problems start happening. 15:10 < fltrz> whats the problem statement? 15:11 < kanzure> distribute a complete copy of all scientific papers to so many people that the data set will be effectively considered public, without violating opsec constraints. 15:11 < kanzure> this is expected to be 50-100 TB total. 15:12 < kanzure> libgen and scihub for the most part don't have high-bandwidth ways of distributing their data. if they started paying for a server in the US, then their identities would be revealed and they would be compelled to pay their $15m judgements. 15:15 < fltrz> yeah, when it comes to intellectual property the western world is screwing itself, those in control of the paywalls preferring to be king of the hill :( 15:15 < kanzure> so you need a way to get the data widely distributed, without screwing over everyone in the process. 15:15 < fltrz> in a way that the hoster can deny responsibility? 15:16 < kanzure> well, in a timelock distribution method, i would assume that by the time that the puzzle is cracked, the users would have long ago abandoned the original hosts. 15:17 < kanzure> e.g. first wave of hosts only handles distribution for ~10 users, those 10 users are responsible for making other hosting arrangements on their own, next round gets ~1000 users, who are then each responsible for finding other hosting providers, and so on. 15:17 < kanzure> and then later the timelock puzzle is solved after sufficient time has passed. 15:18 < fltrz> target audience wants the full x TB dataset, or the target audience is users requesting a specific paper? 15:19 < kanzure> all of it. 15:20 < kanzure> you could imagine (and i know this requires a tremendous feat of imagination) that the target audience includes libraries and librarians that want to offer that data to their communities. 15:20 < fltrz> but not many people have the kind of connection to download 100TB in a home setting 15:21 < fenn> a standard comcast connection can download 10TB/mo if you get the unlimited data cap upgrade 15:22 < fenn> maybe more, not sure 15:22 < kanzure> still, there are many people who would pay $2k to get a cloud server or whatever to host that data. it's fucking important data. 15:23 < kanzure> "humanity global science dividend" 15:23 < fenn> it would be cheaper to get a real server than to wait around for it to download on comcast 15:23 < fltrz> so this would be like a one year operation, first everyone downloads the 100TB popular "secret of the universe" file, after which the private key for decryption is published? 15:23 < kanzure> well anyone can publish the decryption key once the timelock puzzle is solved. 15:24 < fltrz> I think the people who appreciate the importance of the data would not like to depend a "cloud"=='someone elses computer' service 15:25 < kanzure> well.... agreed, but right now having 10,000 "cloud" copies of the data would be way better than the current situation. 15:25 < fltrz> kanzure: but one would want the decryption not to happen until a majority of users has the full file, as soon as its realized what the file is the server would be blacklisted? 15:26 < kanzure> and obv. we would tell users please for the love of god make sure to use an obscure hosting provider, since if everyone just uses amazon then they could go in and delete all of the data. 15:26 -!- g0d355__ [~lmao@104.131.75.159] has joined ##hplusroadmap 15:26 < fltrz> how do I know a server truly is obscure? for all I know it is amazon et al 15:26 < kanzure> fltrz: once the data is decrypted, yes the hosting providers (all 10,000 of those companies) could uniformly go in and search to see if they have copies. it's important for people to make offline backups, yeah. 15:27 < fltrz> but how do the users get an offline backup? by downloading through their comcast connection... 15:27 < fenn> mumble mumble freenet 15:27 < kanzure> once the entire data set is released, theoretically the publishers go out of business (it's a giant bluffing game) because why would any libraries pay subscription fees to something that is so widely available in public? 15:28 < kanzure> fltrz: if you are worried about bandwidth limitations, users can just order hard drives that have the data already installed. 15:28 < kanzure> fltrz: if you are worried that comcast will record a fingerprint of all encrypted data that is transferred, then see my re-scrambling proposal above (to change the hash of the data etc). 15:29 < fltrz> yes, I think hard drives could work, especially if there is a unique partitioning, so one would be encouraged to get a single HDD with only 1TB, if one does not have the money for 100 HDDs 15:30 < fltrz> presumably there will be enough people in the same state or whatever that have the other parts 15:30 < kanzure> problem w/ hard drives is that someone has to actually put that data on the drive. i dunno if you can order data online and get hard drives shipped to you... presumably someone is operating services like that. 15:30 < fltrz> I tried to find such a service but had a really hard time, when I tried to get the ImageNet database 15:31 < kanzure> there's stuff like this but it gets pretty expensive https://aws.amazon.com/snowball/disk/ 15:31 < fenn> fltrz: "fountain codes" let you reassemble a 100TB file from 100TB of data, regardless how it's partitioned and which particular chunks you got 15:31 < fltrz> I had contacted I think amazon, and the day after I had gone to an internet cafe and got just the training set, amazon called me 15:31 < kanzure> there's also a version of snowball that comes with a semi-truck and armed security guards https://aws.amazon.com/snowmobile/ 15:32 < fltrz> yeah it was snowball, never actually got a quote as I thought I had the complete ImageNet dataset and thought I no longer needed them 15:33 < fltrz> I don't know if they would make a problem, but I suspect amazon will pretend problems if your dataset is encrypted 15:33 < fltrz> they'll probably stall and ask more details for copying the dataset more "optimally" etc 15:33 -!- poppingtonic [~brian@unaffiliated/poppingtonic] has quit [Ping timeout: 240 seconds] 15:33 < kanzure> giant corps love encryption. amazon would have a huge public relations problem if it turns out that they don't allow encrypted data. 15:33 < fenn> they actually force it to be encrypted 15:33 < kanzure> anyway, yeah i think there must be another online service that just sends you random hard drives, surely. 15:34 < fltrz> kanzure: I would love to hear about such a service, but I failed to find one 15:34 < kanzure> something like: "here's 20 MB of important encrypted data, icanhas usb thumb drive? please ship it to this address: ...". 15:34 < fltrz> yeah exactly 15:34 < fltrz> a local service would be cheap 15:34 < fenn> how big is ImageNet? 15:35 < kanzure> this one is for upload :\ https://www.idrive.com/idrive-express 15:35 < fltrz> I forgot, I can look it up, the training set was reasonable size but still a lot for home connection 15:35 < kanzure> https://www.backblaze.com/blog/4-tb-usb-restore-drives-are-here-yay/ 15:35 < kanzure> "USB Flash Drive — For $99, you select up to 110 GB of files/folders you want to restore and get your files on 128 GB USB flash drive. We send it to you next day express (within the US) so you get your data fast and you get to keep the drive" 15:35 < kanzure> "USB External Hard Drive — For $189, you select up to 3,500,000 MB of files/folders you want to restore and get your files on an external USB hard drive large enough to hold your data, up to a 4 TB drive. Once prepared, we send it to you next day express (within the US) and you get to keep the drive.." 15:36 < kanzure> 128 GB? what is this, a data backup solution for ants? 15:40 < fenn> not sure if this data is correct, but a local isp charges $400/mo for a 1U slot and $0.16/TB bandwidth 15:41 < fenn> i think it's 10Gbit so that would take about a day to download (?) 15:41 < fenn> you're limited by the other end of the download 15:42 < kanzure> and right now that limitation is libgen + scihub and the torrent seeders. 15:45 < fltrz> whats the difference between libgen and scihub? 15:45 < fenn> libgen has books and scihub has journals 15:45 < fltrz> oh ok 15:45 < fltrz> and the 100TB is scihub or both? 15:45 < kanzure> libgen supposedly has a copy of scihub but it's not clear whether this is true or how good this copy is or what the fuck is going on. 15:46 < kanzure> scihub is about 50 TB these days. libgen without "scimag" was 17 TB last time i looked. 15:46 < fenn> i bet someone is just confused about libgen's "scimag" 15:46 < fltrz> I also remember scihub sometimes fetching a paper for the first time 15:46 < kanzure> yes scihub has its own cache too. 15:47 < kanzure> fenn: maybe. i feel like i might have seen her mention libgen's mirror. dunno. i don't have a reference. 15:57 < kanzure> yeah it would be bad for us to make that assumption and it turns out to be false. 15:59 < kanzure> anyway based on fenn's numbers, it seems like the total replication cost would be <$5,000 USD per copy, if people were making redundantly hosted copies of course. 16:00 < fenn> i've been trying to double check those numbers and it's really frickin hard to get any actual price quotes 16:00 -!- CheckDavid [uid14990@gateway/web/irccloud.com/x-auuprvkrjcbejaik] has quit [Quit: Connection closed for inactivity] 16:01 < kanzure> it would be interesting if there would be a way to let the hosts communicate with each other and decide at a moment whether they should all start seeding a torrent at the same moment. you could interrogte the other hosts to check if they have the chunks they claim to have. 16:01 < kanzure> (or at least: you could move from private seeding to slightly-more-public seeding, at that agreed moment.) 16:02 < kanzure> it's interesting that full encyclopedia sets used to cost like $2k or whatever too. it's about the same cost here. lot of techie families would absolutely put in that money. 16:03 < fenn> you could hire door to door science salesmen to lug briefcases of hard drives around in an optimal manner 16:04 < kanzure> i would need some kind of algorithm for that. 16:04 < nmz787> some kind of travelling salesmen algorithm... hrmm 16:05 < nmz787> or salesman? 16:05 < nmz787> I gues we want parallelism 16:05 < kanzure> (i wouldn't mind that kind of "MLM" scam. at least you get a bunch of data out of it...) 16:06 < kanzure> (hint: it shouldn't be related to payments because then you can trace that. so don't do that.) 16:13 < fltrz> perhaps sort the pdf's by file size, the historical ones that were badly scanned, and the ones with high res pictures could form a separate category 16:14 < fltrz> perhaps a large portion can fit in substantially less space 16:15 < kanzure> yea at this scale, might even make sense to do a shitty pdf -> plaintext attempt. 16:16 -!- preview [~quassel@2407:7000:842d:4000::3] has quit [Remote host closed the connection] 16:17 < fenn> many pdfs also contain pictures at much higher resolution than really necessary 16:18 < fenn> you could trim some fraction of the data by making everything 120 dpi or whatever 16:18 < kanzure> how you gonna determine that? "observe the skin texture" says the text. figure reduced in resolution. now wat? 16:19 < fenn> any modification carries with it some risk of not working of course 16:19 < fenn> i'm more worried about broken/weird pdf files not getting converted right 16:20 < fltrz> I'm curious what the pdf size histogram looks like 16:20 < fenn> papers are intended to be viewed as roughly a A4 paper sheet per page 16:20 < fltrz> perhaps its 5% taking up 50% of dataset size 16:20 < kanzure> fltrz: well you could look at my data set i guess. and paperbot's. 16:20 < kanzure> http://diyhpl.us/~bryan/papers2/paperbot/ 16:20 < fenn> we had arXiv somewhere 16:21 < kanzure> oh damn where did i put arxiv 16:21 < fltrz> oh thats a good point, does sci-hub mix open access papers with the rest? 16:22 < fenn> open access papers are less than a few percent 16:23 < fltrz> of all papers in the world? or of the sci-hub dataset? 16:24 < fenn> kanzure: i copied some stuff from /mnt/arxiv but it's not there anymore 16:24 < fenn> on nov 21 2014 16:27 < fenn> fltrz: here's an arbitrary slice of 1753 arxiv papers sorted by kB http://fennetic.net/irc/arxiv_sizes 16:30 < fenn> sci-hub has 81 million articles; if they had the same average size as this slice of arxiv there would only be 20TB of data 16:31 < fenn> i conclude that 80% of the data is larger than it needs to be (bad scans, supplementary data, etc) 16:34 < fltrz> fenn: units are kB I presume? 16:35 < fenn> ya 16:37 < fenn> the largest paper (cond-mat0001397.pdf) has a bunch of vector art crystal structures embedded in it 16:39 < fenn> like 4 figures 16:39 < fltrz> so I guess the larger ones are due to some institution scanning a physical copy, at too high resolution "someone"-can-always-downsample-later-attitude 16:40 < fltrz> because the physical work of scanning was larger than any digital work afterwards 16:41 < fenn> no in this arxiv paper it's generated directly into the pdf with some molecular structure visualization program 16:41 < fenn> http://fennetic.net/irc/cond-mat0001397.pdf 16:41 < fenn> might make more sense to just see it 16:41 < fltrz> yes, I was referring to the large files in sci-hub 16:43 < fenn> i dont know 16:45 < fltrz> I guess we could ask sci-hub people for histogram? possibly point to or code the histogram generator for them? 16:46 < fltrz> if 20TB covered 80% of articles, then only a bunch of hard-core people need to get the 100TB version 16:47 < fltrz> nmz787: thanks for the intro to 5 part reverse engineering series, I noticed a couple of tricks I hadn't tried before... 16:47 < fenn> or ask the authors of this paper https://peerj.com/preprints/3100/ 16:48 < fenn> .title 16:48 < yoleaux> Sci-Hub provides access to nearly all scholarly literature [PeerJ Preprints] 16:59 < fenn> .title https://zenodo.org/record/472493 16:59 < yoleaux> Data and Scripts for Looking into Pandora's Box: The Content of Sci-Hub and its Usage | Zenodo 16:59 < fenn> this was for a paper that made histograms by publisher and by journal 17:01 < kanzure> you've unzipped pandora's fly. now look what you've done. 17:02 < fenn> i'm over here reading the zip manpage 17:07 -!- Storyteller [~Storytell@unaffiliated/storyteller] has joined ##hplusroadmap 17:08 < jnplx> Would it be possible to obtain the input the authors fed their mol vis program to get those figures? Distributing something like a smiles or an inchi would be much smaller 17:12 < fenn> lossless compression is the biggest rabbit hole of them all 17:12 < fenn> you may wake up one day in a lossless compression program 17:14 < kanzure> tracking down the authors of all 50 TB of papers is going to be pointless. 17:26 < jnplx> compressing the text to generate figures vs. the figures themselves could definitely help, where the input is available/mineable 17:27 < jnplx> I agree that contacting the authors would be a dead-end 17:27 < fenn> there's no way to do that reliably for 80 million papers 17:27 < fenn> there's no metadata about how they constructed the figures 17:28 < fenn> and especially not in any machine parseable format 17:29 < jnplx> Table I of that paper you linked provides the coordinates of those atoms 17:38 < fenn> this list of papers available on scihub may work as a DOI crossref 17:39 < fenn> aw nevermind there's no titles 17:47 < fenn> that pandoras box thing only contains DOI, publisher, and journal 17:47 < fltrz> easiest would be for sci-hub people themselves, or perhaps to check if the torrents have some index file with sizes mentioned? 17:48 < kanzure> https://github.com/petertodd/timelock 17:49 < kanzure> https://github.com/jaibot/timelock 17:51 < kanzure> er wait.. that's not a cryptographic timelock O_O. 17:54 -!- maaku [~mark@173.234.25.100] has quit [Quit: Lost terminal] 17:56 -!- maaku [~mark@173.234.25.100] has joined ##hplusroadmap 17:58 < fenn> ugh 15GB of SQL dump and no newlines 18:02 -!- danfox [~danfox@vulpinedesigns.co.uk] has quit [Ping timeout: 240 seconds] 18:04 -!- danfox [~danfox@vulpinedesigns.co.uk] has joined ##hplusroadmap 18:18 < fltrz> you got the sql dumps? 18:19 < fltrz> is there a description somewhere of what data the sql dumps contain? I assume at least DOI and filehash 18:23 < fenn> i was looking at sci-mag dump 18:26 < fenn> this is the beginning of the file http://fennetic.net/irc/sci-mag.sql 18:30 < fenn> http://fennetic.net/irc/backup_scimag_head10000.sql.tgz 18:32 < fenn> maybe this is useless because it's a different repository entirely 18:35 < fltrz> it has a 'Filesize' int :) 18:37 < fltrz> if I disable text wrapping in my editor each entry is on its own line here 18:40 < fltrz> the line between lines with ID 65 and 67 is a line with ID 21067061 ... weird 18:40 -!- jtimon [~quassel@143.29.134.37.dynamic.jazztel.es] has joined ##hplusroadmap 18:42 < fenn> i added linebreaks 18:42 < fltrz> that can't cause it 18:42 < fltrz> oh I see thanks 18:46 < fltrz> fenn, you have the whole sql? perhaps by checking the number of entries you can find out if its similar to sci-hub or not 18:46 < fltrz> i.e. 80M ish 18:48 < fltrz> where did you find the sql? 18:48 < fenn> sci-hub has been growing; around the time this dump was created there were probably 21 million 18:49 < fenn> i got it from ftp://ftp.libgen.in/dbdumps/scimag_dbbackup-2015-03-07.rar 18:50 < fltrz> I just want to see a plot of percentage of sorted files vs size of sorted files 18:51 < fltrz> sorted by filesize 18:51 < fltrz> hm the link doesn't work here 18:52 < fenn> yeah everything's broken and seized by official officials 18:53 < fenn> http://gnusha.org/~bryan/scimag/ 18:55 < kanzure> not sure you should know about that 18:56 < kanzure> fltrz: take a look real quick before i disable that index. 18:57 < fenn> it's just db dumps 18:57 < kanzure> YES, HUMON. TASTY DATABASE DUMPS. INDEED. HMM. 18:59 < fltrz> https://github.com/greenelab/scihub/tree/master/download/libgen 19:00 < fltrz> results at https://figshare.com/articles/A_user-friendly_extract_of_the_LibGen_scimag_metadata_SQL_dump_on_2017-04-07/5231245 19:02 < fltrz> so the .xz 4.8GB file is smaller than the original sql dump rar from libgen and results in a tab separated format instead of SQL 19:02 < fltrz> might actually be usefull to make a local search engine for titles 19:03 < fltrz> anyway I'll download that to make the histogram and cumulative size plots 19:06 < fltrz> and perhaps also filesize vs publication year scatterplot 19:07 < fltrz> its really possible the big ones are just conservative hi-res scans 19:08 < kanzure> sometimes instead of a pdf of a single article, you get the entire issue. if each paper in the issue was indexed that way, then you get a lot of duplication. 19:09 < fltrz> oh, but those should have the same md5sum, good you point that out 19:10 < kanzure> some papers include an excerpt of the paper from the next page. it's only 10% of the page content, but at scale this adds up to an enormous amount of extra duplicated data. 19:10 < kanzure> they really really need to start using multiple hash functions. md5 for everything is a bad idea. 19:10 < kanzure> it should be md5 + sha1 + sha2 + sha3. 19:10 < fenn> there is sha1 in the db dump 19:10 < fltrz> but I think I did often see some book/ comprehensive review by multiple authors, and you could download the whole as one pdf, but also subsections as one pdf or specific articles as one pdf... 19:10 < kanzure> (and sha1 isn't going to last forever either) 19:11 < fltrz> i wrote md5 but just meant hash function sorry 19:13 -!- g0d355__ [~lmao@104.131.75.159] has quit [Ping timeout: 248 seconds] 19:13 -!- jtimon [~quassel@143.29.134.37.dynamic.jazztel.es] has quit [Ping timeout: 260 seconds] 19:19 * fenn sleeps 19:35 < kanzure> well anyway, i'll see if petertodd has some suggestions for timelock encryption, maybe timelock.git is not ideal for decryption keys (since it has more of a bitcoin timelock purpose). 19:38 < kanzure> "Neural entrainment determines the words we hear" http://www.biorxiv.org/content/early/2017/08/10/175000 19:46 < kanzure> hope superkuh doesn't mind me prosetylizing his http://superkuh.com/spiralantenna.html in #blockstream-satellite 19:46 < superkuh> For most satellite stuff it's going to be better to make something frequency specific. 19:47 < kanzure> 19:47 <@gmaxwell> kanzure: might be hard to tune one of those for 12GHz. you could etch one on a PCB. 19:47 < kanzure> well it's 12178.3 MHz so... https://blockstream.com/satellite/satellite/# 19:48 < superkuh> So, like a 1m dish. 19:48 < kanzure> yes. 19:49 < kanzure> superkuh: they are presently interested in stupid-cheap ways to make the data available. 19:50 < kanzure> and they are also willing to front upfront costs (or recurring expenses, like paying for satellite bandwidth). 19:52 < kanzure> would it work better to just draw a spiral in the dirt and do some casting. 19:55 < kanzure> isn't there a well-known device for turning metal wires. it's some sort of knob on a desk and you just keep wrapping around it. 20:08 < superkuh> I don't think a spiral would be good for this application. 12 GHz low noise block downconverters w/cast horns are extremely cheap. And they're optimized to illuminate the type of cheap reflector dishes you can get. 20:09 < kanzure> i think if you wrap around a cone with increasing width, you can make a spiral more quickly? 20:10 < superkuh> http://superkuh.com/conical-spiral-antenna.html 20:10 < kanzure> welp there you go. 20:11 < superkuh> It's mediocre over a very wide frequency range. 20:11 < superkuh> In my application that's great. 20:12 -!- UnknownRogue [~UnknownRo@2a04:9dc0:c1:113:402:200:0:1927] has joined ##hplusroadmap 20:47 -!- Storyteller [~Storytell@unaffiliated/storyteller] has quit [Quit: Leaving] 20:56 < kanzure> 20:21 yeah, AFAIK timelock.git is the best out there right now 20:56 < kanzure> 20:21 though it's kinda ugly... 21:03 -!- jaboja [~jaboja@jaboja.pl] has quit [Remote host closed the connection] 21:10 -!- UnknownRogueX [~UnknownRo@2a04:9dc0:c1:113:402:200:0:1927] has joined ##hplusroadmap 21:14 -!- UnknownRogue [~UnknownRo@2a04:9dc0:c1:113:402:200:0:1927] has quit [Ping timeout: 276 seconds] 21:31 < fltrz> i'm not entirely sure why you seek a timelock: what is the advantage over the sender broadcasting encrypted data, and only revealing the decryption key after the broadcast is complete? 21:53 -!- Malvolio [~Malvolio@unaffiliated/malvolio] has quit [Quit: Malvolio] 22:03 -!- UnknownRogueX [~UnknownRo@2a04:9dc0:c1:113:402:200:0:1927] has quit [Quit: Leaving] 22:06 -!- y0ur1 [~y0ur1@gateway/tor-sasl/y0ur1] has joined ##hplusroadmap 22:07 -!- Malvolio [~Malvolio@unaffiliated/malvolio] has joined ##hplusroadmap 22:23 -!- CheckDavid [uid14990@gateway/web/irccloud.com/x-hweobyjufygbkmzw] has joined ##hplusroadmap 22:24 -!- helleshin [~talinck@cpe-174-97-113-184.cinci.res.rr.com] has joined ##hplusroadmap 22:26 -!- hehelleshin [~talinck@cpe-174-97-113-184.cinci.res.rr.com] has quit [Ping timeout: 248 seconds] 22:36 -!- Gurkenglas_ [~Gurkengla@dslb-178-000-218-087.178.000.pools.vodafone-ip.de] has quit [Ping timeout: 240 seconds] 23:29 -!- esmerelda [~mabel@174-24-228-29.tukw.qwest.net] has joined ##hplusroadmap 23:29 -!- esmerelda [~mabel@174-24-228-29.tukw.qwest.net] has quit [Changing host] 23:29 -!- esmerelda [~mabel@unaffiliated/jacco] has joined ##hplusroadmap 23:36 < kanzure> fltrz: the owner of the data might not be around to broadcast the decryption key later. 23:45 < fltrz> the entire database transmission should happen in chunks or as an atomic event? 23:46 < kanzure> well, i would prefer atomic so that everyone is forced to get the full thing 23:47 < kanzure> in the past my concern about chunks was always that people would only get the chunk that has the file they want. but if it's an encrypted mystery grab bag, i guess that's not much of an option. so maybe chunking isn't so bad. 23:47 < fltrz> if as an atomic event, then until the downloads were completed, no demonstrable crime has occured (since its just random entropy without the key) and its a matter of seconds or minutes to distribute the key 23:48 < fltrz> I downloaded the tab separated value index of scimag, tomorrow I will spend some time and perhaps create the cumulative plot 23:48 < kanzure> right. i think the trick is going to be: how to get enough attention on the encrypted blob soon enough, while also not attracting too much attention? if the original host is still the primary host by the time the key is unlocked, the whole thing can be trivially traced back to whoever first uploaded it to that single host. whoooops. 23:49 < fltrz> I hope (probably idlely) that a convincing majority of the papers turn out to fit in a convincing minority of data 23:50 < fltrz> if they can fit in a few terrabytes, perhaps more people would be willing to take it 23:52 < fltrz> I think perhaps cryptographically incentivised p2p filesharing schemes are perhaps the best 23:52 < kanzure> well that's why i highlighted taek yesterday. he might have something his sleeve on that front which could even be benefcial to this particular problem. 23:53 < kanzure> http://diyhpl.us/wiki/transcripts/mit-bitcoin-expo-2016/siacoin/ 23:53 < fltrz> or alternatively, divide the papers by subject matter, but only in very wide categorization, i.e. all physics papers 23:55 < fltrz> see it from the perspective of some student, not too well off... might still buy a few TB HDD to store all papers relating to his studies... but not going to spend 50x more... the participating audience would be way larger 23:56 < fltrz> if people are forced to take all or nothing irrespective of subject... I think it would chase more people away 23:57 < fltrz> but within a subject... also, once one student has the db for say physics, he would know all the other people who might want a copy too, without having to pipe it through intertubes --- Log closed Wed Aug 16 00:00:51 2017