--- Log opened Tue Jan 10 00:00:10 2023
02:16 -!- Malvolio [~Malvolio@idlerpg/player/Malvolio] has quit [Ping timeout: 260 seconds]
02:37 -!- Malvolio [~Malvolio@idlerpg/player/Malvolio] has joined #hplusroadmap
03:21 -!- Malvolio [~Malvolio@idlerpg/player/Malvolio] has quit [Ping timeout: 260 seconds]
03:23 -!- TMM_ [hp@amanda.tmm.cx] has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
03:23 -!- TMM_ [hp@amanda.tmm.cx] has joined #hplusroadmap
04:33 -!- test_ is now known as _flood
05:19 -!- yashgaroth [~ffffffff@2601:5c4:c780:6aa0::4249] has joined #hplusroadmap
05:31 -!- TMM_ [hp@amanda.tmm.cx] has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
05:32 -!- TMM_ [hp@amanda.tmm.cx] has joined #hplusroadmap
08:31 < kanzure> "One of [Fathom Computing's] long-term goals is to build the hardware to train neural networks with the same number of parameters as the human brain has synapses (>100 trillion)"
09:34 < nsh> lol
09:35 -!- cthlolo [~lorogue@77.33.23.154.dhcp.fibianet.dk] has joined #hplusroadmap
09:43 < lsneff> yeah, I talked to Andrew Payne about e11 last year
09:44 < lsneff> It’s an FRO
09:44 < lsneff> Essentially a startup
09:44 < lsneff> They were on track to be able to process entire human brains by roughly 2040 iirc
09:47 < lsneff> maybe i'll apply to fathom to see where they are
09:51 < lsneff> that was a while ago, not sure what their estimate is now tho
10:29 -!- stipa_ [~stipa@user/stipa] has joined #hplusroadmap
10:30 -!- stipa [~stipa@user/stipa] has quit [Ping timeout: 260 seconds]
10:30 -!- stipa_ is now known as stipa
10:59 < kanzure> https://www.youtube.com/watch?v=PoDJZ4Ln9Rw
10:59 < Muaddib> [PoDJZ4Ln9Rw] Neuroethology of Toads (Part 3 of 3; English): Interpretation, Verification & Application (12:03)
11:04 < kanzure> from https://joerg-peter-ewert.de/1.html
11:49 -!- L29Ah [~L29Ah@wikipedia/L29Ah] has quit [Read error: Connection reset by peer]
12:02 < kanzure> .wik human accelerated regions
12:02 < saxo> "Human accelerated regions (HARs), first described in August 2006, are a set of 49 segments of the human genome that are conserved throughout vertebrate evolution but are strikingly different in humans." - https://en.wikipedia.org/wiki/Human_accelerated_regions
12:04 < kanzure> mice with humanized HARE5 https://www.sciencedirect.com/science/article/pii/S0960982215000731 and brain size effect
12:17 < kanzure> hmm https://www.licenses.ai/
12:28 -!- L29Ah [~L29Ah@wikipedia/L29Ah] has joined #hplusroadmap
12:29 -!- codaraxis [~codaraxis@user/codaraxis] has joined #hplusroadmap
12:35 < superkuh> youtube just changed their policy so that unless you give them your phone number you cannot include URLs in video descriptions.
12:36 < muurkha> every day the jaws close tighter
12:45 -!- cthlolo [~lorogue@77.33.23.154.dhcp.fibianet.dk] has quit [Read error: Connection reset by peer]
15:49 < kanzure> https://www.righto.com/2023/01/the-8086-processors-microcode-pipeline.html https://news.ycombinator.com/item?id=34329201
16:08 -!- TMM_ [hp@amanda.tmm.cx] has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
16:09 -!- TMM_ [hp@amanda.tmm.cx] has joined #hplusroadmap
16:20 < kanzure> "A generalist agent" https://arxiv.org/abs/2205.06175
16:20 < kanzure> "just throw lots of computation at it" http://www.incompleteideas.net/IncIdeas/BitterLesson.html
16:32 < kanzure> "Towards the neuroevolution of low-level artificial general intelligence" https://arxiv.org/abs/2207.13583
16:35 < kanzure> "A critique of pure learning and what artificial neural networks can learn from animal brains" https://www.nature.com/articles/s41467-019-11786-6 (from zador lab)
16:36 < kanzure> ".. Here we argue that most animal behavior is not the result of clever learning algorithms—supervised or unsupervised—but is encoded in the genome. Specifically, animals are born with highly structured brain connectivity, which enables them to learn very rapidly. Because the wiring diagram is far too complex to be specified explicitly in the genome, it must be compressed through a ...
16:36 < kanzure> ...“genomic bottleneck”."
16:40 < kanzure> "Encoding innate ability through a genomic bottleneck" https://www.biorxiv.org/content/10.1101/2021.03.16.435261v2.abstract (also zador)
16:41 < kanzure> "... Here we formulate the problem of innate behavioral capacity in the context of artificial neural networks in terms of lossy compression of the weight matrix. We find that several standard network architectures can be compressed by several orders of magnitude, yielding pre-training performance that can approach that of the fully-trained network. Interestingly, for complex but not for simple ...
16:41 < kanzure> ...test problems, the genomic bottleneck algorithm also captures essential features of the circuit, leading to enhanced transfer learning to novel tasks and datasets. Our results suggest that compressing a neural circuit through the genomic bottleneck serves as a regularizer, enabling evolution to select simple circuits that can be readily adapted to important real-world tasks."
16:54 < kanzure> "Structure induces computational function in networks with diverse types of spiking neurons" https://www.biorxiv.org/content/10.1101/2021.05.18.444689v4
16:54 < kanzure> ".... Nature endows networks of spiking neurons in the brain with innate computing capabilities. But it has remained an open problem how the genome achieves that. Experimental data imply that the genome encodes synaptic connection probabilities between neurons depending on their genetic types and spatial distance. We show that this low-dimensional parameterization suffices for programming ...
16:54 < kanzure> ...fundamental computing capabilities into networks of spiking neurons. However, this method is only effective if the network employs a substantial number of different neuron types. This provides an intriguing answer to the open question why the brain employs so many neuron types, many more than were used so far in neural network models. Neural networks whose computational function is induced ...
16:54 < kanzure> ...through their connectivity structure, rather than through synaptic plasticity, are distinguished by short wire length and robustness to weight perturbations."
16:54 < kanzure> that's pretty cool.
17:15 -!- HumanG33k [~HumanG33k@dau94-2-82-66-65-160.fbx.proxad.net] has quit [Ping timeout: 260 seconds]
17:17 < kanzure> "A survey of visualization and analysis in high-resolution connectomics" https://diglib.eg.org/xmlui/bitstream/handle/10.1111/cgf14574/v41i3pp573-607.pdf?sequence=1
17:41 -!- codaraxis [~codaraxis@user/codaraxis] has quit [Quit: Leaving]
17:46 -!- yashgaroth [~ffffffff@2601:5c4:c780:6aa0::4249] has quit [Quit: Leaving]
17:55 < kanzure> "It has been estimated that the brain of a three-year-old child has about 10^15 synapses (1 quadrillion)." not familiar with this estimate
18:09 < muurkha> seems like a good ballpark
18:09 < kanzure> .tw https://twitter.com/blennon_/status/1612641938813104128
18:09 < saxo> What are the best Discord servers where researchers and engineers building/training LLMs hangout? (@blennon_) psa from .tw cmd: fuck twitter
18:26 < kanzure> https://www.amazon.com/Corticonics-Neural-Circuits-Cerebral-Cortex/dp/0521376173
19:30 -!- HumanG33k [~HumanG33k@dau94-2-82-66-65-160.fbx.proxad.net] has joined #hplusroadmap
19:31 < lsneff> genomic bottleneck, interesting
19:33 < lsneff> that concept, of squeezing a large amount of data through a low bandwidth channel to pull the essential structure out comes up a lot in ml
19:38 < lsneff> I wonder when we’ll have the compute to do molecular dynamics simulations of an entire neuron
19:38 < lsneff> I could see that being necessary to figure out exactly what neurons are doing before we can pull out the really essential bits for wbe
19:40 < lsneff> or maybe slightly higher than that, protein interaction simulation
19:57 < L29Ah> or collect a lot of data from experiments on neurons and train a model on them!
20:20 < lsneff> possibly, but it seems very difficult to observe everything that neurons are doing
20:20 < lsneff> Electrodes aren’t enough, since there’s more going on than action potentials
23:00 < fenn> andrew payne is now CEO of e11. awkward (for me)
23:01 < fenn> they're still at the stage of figuring out image registration for small slices of tissue, but apparently the RNA sequencing stuff works at least as a proof of concept
23:21 < fenn> .title https://www.prweb.com/releases/foresightinstitute/president/prweb13304878.htm
23:21 < saxo> Foresight Institute Appoints Julia Bossmann as New President
23:23 < fenn> oh that happened in 2016, not today
23:59 < fenn> foresight president is currently allison deuttmann
--- Log closed Wed Jan 11 00:00:11 2023