--- Log opened Mon May 15 00:00:01 2023
03:53 -!- test__ [~flooded@146.70.195.83] has joined #hplusroadmap
03:53 -!- test__ is now known as _flood
05:03 -!- yashgaroth [~ffffffff@c-73-147-55-120.hsd1.va.comcast.net] has joined #hplusroadmap
06:16 < alethkit> muurkha: Existing ANNs are hard to have formal guarantees over
06:17 < alethkit> specially in chaotic environments like a kitchen
06:32 < L29Ah> muurkha: so how do you plan to combine LLMs with motor control systems and sensors? care to provide some examples on it being done with success?
06:32 < L29Ah> "ANNs" is a too huge of a generalization: these all are very domain-specific in their architecture when they do excel at things
06:52 -!- L29Ah [~L29Ah@wikipedia/L29Ah] has left #hplusroadmap []
06:52 -!- L29Ah [~L29Ah@wikipedia/L29Ah] has joined #hplusroadmap
07:49 -!- Pi20 [~Pi@223.190.92.144] has joined #hplusroadmap
08:06 -!- TMM_ [hp@amanda.tmm.cx] has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
08:06 -!- TMM_ [hp@amanda.tmm.cx] has joined #hplusroadmap
08:30 < muurkha> alethkit: it doesn't have to run faster than the bear
08:36 < muurkha> L29Ah: that's a pretty large field going back decades; Bekey and Goldberg published a book surveying it in 02012
08:37 < muurkha> the first edition is from 01993
08:39 < muurkha> well, those ANNS 40 and 50 years ago weren't LLMs, of course
08:40 < L29Ah> interpreting vague answer as "i don't know, there's probably some more r&d required since there aren't readily available examples of useful LLM-controlled robots"
08:42 < hprmbridge> kanzure> lego ATP synthetase https://cdn.discordapp.com/attachments/1064664282450628710/1107694428417687562/BBX443MCEAAtyQp.png
08:42 < L29Ah> i am not questioning the ability of ANNs to grok motor control, but the ability to provide an interface to motors/sensors that is usable for a LLM
08:42 < muurkha> oh, I see
08:44 < muurkha> what the humans do with their natural neural networks is that they train them to provide such an interface
08:45 < L29Ah> they don't have such an interface in the first place
08:52 -!- Pi20 [~Pi@223.190.92.144] has quit [Ping timeout: 245 seconds]
08:54 < muurkha> I think it's pretty common for one human to say to another something like, "Please go to the hardware store just past the obelisk and pick up a bag of cement and two bags of #4 nails, be back by 4 PM" and the other human to carry out a complex sequence of motor actions as a result
08:55 < muurkha> so evidently human language models have some kind of interface to motors and sensors that allows their large language models to control their motor actions sometimes
08:56 < muurkha> on other occasions the humans confabulate language-based rationales for their motor actions, but in examples like this I think the causality is clear
08:58 < muurkha> to develop controllable motor skills like Tetris, QWERTY, or football, they have to go through long training processes
08:59 < muurkha> and in those cases it's clear that the sensorimotor processes have substantial autonomy from the linguistic ones, but they are still sometimes controllable
09:03 < muurkha> by the linguistic networks, I mean.  football players can verbally plan tactics ahead of time and then carry them out during the game, and typists can take dictation, for example
09:03 < L29Ah> there's likely no human neural network that is fed with language tokens and executes them, especially since language is always abstract and lacks detail; and things like "go to the hardware store just past the obelisk" require complex planning and feedback that no ANN i'm aware of can execute down to the motor level regardless of how it was expressed, let alone using the extremely vague and abstract la
09:03 < L29Ah> nguage tokens you provided
09:06 < muurkha> are you claiming that the humans cannot follow verbal instructions, that the verbal instructions are not language tokens, or that the humans use something other than a neural network to execute them, such as an immaterial consciousness belonging to the spiritual realm?
09:07 < muurkha> certainly the humans do not always follow verbal instructions, but sometimes they do, which demonstrates that they have that capability
09:10 < L29Ah> i am claiming that a human-human verbal interface is a cop-out of the problem since it effectively demands another human to do everything for the LLM and tells us nothing how do human LLM interacts with the same human's motor control systems, and if the said LLM is in control due to its LLMy reasoning/thinking capabilities people like to boast here about, or is just serving as a translator-of-sorts for
09:10 < L29Ah>  other modules that are responsible for most of the human intelligence™
09:12 < muurkha> well, it's clearly not in control most of the time, but just as clearly when one human executes instructions they received verbally from another one, it clearly is, isn't it?
09:13 < muurkha> Maybe a sort of delegated control subordinate to things like instinctive dominance hierarchies and operant conditioning
09:14 < muurkha> but if the language processing part of a human was never able to control their sensorimotor capabilities, then in cases like the one I described above, the human being addressed would not go to the hardware store, lift the bag of cement, or return by 4 PM
09:15 < muurkha> they would just engage in some kind of verbal behavior and then continue with whatever motor activities they would have done anyway, or possibly something randomly different
09:16 < muurkha> but at least in my experience interactions like the one outlined above are very common
09:16 < muurkha> anyway I wasn't claiming that you have to replace all your robots' existing motor control PID loops with more ANNs, but rather that existing sensorimotor robotic systems are likely up to the task of enabling LLMs to carry out a plan to do things like cook a meal
09:23 < superkuh> http://erewhon.superkuh.com/library/Neuroscience/I%20of%20the%20Vortex_%20From%20Neurons%20to%20Self_%20Rodolfo%20R%20Llinas.pdf - chapter 7 of this, "Fixed Action Patterns: Automatic Brain Modules that Make Complex Movements" is some great thinking about how animal brain's central prediction centers manage the highly-underconstrained problems of motor control.
09:23 < superkuh> The entire book is well worth a read though.
09:38 -!- flooded [~flooded@146.70.174.147] has joined #hplusroadmap
09:42 -!- _flood [~flooded@146.70.195.83] has quit [Ping timeout: 250 seconds]
10:01 -!- cthlolo [~lorogue@77.33.23.154.dhcp.fibianet.dk] has joined #hplusroadmap
10:28 < hprmbridge> kanzure>  https://cdn.discordapp.com/attachments/1064664282450628710/1107721213616337017/Protein_translation.png
10:28 < hprmbridge> kanzure> from https://commons.wikimedia.org/wiki/File:Protein_translation.gif
10:28 < kanzure> oh, discord doesn't render gifs. weird.
10:30 < superkuh> That link is to an html page. This is the gif link, https://upload.wikimedia.org/wikipedia/commons/9/94/Protein_translation.gif
10:32 < kanzure> discord renders my second link, but the first one it converted from gif to png
11:00 -!- cthlolo [~lorogue@77.33.23.154.dhcp.fibianet.dk] has quit [Read error: Connection reset by peer]
11:43 < muurkha> superkuh: thank you very much for that link!
12:26 < hprmbridge> kanzure> https://homebrew.bio/
12:30 -!- Gooberpatrol66 [~Gooberpat@user/gooberpatrol66] has joined #hplusroadmap
12:32 -!- Guest60 [~Guest60@2a09:bac5:c534:15eb::22f:80] has joined #hplusroadmap
12:38 -!- Chiester [~Chiester@user/Chiester] has quit [Read error: Connection reset by peer]
12:40 -!- Chiester [~Chiester@user/Chiester] has joined #hplusroadmap
13:51 -!- Guest60 [~Guest60@2a09:bac5:c534:15eb::22f:80] has quit [Quit: Client closed]
14:10 -!- pasky [~pasky@nikam.ms.mff.cuni.cz] has quit [Remote host closed the connection]
14:16 -!- Guest80 [~Guest80@2603-8080-2701-6e00-ac77-9942-ee33-868e.res6.spectrum.com] has joined #hplusroadmap
14:18 -!- Guest80 [~Guest80@2603-8080-2701-6e00-ac77-9942-ee33-868e.res6.spectrum.com] has quit [Client Quit]
15:57 -!- test__ [~flooded@146.70.202.67] has joined #hplusroadmap
16:00 -!- flooded [~flooded@146.70.174.147] has quit [Ping timeout: 240 seconds]
16:01 < fenn> L29Ah> muurkha: so how do you plan to combine LLMs with motor control systems and sensors? care to provide some examples on it being done with success?  <-- https://arstechnica.com/information-technology/2023/03/embodied-ai-googles-palm-e-allows-robot-control-with-natural-commands/
16:02 < fenn> i read about similar schemes having robots perform tasks in simulated VR apartments, but i can't remember what it was called
16:13 < L29Ah> looks like some image classifier being slapped on top, that's certainly an improvement; i couldn't understand how do they implement motion control; https://arxiv.org/abs/2209.11302 gives some hints tho
16:13 < L29Ah> https://progprompt.github.io/ examples look badass
16:35 < fenn> i predict you'll need to fine tune the LLM to perform actions
16:36 < fenn> i believe fine tuning GPT-3/4 was needed to peform plugin API queries
16:37 < fenn> prompting just isn't very good at delivering repeatable, reliable, constrained outputs
16:38 < hprmbridge> lachlan> Constrained token sampling methods can help a lot with that
16:42 -!- stipa_ [~stipa@user/stipa] has joined #hplusroadmap
16:43 -!- stipa [~stipa@user/stipa] has quit [Ping timeout: 264 seconds]
16:44 -!- stipa_ is now known as stipa
17:10 < hprmbridge> nmz787> Anyone recall what the token vectorization/similarity-distance thing at the get together was?
17:12 < hprmbridge> lachlan> chroma I think
17:24 < hprmbridge> kanzure> what's a good link for the MDMA/cannabis yeast stuff
17:24 < hprmbridge> nmz787> Gmo mdma? Don't think that exists
17:40 < hprmbridge> kanzure> Maybe I'm thinking of something else.
17:44 < hprmbridge> nmz787> Probably psilocin or psilocybin
19:36 -!- yashgaroth [~ffffffff@c-73-147-55-120.hsd1.va.comcast.net] has quit [Quit: Leaving]
20:16 -!- TMM_ [hp@amanda.tmm.cx] has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
20:16 -!- TMM_ [hp@amanda.tmm.cx] has joined #hplusroadmap
21:48 -!- deltab [~deltab@user/deltab] has quit [Ping timeout: 240 seconds]
21:50 -!- deltab [~deltab@user/deltab] has joined #hplusroadmap
--- Log closed Tue May 16 00:00:02 2023