WEBVTT 00:20.090 --> 00:22.760 Doctor Steven Cap Rogers is our 00:22.760 --> 00:25.840 fearless leader , the founder of Act 3 , 00:26.350 --> 00:28.294 Department of the Air Force senior 00:28.294 --> 00:30.760 technologist for AI , and has been uh 00:30.760 --> 00:32.649 running this group for going on a 00:32.649 --> 00:34.871 couple of decades now . So over to UCAP 00:34.871 --> 00:36.927 to lead us in the Kapersky lecture . 00:37.580 --> 00:39.380 Yeah , just a little context . 00:40.869 --> 00:43.229 So yeah , go , go to the next , go to 00:43.229 --> 00:46.840 the next slide , um . Some years back , 00:46.970 --> 00:49.026 I was a young professor in the early 00:49.026 --> 00:52.810 80s , and I was poking around . I was 00:52.810 --> 00:54.921 really getting into optical computing 00:54.921 --> 00:57.143 and I ran into this guy , Matt Kriski , 00:57.143 --> 00:59.009 Special K , and , and we worked 00:59.009 --> 01:01.009 together for about 30 years doing a 01:01.009 --> 01:03.065 whole bunch of stuff , which you can 01:03.065 --> 01:03.049 see on that plaque . That plaque is 01:03.049 --> 01:06.970 actually a vault . It's inside a vault , 01:07.129 --> 01:10.040 a stiff on bass . We named it after him , 01:10.209 --> 01:12.639 a big auditorium there , and you can , 01:13.010 --> 01:14.954 you probably can't read the plaque 01:14.954 --> 01:17.066 there , but I mean , We did a diverse 01:17.066 --> 01:20.639 set of things and for a decade we had 01:20.639 --> 01:22.861 literally hundreds of graduate students 01:22.861 --> 01:25.028 through the graduate school there . It 01:25.028 --> 01:27.083 was , it was , it was quite a ride . 01:27.083 --> 01:29.306 Then , then I went off for a decade and 01:29.306 --> 01:31.361 did breast cancer detection , AI for 01:31.361 --> 01:33.583 breast cancer detection , and that came 01:33.583 --> 01:35.806 with me . And then um once we took that 01:35.806 --> 01:37.917 company public , um , I retired for a 01:37.917 --> 01:40.370 second time , but Matt and I had some 01:40.370 --> 01:42.610 problems we still cared about . And so 01:42.610 --> 01:45.889 I took this current job in 2006 . And 01:45.889 --> 01:48.169 so one of the deals was as we would 01:48.169 --> 01:50.739 every week get together to talk about 01:50.849 --> 01:53.071 these issues which we didn't think were 01:53.071 --> 01:55.293 being addressed , specifically the area 01:55.293 --> 01:57.349 of consciousness . And so that's how 01:57.349 --> 01:59.571 all this started , and I didn't want to 01:59.571 --> 02:01.793 short shrift the reason we do this . We 02:01.793 --> 02:04.071 lost him in 2011 after 5 years of this , 02:04.071 --> 02:06.238 but we kept it going on in his honor . 02:06.250 --> 02:08.139 If you go to the next slide , the 02:08.139 --> 02:10.306 reason for the Gebersky lecture was in 02:10.306 --> 02:13.130 the original days we would do this over 02:13.130 --> 02:15.639 at AFIT , and so we had this sort of 02:15.639 --> 02:17.929 rotating group of graduate students for 02:17.929 --> 02:20.096 whatever reason , that occasionally we 02:20.096 --> 02:22.262 would let them graduate . You know , I 02:22.262 --> 02:21.919 don't , I don't think we're ever gonna 02:21.919 --> 02:24.086 let Collin graduate , but , you know , 02:24.086 --> 02:25.863 we would let some of these guys 02:25.863 --> 02:28.086 graduate , new ones would come , and so 02:28.086 --> 02:30.252 we would sort of reboot and cause we'd 02:30.252 --> 02:32.363 have people join us during the year . 02:32.363 --> 02:34.475 It also gave us an ability to sort of 02:34.475 --> 02:36.530 pull together our current thoughts . 02:36.530 --> 02:38.752 And so that was this idea that I coined 02:38.752 --> 02:40.808 at the Kabrisky lecture , and here's 02:40.808 --> 02:43.309 what my intent is to cover . What we 02:43.309 --> 02:45.476 did this year , which we've never done 02:45.476 --> 02:47.698 before , we'll see if it works , is put 02:47.698 --> 02:49.865 together a deck of less than 30 slides 02:49.865 --> 02:51.976 to sort of take you from zero all the 02:51.976 --> 02:54.630 way up to , you know , full speed in 02:54.630 --> 02:56.960 terms of our weekly interactions . Uh , 02:56.979 --> 02:59.809 that's quite a task . Um , but the 02:59.809 --> 03:02.031 result of that is we don't have a bunch 03:02.031 --> 03:04.253 of the examples we normally throw in to 03:04.253 --> 03:06.587 justify some of the things we're saying . 03:06.587 --> 03:09.970 So my intent is to say them and try to 03:09.970 --> 03:12.192 answer some of your questions , but ask 03:12.192 --> 03:14.192 them as we go because I want you to 03:14.192 --> 03:16.359 understand how we're using the terms . 03:16.359 --> 03:18.359 And then what we'll do is , is next 03:18.359 --> 03:20.192 week we'll throw in a bunch more 03:20.192 --> 03:21.970 examples to justify some of the 03:21.970 --> 03:21.740 statements or at least tell you how we 03:21.740 --> 03:23.809 got to where we are . But here's the 03:23.809 --> 03:27.630 content . A again , originally the idea 03:27.630 --> 03:29.741 was , is that there's something going 03:29.741 --> 03:31.963 on that nature figured out which people 03:31.963 --> 03:34.186 call consciousness . Um , over the last 03:34.186 --> 03:36.297 couple of years , what we figured out 03:36.297 --> 03:38.186 is as the rest of the AI world is 03:38.186 --> 03:40.352 converged on using this term , the way 03:40.352 --> 03:42.869 we were using it is not standard . The 03:42.869 --> 03:44.813 way a lot of people use it . So we 03:44.813 --> 03:46.702 started converging on where we're 03:46.702 --> 03:48.758 exactly focused , which is this word 03:48.758 --> 03:50.929 Qua . So we're going to start out by 03:50.929 --> 03:53.770 giving you some examples of Qua and why 03:53.770 --> 03:55.992 we currently think you create them , if 03:55.992 --> 03:58.850 you will , the cash value of Qua , and 03:58.850 --> 04:00.739 because I'm an AI guy and we were 04:00.739 --> 04:03.072 always sort of focused in this AI world , 04:03.072 --> 04:05.128 the reason I care about that is It's 04:05.128 --> 04:07.461 our position that there is , there is a , 04:07.461 --> 04:10.020 a role that the creation and experience 04:10.020 --> 04:12.300 you know quality plays in giving you 04:12.300 --> 04:14.300 the intelligence that you have . So 04:14.300 --> 04:16.189 we're gonna go down that path and 04:16.189 --> 04:18.411 specifically with respect to What is it 04:18.411 --> 04:21.290 we can discern about the way we 04:21.290 --> 04:23.401 represent Quaglia and how we might be 04:23.401 --> 04:25.623 able to exploit that in AI ? So we'll , 04:25.623 --> 04:27.790 we'll converge on our set of tenants , 04:27.790 --> 04:30.339 if you will , our , our S3Q theory of 04:30.339 --> 04:32.609 Quaglia . Um , and then we'll try to 04:32.609 --> 04:34.720 say , we'll try to finish up today by 04:34.720 --> 04:36.887 sort of tying that together and saying 04:36.887 --> 04:39.053 what are the implications of that with 04:39.053 --> 04:41.109 respect to sort of the standard ways 04:41.109 --> 04:43.053 you do AI ? And , and when I do AI 04:43.053 --> 04:45.331 generally I , I go through these steps , 04:45.331 --> 04:47.165 you know , I I pick a language . 04:47.165 --> 04:49.109 Sometimes I do it in more than one 04:49.109 --> 04:51.220 representation . It might be a neural 04:51.220 --> 04:53.276 approach where the language is , you 04:53.276 --> 04:53.160 know , connections and weights and 04:53.160 --> 04:55.359 architectures and a learning role to 04:55.359 --> 04:57.192 sort of encode the model in that 04:57.192 --> 04:59.248 language . It could be in symbolic , 04:59.248 --> 05:01.192 you know . Either way , you pick a 05:01.192 --> 05:03.415 language , you encode a model , and you 05:03.415 --> 05:05.581 use the model to do things . The piece 05:05.581 --> 05:07.692 that we haven't emphasized as much as 05:07.692 --> 05:09.915 we should , I think , But in fact , one 05:09.915 --> 05:11.803 of the key pieces of this Quaglia 05:11.803 --> 05:14.026 lecture is that that we're in in nature 05:14.026 --> 05:16.549 always form a new , newua and we'll 05:16.549 --> 05:18.771 we'll work our way through that . And I 05:18.771 --> 05:20.549 think that idea of changing the 05:20.549 --> 05:22.605 vocabulary as you go along , um , is 05:22.605 --> 05:24.827 sort of a key piece here . So those are 05:24.827 --> 05:26.938 sort of the four things . Um , by the 05:26.938 --> 05:28.993 way , um , again , yell at me if you 05:28.993 --> 05:31.216 have questions , put them in the chat . 05:31.216 --> 05:30.700 Other people can answer them probably 05:30.700 --> 05:32.756 better than me in the chat if you're 05:32.756 --> 05:34.922 comfortable with that , or just say it 05:34.922 --> 05:37.144 out loud and we'll go from there . So , 05:37.144 --> 05:40.239 uh , go to the next slide , please . So 05:40.239 --> 05:42.406 you know what is this cu word ? What a 05:42.406 --> 05:44.869 call you . So look at this , these 05:44.869 --> 05:47.091 squares in the top right , the two sort 05:47.091 --> 05:49.202 of reddish squares in the top right . 05:49.202 --> 05:51.258 If you look at one of them , there's 05:51.258 --> 05:53.480 something you actually are experiencing 05:53.480 --> 05:55.591 that you see in your mind's eye , and 05:55.591 --> 05:57.758 you look at the other square , there's 05:57.758 --> 05:59.980 a slightly different experience looking 05:59.980 --> 06:02.147 at that one versus the first one . And 06:02.147 --> 06:04.258 so what those are , the key word is , 06:04.258 --> 06:06.258 it's the quality , it's what you're 06:06.258 --> 06:08.091 actually seeing as a result of a 06:08.091 --> 06:10.036 stimuli coming in from the outside 06:10.036 --> 06:12.147 world . And it , you know , of course 06:12.147 --> 06:12.059 it isn't just visual . There's a 06:12.059 --> 06:14.115 discernible difference between those 06:14.115 --> 06:16.337 two squares , hopefully . Hopefully you 06:16.337 --> 06:18.448 can see it . Um , but similarly , you 06:18.448 --> 06:20.448 know , if I , if I hear , if I hear 06:20.448 --> 06:22.503 something like if rock stars playing 06:22.503 --> 06:24.448 Rosamunde for me , you know , from 06:24.448 --> 06:26.281 Schubert on a piano , you know , 06:26.281 --> 06:28.281 there's sounds I hear , and there's 06:28.281 --> 06:28.220 that I hear it in my mind's , I'll say 06:28.220 --> 06:30.700 my mind's ear , or if I'm Eating a 06:30.700 --> 06:32.922 Giovanni's pizza over there on the left 06:32.922 --> 06:35.459 there's a taste I have . Those all are 06:35.459 --> 06:37.649 experiences , experiences I'm having . 06:37.940 --> 06:40.540 They're qualia that are evoked . And 06:40.540 --> 06:42.762 there's something discernibly different 06:42.762 --> 06:44.873 between the ones I'm experiencing and 06:44.873 --> 06:46.818 ones that could have been in their 06:46.818 --> 06:48.984 places . That square I could have made 06:48.984 --> 06:48.299 bluish , you know , that could have 06:48.299 --> 06:50.077 changed the huge saturation and 06:50.077 --> 06:51.855 blindness , huge saturation and 06:51.855 --> 06:54.077 brightness , and the quality evoked and 06:54.077 --> 06:56.355 you would have changed , OK . So there , 06:56.355 --> 06:58.077 there is in fact a discernible 06:58.077 --> 07:00.188 difference between the different quaa 07:00.188 --> 07:02.410 that can represent different aspects of 07:02.410 --> 07:04.355 the things that are in this , this 07:04.355 --> 07:06.521 theater you're , you're creating , and 07:06.521 --> 07:08.577 that's that . That Cartesian theater 07:08.577 --> 07:10.632 picture in the middle there from our 07:10.632 --> 07:12.966 former colleague , the late Dan Dennett , 07:12.966 --> 07:15.188 you know , he came up with this idea of 07:15.188 --> 07:17.021 you have this illusory Cartesian 07:17.021 --> 07:19.243 theater . It's as if you're have this a 07:19.243 --> 07:21.132 little homonculus sitting in , in 07:21.132 --> 07:23.299 Looking at an environment through your 07:23.299 --> 07:25.521 eyes , listening to it with your ears , 07:25.521 --> 07:25.309 tasting it with your , you know , 07:25.429 --> 07:27.707 ability to taste and touching it , etc . 07:27.869 --> 07:30.091 and you're having thoughts while you're 07:30.091 --> 07:32.313 in that theater . And so that's sort of 07:32.313 --> 07:34.480 the mental image . It's as if you're , 07:34.480 --> 07:36.425 you're creating this sensory motor 07:36.425 --> 07:38.313 world model , and it isn't just a 07:38.313 --> 07:40.536 sensory data , you know , it's , it's , 07:40.536 --> 07:42.647 for example , in humans , it includes 07:42.647 --> 07:44.813 moods . So they're on the bottom right 07:44.813 --> 07:46.813 every morning at dawn , I'm walking 07:46.813 --> 07:48.980 that dog at that lake , and It puts me 07:48.980 --> 07:51.202 in a mood , OK ? Um , you know , it's , 07:51.202 --> 07:53.313 it's the emotions I feel , you know , 07:53.313 --> 07:55.480 they're in the middle when I buried my 07:55.480 --> 07:57.702 dad , former Marian Gray at Arlington , 07:57.702 --> 07:59.925 those emotions , you know , are , are , 07:59.925 --> 07:59.459 you know , you put them all together 07:59.459 --> 08:01.730 with respect to how I spend my favorite 08:01.730 --> 08:03.619 days with , with my granddaughter 08:03.619 --> 08:05.850 Campbell , and I'm sorry about the blue 08:05.850 --> 08:07.739 things on her face , but that's a 08:07.739 --> 08:09.961 public affairs thing . You got to cover 08:09.961 --> 08:12.128 up children's faces . They don't worry 08:12.128 --> 08:14.295 about covering up my face . Who the he 08:14.295 --> 08:16.128 cares ? But everything gets tied 08:16.128 --> 08:18.239 together in in terms of this cohesive 08:18.239 --> 08:20.350 representation , OK ? And that's what 08:20.350 --> 08:22.628 quality are about . They populate that . 08:22.628 --> 08:24.795 A quality is any discernible aspect of 08:24.795 --> 08:26.961 those experiences . So it's , it's the 08:26.961 --> 08:29.183 redness of that square or the mood that 08:29.183 --> 08:31.239 I , I can say I am sort of in this . 08:31.239 --> 08:33.461 mood versus that or the emotions I feel 08:33.461 --> 08:35.628 that , hey , I love that granddaughter 08:35.628 --> 08:37.739 and I don't care about my grandsons , 08:37.739 --> 08:37.549 you know , it's all those discernible 08:37.549 --> 08:39.660 differences . Those are the different 08:39.660 --> 08:41.660 qualia and they populate this world 08:41.660 --> 08:44.469 model for you or it makes it seem as if 08:44.710 --> 08:46.877 they are , they are your world model . 08:46.877 --> 08:49.099 Now , if you go to the next slide , you 08:49.099 --> 08:51.250 know , I'm , I want With no 08:51.250 --> 08:53.361 justification , but at this point , I 08:53.361 --> 08:55.361 want you to sort of accept the fact 08:55.409 --> 08:57.650 that where I'm headed is all that stuff 08:57.650 --> 08:59.872 that you're creating in your quality of 08:59.872 --> 09:02.570 space is , is not really driving the 09:02.570 --> 09:04.514 world . It's not really , it's not 09:04.514 --> 09:06.848 reality . So for example , that redness , 09:06.848 --> 09:08.626 there's no red out there , OK ? 09:08.626 --> 09:10.737 Whatever you see in your mind's eye , 09:10.737 --> 09:12.792 that's not what's out there . You're 09:12.792 --> 09:14.959 creating that to represent some aspect 09:14.959 --> 09:16.848 of the environment . The thoughts 09:16.848 --> 09:19.070 you're having , you're creating in this 09:19.070 --> 09:21.181 Qua theater to represent , you know , 09:21.181 --> 09:23.348 this . Sort of mood versus that , this 09:23.348 --> 09:25.348 emotion versus that . So it in many 09:25.348 --> 09:27.570 ways is a tin god . It's not reality at 09:27.570 --> 09:29.459 all . And , and that's sort of an 09:29.459 --> 09:31.669 important leap to make , you know , 09:31.820 --> 09:34.042 you're creating a stable , consistent , 09:34.042 --> 09:35.931 and useful representation for the 09:35.931 --> 09:38.020 purpose of , of surviving and 09:38.020 --> 09:40.242 procreating . And if you go to the next 09:40.242 --> 09:42.298 slide , you , you know , that's this 09:42.298 --> 09:44.099 idea of a world model . You are 09:44.099 --> 09:46.460 creating this inside you as an agent . 09:47.130 --> 09:49.309 A representation that when things come 09:49.309 --> 09:51.309 in from the outside or you generate 09:51.309 --> 09:53.420 things from the inside , you can make 09:53.420 --> 09:55.031 sense out of it . You do the 09:55.031 --> 09:57.253 sense-making , OK ? And , and , and you 09:57.253 --> 09:59.476 do that independent of the uh I'm gonna 09:59.476 --> 10:01.698 contend , independent of the quality of 10:01.698 --> 10:03.698 formation . Quality of formation is 10:03.698 --> 10:05.698 going to play a role here , but I'm 10:05.698 --> 10:07.865 gonna contend that it's gonna play the 10:07.865 --> 10:09.920 role of populating and updating that 10:09.920 --> 10:12.087 world model . It is not , it is not in 10:12.087 --> 10:14.031 charge of it . It's not making any 10:14.031 --> 10:16.198 decisions for you , but it is critical 10:16.198 --> 10:18.309 in forming those memories in the in a 10:18.309 --> 10:20.476 representation that you can use in a , 10:20.476 --> 10:22.365 in a useful manner . So I'm gonna 10:22.365 --> 10:24.659 define who you are , yourself , if you 10:24.659 --> 10:26.770 will , as your personal world model . 10:26.770 --> 10:28.880 It's your consistent perspective . I 10:28.880 --> 10:31.102 don't , I have recognized the hand from 10:31.102 --> 10:33.380 Kevin Smith , so we'll keep going . No , 10:33.380 --> 10:35.491 go ahead , Nagle . I'm sorry . What , 10:35.491 --> 10:37.602 what , what's populated mean ? Yeah , 10:37.602 --> 10:40.190 so , so , um , as I was pointing out on 10:40.190 --> 10:42.412 those , for example , the red squares , 10:42.429 --> 10:44.869 there could be an insertion of the of 10:44.869 --> 10:47.010 the quality you see at that square on 10:47.010 --> 10:49.066 the left , or I could similarly have 10:49.066 --> 10:51.288 moved that square on the right into its 10:51.288 --> 10:53.066 place and I populated it with a 10:53.066 --> 10:55.232 different quality . So what I'm saying 10:55.232 --> 10:57.399 is there , there is alternatives which 10:57.399 --> 10:59.677 you can discern the difference between . 10:59.677 --> 11:01.788 That's what I mean by populated . You 11:01.788 --> 11:04.066 populated your , your experiences , um , 11:04.090 --> 11:06.570 with , with very specific quality 11:06.570 --> 11:09.409 experiences , and then that how you did 11:09.409 --> 11:11.576 that and how they relate to each other 11:11.576 --> 11:13.631 and how they interact is what you're 11:13.631 --> 11:15.853 going to use to update your knowledge . 11:15.853 --> 11:18.187 So means you have a specific experience . 11:18.187 --> 11:20.242 But that's , it's only populated for 11:20.242 --> 11:22.187 the learning because you were just 11:22.187 --> 11:24.187 making the decision making learning 11:24.187 --> 11:24.140 distinction . It's the pop the , it's 11:24.140 --> 11:26.309 not the populated thing isn't used for 11:26.309 --> 11:28.642 the decision making . What no what yeah , 11:29.070 --> 11:31.014 that's a great question . What I'm 11:31.014 --> 11:33.181 suggesting to you , and this is , this 11:33.181 --> 11:35.403 is going to be quite controversial , is 11:35.403 --> 11:37.626 decision making is done outside of that 11:37.626 --> 11:39.848 Qua loop . The Qua loop is nothing more 11:39.848 --> 11:41.514 than to put everything into a 11:41.514 --> 11:43.348 consistent representation in the 11:43.348 --> 11:45.514 context of everything else , so it can 11:45.514 --> 11:47.348 update your knowledge . OK . The 11:47.348 --> 11:49.403 knowledge is your world model , OK ? 11:49.403 --> 11:51.459 And that's what you're using for all 11:51.459 --> 11:53.626 the decision making and sense making . 11:53.626 --> 11:55.681 And that's doing , that's going on , 11:55.681 --> 11:57.848 that's stored away , that's outside of 11:57.848 --> 11:59.848 your , your quality of creation and 11:59.848 --> 12:01.903 experience loop . You're getting the 12:01.903 --> 12:03.848 illusion as if that is the world . 12:03.848 --> 12:05.959 You're getting the illusion is I made 12:05.959 --> 12:05.750 this conscious decision to do this or 12:05.750 --> 12:08.028 that , but that happens after the fact . 12:08.028 --> 12:10.194 I contend there's just so much data to 12:10.194 --> 12:12.250 suggest that that comes afterwards , 12:12.250 --> 12:14.250 but it has to be created for this . 12:15.000 --> 12:18.650 Illusion to be consistent . Rockstar . 12:19.109 --> 12:21.570 Oh wait , uh , Michael is first . 12:21.780 --> 12:24.099 Ancient Mike . And one of the few 12:24.099 --> 12:26.539 people around who knew , you knew Matt . 12:27.320 --> 12:29.429 How is the quality of representation 12:29.840 --> 12:31.507 differ from just the standard 12:31.507 --> 12:33.507 representation ? Is there something 12:33.507 --> 12:36.150 unique about it ? Yeah , and so , so 12:36.150 --> 12:38.372 you know that the whole idea of the S3Q 12:38.372 --> 12:40.483 model is there are characteristics of 12:40.483 --> 12:42.317 that representation which we can 12:42.317 --> 12:44.372 discern based upon how we experience 12:44.372 --> 12:47.299 them , and we need to capture those in 12:47.299 --> 12:49.521 the way we're currently trying to do AI 12:49.521 --> 12:53.150 if that's what you're asking . OK , 12:53.260 --> 12:57.059 OK , good . Rockstar ? Yes , this 12:57.059 --> 13:00.099 real short . Would you elaborate a 13:00.099 --> 13:03.219 little bit more by the last two words 13:03.219 --> 13:05.900 of this slide ? No , Claudia . Yeah , 13:06.020 --> 13:08.187 go , go down to the next slide and I , 13:08.187 --> 13:10.187 I think I'm going to parse it apart 13:10.187 --> 13:12.464 here in the next , yeah . So , so yeah , 13:12.464 --> 13:14.631 thanks . So , so here in this slide on 13:14.631 --> 13:16.742 the upper above that squiggly line in 13:16.742 --> 13:18.853 the upper right , there's a world out 13:18.853 --> 13:20.964 there , OK ? Everything below that is 13:20.964 --> 13:22.964 inside my mind , all right ? So out 13:22.964 --> 13:24.964 there in the world , there's things 13:24.964 --> 13:26.909 that happen and I sense it with my 13:26.909 --> 13:29.187 sensors that comes into this knowledge , 13:29.187 --> 13:31.242 my world model , if you will , and I 13:31.242 --> 13:33.464 can make sense of that and maybe decide 13:33.464 --> 13:35.687 what to do about it and go do it , OK ? 13:35.687 --> 13:37.853 I'm saying that happens and , and that 13:37.853 --> 13:39.909 is , is , I , um , you know , what , 13:39.909 --> 13:41.909 what I think our colleague Robert , 13:41.909 --> 13:41.799 who's sitting across the room from me 13:41.799 --> 13:43.632 right now , might call intuitive 13:43.632 --> 13:45.688 cognition . I make sense of it and I 13:45.688 --> 13:47.910 act upon it , OK ? But when I do that , 13:47.910 --> 13:49.743 you know , I have to , I have to 13:49.743 --> 13:51.855 account for that in , in my knowledge 13:51.855 --> 13:54.077 representation . So I might evoke these 13:54.077 --> 13:56.410 things on the left side of this diagram , 13:56.410 --> 13:58.521 those two ovals in the back . I might 13:58.521 --> 14:00.688 evoke the Qua creation system to say , 14:00.688 --> 14:02.688 hey , you , you just saw this , you 14:02.688 --> 14:04.577 just did that , you just had this 14:04.577 --> 14:06.299 thought , Go create Quaglia to 14:06.299 --> 14:08.410 represent that and have an experience 14:08.410 --> 14:10.577 of it , but what it is like . And then 14:10.577 --> 14:12.910 when you have that experience , you say , 14:12.910 --> 14:14.855 now that I've had that , I want to 14:14.855 --> 14:14.799 incorporate that back in my knowledge 14:14.799 --> 14:17.159 representation . So you , you now are 14:17.159 --> 14:19.450 able to account for experiences and 14:19.450 --> 14:21.228 update your knowledge via those 14:21.228 --> 14:23.719 experiences . So , What I'm suggesting 14:23.719 --> 14:26.359 to you , uh , Rockstar , is that I can 14:26.359 --> 14:28.581 come into my knowledge representation , 14:28.581 --> 14:30.637 my world model , and respond with no 14:30.637 --> 14:32.748 Qua . And we're going to see examples 14:32.748 --> 14:34.859 of that with an example in Blindsight 14:34.859 --> 14:37.081 or , or incongruent images . We'll give 14:37.081 --> 14:39.192 you examples of that . And you can do 14:39.192 --> 14:38.960 that without Qua and do very 14:38.960 --> 14:41.071 complicated processing . I mean , the 14:41.071 --> 14:43.016 discernment there is , is freaking 14:43.016 --> 14:45.080 amazing . But if you want to update 14:45.080 --> 14:47.229 that representation to be able to use 14:47.229 --> 14:49.451 in the long term , then you're going to 14:49.451 --> 14:51.562 have to go through this Qua process . 14:51.562 --> 14:54.799 Glenn , thank you . Glenn . Welcome , 14:54.809 --> 14:56.698 by the way . Yeah , I'm a , I'm a 14:56.698 --> 14:59.690 simple guy . So I just am reading an 14:59.690 --> 15:02.650 email , and whether it's all caps or 15:02.650 --> 15:05.640 whatever , and maybe there was , you 15:05.640 --> 15:07.862 know , email traffic leading up to it . 15:08.210 --> 15:10.969 But I take it on , you know , I see the 15:10.969 --> 15:14.909 text , but then all my biases and pre , 15:15.010 --> 15:17.559 uh , you know , this is what it means , 15:17.570 --> 15:20.130 or now this person , all the emails in 15:20.130 --> 15:22.719 caps . You know , the caps don't speak 15:22.719 --> 15:25.570 for themselves , so comment on , uh , 15:25.960 --> 15:28.127 you know , I'm taking this whole world 15:28.127 --> 15:30.238 reality matrix thing and just boiling 15:30.238 --> 15:33.130 it down to a simple how we perceive an 15:33.130 --> 15:36.530 email over . So you , you get inputs in . 15:36.640 --> 15:39.549 I mean that's visual data . It comes in . 15:40.039 --> 15:42.679 It , you , you look at those , your 15:42.679 --> 15:44.901 eyes are looking at that . It takes the 15:44.901 --> 15:47.235 information , puts it up on your retina , 15:47.235 --> 15:49.457 gets up on your visual cortex . You now 15:49.457 --> 15:51.512 use your knowledge that is stored to 15:51.512 --> 15:53.679 sort of interpret those words . Here's 15:53.679 --> 15:55.846 what those words mean . Um , the facts 15:55.846 --> 15:58.068 that they're in caps or not may in fact 15:58.068 --> 16:00.346 bring as as Kevin just put in the chat , 16:00.346 --> 16:02.599 exclamation , information that isn't in 16:02.599 --> 16:04.640 the text itself , it's additional 16:04.640 --> 16:06.696 information that brings , uh , sense 16:06.696 --> 16:08.751 making in a different direction , if 16:08.751 --> 16:11.084 you will . Um , that , that all goes on , 16:11.084 --> 16:13.400 but because it is language , because it 16:13.400 --> 16:15.567 is language and the way , the way this 16:15.567 --> 16:17.567 is going to work is you're going to 16:17.567 --> 16:19.678 come in and explore . It , but you're 16:19.678 --> 16:19.380 going to say , hey , I want to put this 16:19.380 --> 16:21.900 in context . So I create Gev . I read 16:21.900 --> 16:24.011 this word , that word . I create this 16:24.011 --> 16:26.130 sequence of , of Gwalia experiences 16:26.340 --> 16:28.284 associated with those sequences of 16:28.284 --> 16:30.284 words and what was in the paragraph 16:30.284 --> 16:32.340 before and what I did right before I 16:32.340 --> 16:34.396 got on this email . I had that phone 16:34.396 --> 16:36.229 call from uh from Neils , and so 16:36.229 --> 16:38.451 therefore I was in this frame of mind . 16:38.451 --> 16:40.451 All that goes into the sense making 16:40.451 --> 16:39.419 part of it . 16:44.320 --> 16:46.264 I'll come back to you , Glenn . Go 16:46.264 --> 16:48.209 ahead , Patrick . I got it . I was 16:48.209 --> 16:50.376 wondering if we could uh talk about uh 16:50.376 --> 16:52.859 cases of perception that don't lead to 16:52.859 --> 16:54.803 Quala . So something that might be 16:54.803 --> 16:57.026 perceived , but perhaps it's not , it's 16:57.026 --> 16:59.460 not the system is not capable of 16:59.460 --> 17:01.571 simulating it , or the representation 17:01.571 --> 17:03.682 isn't structurally coherent , right ? 17:03.682 --> 17:05.849 There are probably instances of things 17:05.849 --> 17:08.071 you perceive , maybe you can even react 17:08.071 --> 17:10.016 to them . But you may may not have 17:10.016 --> 17:12.127 formedqualia and therefore you're not 17:12.127 --> 17:14.238 able to create knowledge about them . 17:14.238 --> 17:16.460 Are there examples of that ? Yeah , but 17:16.460 --> 17:18.516 I'm gonna give you , I'll give you , 17:18.516 --> 17:20.571 well , I'm gonna give you a slightly 17:20.571 --> 17:19.959 different version which I think will 17:19.959 --> 17:21.903 sneak up on the same . Go down two 17:21.903 --> 17:24.729 slides . To the blind side . No , oh , 17:25.680 --> 17:27.680 there should be a , I'm sorry , one 17:27.680 --> 17:29.791 more slide , the blind slide , yeah , 17:29.791 --> 17:32.069 yeah , yeah , OK , yeah . So , so this , 17:32.069 --> 17:34.291 this poor critter here has had a stroke 17:34.291 --> 17:36.819 and his , his V1 has been destroyed 17:36.819 --> 17:39.390 bilaterally . So this individual , if 17:39.390 --> 17:41.446 you , if you stand him at the end of 17:41.446 --> 17:43.557 this hallway and you say , Hey , what 17:43.557 --> 17:45.279 do you see ? You're asking him 17:45.279 --> 17:47.446 consciously what quality you are being 17:47.446 --> 17:49.557 evoked as a result of the visual data 17:49.557 --> 17:51.834 coming in . And his answer will be , I , 17:51.834 --> 17:53.946 I , I don't see anything , you know , 17:53.946 --> 17:56.001 the world is black . I'm blind . And 17:56.001 --> 17:55.729 you say , walk down the hallway , but 17:55.729 --> 17:57.951 be careful . I put things in your way . 17:57.951 --> 18:00.173 And the person says , Give me my cane , 18:00.173 --> 18:02.229 and I'll tap my way through them and 18:02.229 --> 18:04.396 you say , No , walk down the hallway . 18:04.396 --> 18:06.562 All right ? This person will walk down 18:06.562 --> 18:08.562 the hallway and dodge the objects , 18:08.562 --> 18:10.785 even though if you ask him , you know , 18:10.785 --> 18:13.007 what does he see ? He sees nothing . He 18:13.007 --> 18:15.173 is not generating visual qua , but yet 18:15.173 --> 18:17.285 he's getting visual data in and being 18:17.285 --> 18:19.562 able to react appropriately to it , OK ? 18:19.562 --> 18:21.673 Similarly , there are people who have 18:21.673 --> 18:23.840 strokes and , and they can't recognize 18:23.840 --> 18:25.840 faces . So they look at that , they 18:25.840 --> 18:28.007 look at Patrick there , and they say , 18:28.007 --> 18:30.118 Hey , he's got glasses , he doesn't , 18:30.118 --> 18:32.062 he has more hair than Cap , but he 18:32.062 --> 18:34.062 doesn't have a lot . He smiles , so 18:34.062 --> 18:36.173 he's not like Cap at all , you know , 18:36.173 --> 18:38.340 he looks like a good guy , but I don't 18:38.340 --> 18:40.507 know who it is , and it turns out it's 18:40.507 --> 18:42.729 your mother , OK ? You know , she has , 18:42.729 --> 18:44.729 you know , this prosopagnosia . She 18:44.729 --> 18:46.896 can't put it together into the patrick 18:46.896 --> 18:49.007 quality , OK ? She can do the quality 18:49.007 --> 18:51.062 of the individual pieces , but loses 18:51.062 --> 18:53.062 the quality of patrickness , if you 18:53.062 --> 18:55.285 will . You know , the example I used to 18:55.285 --> 18:57.507 use when I used to give this lecture in 18:57.507 --> 18:57.219 person , and I'll hold this to my 18:57.219 --> 18:59.800 camera here , was this device here , OK ? 19:00.060 --> 19:02.270 And I ask people what is that ? All 19:02.270 --> 19:03.770 right . And what it's like 19:03.770 --> 19:05.937 prosopagnosia . You describe it . Oh , 19:05.937 --> 19:08.159 it's got a blue , it can go up and down 19:08.159 --> 19:10.159 on its cylinder . It's got a little 19:10.159 --> 19:12.159 pink part . I , they can't make the 19:12.159 --> 19:14.214 quality of what it is , but yet they 19:14.214 --> 19:16.270 can describe the pieces . So quality 19:16.270 --> 19:18.103 isn't just in the , in the , the 19:18.103 --> 19:20.270 sensory channel , it's in making sense 19:20.270 --> 19:22.214 of . And so again , that's for the 19:22.214 --> 19:24.381 purpose of , that's for the purpose of 19:24.381 --> 19:26.492 creating this um world model in a way 19:26.492 --> 19:28.437 that it can be exploited in a very 19:28.437 --> 19:30.659 general way . OK , I don't know if that 19:30.659 --> 19:32.714 got me to where you wanted to get to 19:32.714 --> 19:34.770 Patrick . It does , it does , yeah . 19:34.770 --> 19:37.739 Yeah , Robert . I'm just gonna mention 19:37.739 --> 19:41.339 the um study by Irving Biederman , 1972 , 19:41.349 --> 19:43.293 published in the journal Science , 19:43.293 --> 19:45.750 where he showed a 100 millisecond 19:45.750 --> 19:48.420 exposure . Of course , your brain is 19:48.420 --> 19:50.476 processing it longer than that , but 19:50.476 --> 19:52.309 100 millisecond , 10 of a second 19:52.309 --> 19:55.609 exposure of a street scene , people can 19:55.609 --> 19:58.660 uh If , if you ask them , can you 19:58.660 --> 20:02.060 detect uh uh a um a um 20:02.060 --> 20:05.060 mailbox in the street scene , if it's 20:05.060 --> 20:07.060 on the roof of a house , people are 20:07.060 --> 20:09.171 very poor at doing it , or if it's on 20:09.171 --> 20:11.171 the sidewalk near where it normally 20:11.171 --> 20:13.116 would be , people are very good at 20:13.116 --> 20:12.859 doing it . So in other words , you can 20:12.859 --> 20:14.900 get in the sense this self . 20:16.089 --> 20:18.810 Self-derived meaning , uh , in about 20:18.810 --> 20:20.560 10% of the exposure over . 20:22.810 --> 20:24.921 Yeah , we actually used to do that to 20:24.921 --> 20:27.088 our grad students . I would bring into 20:27.088 --> 20:29.319 the class a shutter from a camera , and 20:29.319 --> 20:31.541 we'd flash things very quickly for them 20:31.541 --> 20:33.541 and , and we would , you know , ask 20:33.541 --> 20:35.763 them about them , and it's , you know , 20:35.763 --> 20:37.986 far too quick because of the scatting . 20:37.986 --> 20:37.800 You can't scatter around it , but yet 20:37.800 --> 20:40.959 you do get a lot of information . Go up 20:40.959 --> 20:43.630 two slides to the bat slide right there . 20:43.760 --> 20:47.699 Oh , down 1 . Thank you . So 20:47.699 --> 20:49.866 the point I'm trying to make here is , 20:49.866 --> 20:51.977 you know , that , you know , so Kevin 20:51.977 --> 20:53.977 Schmidt there , I gave him the call 20:53.977 --> 20:55.866 sign Nagle for the reason of this 20:55.866 --> 20:55.650 article here . You know , it's focused 20:55.650 --> 20:58.959 on experience . It's also onua . And so , 20:59.010 --> 21:01.288 you know , what is it like to be a bat . 21:01.288 --> 21:03.719 And this , this used to bother me a lot , 21:03.839 --> 21:05.839 but honestly , it doesn't bother me 21:05.839 --> 21:08.117 anymore . I , I'm completely convinced . 21:08.117 --> 21:10.228 I , what it would be like to be a bat 21:10.228 --> 21:12.800 is this bat has to have created a 21:12.800 --> 21:15.670 sensory motor world model for bat space , 21:15.959 --> 21:18.479 you know . So , so this bat is , I 21:18.479 --> 21:20.719 suspect , creating Qua . Associated 21:20.719 --> 21:22.997 with the way it senses the environment , 21:22.997 --> 21:24.663 the way it interacts with the 21:24.663 --> 21:26.830 environment , and then it creates this 21:26.830 --> 21:29.052 world model that as it's flying through 21:29.052 --> 21:29.000 real fast , it doesn't have to create 21:29.000 --> 21:31.056 the quality to react appropriately , 21:31.056 --> 21:33.222 but yet it , it can create the quality 21:33.222 --> 21:35.222 to update its knowledge . And , you 21:35.222 --> 21:37.389 know , I , I put this little quote , I 21:37.389 --> 21:39.167 just came about recently , um , 21:39.400 --> 21:41.400 Wittgenstein , you know , suggested 21:41.400 --> 21:43.678 that , you know , if a lion could talk , 21:43.678 --> 21:46.011 we wouldn't understand what it's saying . 21:46.011 --> 21:48.289 You know , and I , I think what , what , 21:48.289 --> 21:48.030 what all these people have realized is , 21:48.349 --> 21:51.229 is that it's so , and dreamer , Bob 21:51.229 --> 21:53.396 Mash sitting across the room from me , 21:53.396 --> 21:55.562 you know , he pointed up that I needed 21:55.562 --> 21:57.729 to fix this to say agent centric , you 21:57.729 --> 21:59.785 know , so I think I think this makes 21:59.785 --> 22:01.896 the sense of the quality I experience 22:01.896 --> 22:04.160 are unique to me and probably there's 22:04.160 --> 22:06.271 some similarity with other members of 22:06.271 --> 22:08.493 my species , but the quality created by 22:08.493 --> 22:10.549 these other species is unique to the 22:10.549 --> 22:12.493 way they interact and the way they 22:12.493 --> 22:12.280 sense the world , and it would make 22:12.280 --> 22:14.447 sense to us , even if I could see your 22:14.447 --> 22:16.558 quality . If you're , if you're , for 22:16.558 --> 22:18.891 example , a lion , you know , and so it , 22:18.891 --> 22:20.891 it does me no good . So go down one 22:20.891 --> 22:22.947 more slide , Katrina , and then I'll 22:22.947 --> 22:25.058 get to you . One second here . I just 22:25.058 --> 22:27.336 want to hit this , this one right here , 22:27.336 --> 22:27.074 and that , so where I , where I've 22:27.074 --> 22:29.635 landed is there is a reason forua 22:29.635 --> 22:32.359 creation . And the reason is For the 22:32.359 --> 22:34.526 creation and maintenance of that world 22:34.526 --> 22:36.748 model , the thing that is , is deciding 22:36.748 --> 22:38.970 what you're doing , doing what you need 22:38.970 --> 22:40.581 to do , interacting with the 22:40.581 --> 22:42.803 environment , executing those actions , 22:42.803 --> 22:44.803 interacting with other agents . Qua 22:44.803 --> 22:46.915 puts everything into a representation 22:46.915 --> 22:49.137 that allows you to keep it in context , 22:49.137 --> 22:51.930 and Katrina . Yeah , just on that 22:51.930 --> 22:54.290 previous slide , I think you could add 22:54.290 --> 22:56.739 the word umwelt there , which is that 22:56.739 --> 23:00.250 Jacob von Uskal notion that like our 23:00.250 --> 23:02.361 biology constrains our quality . So I 23:02.361 --> 23:04.528 think that relates back to the earlier 23:04.528 --> 23:06.694 slide you said about self , where your 23:06.694 --> 23:08.750 self is giving you kind of the frame 23:08.750 --> 23:10.639 through which any of that sensory 23:10.639 --> 23:12.639 information available to you in the 23:12.639 --> 23:14.861 world is used to create a world model . 23:17.880 --> 23:19.824 Perfect , perfect . That's exactly 23:19.824 --> 23:21.991 right . So the point is , the point of 23:21.991 --> 23:24.324 all this is what nature figured out was , 23:24.324 --> 23:26.047 there is a way to create a , a 23:26.047 --> 23:29.109 agent-centric semantic . World model 23:29.109 --> 23:31.770 something that makes meaning for you as 23:31.770 --> 23:33.826 an individual . I can compensate for 23:33.826 --> 23:35.714 the deficiencies in your personal 23:35.714 --> 23:37.937 sensors if I have this general approach 23:37.937 --> 23:40.103 to creating this representation of the 23:40.103 --> 23:42.159 way the environment is and where you 23:42.159 --> 23:44.103 fit in that environment . Thoughts 23:44.103 --> 23:46.214 you're having , how do they relate to 23:46.214 --> 23:48.449 the environment ? And , and , and so if 23:48.449 --> 23:51.569 you , if you now go down , um , go down 23:51.569 --> 23:55.510 a slide , one more . Um , you 23:55.510 --> 23:57.843 know , this is just pointing out , yeah , 23:57.843 --> 23:59.843 let's stay here for just a second . 23:59.843 --> 24:01.732 This is just pointing out for the 24:01.732 --> 24:03.954 blindside patient , how does my diagram 24:03.954 --> 24:06.121 change ? My diagram suggests that that 24:06.121 --> 24:07.954 that guy is interacting with the 24:07.954 --> 24:09.954 environment , using his knowledge , 24:09.954 --> 24:12.177 using his world model , and on the left 24:12.177 --> 24:14.343 side he's not creating quality for the 24:14.343 --> 24:16.699 visual input . But I'm sure that he's 24:16.699 --> 24:18.866 creatingqua with respect to , oh , now 24:18.866 --> 24:20.977 I'm moving to the left . Oh , now I'm 24:20.977 --> 24:23.088 ducking down underneath the , I think 24:23.088 --> 24:24.921 he has to have created those Qua 24:24.921 --> 24:27.088 because he has , you know , he's still 24:27.088 --> 24:29.310 interacting and experiencing those . He 24:29.310 --> 24:31.477 just can't say , Hey , that's a ladder 24:31.477 --> 24:33.699 right in front of me , or you've left a 24:33.699 --> 24:33.420 printer laying on the floor and I'm 24:33.420 --> 24:35.739 dodging it . He can't do thatua 24:35.739 --> 24:37.961 formation , but I suspect he's creating 24:37.961 --> 24:40.017 the other Quaglia . Um , go down one 24:40.017 --> 24:43.520 more slide . And so this , this is the 24:43.520 --> 24:45.631 white quest , you know , this is what 24:45.631 --> 24:47.631 we're after here . We're after this 24:47.631 --> 24:49.687 idea of , of , of the creation . You 24:49.687 --> 24:51.909 know , the maintenance , creation , the 24:51.909 --> 24:54.076 exploitation of , of world models that 24:54.076 --> 24:55.964 will allow us to make journalists 24:55.964 --> 24:57.964 agents , agents that can do lots of 24:57.964 --> 24:59.798 stuff , not just one thing , not 24:59.798 --> 25:02.199 memorize the idiosyncratic details of 25:02.199 --> 25:04.255 some set of data and then be able to 25:04.255 --> 25:06.366 replicate that and hopefully that the 25:06.366 --> 25:08.366 data is close enough to what you're 25:08.366 --> 25:10.532 going to do in the real world . That's 25:10.532 --> 25:12.699 OK . That is bringing value , but it's 25:12.699 --> 25:12.319 not what we're really after . What 25:12.319 --> 25:14.400 we're after is this idea of creating 25:14.400 --> 25:16.400 this , uh , more generalist agent , 25:16.400 --> 25:18.622 which requires creating a world model . 25:18.622 --> 25:21.119 And , and go down the next slide . And 25:21.119 --> 25:23.479 so that's what ties us to this idea of , 25:23.670 --> 25:25.392 of intelligence and artificial 25:25.392 --> 25:27.614 intelligence . This is my definition of 25:27.614 --> 25:29.781 intelligence . It's this ability of an 25:29.781 --> 25:32.003 agent to create knowledge . The way you 25:32.003 --> 25:34.410 create knowledge is by experience , 25:34.489 --> 25:36.770 interacting with the environment , by 25:36.770 --> 25:38.826 reasoning , thinking , if you will , 25:38.826 --> 25:40.992 manipulating the knowledge you already 25:40.992 --> 25:43.048 have , by using your sensors , or by 25:43.048 --> 25:45.048 other agents giving you knowledge , 25:45.048 --> 25:47.729 culture . And AI is just these days a 25:47.729 --> 25:49.785 computer that could do that . Notice 25:49.785 --> 25:51.673 I've completely taken out of this 25:51.673 --> 25:53.951 definition of AI . any idea , you know , 25:53.969 --> 25:56.136 for whatever reason , the daf likes to 25:56.136 --> 25:58.358 tie it to doing things that humans do . 25:58.358 --> 26:00.650 I , I think that's just a silly 26:00.650 --> 26:02.872 measurement . That's just the wrong way 26:02.872 --> 26:04.983 to think about . stuff . I think it's 26:04.983 --> 26:07.094 all about creating these world models 26:07.094 --> 26:09.261 which allows this agent , whatever you 26:09.261 --> 26:11.261 want to do , uh , interact with the 26:11.261 --> 26:13.261 world in that meaningful way . That 26:13.261 --> 26:15.150 that's what we're after . I'm not 26:15.150 --> 26:17.261 conflating . Now here's a big thing , 26:17.261 --> 26:17.135 and this is where Ancient Mike usually 26:17.135 --> 26:19.357 throws stuff at me and he's right , and 26:19.357 --> 26:21.468 he's made me really focus this down . 26:21.468 --> 26:23.734 I'm not conflating intelligence with 26:23.734 --> 26:26.012 Qua or intelligence with consciousness . 26:26.180 --> 26:28.402 But I am absolutely going down the path 26:28.402 --> 26:31.469 of quality formation is the key to the 26:31.469 --> 26:33.525 way that we've created the knowledge 26:33.525 --> 26:35.747 that we're using to do the things we're 26:35.747 --> 26:37.802 doing . And so therefore , I'm going 26:37.802 --> 26:39.913 down this path of it in fact is a key 26:39.913 --> 26:42.080 to making , you know , more generalist 26:42.080 --> 26:44.890 AI agents . The lessons I can learn 26:44.890 --> 26:47.640 based upon the studying of Qua . Next 26:47.640 --> 26:50.699 slide . Yeah , this is Bogdan , if I 26:50.699 --> 26:53.579 may , uh , jump in a bit . I have a 26:53.579 --> 26:55.579 question related to actually to the 26:55.579 --> 26:58.060 example of the , of the mailbox that 26:58.060 --> 27:00.140 people cannot find on the top of the 27:00.140 --> 27:02.609 buildings if they are exposed for 100 27:02.609 --> 27:04.939 milliseconds . And I think what Katrina 27:04.939 --> 27:07.609 said cemented the choira for me . Uh , 27:07.849 --> 27:10.071 but let me ask you , if , if this makes 27:10.071 --> 27:12.238 sense . So we have , we expose a photo 27:12.238 --> 27:14.349 to somebody for 100 milliseconds . We 27:14.349 --> 27:16.349 know that we have , uh , maybe 1010 27:16.349 --> 27:18.516 milliseconds per sarca , right ? And , 27:18.516 --> 27:20.627 and then , and then we have 10 sarcas 27:20.627 --> 27:22.738 that people can do in those , uh , in 27:22.738 --> 27:25.010 that time . Uh , if you ask somebody to 27:25.010 --> 27:27.232 look for a mailbox , they're gonna look 27:27.232 --> 27:29.399 on the street , then I'm gonna look on 27:29.399 --> 27:31.621 the top of the building . So that's why 27:31.621 --> 27:31.569 they miss it most of the time . Is that 27:31.569 --> 27:33.513 the , is that the correct approach 27:33.513 --> 27:35.513 where the biology comes in place as 27:35.513 --> 27:38.459 embodiment , over ? No , humans are 27:38.459 --> 27:40.130 very slow . It takes like 200 27:40.130 --> 27:42.579 milliseconds , so they have to take in 27:42.579 --> 27:45.300 the scene . Yeah , yeah , it's not like 27:45.300 --> 27:47.780 even better , thanks , yeah . 27:48.750 --> 27:49.349 Glenn 27:52.949 --> 27:55.469 Uh , question , is an American Qua 27:55.469 --> 27:57.619 different from a Chinese Quala ? 27:59.430 --> 28:02.349 Yeah , it is absolutely true 28:02.640 --> 28:05.520 that that your world model is based 28:05.520 --> 28:07.880 upon , it is updated dynamically . It's 28:07.880 --> 28:10.047 not that you , you evolution has given 28:10.047 --> 28:12.213 you a world model . Evolution gave you 28:12.213 --> 28:14.491 the ability to create your world model . 28:14.491 --> 28:16.658 So absolutely , as you're raised , the 28:16.658 --> 28:18.936 cultural impacts you have changes your , 28:18.936 --> 28:20.991 your , the way you create your world 28:20.991 --> 28:23.158 model . So absolutely people raised in 28:23.158 --> 28:25.324 different cultures will have different 28:25.324 --> 28:28.699 world models . True . Igor . Uh , uh , 28:28.770 --> 28:30.810 soup . Do you believe that the 28:30.810 --> 28:33.030 existence of koala is a necessary , 28:33.140 --> 28:37.099 necessary factor for achieving general 28:37.099 --> 28:39.579 consciousness or artificial general 28:39.579 --> 28:42.449 intelligence ? Uh , from , uh , 28:42.930 --> 28:45.170 information theory , I understand that 28:45.170 --> 28:47.880 we need to , uh , compress all this 28:47.880 --> 28:50.800 knowledge and uh put it together , uh , 28:50.810 --> 28:54.400 in one , easy to process units , 28:54.689 --> 28:57.369 but , uh , do , uh , do you believe 28:57.369 --> 29:00.569 it's necessary for artificial general 29:00.569 --> 29:03.810 intelligence and , uh , can we do it 29:03.810 --> 29:05.650 different way and still achieve 29:05.650 --> 29:08.569 consciousness or . It's a good question . 29:08.680 --> 29:10.680 Go up one slide , the slide we were 29:10.680 --> 29:12.680 just at there on the bottom in that 29:12.680 --> 29:14.839 Carolina blue , you'll see that I'm 29:14.839 --> 29:16.950 suggesting to you we can't get to AGI 29:16.950 --> 29:19.800 without this idea , and I , and whether 29:19.800 --> 29:22.000 I'm right or that is one thing , but 29:22.000 --> 29:24.000 I'm telling you absolutely we can't 29:24.000 --> 29:26.056 have no hope for alignment unless we 29:26.056 --> 29:28.167 crack this nut , unless we figure out 29:28.167 --> 29:30.333 how people , you know , think how they 29:30.333 --> 29:32.500 actually do sense making , we'll never 29:32.500 --> 29:34.667 be able to align AI . Even if it isn't 29:34.667 --> 29:37.469 based on quest ideas to humans . So in 29:37.469 --> 29:39.747 either case , in the short and midterm , 29:39.747 --> 29:41.858 we as the Department of the Air Force 29:41.858 --> 29:44.025 are focused on human machine teaming , 29:44.025 --> 29:46.191 and I suggest we can't get to where we 29:46.191 --> 29:48.247 need to get to without cracking this 29:48.247 --> 29:50.191 nut of how people form their world 29:50.191 --> 29:52.302 models , how they update them , etc . 29:52.302 --> 29:54.469 Now I personally , now to to really go 29:54.469 --> 29:56.802 way out on a small limb with you , Igor , 29:56.802 --> 29:58.969 I am absolutely committed that this is 29:58.969 --> 30:01.790 the way forward to get to AGI . All 30:01.790 --> 30:05.349 right , let's go down a slide . So in 30:05.349 --> 30:07.405 Pedro Dominguez's book when he talks 30:07.405 --> 30:09.571 about knowledge , he suggests just the 30:09.571 --> 30:11.849 three things on the left on this slide . 30:11.849 --> 30:13.960 There's 3 ways nature has figured out 30:13.960 --> 30:16.071 how to put knowledge into a critter . 30:16.071 --> 30:18.293 One is you put it in their DNA , and so 30:18.293 --> 30:20.405 at evolution , you know , you sort of 30:20.405 --> 30:22.627 captured all the best practices and you 30:22.627 --> 30:22.380 provide the critter that when they're 30:22.380 --> 30:24.324 born . And the second thing is you 30:24.324 --> 30:26.436 might make a critter that is a little 30:26.436 --> 30:28.491 more sophisticated and based upon it 30:28.491 --> 30:30.491 interacting with the world , it can 30:30.491 --> 30:32.602 pick up knowledge and then store that 30:32.602 --> 30:34.769 away for subsequent use . And then the 30:34.769 --> 30:36.713 third way he talks about is if the 30:36.713 --> 30:38.824 critter is , is so sophisticated , it 30:38.824 --> 30:40.880 can interact with other critters and 30:40.880 --> 30:42.880 via culture get knowledge from that 30:42.880 --> 30:44.936 other critter's experiences . And so 30:44.936 --> 30:44.750 those are the three ways nature has 30:44.750 --> 30:47.150 figured out on how to do knowledge . OK , 30:47.310 --> 30:50.810 creation , and , and , um , the 30:50.810 --> 30:52.810 experience thing , there's a little 30:52.810 --> 30:54.921 thing there in red , which is hard to 30:54.921 --> 30:56.921 read . I think , you know , there , 30:56.921 --> 30:56.910 Pedro was talking about interacting 30:56.910 --> 30:58.910 with the environment . I include in 30:58.910 --> 31:00.854 that experience word there , and I 31:00.854 --> 31:03.188 apologize for that because I keep using . 31:03.188 --> 31:05.021 The experience word with respect 31:05.021 --> 31:07.243 toqualia that what it is like for you , 31:07.243 --> 31:09.188 but these experiences are events , 31:09.188 --> 31:08.699 things that you're interacting with the 31:08.699 --> 31:11.699 world in , but also there's experiences 31:11.699 --> 31:13.643 that you have by by reasoning , by 31:13.643 --> 31:15.810 manipulating your internal knowledge . 31:15.810 --> 31:18.032 Those are experiences . So that sort of 31:18.032 --> 31:20.032 comes in that second category . But 31:20.032 --> 31:21.755 what Jan and , and some of our 31:21.755 --> 31:23.866 colleagues will tell you is from this 31:23.866 --> 31:23.630 point forward on the Earth , most of 31:23.630 --> 31:25.463 the knowledge will be created by 31:25.463 --> 31:27.630 machines and , and that's because they 31:27.630 --> 31:29.852 can just keep manipulating the existing 31:29.852 --> 31:31.908 knowledge and create new knowledge . 31:31.908 --> 31:34.019 You can argue about the usefulness or 31:34.019 --> 31:36.241 not , whether humans can make sense out 31:36.241 --> 31:38.297 of it or not , whether it will evoke 31:38.297 --> 31:40.241 into us things we want to . That , 31:40.241 --> 31:42.463 that's a different challenge , but it's 31:42.463 --> 31:44.574 absolutely true . It will be creating 31:44.574 --> 31:46.630 lots of stuff . So if you keep going 31:46.630 --> 31:48.741 left to right on this diagram , where 31:48.741 --> 31:50.686 we're really headed is this can we 31:50.686 --> 31:52.686 create a knowledge of machines that 31:52.686 --> 31:54.963 humans can use both in teaming and not , 31:54.963 --> 31:57.186 and then , then you , so you have human 31:57.186 --> 31:59.019 machine culture and then machine 31:59.019 --> 32:01.074 machine culture , you know , machine 32:01.074 --> 32:00.689 machines that weren't designed to work 32:00.689 --> 32:02.522 together , can they in fact give 32:02.522 --> 32:04.578 knowledge to each other ? So I think 32:04.578 --> 32:06.633 this is sort of the trajectory we're 32:06.633 --> 32:08.839 all on . Go ahead , rockstar . Yes , I 32:08.839 --> 32:11.640 will respond with this thought 32:11.640 --> 32:13.680 throughout this year with the whole 32:13.680 --> 32:17.329 team . But just to see this 32:17.329 --> 32:21.229 idea . The idea of critics to 32:21.229 --> 32:25.170 determine the value of different uh 32:25.310 --> 32:28.839 arbitrary topologies of coa 32:29.670 --> 32:33.589 inside the machine is important . How 32:33.589 --> 32:36.819 do the critics decide what is a good 32:37.189 --> 32:40.910 connection of uh subcomponents of 32:40.910 --> 32:44.339 knowledge for a given moment in time 32:44.339 --> 32:44.829 over . 32:49.359 --> 32:51.637 All right , let's go to the next slide . 32:52.130 --> 32:54.449 So over the years , over these last 32:54.449 --> 32:57.050 couple of decades , we , we keep , you 32:57.050 --> 32:58.828 know , um , in the , you know , 32:58.930 --> 33:01.152 reviewing literature and coming up with 33:01.152 --> 33:03.374 examples of experiences people havequaa 33:03.374 --> 33:05.486 that are evoked in people as a result 33:05.486 --> 33:07.652 of different stimuli . And , and by us 33:07.652 --> 33:09.930 capturing the characteristics of those , 33:09.930 --> 33:12.152 we've , we've captured a whole bunch of 33:12.152 --> 33:15.170 them into a long list , and it isn't in 33:15.170 --> 33:17.329 this quick pass version , but in the 33:17.329 --> 33:19.551 more extended version we can go through 33:19.551 --> 33:21.773 next week , we've pulled those together 33:21.773 --> 33:23.829 in these categories and we call this 33:23.829 --> 33:25.829 sort of our theory of Quaglia . And 33:25.829 --> 33:27.773 what it is is , is we've , we've , 33:27.773 --> 33:29.940 we've suggested that that Qua in terms 33:29.940 --> 33:32.420 of representations are simulated , um , 33:32.630 --> 33:35.979 there , there , there is a There is a 33:35.979 --> 33:38.760 discernible confabulation aspect to the 33:38.760 --> 33:40.927 to what you're experiencing . It's not 33:40.927 --> 33:42.982 reality . There's a simulation , but 33:42.982 --> 33:45.093 it's , it is captures some aspects of 33:45.093 --> 33:46.816 reality in the sense that it's 33:46.816 --> 33:49.310 structurally coherent . And it's 33:49.310 --> 33:52.469 situated that is . Every quality is 33:52.469 --> 33:54.680 only you can only define it by its 33:54.680 --> 33:56.736 relationship and how it can interact 33:56.736 --> 33:58.847 with other Qua , and it's all made up 33:58.847 --> 34:01.013 of this , the vocabulary , which we'll 34:01.013 --> 34:03.069 call the Qua vocabulary . So this is 34:03.069 --> 34:05.420 our S3Q theory of Qua , situated , 34:05.430 --> 34:07.869 simulated , structurally coherent set 34:07.869 --> 34:09.813 of vocabulary elements which we're 34:09.813 --> 34:12.036 going to call aqualia , and we're going 34:12.036 --> 34:11.870 to work our way through each of these 34:11.870 --> 34:14.429 pieces . Go to the next slide . I was 34:14.429 --> 34:16.540 going to say , in the mailbox example 34:16.540 --> 34:18.707 is situatedness , the problem . Yeah , 34:18.719 --> 34:20.830 yeah , so you , you'll see it when we 34:20.830 --> 34:23.052 get to the situated piece . What you do 34:23.052 --> 34:24.941 is you store knowledge based upon 34:24.941 --> 34:27.219 relationships and so therefore mailbox , 34:27.219 --> 34:29.275 it's sort of like seeing that person 34:29.275 --> 34:31.275 who's your butcher on the bus . You 34:31.275 --> 34:31.159 don't recognize them , you know , you 34:31.159 --> 34:33.679 don't have the all the links . Now , 34:33.719 --> 34:35.663 there's a real advantage when that 34:35.663 --> 34:37.552 happens , by the way , because it 34:37.552 --> 34:39.719 allows you to efficiently search . You 34:39.719 --> 34:41.830 know , if I'm in the butcher shop , I 34:41.830 --> 34:43.830 see these people there . Go ahead , 34:43.830 --> 34:45.386 ain't you Mike . What about 34:45.386 --> 34:48.060 recreational drug use ? Are you saying 34:48.060 --> 34:50.116 that those are not quality , and I'm 34:50.116 --> 34:52.004 not sure they would meet all your 34:52.004 --> 34:54.509 criteria . No , yeah , absolutely . I 34:54.509 --> 34:56.731 don't care if it's artificially induced 34:56.731 --> 34:58.898 or not . It doesn't matter . Those are 34:58.898 --> 35:00.787 quality that are being informed . 35:00.787 --> 35:00.589 You're only arguing about what evoked 35:00.589 --> 35:02.756 them . So , you know , I don't care if 35:02.756 --> 35:04.645 you're , I don't care if you're a 35:04.645 --> 35:06.811 habitual drug user . Those are , those 35:06.811 --> 35:08.978 are quality your mind is generating as 35:08.978 --> 35:10.922 a result of whatever your internal 35:10.922 --> 35:13.256 manipulation , your chemical inducement , 35:13.256 --> 35:15.478 or , you know , uh , sensory data . And 35:15.478 --> 35:17.367 they meet your criteria for Qua . 35:17.820 --> 35:19.820 Absolutely . They may be completely 35:19.820 --> 35:21.709 unique . They may be there may be 35:21.709 --> 35:23.931 quality that you can evoke in your mind 35:23.979 --> 35:25.868 that , that , that as a result of 35:25.868 --> 35:28.179 chemical , uh , you know , involvement 35:28.379 --> 35:30.379 that you cannot do without that . I 35:30.379 --> 35:32.490 don't doubt that at all . All right , 35:32.490 --> 35:35.620 thanks . Sure . I am not condoning , by 35:35.620 --> 35:39.260 the way , if you ever ask . So , so 35:39.260 --> 35:41.538 here's the idea of the cue , the , the , 35:41.538 --> 35:43.760 you know , that there's this vocabulary 35:43.760 --> 35:45.871 of thought . And over the years we've 35:45.871 --> 35:47.871 actually on many occasions tried to 35:47.871 --> 35:49.982 hone down on how many of these do you 35:49.982 --> 35:49.909 need , you know , what , what , what is , 35:50.229 --> 35:52.396 what is your cu vocabulary ? How , how 35:52.396 --> 35:54.562 big is it ? And it's changing , by the 35:54.562 --> 35:56.673 way . You know , you regularly , uh , 35:56.673 --> 35:59.007 you know , we'll get back to that . You , 35:59.007 --> 36:01.285 you regularly create new call you , uh , 36:01.285 --> 36:03.507 as you , uh , as you sort of grow , but 36:03.507 --> 36:05.840 as you get older like me , I think that , 36:05.840 --> 36:08.007 that sort of , that ability . has been 36:08.007 --> 36:07.879 reduced , but this idea that you need 36:07.879 --> 36:11.199 this compressed view of , of this world 36:11.199 --> 36:14.110 model , it forces you to sort of , uh , 36:14.120 --> 36:16.342 find an abstraction of , of things that 36:16.342 --> 36:18.820 are useful for you . And again , uh , 36:18.919 --> 36:20.919 it just takes something simple like 36:20.919 --> 36:23.141 discernment of colors . If I change the 36:23.141 --> 36:25.086 you saturation and brightness very 36:25.086 --> 36:26.808 slightly , you might not get a 36:26.808 --> 36:28.919 different quality evoked in you , but 36:28.919 --> 36:31.086 there'll be eventually some noticeable 36:31.086 --> 36:33.197 difference for whatever reason , from 36:33.197 --> 36:32.959 maybe from an evolutionary perspective . 36:33.139 --> 36:35.028 You've been able to , your , your 36:35.028 --> 36:36.917 sensors were able to discern that 36:36.917 --> 36:39.139 difference . It helps somebody sometime 36:39.139 --> 36:40.972 and so therefore , now that is a 36:40.972 --> 36:40.810 discernible difference in quality of 36:40.810 --> 36:42.977 formation for you . But the whole goal 36:42.977 --> 36:45.330 of this vocabulary is , I think what 36:45.330 --> 36:47.274 Jeff Hawkins used to call reliable 36:47.274 --> 36:49.274 predictability . You need to create 36:49.274 --> 36:51.274 this world model that allows you to 36:51.274 --> 36:53.441 anticipate and thus be able to take in 36:53.441 --> 36:55.386 data and , and quickly put it into 36:55.386 --> 36:57.608 context and know what to do , etc . And 36:57.608 --> 36:59.774 so that requires this sort of level of 36:59.774 --> 37:02.219 abstraction . The stable , consistent 37:02.540 --> 37:04.620 world model objective is one that I 37:04.620 --> 37:06.787 don't think we've exploited as much as 37:06.787 --> 37:08.509 we should have in creating the 37:08.509 --> 37:10.787 vocabularies of our current , you know , 37:10.787 --> 37:10.330 the models that we're creating like in 37:10.330 --> 37:12.441 LLMs . You know , I don't think we've 37:12.441 --> 37:14.774 done , we , we used the wrong objective . 37:14.774 --> 37:16.941 Our objective is reconstruction of the 37:16.941 --> 37:19.108 sequence . The objective really should 37:19.108 --> 37:21.163 be predictability of the meaning and 37:21.163 --> 37:23.274 the sequence of meanings , and that , 37:23.274 --> 37:25.552 that is a different objective function , 37:25.552 --> 37:27.774 one of which I think we , we can pursue 37:27.774 --> 37:30.108 aggressively to make this more testable . 37:30.108 --> 37:32.108 OK , go , go to the next slide that 37:32.108 --> 37:34.386 this , this idea of queuing is not new . 37:34.386 --> 37:34.219 I mean , just a week or so ago , we 37:34.219 --> 37:36.441 were talking about , you know , Chomsky 37:36.441 --> 37:38.163 and Newell , uh , both of them 37:38.163 --> 37:40.275 concluded that , you know , this is , 37:40.275 --> 37:42.441 this is the great breakthrough in , in 37:42.441 --> 37:44.663 their case , they were suggesting , you 37:44.663 --> 37:47.409 know , humanity , but , but You know , 37:47.489 --> 37:49.711 I would suggest , you know , all nature 37:49.711 --> 37:51.878 figured this out . You know , having a 37:51.878 --> 37:53.969 fixed representation is really 37:53.969 --> 37:56.191 difficult to handle everything that you 37:56.191 --> 37:58.469 can , you could possibly be faced with . 37:58.469 --> 38:00.302 So you need this ability to take 38:00.302 --> 38:02.247 existing primitives , combine them 38:02.247 --> 38:04.358 together to make new primitives , and 38:04.358 --> 38:06.413 then not destroy the old ones . They 38:06.413 --> 38:08.580 may still be useful , but eventually , 38:08.580 --> 38:10.802 you know , you don't use them as much . 38:10.802 --> 38:10.459 Maybe they , maybe they are quote 38:10.459 --> 38:13.129 forgotten . But this idea of this 38:13.129 --> 38:15.429 movement towards this compression , the 38:15.429 --> 38:17.540 tie of compression and intelligence , 38:17.540 --> 38:19.707 those are not new ideas . They've been 38:19.707 --> 38:22.040 around a long time and , and , you know , 38:22.040 --> 38:24.207 these , these great minds , you know , 38:24.207 --> 38:26.429 were , were right there . Uh , let's go 38:26.429 --> 38:28.596 to the next slide . So , so , when you 38:28.596 --> 38:30.651 have the cues , how do you represent 38:30.651 --> 38:30.350 them ? Now , there's , there's lots of 38:30.350 --> 38:32.949 data we have in , in this space , um . 38:34.179 --> 38:36.419 My favorite example of it was , was , I 38:36.419 --> 38:38.530 was actually watching an interview of 38:38.530 --> 38:40.790 that guy on the right , Kim Peek , and 38:40.790 --> 38:42.957 for the older people in the audience , 38:42.957 --> 38:45.234 if you've ever seen the movie Rain Man , 38:45.234 --> 38:47.346 it was , it was a patterned after Kim 38:47.346 --> 38:49.457 Peek . And Kim , Kim Peek is a , is a 38:49.457 --> 38:52.219 savant , and he , you know , he , he's 38:52.219 --> 38:54.108 not only a memory savant , he's a 38:54.108 --> 38:56.163 musical savant . And if you , if you 38:56.370 --> 38:58.426 You know , he was playing a piece of 38:58.426 --> 39:00.648 music that somebody just played for him 39:00.648 --> 39:02.870 once and he could immediately replay it 39:02.870 --> 39:02.780 and then they were interviewing him and 39:02.780 --> 39:04.891 as they were interviewing him , I was 39:04.891 --> 39:07.169 watching him and I was just fascinated , 39:07.169 --> 39:09.113 you know , he was , he was sitting 39:09.113 --> 39:09.000 there talking about the piece of music 39:09.000 --> 39:10.944 and it because it was a , it was a 39:10.944 --> 39:12.944 Beethoven piece , he , he , he , he 39:12.944 --> 39:15.167 said Beethoven and I heard him mumbling 39:15.167 --> 39:17.278 Beethoven , Beethoven , Beethoven's , 39:17.278 --> 39:17.169 and he mumbled , mumble , and then I 39:17.169 --> 39:19.570 heard him say Edward R . Murrow , and 39:19.570 --> 39:21.459 I'm thinking , what the hell just 39:21.459 --> 39:23.681 happened ? And so this was in DCR day , 39:23.681 --> 39:25.570 so I backed it up and I played it 39:25.570 --> 39:27.737 slowly , backed and played it slowly , 39:27.737 --> 39:29.570 and when I listened to him , the 39:29.570 --> 39:31.626 mumbling , he went from Beethoven to 39:31.626 --> 39:34.060 Beethoven's 5th , and he would then as 39:34.060 --> 39:36.171 he said , Beethoven's 5th , and his , 39:36.171 --> 39:38.338 he said verbally , he said da da da da 39:38.338 --> 39:40.850 da da da da . And when he said that , 39:40.929 --> 39:43.179 it evoked in him the link of Morse's 39:43.179 --> 39:47.100 code . Did it hold it . Morse 39:47.100 --> 39:49.156 code . What is de da in Morse code ? 39:49.156 --> 39:52.379 That is B V V V is tied to V for 39:52.379 --> 39:54.601 victory , and you can hear him mumbling 39:54.601 --> 39:56.768 these things , but he did them so fast 39:56.768 --> 39:58.935 he went from B to victory to World War 39:58.935 --> 40:01.101 II to the Blitz to Edward R . Murrow , 40:01.101 --> 40:02.935 and he did it in a flash . And I 40:02.935 --> 40:05.046 started thinking to myself , what the 40:05.046 --> 40:07.268 heck just happened ? And is this just , 40:07.268 --> 40:09.490 uh , is this a savant thing , or are we 40:09.490 --> 40:11.657 all doing this ? And , and then , uh , 40:11.657 --> 40:11.610 you know , one of our early , you know , 40:11.750 --> 40:13.830 quest , uh , you know , people , you 40:13.830 --> 40:15.941 know , we started talking about ideas 40:15.941 --> 40:17.997 that pop into your head , you know , 40:17.997 --> 40:19.886 where do they come from ? I don't 40:19.886 --> 40:19.360 believe in magic . Where the hell did 40:19.360 --> 40:21.471 they come from ? And what , so what I 40:21.471 --> 40:23.638 forced people to do , and I'm going to 40:23.638 --> 40:25.749 give you all homework , is there will 40:25.749 --> 40:27.916 be something that happens to you today 40:27.916 --> 40:27.830 that that later you're going to try to 40:27.830 --> 40:30.052 remember to tell somebody . And you may 40:30.052 --> 40:32.219 or may not remember that you wanted to 40:32.219 --> 40:34.386 tell them , but if you , when you do , 40:34.386 --> 40:36.441 when it pops into your mind that you 40:36.441 --> 40:38.663 wanted to tell them , stop , don't tell 40:38.663 --> 40:40.830 them , and think about all the sensory 40:40.830 --> 40:42.997 data you're getting and what evoked in 40:42.997 --> 40:45.108 you the links back to that thing that 40:45.108 --> 40:47.386 you're trying to tell them . And again , 40:47.386 --> 40:47.129 in the longer version of this lecture , 40:47.139 --> 40:49.689 I get some examples of of our previous 40:49.689 --> 40:51.689 quest people doing that thing . I'm 40:51.689 --> 40:53.800 telling you that this idea of quality 40:53.800 --> 40:56.379 experiences all tied to links . This 40:56.379 --> 40:58.546 linking allows us a really interesting 40:58.546 --> 41:01.330 way to access relevant information . 41:01.540 --> 41:03.762 You might argue in this case a bunch of 41:03.762 --> 41:05.929 irrelevant information . And so how do 41:05.929 --> 41:08.096 you prune that space ? That's a really 41:08.096 --> 41:10.262 interesting question . But this is the 41:10.262 --> 41:10.219 idea I want you to think of when I say 41:10.219 --> 41:13.260 situated . The Qua are situated . They 41:13.260 --> 41:15.427 have value with respect to how they're 41:15.427 --> 41:17.538 related to other Qua and how they can 41:17.538 --> 41:19.538 interact with them . Go to the next 41:19.538 --> 41:21.340 slide . Go ahead , Rockstar . 41:23.709 --> 41:27.580 I just quickly . I sometimes think 41:27.580 --> 41:31.530 of this as with the Beethoven example . 41:32.290 --> 41:36.219 As a chance to understand value 41:36.530 --> 41:39.679 brought to the table by the agents 41:39.929 --> 41:43.760 slash critics that attenuated 41:44.010 --> 41:47.459 to the simpler parts of 41:47.459 --> 41:50.409 coa that represent what's in the world 41:50.409 --> 41:54.389 of that individual , or we'll 41:54.389 --> 41:58.120 get to that machine by a computer 41:58.479 --> 42:01.620 that Builds these different building 42:01.620 --> 42:05.169 blocks of knowledge in a uh various 42:05.510 --> 42:08.530 partial ordering type of way over . 42:09.489 --> 42:11.840 Copy . So the second S is for 42:11.840 --> 42:14.149 simulation and , and , you know , in 42:14.149 --> 42:15.982 these days where we have so many 42:15.982 --> 42:18.038 virtual worlds , I mean , you know , 42:18.038 --> 42:20.149 all my grandkids that were around for 42:20.149 --> 42:22.371 the holidays , they're all , you know , 42:22.371 --> 42:24.593 in these VR goggles and , you know , so 42:24.593 --> 42:26.816 when I say simulation to them , they're 42:26.816 --> 42:25.949 all in . They understand look left , 42:25.989 --> 42:28.156 you see what's left , you know , the , 42:28.156 --> 42:30.045 the , the point on the simulation 42:30.045 --> 42:32.045 tenant is what you're perceiving in 42:32.045 --> 42:34.211 your quality of space is not reality . 42:34.211 --> 42:36.378 It's created for you , um . You know , 42:36.459 --> 42:38.626 it , it , you know , a , a phrase that 42:38.626 --> 42:40.737 we have in the longer version of this 42:40.737 --> 42:42.959 lecture is , you see what you believe . 42:42.959 --> 42:45.181 It's not you believe what you see . You 42:45.181 --> 42:45.050 create in the visual quality , um , 42:45.379 --> 42:47.601 information that's consistent with what 42:47.601 --> 42:49.379 you think is there , and you'll 42:49.379 --> 42:51.435 convince yourself of that , uh , but 42:51.435 --> 42:53.435 it's not reality . Uh , it goes way 42:53.435 --> 42:55.546 beyond what you're . And just a piece 42:55.546 --> 42:54.895 of homework for you if you go to the 42:54.895 --> 42:57.173 next slide . If you've never done this , 42:57.173 --> 42:59.284 this is a great thing to do with kids 42:59.284 --> 43:01.228 and grandkids , is show them their 43:01.228 --> 43:03.451 blind spot . And if you don't know what 43:03.451 --> 43:03.254 I'm talking about there , just go look 43:03.254 --> 43:05.254 it up on the internet . But you get 43:05.254 --> 43:07.365 when you get a kid to see their blind 43:07.365 --> 43:09.587 spot , and they say , man , oh man , if 43:09.587 --> 43:11.587 I do that right , then . The finger 43:11.587 --> 43:13.587 disappears . It blows their mind is 43:13.587 --> 43:15.532 that because you , you've got this 43:15.532 --> 43:15.520 illusion , even when you close one eye 43:15.520 --> 43:17.576 that you're seeing the whole world , 43:17.576 --> 43:19.631 you don't realize there is in fact a 43:19.631 --> 43:21.909 blind spot there , which is where that , 43:21.909 --> 43:21.760 you know , the optic nerve goes out of 43:21.760 --> 43:23.816 the back of the retina . Um , so the 43:23.816 --> 43:26.093 point is , is that you , you , in fact , 43:26.159 --> 43:28.270 fill in , when people say do you fill 43:28.270 --> 43:30.381 in your blind spot , I would continue 43:30.381 --> 43:32.715 to fill in the entire visual field . It , 43:32.715 --> 43:34.826 it's all filled in , it's all in fact 43:34.826 --> 43:36.992 confabulate . And that's so that's the 43:36.992 --> 43:38.826 2nd S . You're situated , you're 43:38.826 --> 43:40.881 simulated . Go to the next slide and 43:40.881 --> 43:42.937 we'll get the 3rds . Which is you're 43:42.937 --> 43:44.937 structurally coherent . And again , 43:44.937 --> 43:47.159 what I mean by that is , you know , I , 43:47.159 --> 43:46.780 I just told you , you're making up this 43:46.780 --> 43:48.669 visual scene , but there's enough 43:48.669 --> 43:50.891 reality to it that when I reach over to 43:50.891 --> 43:53.058 grab this bottle of caffeine here , it 43:53.058 --> 43:55.280 is there and I can grab it . I can take 43:55.280 --> 43:57.502 a drink , OK ? So there is , there is a 43:57.502 --> 44:00.060 coherence required for this information 44:00.060 --> 44:01.893 that you're representing in your 44:01.893 --> 44:04.116 quality stream because it's going to go 44:04.116 --> 44:06.282 into your world model . So you have to 44:06.282 --> 44:08.449 have that coherence associated with it 44:08.449 --> 44:10.449 and , and As , as Rod Brooks always 44:10.449 --> 44:12.671 taught us , you know , the world is its 44:12.671 --> 44:14.782 own best representation . Interacting 44:14.782 --> 44:16.893 with the world is a great way to make 44:16.893 --> 44:19.004 sure you've honed your representation 44:19.004 --> 44:21.116 into a , a useful , useful form . And 44:21.116 --> 44:23.227 so that's a structural coherence . If 44:23.227 --> 44:25.338 we go one more slide . Another aspect 44:25.338 --> 44:27.560 of structural coherence is , although I 44:27.560 --> 44:29.727 cannot , you know , call you is spooky 44:29.727 --> 44:29.540 in the sense that you're , you're 44:29.540 --> 44:32.050 experiencing your Qua , I could measure 44:32.050 --> 44:34.272 everything associated with what's going 44:34.272 --> 44:36.050 on in your physiology , in your 44:36.050 --> 44:38.161 neurophysiology , and I don't think I 44:38.161 --> 44:37.750 would experience what you're 44:37.750 --> 44:40.550 experiencing , but I do believe there's 44:40.550 --> 44:42.989 a way I can get at aspects of your 44:42.989 --> 44:45.429 representation by relationships . And 44:45.429 --> 44:47.429 so my favorite example of that is a 44:47.429 --> 44:49.596 color wheel . If you have quote normal 44:49.596 --> 44:51.762 color vision . And you take two colors 44:51.762 --> 44:53.762 which are adjacent to each other on 44:53.762 --> 44:55.985 this color wheel , they look similar to 44:55.985 --> 44:58.096 you , at least more similar than ones 44:58.096 --> 44:58.050 that are on the other part of the 44:58.050 --> 45:00.217 collar wheel , OK ? So I think there's 45:00.217 --> 45:02.770 ways we can get at your personal qua by 45:02.770 --> 45:04.937 looking at relationships that , that , 45:04.937 --> 45:06.992 that might be critical . All right , 45:06.992 --> 45:08.992 let's go to the next slide . So now 45:08.992 --> 45:11.340 we've got these ideas of You've created 45:11.340 --> 45:14.149 this thisquaa consistent representation 45:14.149 --> 45:16.316 which we're calling in a world model . 45:16.316 --> 45:18.482 What do we do with that as AI people ? 45:18.482 --> 45:20.538 What do we do with that ? And so you 45:20.538 --> 45:22.482 just look historically in AI where 45:22.482 --> 45:24.482 we've been and you know , the , you 45:24.482 --> 45:26.705 know , the , the , the rock stars , and 45:26.705 --> 45:28.760 I have been around a long time . You 45:28.760 --> 45:30.871 know , we started out by writing very 45:30.871 --> 45:32.760 complicated programs because we , 45:32.760 --> 45:34.982 because we're arrogant , we're computer 45:34.982 --> 45:37.205 people , we're all arrogant , and so we 45:37.205 --> 45:36.560 assume we understand everything , we 45:36.560 --> 45:38.729 just write it into code . And so , you 45:38.729 --> 45:40.899 know , there's there , this first wave 45:40.899 --> 45:42.899 of AI was sort of dominated by this 45:42.899 --> 45:45.659 idea of handcrafted knowledge , you 45:45.659 --> 45:48.770 know , uh , I spent many , many a day 45:49.020 --> 45:50.853 out here on base doing knowledge 45:50.853 --> 45:52.742 engineering where I would talk to 45:52.742 --> 45:54.964 people who are doing a job and I try to 45:54.964 --> 45:57.187 encode in , in a production system what 45:57.187 --> 45:59.298 they're doing and then be able to use 45:59.298 --> 46:01.353 those rules to , to help , help , uh 46:01.353 --> 46:03.576 help them do , do the job or do the job 46:03.576 --> 46:05.798 in lieu of them . And then in the 80s , 46:05.798 --> 46:07.853 we had this sort of Machine learning 46:07.853 --> 46:10.076 revolution piece and we started doing , 46:10.076 --> 46:12.020 hey , let's instead of complicated 46:12.020 --> 46:14.187 algorithms , let's hand craft the data 46:14.187 --> 46:16.409 instead of handcrafting the knowledge . 46:16.409 --> 46:18.353 And so if we have a representation 46:18.353 --> 46:18.070 approach , it could be we could extract 46:18.070 --> 46:20.126 from the data and we're still riding 46:20.126 --> 46:22.292 that second wave , you know , I mean , 46:22.292 --> 46:24.348 around 2010 when we had the big deep 46:24.348 --> 46:26.626 learning breakthrough , etc . you know , 46:26.626 --> 46:28.848 we're , we're , we're doing really cool 46:28.848 --> 46:28.590 stuff with that . But there's a bunch 46:28.590 --> 46:30.590 of things which we are running into 46:30.590 --> 46:33.135 walls on that know we have to solve and 46:33.135 --> 46:35.191 so people like to talk about the 3rd 46:35.191 --> 46:37.413 wave and what that will look like , you 46:37.413 --> 46:39.468 know , what will it would consist of 46:39.468 --> 46:41.468 and I don't want to bog down on the 46:41.468 --> 46:41.395 details of the 3rd wave that I've 46:41.395 --> 46:43.764 written there because Burt , El Burt , 46:43.834 --> 46:45.834 one of our colleagues , is going to 46:45.834 --> 46:47.834 help me in before next Friday we're 46:47.834 --> 46:49.612 going to update that third wave 46:49.612 --> 46:51.667 characteristics which we think we're 46:51.667 --> 46:51.264 missing , but I wanted to hit a couple 46:51.264 --> 46:53.486 of them today because I don't think the 46:53.486 --> 46:55.597 lecture is complete without them . So 46:55.597 --> 46:55.485 if you go to the next slide , you know , 46:55.514 --> 46:57.800 one of the particular areas which I'm 46:57.800 --> 46:59.800 finding particularly interesting is 46:59.800 --> 47:01.911 this idea of interesting , you know , 47:01.911 --> 47:03.967 the , the Ken Stanley idea of , it's 47:03.967 --> 47:06.022 easy , you know , virtually every AI 47:06.022 --> 47:08.133 system we're doing today is trying to 47:08.133 --> 47:10.244 figure out a way to capture or create 47:10.244 --> 47:12.709 human . Human curated knowledge , you 47:12.709 --> 47:14.876 know , how do I capture if you tell me 47:14.876 --> 47:16.987 this is a truck and that's a Jeep and 47:16.987 --> 47:18.765 that's a kitty cat and that's a 47:18.765 --> 47:18.419 Dalmatian and there's a golden 47:18.419 --> 47:20.141 retriever , how do I take that 47:20.141 --> 47:22.252 knowledge and represent it versus how 47:22.252 --> 47:24.141 do you create new knowledge ? Now 47:24.141 --> 47:26.363 there's a bunch of work in the , in the 47:26.363 --> 47:28.252 event that Rockstar held for us a 47:28.252 --> 47:30.475 little over a year ago . Where we got a 47:30.475 --> 47:32.530 bunch of thinkers together to say AI 47:32.530 --> 47:34.697 for humanity in 2050 . There was a lot 47:34.697 --> 47:34.604 of people talking about the AI 47:34.604 --> 47:36.826 scientists , you know , how do we , how 47:36.826 --> 47:38.882 do we automate that process ? I find 47:38.882 --> 47:41.160 that really fascinating pieces of work , 47:41.160 --> 47:43.160 but I think what what it's going to 47:43.160 --> 47:45.160 require is it's , it requires us to 47:45.160 --> 47:46.937 come up with this definition of 47:46.937 --> 47:48.993 interestingness . You know , it , it 47:48.993 --> 47:50.771 right now we are really good at 47:50.771 --> 47:50.540 generating things that are different 47:50.729 --> 47:52.951 than things we've done before , but how 47:52.951 --> 47:55.118 do you know that they're interesting ? 47:55.118 --> 47:57.285 And I've hit my act 3 people with this 47:57.285 --> 47:59.229 multiple times because I need them 47:59.229 --> 48:01.451 doing research , but how do we quantify 48:01.451 --> 48:03.729 the value of their research , you know , 48:03.729 --> 48:05.840 and it , it basically comes down to I 48:05.840 --> 48:07.785 tell them do things that they find 48:07.785 --> 48:09.896 interesting . Let's figure out how to 48:09.896 --> 48:11.896 capture them and in the Ken Stanley 48:11.896 --> 48:13.840 sense , how we , how we will build 48:13.840 --> 48:15.785 stepping stones that when somebody 48:15.785 --> 48:18.007 comes to us with a new problem , we can 48:18.007 --> 48:17.729 use those stepping stones to get where 48:17.729 --> 48:19.959 we're after . And so this idea , I 48:19.959 --> 48:22.679 think , is that that if I come up with 48:22.679 --> 48:25.439 an idea which allows me to situate this 48:25.439 --> 48:27.606 new piece of knowledge with respect to 48:27.606 --> 48:29.495 other pieces of knowledge , if it 48:29.495 --> 48:31.550 allows me to simulate some aspect of 48:31.550 --> 48:31.389 the environment which I couldn't 48:31.389 --> 48:33.639 simulate before , they become very 48:33.639 --> 48:35.806 interesting . Now , I think Igor , you 48:35.806 --> 48:37.806 got a question . Uh , yes , I would 48:37.806 --> 48:39.972 like to ask you about Dark side of the 48:39.972 --> 48:43.120 caller . Uh , I , I understand it's 48:43.120 --> 48:45.689 extremely valuable for human-computer 48:45.689 --> 48:48.820 interaction , but it brings the burden 48:49.000 --> 48:52.639 of , uh , uh , human thinking such as 48:52.639 --> 48:54.800 jumping to conclusion , prejudice , 48:54.850 --> 48:56.719 misconceptions , delusions , 48:56.800 --> 49:00.070 hallucinations , finally fake realities , 49:00.760 --> 49:04.389 and so on . So , uh . By 49:04.389 --> 49:07.800 developing AI qualities we actually 49:07.800 --> 49:11.520 would uh develop all these uh 49:11.520 --> 49:14.669 scenes too . Oh yeah , and I , and I 49:14.669 --> 49:16.502 think , I think the key to those 49:16.502 --> 49:19.580 abilities we have is this quality-based 49:19.580 --> 49:21.691 representation . The fact that we can 49:21.691 --> 49:23.747 follow those links and suggest , you 49:23.747 --> 49:25.858 know , hey , now this , this is going 49:25.858 --> 49:27.969 to take me down a path that I haven't 49:27.969 --> 49:29.580 been before and it has these 49:29.580 --> 49:31.524 characteristics that might make it 49:31.524 --> 49:33.691 useful . So I , I think that , I think 49:33.691 --> 49:35.858 there's , there is value by projecting 49:35.858 --> 49:37.858 those quote breakthroughs into this 49:37.858 --> 49:40.080 sort of a space . Um , now what Chollet 49:40.080 --> 49:41.913 says , we go to the next slide . 49:41.913 --> 49:43.691 Chollet likes to break this the 49:43.691 --> 49:45.802 problems into how do you do the sense 49:45.802 --> 49:47.913 making and then how do you create the 49:47.913 --> 49:50.136 new vocabulary . Now , if you just look 49:50.136 --> 49:52.247 at the sense making piece , uh , if , 49:52.247 --> 49:54.191 if we had an objective function to 49:54.191 --> 49:56.302 generate these link representations , 49:56.302 --> 49:56.199 then that would help us , I think , 49:56.399 --> 49:58.566 with respect to sense making . Our our 49:58.566 --> 50:00.566 colleague Robert Patterson likes to 50:00.566 --> 50:02.732 talk about , you know , situated , you 50:02.732 --> 50:04.677 know , intuitive cognition , and I 50:04.677 --> 50:06.843 think , I think that that is , that is 50:06.843 --> 50:08.843 absolutely what we're talking about 50:08.843 --> 50:10.732 here . You know , if I ask you my 50:10.732 --> 50:10.252 favorite question is , have you ever 50:10.252 --> 50:12.452 driven a Roy Rolls-Royce ? You don't , 50:12.613 --> 50:14.446 you don't go through the tree of 50:14.446 --> 50:16.724 checking everything you've ever driven . 50:16.724 --> 50:18.669 You're you're able to very quickly 50:18.669 --> 50:20.891 check those links , don't . Rolls-Royce 50:20.891 --> 50:23.057 and me driving , it doesn't exist . So 50:23.057 --> 50:25.169 that representation is , is , I think 50:25.169 --> 50:27.502 different . And , and this idea of , of , 50:27.502 --> 50:29.557 of this objective function to create 50:29.557 --> 50:31.557 new vocabulary elements , uh , this 50:31.557 --> 50:33.502 quaization idea , those left three 50:33.502 --> 50:35.613 bubbles , I , I think that's , that's 50:35.613 --> 50:37.724 sort of what we have to do . Now , we 50:37.724 --> 50:39.891 got one more meat slide here I want to 50:39.891 --> 50:39.845 hit because I can't , I can't end this 50:39.845 --> 50:41.789 slide , in this deck , or one more 50:41.789 --> 50:43.456 slide , uh , without saying . 50:44.209 --> 50:47.030 Responsible AI , AI ethics , you know , 50:47.189 --> 50:49.300 I , this is , this is an area I spend 50:49.300 --> 50:51.578 an enormous amount of bandwidth on and , 50:51.578 --> 50:53.745 and I don't think I can get to where I 50:53.745 --> 50:56.770 need to get to with B&B doing 50:56.770 --> 50:58.810 validation , verification . I will 50:58.810 --> 51:00.929 never get to where I need to get to 51:00.929 --> 51:03.096 until I can crack the nut of how is it 51:03.096 --> 51:05.250 we represent morals , how do we 51:05.250 --> 51:07.361 represent ethics , you know , what is 51:07.361 --> 51:09.472 that ? Those are , those are feelings 51:09.472 --> 51:11.583 to us , or quality to us , you know , 51:11.583 --> 51:13.806 how do I do that ? And , and I I just , 51:13.806 --> 51:15.750 I keep running against the wall of 51:15.750 --> 51:17.806 people saying , Hey , we're going to 51:17.806 --> 51:20.028 let your AI do this or that , and we're 51:20.028 --> 51:19.409 going to put a human on the loop to 51:19.409 --> 51:21.520 decide if it's OK to do it . And that 51:21.520 --> 51:23.353 just drives me crazy for all the 51:23.353 --> 51:25.576 reasons I just spent an hour talking to 51:25.576 --> 51:27.770 you about , because you , you're 51:27.770 --> 51:29.881 creating this representation which is 51:29.881 --> 51:32.340 not reality . And , and , and you're 51:32.340 --> 51:34.562 going to make a decision and you will , 51:34.562 --> 51:36.673 your decision will be biased based on 51:36.673 --> 51:38.951 all the stuff my AI is throwing at you . 51:38.951 --> 51:40.951 It just , it , I think we're really 51:40.951 --> 51:43.007 going down the wrong path . We think 51:43.007 --> 51:42.780 put humans on the loop is going to 51:42.780 --> 51:44.836 solve this problem . My two favorite 51:44.836 --> 51:48.090 examples are one , recidivism . So , 51:48.139 --> 51:50.083 you know , you have judges , human 51:50.083 --> 51:52.306 judges who are deciding whether a , a , 51:52.370 --> 51:54.481 a person should get a parole or not , 51:54.481 --> 51:56.703 and they look at the data , all right ? 51:56.703 --> 51:58.481 That data has been Massaged and 51:58.481 --> 52:00.648 manipulated by artificial intelligence 52:00.648 --> 52:02.759 algorithms which are trying to decide 52:02.759 --> 52:04.814 the probability of this person going 52:04.814 --> 52:06.981 back to crime . And what you find is , 52:06.981 --> 52:08.814 is because of the way the AI was 52:08.814 --> 52:10.870 created , it's creating a , a biased 52:10.870 --> 52:12.870 view of , of whether this person is 52:12.870 --> 52:15.037 going to go back to crime or not . And 52:15.037 --> 52:14.600 so there , even though you have a human 52:14.600 --> 52:16.878 on the loop , they're believing the AI , 52:16.878 --> 52:18.933 you could say it's because the AI is 52:18.933 --> 52:21.044 wrong . I , I think the AI could be , 52:21.080 --> 52:23.139 you know , perfect , but it's humans 52:23.139 --> 52:25.195 are going to be swayed . And so what 52:25.195 --> 52:27.417 we've got to figure out is how do we do 52:27.417 --> 52:29.583 that human AI teaming better than what 52:29.583 --> 52:31.695 we're currently doing ? That's a real 52:31.695 --> 52:33.861 challenge . My second example is , you 52:33.861 --> 52:35.917 know , I don't know what the Israeli 52:35.917 --> 52:38.139 Defense Force , you know , the sequence 52:38.139 --> 52:40.028 that led to them bombing that aid 52:40.028 --> 52:42.159 convoy of 3 vans , the kitchen aid 52:42.159 --> 52:44.439 convoy , but I'm going to suggest to 52:44.439 --> 52:46.550 you that there was some data analysis 52:46.550 --> 52:48.717 went on there and there was a human in 52:48.717 --> 52:50.828 the loop that made the decision , but 52:50.828 --> 52:52.772 they were swayed because of the AI 52:52.772 --> 52:54.995 saying , Hey , these sequences of stops 52:54.995 --> 52:54.949 with that set of vehicles , that's , 52:55.000 --> 52:57.167 that's Hamas . I'm going to hit them . 52:57.167 --> 52:59.333 So again , I don't think we put humans 52:59.333 --> 53:01.556 on the loop is going to get me to where 53:01.556 --> 53:03.778 I need to get to . I , right now , what 53:03.778 --> 53:05.889 I , my , my current approach to doing 53:05.889 --> 53:08.379 this is We're using AI to enable us to 53:08.379 --> 53:11.520 do things on behalf of humans , so a 53:11.520 --> 53:13.687 human is delegating bounding authority 53:13.687 --> 53:15.631 for these autonomous systems to do 53:15.631 --> 53:17.798 something on their behalf . So I could 53:17.798 --> 53:19.853 go to that human and say , if you're 53:19.853 --> 53:21.964 going to do that task , what are your 53:21.964 --> 53:24.131 moral and ethical concerns and capture 53:24.131 --> 53:26.242 those . Some of those I can handle in 53:26.242 --> 53:28.187 code . Like don't go over that geo 53:28.187 --> 53:30.409 fence . zero problem with that , OK ? I 53:30.409 --> 53:32.576 can make sure this AI doesn't function 53:32.576 --> 53:34.576 in that part of the space . Easy to 53:34.576 --> 53:36.798 handle . But there'll be others , hey , 53:36.798 --> 53:38.964 do no collateral damage . That becomes 53:38.964 --> 53:41.187 a little more general . And so now I've 53:41.187 --> 53:40.080 got to figure out , you know , which 53:40.080 --> 53:42.413 ones can I handle , which one can I not . 53:42.413 --> 53:45.429 How do I be a training , get that human 53:45.429 --> 53:47.310 to get that , that world model 53:47.310 --> 53:49.532 appropriately created so they know when 53:49.532 --> 53:51.754 and how much have you will to calibrate 53:51.754 --> 53:53.532 the trust ? Because I need that 53:53.532 --> 53:55.643 appropriate level of human judgment , 53:55.643 --> 53:57.810 you know , that's the phrase we use in 53:57.810 --> 53:59.977 the DUD directive , which it says when 53:59.977 --> 53:59.669 we can use these things and when we 53:59.669 --> 54:01.810 can't . Now , the last slide , you go 54:01.810 --> 54:04.199 to the last slide . Um , you know , 54:04.280 --> 54:06.613 here , here's some things that , I mean , 54:06.613 --> 54:08.669 next week , um , what I give or take 54:08.669 --> 54:10.836 what you guys feedback to , to Nagle , 54:10.836 --> 54:13.058 um , we can pack in examples around any 54:13.058 --> 54:15.113 of these things that you think are a 54:15.113 --> 54:17.336 little tenuous . of why we landed where 54:17.336 --> 54:19.502 we did , um , as a group , some of the 54:19.502 --> 54:21.558 things I'd like to see this year , I 54:21.558 --> 54:23.613 had this same slide last year and we 54:23.613 --> 54:25.724 did none of them , or maybe I did the 54:25.724 --> 54:27.891 types of quality one , but , but , but 54:27.891 --> 54:30.002 slide , I got to get slider to do the 54:30.002 --> 54:32.169 mathematics requests . Now we did some 54:32.169 --> 54:34.336 really interesting things this year in 54:34.336 --> 54:34.270 terms of manifolds and everything which 54:34.270 --> 54:36.381 are going down that path . Previously 54:36.381 --> 54:38.492 we've done some category theory , but 54:38.492 --> 54:40.492 I'm going to rely on slider pulling 54:40.492 --> 54:42.381 together , you know , some , some 54:42.381 --> 54:44.548 thoughts there , um . And by the way , 54:44.548 --> 54:46.770 Jason , I'm very familiar with Asimov . 54:46.770 --> 54:48.714 Uh , TJ and I go way back , so I'm 54:48.714 --> 54:50.881 happy to talk to you about that . Um , 54:50.881 --> 54:53.214 the moonshine , you know , I , you know , 54:53.214 --> 54:52.629 how do we change the current things 54:52.629 --> 54:54.573 we're doing , you know , the large 54:54.573 --> 54:56.573 pre-trained models , etc . and make 54:56.573 --> 54:58.796 them more quest compliance . And then , 54:58.796 --> 55:00.796 of course , Nagle , you know , keep 55:00.796 --> 55:02.962 pushing us down the neurophysiology of 55:02.962 --> 55:04.796 Quest , and then , um , expanded 55:04.796 --> 55:06.796 thoughts on this thing . Um , let's 55:06.796 --> 55:08.518 spend 5 minutes on questions . 55:10.449 --> 55:14.399 Anything I missed . This 55:14.399 --> 55:16.566 is both done , and I , I'm , I'm , I'm 55:16.566 --> 55:18.788 sorry I'm jumping in , but I'm , I'm on 55:18.788 --> 55:20.843 a phone because we're out of power . 55:20.843 --> 55:22.955 Can I ask you a quick question , um . 55:22.979 --> 55:26.360 About morals . So , would you say that 55:26.360 --> 55:28.582 in , in a simple vision , morals can be 55:28.582 --> 55:31.280 either driven by fear or by empathy 55:31.280 --> 55:33.580 over ? Yeah , 55:35.260 --> 55:37.204 that can't be answered trivially , 55:37.204 --> 55:39.260 Molly . Let me send you some stuff I 55:39.260 --> 55:41.371 have on that , OK ? Um , we've looked 55:41.371 --> 55:43.482 at that in . OK , go ahead , George . 55:43.679 --> 55:46.000 Yeah , so , you know , I , I'm sort of 55:46.000 --> 55:49.429 going back to your uh uh comment 55:49.429 --> 55:52.840 about Chinese world models and American 55:52.840 --> 55:54.896 world models and so on . So I'm just 55:54.896 --> 55:57.479 gonna make an assertion that uh like 55:57.479 --> 55:59.423 basic training , which I didn't go 55:59.423 --> 56:01.423 through , but some of the people on 56:01.423 --> 56:05.040 this call did , is uh is part of that 56:05.040 --> 56:08.919 to create a common world model among . 56:09.629 --> 56:12.350 You know , infantry or sailors or 56:12.719 --> 56:16.080 airmen in your case , um , so that they 56:16.080 --> 56:18.580 can work effectively as a team . So you , 56:18.679 --> 56:20.623 you want , you don't want all this 56:20.623 --> 56:22.879 creativity and random interpretation of 56:22.879 --> 56:25.239 what the world is when , when you're in 56:25.239 --> 56:28.520 the military , it seems . So , I'm , I 56:28.520 --> 56:30.853 just made that assertion if that's true . 56:30.959 --> 56:34.639 Um , now , as we move into man machine 56:34.639 --> 56:38.459 teaming , Can , do you have 56:38.459 --> 56:40.790 some advice to people who are gonna be 56:40.790 --> 56:44.530 training future airmen , sailors , 56:44.620 --> 56:48.010 infantry uh about how to build 56:48.379 --> 56:51.020 common world models with machines . So 56:51.020 --> 56:53.242 we're gonna start using machines more . 56:54.320 --> 56:57.629 Is it too early to start or what would 56:57.629 --> 56:59.796 you do ? What's your recommendation on 56:59.796 --> 57:02.590 that ? So a former colleague of ours , 57:02.679 --> 57:04.949 Dave Woods of Ohio State , convinced me 57:04.949 --> 57:08.010 some time ago . We spent some time 57:08.010 --> 57:10.209 looking at failures we've had in 57:10.209 --> 57:13.330 fielding AI in different domains , and 57:13.330 --> 57:15.552 we concluded that , you know , that the 57:15.552 --> 57:17.719 human , because exactly the reason you 57:17.719 --> 57:20.449 say it has no common representation to 57:20.449 --> 57:23.330 the AI solutions and , and therefore 57:23.330 --> 57:24.997 they have a real problem with 57:24.997 --> 57:26.997 interpredictability knowing when it 57:26.997 --> 57:29.163 does this and when it does that . That 57:29.163 --> 57:28.790 you have to go down this path of what 57:28.790 --> 57:30.957 is it doing ? why is it doing it , and 57:30.957 --> 57:33.290 what's it going to do when this happens ? 57:33.290 --> 57:35.290 And , and so there is , there is in 57:35.290 --> 57:37.346 fact been lots of progress on how do 57:37.346 --> 57:39.568 you train people , but , but those were 57:39.568 --> 57:42.229 in very small uh use cases in very 57:42.229 --> 57:44.510 specific narrow use cases . And so in 57:44.510 --> 57:46.621 that , in those cases , I think we're 57:46.621 --> 57:48.788 probably OK , and I , I see Lorraine , 57:48.788 --> 57:50.899 Lorraine's on that she could probably 57:50.899 --> 57:53.177 help us on this . Um , the , the , the , 57:53.177 --> 57:55.343 those , I think we're probably OK on . 57:55.343 --> 57:57.677 I , I , what I'm far more worried about , 57:57.677 --> 57:57.530 George , is the one you're pushing us 57:57.530 --> 57:59.586 on . As they make these systems more 57:59.586 --> 58:01.641 capable where they can do a range of 58:01.641 --> 58:03.697 things , then , then how do I create 58:03.697 --> 58:05.697 that interpredictability ? How do I 58:05.697 --> 58:07.863 create what I'll call the wingman AI ? 58:07.863 --> 58:10.030 And , and that is , that's a wild west 58:10.030 --> 58:12.197 right now . And the part of the reason 58:12.197 --> 58:14.363 is , is our VNV approach is . are just 58:14.363 --> 58:16.530 absolutely terrible . What we do is we 58:16.530 --> 58:18.419 take the AI and test it by itself 58:18.419 --> 58:20.752 instead of doing what you're suggesting , 58:20.752 --> 58:20.364 which is how do I take this diverse set 58:20.364 --> 58:23.334 of potential users of this AI and , and , 58:23.514 --> 58:25.847 and look at all the variations of , hey , 58:25.847 --> 58:28.014 this novice versus this guy who's more 58:28.014 --> 58:30.125 sophisticated . What are the things I 58:30.125 --> 58:32.070 have to do to interface to each of 58:32.070 --> 58:34.236 those variations ? Um , that is , that 58:34.236 --> 58:36.347 is a , that is a real challenge . And 58:36.347 --> 58:38.403 I've lost the buckle of who's next . 58:41.060 --> 58:42.560 Igor , you wanna jump in ? 58:46.469 --> 58:49.550 Can you hear me ? Yeah , uh , I , I 58:49.550 --> 58:51.717 just have a comment on , uh , George , 58:51.717 --> 58:55.050 uh , uh , uh , commentary regarding the 58:55.050 --> 58:57.979 what model from Chinese or , uh , well , 58:58.219 --> 59:01.870 American . Uh , work model . Uh , so , 59:02.229 --> 59:04.396 for much of years it is very important 59:04.396 --> 59:07.290 and I , I believe that , uh , uh , work 59:07.290 --> 59:09.840 model from , uh , totalitarian state 59:10.050 --> 59:12.050 would be very different from , uh , 59:12.050 --> 59:14.217 free world . And also there are some , 59:14.409 --> 59:17.169 uh , differences in religious beliefs . 59:17.409 --> 59:19.187 On high level , it will be very 59:19.187 --> 59:21.298 different on the low level , it could 59:21.298 --> 59:23.353 be quite same because it's obviously 59:23.353 --> 59:25.709 could be tested . So , uh , in my view , 59:25.919 --> 59:27.808 uh , yes , it's important to have 59:27.879 --> 59:30.610 indoctrination and , uh , uh , just 59:30.610 --> 59:32.439 joint training all the time and 59:32.439 --> 59:34.919 discussing on high levels , uh , for 59:34.919 --> 59:37.929 any artificial intelligence , while , 59:37.939 --> 59:40.750 uh , Satan . I got to tell you , I get , 59:40.870 --> 59:42.870 I get that question a lot from that 59:42.870 --> 59:45.037 funny shape building . You know , what 59:45.037 --> 59:47.259 is the price we're going to pay because 59:47.259 --> 59:49.426 of the potential Chinese employment of 59:49.426 --> 59:51.481 AI will be different than ours based 59:51.481 --> 59:53.537 upon the cultural differences . Um , 59:53.537 --> 59:55.814 Glenn , you , you want to take us home ? 59:55.814 --> 59:58.037 Yeah , I'm , I'm thinking that you keep 59:58.037 --> 01:00:00.899 hitting like I have one ring that rules 01:00:00.899 --> 01:00:04.560 them all AI model . And , you 01:00:04.560 --> 01:00:08.100 know , uh , we have debates in our own 01:00:08.100 --> 01:00:10.649 minds over what to do , you know , the , 01:00:10.659 --> 01:00:13.219 uh , the ethical AI says , well , you 01:00:13.219 --> 01:00:15.870 better be careful . Uh , the other one 01:00:15.870 --> 01:00:17.926 goes , no , no , these rules , I can 01:00:17.926 --> 01:00:19.981 get away with this . It's , uh , and 01:00:19.981 --> 01:00:21.981 then another , another part of your 01:00:21.981 --> 01:00:23.981 brain says , yeah , but how is this 01:00:23.981 --> 01:00:26.203 gonna read in the New York Times or The 01:00:26.203 --> 01:00:28.370 Washington Post ? What are you doing ? 01:00:28.370 --> 01:00:30.790 And so do we end up training ? A whole 01:00:30.790 --> 01:00:34.399 bunch of voting AIs to uh give you 01:00:34.399 --> 01:00:36.399 different perspectives and then let 01:00:36.399 --> 01:00:39.840 another AI sit back and dispassionately 01:00:39.840 --> 01:00:42.469 rule the courses of action and say , 01:00:42.479 --> 01:00:45.030 yeah , there's too much risk here , 01:00:45.159 --> 01:00:47.520 rather than trying to have one AI do it 01:00:47.520 --> 01:00:50.169 all over . Yes , certainly most of the 01:00:50.169 --> 01:00:52.336 breakthroughs we're getting these days 01:00:52.336 --> 01:00:54.280 are with what we call a mixture of 01:00:54.280 --> 01:00:54.169 experts , where you have lots of 01:00:54.169 --> 01:00:55.669 different variations . Now 01:00:55.669 --> 01:00:57.669 unfortunately , most of that is the 01:00:57.669 --> 01:00:59.669 experts are specialized versus what 01:00:59.669 --> 01:01:01.502 you're suggesting , but the same 01:01:01.502 --> 01:01:03.669 concept works . What that architecture 01:01:03.669 --> 01:01:05.891 is going to look like , how do you test 01:01:05.891 --> 01:01:08.113 it , how do you employ it . Wide open . 01:01:08.129 --> 01:01:10.351 Um , but the bottom line is , in fact , 01:01:10.351 --> 01:01:12.462 we've had an internal debate going on 01:01:12.462 --> 01:01:14.649 on creating AI solutions that lie to 01:01:14.649 --> 01:01:17.050 the humans because it was able to 01:01:17.050 --> 01:01:19.919 deduce that by giving it these false 01:01:19.919 --> 01:01:21.808 information , they'll end up in a 01:01:21.808 --> 01:01:23.975 better place in terms of performance , 01:01:23.975 --> 01:01:26.141 you know , is that , is that something 01:01:26.141 --> 01:01:25.770 we're going to allow to happen ? It's 01:01:25.770 --> 01:01:28.090 really , really interesting days . All 01:01:28.090 --> 01:01:30.312 right guys , it's one o'clock . I don't 01:01:30.312 --> 01:01:32.368 want to run over . I can't afford to 01:01:32.368 --> 01:01:34.479 pay you the overtime , so , uh , send 01:01:34.479 --> 01:01:34.050 questions or comments , and we'll be 01:01:34.050 --> 01:01:35.850 ready next week . Thanks , Kyle .