00:05sawcon democratized learning with his
00:07educational YouTube videos that turned
00:09into Khan Academy which has 150 million
00:11Learners from all over the world today
00:1315 years later he's as energized as ever
00:15about making learning personalized with
00:17artificial intelligence this week on the
00:19podcast a lot and I talked with Saul
00:21about the impact of AI and education he
00:24says surprisingly we're on the cusp of
00:26the biggest positive transformation that
00:28education has ever seen Khan Academy has
00:30recently created conmigo a chatbot tutor
00:33that can nudge Learners in the right
00:34direction if they get stuck so welcome
00:37to no priors thanks for having me so can
00:39you start by giving us some background
00:40on yourself and how you ended up
00:42starting Khan Academy sure you know you
00:45go way back my original background was
00:47in tech I go to business school I end up
00:49being an analyst at a small hedge fund
00:51and it was shortly into that it was 2004
00:54I was a year out of business school I
00:55had just gotten married I was based in
00:57Boston at the time I was born and raised
00:59in New Orleans my family was visiting me
01:01up in up in Boston after the wedding and
01:03it just came out of conversation my 12
01:04year old cousin Nadia was having trouble
01:06with school math in particular she was
01:08being placed into a slower math track
01:10her parents my aunt and uncle hadn't
01:12gone to school in this country so I
01:13don't think they understood the
01:14implications that on that track she
01:16wouldn't end up taking calculus in high
01:17school etc etc so I took it pretty
01:19seriously I said hey I'm up for tutoring
01:22you Nadia if you're up for receiving it
01:24and she agreed so she goes back to New
01:26Orleans I start tutoring her remotely
01:27she actually gets caught up with her
01:29class she was actually having trouble
01:30with unit conversion she gets a little
01:32ahead of her class I call up her school
01:34I said I really think Nadia Raymond
01:35should be able to retake that placement
01:37exam they say who are you I say I'm her
01:39cousin and they let her and that same
01:41Nadia that started off as for lack of a
01:44word a remedial student was then put
01:45into an advanced math track so I was I
01:48was hooked I it was fun to stay
01:50connected with family I really enjoyed
01:52geeking out on the math and it was it
01:55seemed to be really helping my family
01:56and so I started tutoring her younger
02:00word spreads in my family free tutoring
02:02is going on before I know it I'm
02:04tutoring 10 15 cousins and you know my I
02:07still had the day job I was working as
02:08an analyst at a hedge fund and I saw a
02:11common pattern with my cousins they just
02:13the main reason they were struggling was
02:15they had gaps in their knowledge often
02:17times the reason they were having
02:18trouble with that algebra equation was
02:20because they weren't fluent in dividing
02:21decimals or negative numbers or
02:23exponents and so I started making
02:26software for them which I did for fun
02:28because I that that part of my brain
02:30wasn't being useful in my hedge fund job
02:32I started making software that would
02:34generate problems for them give them
02:36immediate feedback provide hints if they
02:38needed them allowed me to keep track I
02:40put a little database behind it and
02:42allowed me to keep track of what they
02:43were doing and when they were doing it
02:45and what they were getting wrong and
02:45right and I called that Khan Academy
02:48that was the domain name it was
02:49available it was kind of a fun family
02:51project and about a year later now we're
02:54about 2006 and I was showing this off at
02:56a dinner party and by this point my
02:58family had moved out here to Silicon
02:59Valley did you say of the hedge fund job
03:01the same hedge fund job but actually
03:02it's interesting I have a whole theory
03:04that we can keep referring to that
03:06benevolent aliens are using Khan Academy
03:09to prepare Humanity for First Contact
03:11and okay because there's a series of
03:14things in in my journey that seem like
03:18it was almost like you know people were
03:20moving me around in a way that that
03:22benefited this but one of them was my
03:25boss I mean it was a two-person hedge
03:26fund at the time his wife decided to
03:29become a law professor and it was
03:32interesting because At first she thought
03:33she was going to go to UVA and in case
03:36you know we would have ended up living
03:37on a farm in Charlottesville or
03:38something but at the last minute she
03:40decided that she wanted to take the job
03:41at Stanford and so my wife and I came
03:44out here and my wife ended who's a
03:46physician she ended up doing her
03:48fellowship at Stanford because of my
03:50boss's wife not wise choice but that
03:52that landed us here in Silicon Valley
03:54which I think ended up becoming very
03:56important for the future of Khan Academy
03:57and then I was showing this off at a
04:00dinner party and in in Silicon Valley to
04:02a friend and he said well you know this
04:03is cool but how are you Skilling your
04:04lessons I'm like well I'm not like it's
04:06hard to do with 15 cousins what I was
04:07doing with one he said you know there's
04:09this YouTube thing why don't you record
04:10some your lessons is that
04:13and I immediately thought that was a
04:15horrible idea I thought YouTube was
04:17somewhat frivolous for dogs on
04:19skateboards cats playing piano Etc uh
04:21and you know I was like yeah what how do
04:24I even record that this was before
04:25phones had cameras Etc but I decided oh
04:28there's got to be a way to record your
04:30screen while you're drawing something
04:31and it's called screen capture and and I
04:34just uh started using it and I started
04:37just covering things that I found myself
04:38repeating a lot for my cousins and my
04:42cousins in famously or infamously told
04:44me that they like me better on YouTube
04:45than in person I think what they were
04:48saying is they they liked having an
04:50on-demand version there's no judgment no
04:52shame if they had to review things it
04:54was available all the time but they
04:56really still appreciated having me in
04:57their life I think uh but then you know
05:00YouTube Just A lot of people started
05:01discovering it a lot of people were also
05:03discovering the the web app I actually
05:05had to shut down registrations because
05:06it was crashing my 30 a month Java
05:12um the I was using at the time at what
05:16you believe that wasn't a custom well
05:18YouTube videos by Def you know I I saw
05:20no reason to make them private right I
05:23meant the like Java servlet app the Java
05:25servlet app initially I had it opened up
05:27to anyone who wanted register and I was
05:28actually you know my day job I was an
05:30endless at a hedge fund but I was
05:31emailing local schools and seeing if
05:34they wanted to use it and some of them
05:36were interested in using it famously the
05:38first school that used Khan Academy this
05:41was in 2007 was Sidwell Friends which is
05:44I didn't even know they're very elite
05:46school in DC it's where all of the you
05:48know senators and presidents kids go but
05:50they were actually a fifth grade
05:51classroom this teacher was John romino
05:53was one of the first and he was just
05:54using this homebrew software that I was
05:56I was creating but I actually stopped
05:58registrations on that at 10 000 because
06:00once again it was crashing my 30 a month
06:03web hosting and my cousins weren't able
06:05to use it so I mean and talk about you
06:08know backwards silicon you know so many
06:09things in Silicon Valley people would
06:10love to have the problem of an app that
06:13more people want to use I was like nope
06:15no more for anyone else but by 2008 it
06:18there were about 50 000 folks who were
06:20using the YouTube videos on a monthly
06:22basis which you know compared to where
06:24we are now seems mini school but at the
06:26time seemed like a lot compared to my 15
06:28cousins and I at the time I didn't think
06:31I was going to quit my day job I
06:32actually liked it I was getting paid
06:35but I did set up as a not-for-profit
06:37thinking that maybe I could get some
06:39funding maybe do this on the side hire
06:42some folks Etc et cetera so I set it up
06:44as a not-for-profit mission free
06:45world-class education but then a year
06:48fast forward it kind of took over my
06:50life at that point it's like you know
06:51100 000 or something folks were using it
06:53this is all I was thinking about and uh
06:57you know I I kind of came to terms with
07:00we were trying to save up money for a
07:01down payment on a house in Silicon
07:03Valley which we all know is not a joke
07:04even back in 2009 but at the end of the
07:07day you know real real wealth is being
07:09able to do what you you feel like is
07:11your is your purpose in life I think and
07:12as long as you have enough money to you
07:15know put food on the table go to a
07:16restaurant every now and then hopefully
07:18send your kids to college etc etc
07:21if you can do what you care about and
07:22feel a sense of purpose that's probably
07:24a bigger deal and so that's that's what
07:25I told myself and my wife and then we
07:28get and I took the plunge to do it full
07:29time this is amazing story at some point
07:32did you like learn more about education
07:34because this feels very much like oh I
07:36know algebra and I have some instincts
07:38about what these kids don't yet know
07:39about algebra like was there like
07:41reading about you know the two Sigma
07:44problems somewhere in there like
07:45pedagogy any of it Sim launchers in the
07:48early stages know in the early stages it
07:50was very intuitive driven and it was
07:53really you know I think a lot of I'm
07:55guessing that both of y'all we were all
07:57in the category of kids that did pretty
07:59well academically when we were in school
08:01and I think when you do well
08:03academically in school there's one of
08:05two narratives you can tell yourself one
08:07narrative is hey I'm just special I'm
08:08just good at this I'm just gifted the
08:11other narrative is you know what I'm
08:13it doesn't really feel all that special
08:16and some of my peers that are flunking
08:18the same classes that I'm getting A's in
08:20they can beat me at chess they can you
08:23know learn a video game or a puzzle or
08:24solve a puzzle faster than me so I don't
08:26think it's about like pure cognitive
08:28capability and then the more I used to
08:30tutor peers in high school as well and
08:32the more I did that I just realized it
08:34was the same thing I saw with my cousins
08:35and and the benefit I had and a lot of
08:38us that did well in topics like math and
08:40science we we didn't allow those gaps to
08:42fill whenever we felt a little bit
08:44uneasy about how fluent we were with
08:46something we we gave ourselves a little
08:47bit of extra practice and we weren't
08:49just memorizing things we always wanted
08:51to understand why you know why does the
08:53long division algorithm work you know
08:55let me think about that does it okay
08:56that makes intuitive sense why am the
08:58place values work etc et cetera so I
09:00think when you do that then later and
09:01later math frankly gets easier and
09:03easier not harder and harder and if you
09:05do the opposite uh where you where you
09:08don't understand the the conceptual
09:09ideas behind the algorithms and the
09:11equations and then you also have fluency
09:14gaps I mean the number of classrooms I
09:16now visit were seventh graders they
09:18understand exponents but they literally
09:20you know three times seven they're grab
09:22grabbing their character I'm not
09:23exaggerating I literally saw a seventh
09:25grader in the Bronx for three times
09:27seven grabbing their calculator and I
09:29literally slapped their hand I don't
09:30know if that can get in trouble for that
09:31anymore but like I literally slapped in
09:33the house like no you figure out three
09:34times seven and the kid he started to
09:36use his fingers and I'm like this is on
09:39some level malpractice that no one sat
09:40down with this kid for like an evening
09:42and said you have got like it has to be
09:45automatic it's got to be in milliseconds
09:47that you know what three times seven is
09:49because he he understood all the
09:51concepts of exponents and algebra and
09:53all that but the cognitive load of
09:55having to do that so anyway I was
09:56experiencing that from a very intuitive
09:58point of view obviously once Khan
10:00Academy got more traction started
10:01working with more Educators people would
10:03tell me oh you're just talking about
10:05Mastery learning I was like yes that's
10:08exactly what I'm talking about which is
10:10just that you always have the
10:11opportunity and incentive to fill in
10:12your gaps oh you're talking talking
10:13about differentiation that everyone is
10:16learning at different paces and you have
10:18I was like that's exactly what I'm
10:20talking about and then people are
10:21talking about well yeah Mastery learning
10:22Benjamin Bloom even though it's arguably
10:25the oldest way of learning that you keep
10:27working on something if you haven't
10:28mastered it yet he coined it his Mastery
10:30learning he also wrote this whole you
10:32know two Sigma problem that if someone
10:33gets one-on-one tutoring personalization
10:35Mastery learning can they get can get
10:37accelerated dramatically so yes it was
10:39intuition for me that seemed to work but
10:42now I'm very familiar with a large body
10:44of literature that actually backs it up
10:46from a from Pretty Pretty rigorously
10:48could you talk a little bit about the
10:50range of things that Khan Academy does
10:51now because you have everything from con
10:53labs to Amigo to a variety of other
10:56efforts it'd be great to just kind of
10:57get a view of you know broadly what you
10:59focus on currently and what are the
11:01areas you're most excited about yeah and
11:02the answer I'm going to give you will
11:03probably make any like textbook strategy
11:08because you know oftentimes people say
11:10just focus on one thing and maybe it's
11:13coming from me that I feel like the the
11:15opportunity to transform education is
11:17not going to happen if you just do it
11:19narrowly so I view our Charter you know
11:22the mission free world-class education
11:24for anyone anywhere which by itself is a
11:25very big statement and almost delusional
11:28when I was one guy operating out of a
11:29closet you know a little over a decade
11:31ago but we are we are covering all of
11:34the core academic material from pre-k
11:36through the core of college that's the
11:37goal we want to do it so it's
11:39personalized we actually even want to do
11:41it so it can eventually result in
11:42credentials and so what we are doing
11:45already on Khan Academy so we have Khan
11:47Academy kids which is used by nearly 2
11:49million kids primarily in America it's
11:51actually kind of turning into like the
11:52Sesame Street of this of this generation
11:54really great Africa studies kids are
11:56spending an average of 90 minutes a
11:57month on it which is pretty high for an
11:59average that's a large chunk of all of
12:01the kids in America between the ages of
12:03three and seven who are using that
12:04that's all subjects that's math Reading
12:06Writing character development social
12:08emotional whatever you want to call it
12:09then as you get into older grade levels
12:11it's called big con you know that that's
12:13where we just keep going from just the
12:15basics of math all the way through
12:17calculus and statistics and
12:19multivariable and and on and on science
12:22we have science a pretty strong
12:24progression from I would say late
12:25Elementary School through Early College
12:28as well going all the way to biology
12:30chemistry physics and this is not just
12:32videos this is most of our resources are
12:34actually behind the exercise platform
12:35where you can get as much practice as
12:37you need deep item Banks immediate
12:39feedback teachers can keep track of what
12:41they're what's going on and and assign
12:43through the platform we've added
12:45Humanities so you know American history
12:47Civics and government world history we
12:49have art history so we're filling out
12:51that grid of all of the core academic
12:52material financial literacy computer
12:54science so so all of that economics is
12:57is happening early on in
13:002012 2011 I wrote a book The One World
13:04Schoolhouse and the first third of the
13:06book was kind of like the history of
13:07education the middle third of the book
13:08was my journey getting here the last of
13:11the book was in a world where things are
13:14changing and tools like Khan Academy
13:15exist what should education look like
13:18just it's almost like a first principles
13:20exercise like could we have full year
13:21schooling mixed age classrooms
13:22personalization Mastery throughout more
13:25time for Hands-On if you can do the
13:27other stuff more efficiently one thing
13:28to write about it a whole other thing to
13:30implement it on top of that my oldest
13:32child was about to enter kindergarten
13:34and I felt like a hypocrite if if I was
13:36telling everyone to do Mastery learning
13:38and if my kid just went to the local
13:40school that was not doing Mastery
13:41learning so we started con lab school
13:43literally underneath on the first floor
13:45of the offices of Khan Academy that was
13:47back in 2014 that is now a K-12 program
13:50we've had three graduating classes
13:52already so I'm happy to talk more about
13:54that we started a another online school
13:57with Arizona State called con World
13:59School we just started last year with
14:00that but that's now sixth through 12th
14:03grade and there's some very exciting
14:04things happening there and then most
14:07well I'm glazing over a lot we are
14:10working on credentials so we have a
14:11pilot with Howard University where kids
14:13in town Title One high schools are
14:14getting Mastery on Khan Academy and
14:16that's resulting in college algebra
14:18credit which is solves a major need
14:21happy to talk more about that and then
14:22most recently we're doing a lot with
14:24generative AI we started partnering with
14:26openai about a year ago so well before
14:28it was really a big thing and we had to
14:31keep it quiet for a while but conmigo
14:34you know it's a in our minds it's a
14:37tutor for every student it's a teaching
14:38assistant for every teacher it can
14:40support students while they do
14:41traditional work on Khan Academy
14:43answering questions making it relevant
14:45motivating them even advising them that
14:47we have functionality where it can be a
14:48guidance counselor it can be a academic
14:51coach but they can also debate the AI
14:53and practice their fine you know
14:55fine-tune their argumentative skills
14:56they can talk to simulations of
14:57historical or fictional characters and
15:00on teachers they can use it to create
15:02lesson plans grade create rubrics
15:04refresh refresh their own knowledge
15:06you've said that um we're on the casa
15:09the biggest positive transformation that
15:10education has ever seen how important do
15:12you think AI is relative to the broader
15:14set of access that you have through
15:16things for YouTube and online coursework
15:17and all the rest of it is this a
15:19complete Game Changer is it an add-on
15:21like what's the relative degree of
15:23importance of this shift right now
15:25I think in the very short term it is
15:28going to be a meaningful add-on I think
15:30if you go three to five years in the
15:32future it will be a game changer and the
15:34reason I say that is going back to
15:37Benjamin Bloom but I think this even
15:38predates been the gold standard was
15:40always to have a personal tutor you go
15:41back to if you were a prince in most of
15:44if you were Alexander the Great 2300
15:46years ago you had Aristotle as your
15:47personal tutor and Aristotle would speed
15:49up slow down motivate you when you're
15:51feeling down like do all of these things
15:52with you and that was always the gold
15:54standard two 300 years ago utopian idea
15:57of Mass public education but in order to
15:59do that economically we had to make
16:01compromises one of which is you don't
16:03get a personal tutor you don't get a
16:04one-on-one teacher we're gonna batch you
16:06into groups of 30. we're going to move
16:08you at a set timer Pace we're going to
16:10apply some lectures and standards and
16:13on the test some of you are going to get
16:15a a hundred percent some of you are
16:16going to get 80. some of you are going
16:18to flunk it too bad the the batch needs
16:21to move forward and somehow we expect
16:23those of you who didn't know 20 or 30 or
16:2540 of the material on the simpler stuff
16:27to understand the more advanced stuff
16:29and what a what what happens those gaps
16:31accumulated and kids start falling off
16:33and then you eventually put kids on
16:35different assembly lines uh when you
16:36start tracking them and that kind of
16:38worked during the Industrial Revolution
16:40where you didn't need a lot of people in
16:42the knowledge economy you need to kind
16:44of basically educated people to work in
16:46factories Etc et cetera you know not not
16:48so acceptable anymore but that was a
16:50compromise that we had to make Benjamin
16:52Bloom 1984 comes up with the writes this
16:55two Sigma problem where you showed
16:56one-on-one tutoring two standard
16:58deviation Improvement two standard
16:59deviations takes someone from the 50th
17:01percentile to the 96th percentile so an
17:03average student then becomes a very
17:05strong becomes an exceptional student he
17:07calls it a two Sigma problem because
17:10there's no way you're going to be able
17:11to do that in a real classroom like we
17:13don't have resources to give everyone a
17:14tutor and then he even back in 1984
17:17tries to see well what what could you
17:18potentially do if you approximate a
17:20tutor using technology and he he kind of
17:23throws out the conjecture that you could
17:24get one standard deviation of
17:25improvement and to the large degree
17:27everything that we've been doing at Khan
17:29Academy was how can we approximate start
17:32to scale pieces of what a tutor could do
17:35so a micro explanation that's what a
17:37video does and it's on demand whenever
17:39you want it exercises with immediate
17:41feedback that also starts to approximate
17:44and then give teachers and give tutors
17:45and give parents information so that
17:47they can also act more like tutors as
17:50opposed to just giving a lecture to
17:51everybody they can look at the data
17:53where kids are and then do more Focus
17:55interventions have groups of three or
17:57four the kids that need help with the
17:58negative numbers while the other kids
17:59keep working now work with the five kids
18:01who have need help with decimals while
18:03the others so it was all about okay in a
18:05class of 30 can we help approximate
18:07tutoring what generative Ai and I didn't
18:09think this was going to happen in my
18:10lifetime so when we started we're
18:12playing with g ppt4 about a year ago it
18:15blew my mind that you could actually get
18:17it not only to pretend to be a tutor it
18:19actually had good quote Twitter moves it
18:21was it was being truly Socratic it had
18:24some big issues with it around the math
18:25and the hallucinations we've been
18:27working pretty feverishly to mitigate
18:29those pretty significantly that it's
18:30pretty darn good now and so it's already
18:33and we've already started putting it on
18:34in real schools out there we're already
18:37seeing it's increasing engagement more
18:39kids are not only engaged but they're
18:41getting they're not getting as blocked
18:43teachers are definitely you know we
18:44haven't done rigorous studies on it yet
18:46it's very early but anecdotal teachers
18:48like that's answering questions that the
18:50students either were afraid to ask or
18:52that I as a teacher would not have been
18:54able to get to teachers are already
18:55really enjoying the teaching assistant
18:57functionality it's saving them a ton of
18:59time writing lesson friends rubrics they
19:01kind of feel less alone as a teacher and
19:03and now we're looking at ways that it
19:05can even help address some of the
19:06problems that generally introduced where
19:08we actually just made an announcement
19:10today with instructor the folks who make
19:11canvas the learning management system
19:13that you know goes out to its most of
19:15higher ed and a large chunk of K-12
19:16we're like well if if you use the AI to
19:19create rubrics lesson plans and then the
19:22AI essentially administers the
19:24assignment with the student and doesn't
19:25do it for them but does it alongside
19:27them then the AI can report back not
19:30just on the outcome of the assignment
19:32but actually the process like hey yeah I
19:34worked with the student we brainstormed
19:35thesis statements and then they were
19:37having a little bit of trouble backing
19:38it up but that we eventually got there
19:39and it took us about four hours and so
19:41you could be pretty confident in that
19:43situation that the kid didn't just copy
19:44and paste from chat GPT so I think in
19:46the next year or two it's going to be a
19:48really useful tool that's going to
19:49increase teacher productivity and
19:51support a lot of students I think when
19:53you start going about three years out
19:55we're already working on the notion of
19:57of memory for the tutor so the tutor if
20:00you tell it if you ask the AI why should
20:03I learn this and it says well what do
20:04you care about and you say well I love
20:06women's soccer the World Cup's going on
20:08so that's front of mind so I love
20:09women's soccer we'll say okay it'll make
20:12a connection to that but the next time
20:14it should remember that you like women's
20:16soccer so that it doesn't have to ask
20:18you again and so it's crazy but it could
20:20you know you like language in a certain
20:22you like more casual language or more
20:23formal language or your reading level
20:25whatever it might be so we're actually
20:27already prototyping that it remembers
20:29these things about you we're making that
20:31very transparent to the user so that
20:32they can say no I'm not into women's
20:33soccer anymore I'm into gymnastics or
20:35whatever it might be but we think that
20:37type of memory and the type of memory
20:40where it can refer to previous
20:41conversations will allow it to be more
20:44allowed to to graduate from being just a
20:47on-demand help from for certain tasks to
20:50being something that could have a long
20:51journey with you have a narrative I know
20:54it sounds a little bit wild but I could
20:55imagine in five or ten years you're
20:57going to get your best college
20:58recommendation from conmigo because
21:01kanmigo's like I've been working with
21:02this kid for hours for the last 12 years
21:04and let me tell you about this and and
21:06by the way they love gymnastics like and
21:09so and you have a great gymnastics
21:10program at whatever uh so so I think
21:13that's and and you're gonna have you
21:14know we're already playing with the
21:15text-to-speech I know people have heard
21:17text-to-speech before what's about to
21:19come will blow your mind it's hard to
21:21differentiate from a real human being
21:23which is scary on other dimensions of
21:24life but it's good for tutors and then I
21:26think in three to five years yeah you're
21:28going to be able to video conference
21:29with with with your tutor which then
21:32makes it like a real thing that you can
21:34have a long a long journey with have you
21:36ever read the the Neil Stevenson book
21:37The Diamond age not only have I read it
21:40I used to give it away to people every
21:42employee at Khan Academy uh used to get
21:45the diamond age and you know you're
21:46referring to it because you know this it
21:48takes place in this like neo-victorian
21:50not too far off future in like China and
21:54this member of this like neo-nobility
21:56gets this AI tablet app for his
21:59granddaughter to educate her the young
22:00lady's Illustrated primer and it gets
22:04bootlegged and it gets in the hands of
22:06200 000 orphan girls who live in barges
22:09and then they essentially just take over
22:11and so I've always used young lady young
22:14ladies Illustrated primer to our team at
22:16Khan Academy he's like this is what
22:17we're hoping to build in the long run
22:19and I never thought we were going to
22:21fully build it I thought we were going
22:23to be able to approximate it but already
22:25conmigo can do some things that are
22:28maybe even beyond what the young ladies
22:30Illustrated primer did where you can
22:32talk to Don Quixote in any way language
22:36that you want uh you can get into
22:39debates with it etc etc and I think in
22:42the next three to five years almost
22:44everything that Neil Stevenson imagined
22:46I think he wrote the book in 1994
22:48um I think is actually going to be a
22:49reality which I didn't think was going
22:51to happen in my lifetime how does that
22:52impact the structure of the actual
22:54school so say that you think I had five
22:56and you're in the context of an advanced
22:58K-12 maybe it's con Labs maybe it's
23:01what happens or what changes or how do
23:03you interact with the tutor versus the
23:05classroom environment and I know it's
23:07hard to predict the future but I'm sure
23:08you've thought a lot about this stuff I
23:10think big picture at a place like con
23:11lab school where you know even before
23:14generative AI we've been and it kind of
23:16World School actually we just documented
23:18we I we were almost embarrassed to say
23:21um Steve Levitt of Freakonomics Fame he
23:23actually looked at it himself because he
23:24was like this is this real and he's like
23:26oh it's real I mean these kids are
23:28learning about three to four times
23:29faster not three percent faster or 30
23:32percent faster three to four times
23:34faster wow and and At Con lab school
23:36we've consistently seen it you know at
23:39least 1.5 to 2 grade levels in a year
23:41and this is before before generative AI
23:44so I think in some of these settings we
23:46haven't had the same issues that the you
23:49know in the Traditional School System
23:51kids are getting about 0.7 grade levels
23:55now I think if you go to more fluent
23:57areas FL areas where parents are college
24:00educated and they can afford more
24:02supports for their kids you're probably
24:04seeing at least a grade level or maybe
24:05closer to what we're seeing At Con lab
24:07school or like 1.5 two grade levels a
24:09year but I think in environments where
24:12basic fluency has been less of an issue
24:15you're you're now going to free up more
24:17time for teachers so they're going to
24:18spend less time doing things like
24:20grading and lesson most teachers spend
24:22about half their time doing that kind of
24:23stuff so that that gives them more
24:25energy for themselves but also more
24:27energy just be with the students which I
24:28think is really powerful students are
24:30going to feel a lot more supported and I
24:32think if you if you fast forward three
24:35to five years kids are going to feel
24:37very supported in their that kind of
24:39that basic fluency and that core
24:40academic but I think it's going to be
24:42like the Magic School Bus like you're
24:43going to be able to put on your like
24:44apple visors or whatever that you know
24:46and you're going to be able to like go
24:48on with the with the teacher and like
24:49you know jump into the the circulatory
24:52system or time machine yourself to
24:53ancient Greece and and have a debate
24:56with Socrates and you're gonna be like I
24:59I think you're gonna have and you're
25:00gonna have uh you know one of the things
25:03we do at con lab school is just try to
25:04create more time and space for students
25:05passions and so you're going to have a
25:07world where a middle school student is
25:09going to be able to create like studio
25:11quality techno music or you know Lord of
25:15the Rings quality you know epic epic
25:17science fiction movie or fantasy movie
25:19so so I I actually think it's going to
25:21be very exciting and I think if you go
25:22into let's call it more mainstream
25:25I think we're going to go a lot further
25:27towards solving the just the core
25:29fluency issue and not just in math but
25:30in writing as well because that's the
25:32other thing the main problem with
25:33writing it's very resource intensive
25:35it's hard to get feedback it's very
25:37expensive for the teacher to give
25:38feedback for 40 kids it's mind-numbing
25:40frankly kids have to wait a week to get
25:42that feedback then they may or may not
25:45get a chance to iterate on it and so
25:47most people aren't getting much practice
25:48and that's why you're seeing a big
25:49problem with writing on top of the
25:51problem with math so I feel confident I
25:53mean already this year in Newark New
25:55Jersey which you know is not a special
25:57case School District the north Ward had
26:00just started using Khan Academy pretty
26:02intensively in November this is
26:03pre-generative AI as of June 70 of those
26:086 000 kids use Khan Academy at a level
26:10that's associated with pretty profound
26:13efficacy gains we're still waiting on
26:15those test scores but we're I think
26:17we're going to show for the first time
26:18in like U.S history or recent history a
26:21pretty large acceleration of a large
26:23Urban School District in math and that's
26:25pretty generative AI so I'm feeling
26:28actually very optimistic about that and
26:30then yes especially for those students
26:31whose families might not get an
26:33opportunity to you know put them in AI
26:36summer camps like we do with our kids or
26:38you know take trips or you know I think
26:41AI is going to bring the world to them
26:43um in really in really powerful ways
26:44they're going to be able to practice
26:45their debating skills they're going to
26:46be able to connect things to other ideas
26:48they're going to be able to create
26:49pretty profound things they're going to
26:51be able to also go on that Magic School
26:52Bus and it won't gonna it's not going to
26:54be something that requires a ton of
26:56resources so I'm I'm overall hopeful you
26:59know there's always forces of friction
27:00and cynicism and et cetera et cetera but
27:03because teachers have a lot to gain here
27:06it's going to save them a lot of time
27:07and energy I think you're going to see a
27:09faster uptake of this than you see of
27:11other other Solutions there's the old
27:13saying that the future is here is just
27:14not equally distributed and with things
27:17where you have sort of strong structure
27:20in place like the education system is
27:22this a five-year transformation a
27:2420-year transformation afford it like
27:25how long will it take for schools to
27:28adopt this despite all the incentives to
27:30do so yeah I've learned not to be too
27:32aggressive in my predictions but I think
27:34it will be safe to say that in five
27:36years almost all teachers will be using
27:39generative AI for lesson planning
27:41grading progress reports rubric creation
27:44I think that is all right I I that might
27:48happen by by like late next year
27:49honestly so that's that's going to
27:52so I think in a world where almost all
27:55teachers are already using the tool
27:57I think having a generative AI tutor
28:00supporting the student I think that's
28:02for sure going to be mainstream in three
28:04to five years I think you know because
28:07chat gbt introduced the emergency around
28:09cheating like it's not like it's so nice
28:12to have it's like a problem it's like
28:13broke the education system people need a
28:16solution and so I think that has to be
28:17fixed and then and I think it's actually
28:18going to be fixed with generative AI as
28:20well that's that hashtag come into play
28:23in the next year or two so you know five
28:26years in the future you might when you
28:28just visit a classroom you might not
28:29superficially see a lot of differences
28:31but I think when you start double
28:32clicking on it and you realize that wow
28:34teachers have a lot more time now for
28:37student-facing things a lot fewer
28:39students are stuck there's a lot more
28:41space for differentiation now I think
28:43it's that that will be transformative
28:45and I think you're going to see it in
28:46student engagement motivation and and in
28:48things like test scores so
28:58pass you're working on and they're
28:59engaging because they have narratives
29:00and the dragons that's like teaching you
29:02calculus with in physics with like your
29:05environment there's a there's a lot of
29:06Engagement with that and much less
29:08interest in some cases in sort of like
29:11group schooling settings like can you
29:14talk a little bit about how you think
29:16about that because you also started
29:19schoolhouse.world with our mutual friend
29:21shashir founder of Coda and and you know
29:23this is like small groups online and
29:26believing that there's still the
29:28importance of this can you can you talk
29:29about sort of how those pieces fit
29:30together and what the motivation was
29:32there yeah absolutely and you're just a
29:34little bit more detail in Schoolhouse we
29:35started during the pandemic you know
29:37that was a moment where like Khan
29:38Academy's usage went up 3x and it was
29:41clear that it was great if people are
29:43using Khan Academy but they they needed
29:44everyone was isolated and they needed
29:46more human support as well and I'm
29:49always thinking about okay how can you
29:50give something at scale that used to
29:53feel really expensive and tutoring live
29:56Human tutoring for the most part is
29:58expensive but I was like well what if
30:00you could leverage volunteerism and so
30:02it was a bit of a you know optimistic
30:03idea that there would probably be a lot
30:04of people who want to do you know what I
30:06did with Nadia that a lot of people
30:08would like to be loved to do that and so
30:10we set up schoolhouse.world as a way you
30:12know it's it's another non-profit a
30:14sister nonprofit to Khan Academy its
30:16mission is to connect the world through
30:17learning and the whole idea is is that
30:20you get free tutoring and who gives the
30:21tutoring it's it's volunteers and a lot
30:24of those volunteers could be near peers
30:26they could be high school students
30:27college students but some of them also
30:29are teachers who teach at a fancy
30:30private school and want to give back
30:32they might be a retired Professor et
30:33cetera et cetera it's it's gone well I
30:36mean it's def definitely at a smaller
30:37scale than Khan Academy it's on the
30:39order of schoolhouses on the order about
30:4110 000 folks per month but it's doing
30:43some pretty powerful things already a
30:45large fraction of the kids who take any
30:46given sat are already using Schoolhouse
30:49to get live Human tutoring yeah Khan
30:51Academy had been the official has been
30:53the official practice for kind of the
30:55the asynchronous tutoring aspect of the
30:57SAT or the the individual practice but
30:59now you can get you can join cohorts and
31:01be become part of that and you know I
31:04think the power of Schoolhouse is as
31:06powerful as it is to get help
31:08academically I think that social
31:09connection especially social connection
31:11across borders etc etc hugely powerful
31:14same thing with con lab school con World
31:17School we think it's all about the human
31:19to human connection Khan World School
31:20which is an online high school but its
31:22anchor is a Socratic seminar where kids
31:24are getting together almost every day
31:26and they're debating things now it's not
31:28five hours on Zoom we think that's wrong
31:31that's mind-numbing that was what was
31:32wrong with pandemic schooling like
31:34that's that's literally going to ruin
31:35kids eyes and like ruin them
31:38um but you know an hour two hours but
31:40you also don't want completely
31:40asynchronous you want some interaction
31:42and At Con lab school even though people
31:45think it's Khan Academy is downstairs
31:46from that of course yes we do use Khan
31:48Academy but if anyone visits you're
31:50going to see far more collaboration and
31:53interaction amongst these students and
31:55far more talking amongst the students
31:57than you will see pretty much at any
31:58other school and I've become a Believer
32:01more and more and more yes it is
32:03important to improve test scores Etc et
32:04cetera and we are doing that but that
32:07social emotional that character
32:08development that ability to collaborate
32:10with other folks that's arguably the
32:13more the more important thing and if I I
32:15almost didn't do it by Design out the
32:17gate but if I'm one of the things I'm
32:18proudest about about it at all of these
32:20projects we're doing is that element is
32:22that the the collaborative element the
32:24connected element you know whenever we
32:26bring in new faculty to con lab school
32:28they're they're always like it's weird
32:29how collaborative these kids are even
32:31when you get into High School even in
32:32Silicon Valley in high school where a
32:34lot of other schools get quite
32:35competitive secretly quite competitive
32:37but everyone knows it that's not
32:39happening at careless the kids are
32:41genuinely collaborative I think part of
32:42that is more time and space to pursue
32:44your passions I think a lot of part of
32:46it is Mastery learning it isn't about
32:48you know elbowing out your peer it's
32:51about hey everyone here should learn it
32:52and I think a lot of it's about the peer
32:54support which we we're trying to scale
32:56out Schoolhouse but has always been part
32:59how did their relationship with openai
33:01and gpt4 start and the the origins of
33:04kanmika was that something you came up
33:05with and you went to open AI did they
33:07come to you like I'm a little bit
33:08curious about the origins of both the
33:10product and the relationship yeah you
33:11know this story is going to be just
33:12another one of the the data points
33:14behind the benevolent alien Theory uh
33:16they it was uh about a year ago end of
33:20summer of 2022 I got an email from Greg
33:23Brockman and Sam Altman
33:25um who I was familiar with I didn't I
33:26didn't know super well saying hey we're
33:29working on our newest model we'd love to
33:32to talk to you about it and I was
33:35skeptical whether it would have any real
33:39strategic bearing on what we do at Khan
33:41Academy but just as someone who's
33:42followed the industry and has always
33:44been fascinated by this you know I I
33:46knew I had been following gpt2 gpt3
33:49Dolly so I was like oh are they talking
33:52about gpt4 that's kind of cool if I get
33:54to be on the earlier get to see what's
33:57happening there so we had that meet and
33:59they hadn't even finished the first
34:01training run of gbt4 but they said you
34:03know we think it's going to be done in
34:04about two weeks and we think this is
34:06going to be the model that really wakes
34:07up people to the power of generative AI
34:09we want two reasons why they were
34:12talking to us the first was
34:15we want to to launch with some social
34:17positive use cases to us the most
34:20obvious ones are education and health
34:21care and y'all are the first call we're
34:24making like and we want to do it with
34:25organizations that people trust and are
34:27going to like do it in a thoughtful way
34:28and so as far as I know we were
34:31literally the first people they called
34:32and and and I said well you know I'm I'm
34:35up for looking at it oh and the second
34:38reason why they want to chat with us is
34:39it turns out that Bill Gates when he saw
34:42gpt3 he's like oh this is cool but we
34:45all know gbt3 really didn't have a good
34:47handle on knowledge and he he told the
34:49open AI teams like I'll be impressed if
34:51this could pass the AP biology exam he
34:53literally said that and so I think part
34:55of the reason they also reached out to
34:57us is like y'all have a lot of AP
34:58biology items could we test this
35:01and we're like yeah sure I was a little
35:04bit of like what's in it for us but like
35:05yeah sure that's kind of I I mean I knew
35:07what was in it for me I just wanted to
35:08see what this was so about two or three
35:11weeks later they said oh you know can we
35:12give you a demo I was like yeah sure so
35:14it's me my chief learning officer we get
35:16in and they had a uh AP Bio question and
35:19they said Sal what's the answer to this
35:21and I'm like okay it was about osmosis
35:22something I could see and they said all
35:24right let's see what it says and you
35:25know it says it's C I'm like oh yeah
35:27that's that's maybe it's like I was like
35:29ask it to explain why it's C and I know
35:31everyone's used to chat GPT and even
35:33gpt4 now but this was before anyone had
35:35seen any of this stuff and I asked it to
35:39explain and it explained it explained it
35:40quite well and then I said explain why
35:43the other choices aren't right explain
35:44that quite well I said and that's when I
35:46started to get the Goosebumps and I I
35:47said create five more questions like
35:49this one it did it and that's what I'm
35:52like okay this this is this is a game
35:54changer and then they're like would you
35:55like access I'm like yup please give us
35:58access so that that weekend I literally
36:00couldn't sleep and you know my myself
36:02our chief technology our chief learning
36:04officer we're literally slacking each
36:05other like four in the morning with like
36:06things that we had gotten it to do that
36:08like seemed like science fiction the
36:10most notable was act ticking on personas
36:13like acting as a tutor and being able to
36:16engage in a Socratic dialogue and we
36:19also were discovering some math errors
36:20and stuff like that we were surfacing
36:22that to open it turns out that they did
36:23have some mistakes in their training
36:24data which we you know so that was our
36:26service to humanity above and beyond the
36:27corecon academy work they realized like
36:29oh yeah there's some math but then we
36:31started to say like look if we can if we
36:32can figure out ways to put right guard
36:34rails around this make sure their safety
36:36security we can mitigate the math errors
36:39we can mitigate the hallucinations this
36:41is a game changer so that's when we
36:42started working chat GPT comes out a few
36:45months later I slack Greg Brockman I'm
36:47like hey I thought we were an under ND
36:48like what are you all launching it's
36:50like no this is just on GPT 3.5 but
36:52everyone's you know seems to be taken by
36:54this even though what we're working on
36:56for was so much better
36:57I I was bummed at first but I was like
36:59oh in hindsight it was a good thing
37:01because it threw out this very imperfect
37:03thing that was not designed for
37:04Education the education World took it
37:06the hardest because kids are going to
37:07cheat now et cetera Etc and then when we
37:10launched with gpt4 in on March 15th of
37:122023 it almost allowed us to directly
37:15address all of the concerns that people
37:17were already having around chat GPT and
37:19so that's yeah that's how we got to
37:21where we got to that's for the call how
37:23did you end up thinking about safety as
37:25you mentioned for example you want to
37:26ensure there were safety around the tool
37:27and probably certain answers or certain
37:30questions could necessarily be asked
37:31like how did you end up approaching that
37:33so step one we as soon as we saw it
37:36we're like oh this could be used to
37:37cheat and so like okay how do you help
37:39without cheating and so that's when we
37:41said we started putting guard rails in
37:43the the system prompts so to speak where
37:46it says don't give the answer make sure
37:48the student does the bulk of the work
37:49it's okay to help them but primarily ask
37:52leading questions you know we have we
37:54have a lot we've tested a lot of prompts
37:56to just get to that sweet spot and we're
37:58constantly tweaking it so that's the
38:05you know this is a new environment and
38:07we don't know where kids are going to
38:08take this so let's make it transparent
38:11to other stakeholders what's going on so
38:13we said okay we're going to record every
38:14interaction with the AI and make it
38:16accessible to parents and teachers if
38:19the student is under 18. and you know we
38:21can debate that for you know there's
38:23good arguments why older students there
38:25might be some constructive stuff that
38:26they could use the AI for that's
38:27probably not good to share with parents
38:29but anyway that's it we took the most
38:31conservative stance out the gate
38:33especially for under 18 there and then
38:36the the next one is you know we have a
38:38second AI That's moderating the
38:39conversations and if it looks like it's
38:41going into an unhealthy place not only
38:43will it kind of shut down that that
38:45threat of the conversation but it will
38:47actively notify parents and teachers and
38:49so that's those are and then we we are
38:51trying this best to do to do digital
38:53literacy as well to students to
38:55recognize like look here's what you
38:56could use generative AI for here's where
38:58you should be skeptical of generative AI
39:00Etc and I we we are feeling increasingly
39:05and also none of the interactions and
39:07this is to credit to open AI none of the
39:09interactions are being used to train the
39:13AI and and arguably there could be
39:16benefit in the future for that to happen
39:17but because this is all new frontier and
39:19it's all very sensitive
39:21we and open AI just said yes let's just
39:23keep this as kosher as possible let's
39:25just make this like you know as safe as
39:30now going Beyond Hanukkah spent 15 years
39:33generally thinking about education do
39:35you have a prediction for what happens
39:36to like University with all of these
39:38Technologies yeah I think I I don't
39:41think generative AI dramatically changes
39:43what was already going to happen to
39:44universities I I think it's not news to
39:47anyone that the return on investment for
39:50for University especially if you have to
39:52take on debt is is mixed depending on on
39:55who you are and what you do with it
39:56obviously we have a student debt crisis
39:58and even if we were able to forgive some
40:00it's not going to solve it for the for
40:01the next generation of students and so
40:03universities are bloated with cost
40:05they're not particularly good at you
40:09know either informing or preparing
40:10students for you know in certain cases
40:12they are and there's other cases they're
40:14not of preparing students for what they
40:16need I think you're going to have a
40:17world where the very elite universities
40:21people join those yes it's it's almost
40:25like you're joining a cast you're
40:26joining a club so I think they're going
40:29to be fine I think they're you know
40:31whether or not they can justify the ROI
40:33they're going to be fine I think now
40:35there's some interesting things that are
40:37happening around you know legacy
40:38admissions Etc et cetera so that'll
40:40that'll be interesting I think the
40:42community colleges at the other end of
40:44the spectrum they're very flexible and
40:46they are very good at thinking about
40:47what are the needs like okay are there
40:49are there are there industry
40:51certifications that could be useful okay
40:52we can support students for that are
40:54there other types of you know mini
40:56badges that could help someone get a job
40:58we can do that so I think the community
40:59colleges that are low-cost and Nimble
41:03and are able to adapt to the changing
41:06environment I actually think they're
41:07going to do fine too I think that vast
41:09middle of universities especially the
41:10ones that are charging you know sixty
41:12seventy thousand dollars when you
41:13include room and board per year and a
41:16lot of the kids are graduating with debt
41:18that is very hard to even cancel with
41:20bankruptcy and and they're not they're
41:22not getting the pay to be able to you
41:25know I I I tell everyone you know all
41:27the student debt stuff they should be
41:29holding the universities accountable
41:30like where did that money go like who
41:32got that money right someone lent it to
41:34the student and then that went someplace
41:35it went to the university they took the
41:38money thank you very much and now
41:40they're just kind of hiding when you
41:41have trillion dollars or two trillion
41:43dollars of debt and it's ruining these
41:45kids lives I think if you if you if you
41:48just said you know what the the
41:49universities are accountable for 10 of
41:51that their their tune will change
41:53overnight overnight they will start
41:56informing students much more about the
41:58career Outlook if you were to like yeah
42:00not everyone's going to become a museum
42:02curator let me just tell you that right
42:04now and by the way the kid who you heard
42:06about who who just became a museum
42:08curator majoring in this major it turns
42:11out that their grandmother like built
42:13the museum right like it's a hard job to
42:15get so don't think that this is going to
42:17lead there just be informed and I think
42:18the universities if they were on the
42:20hook for even a small fraction of that
42:22debt would almost like not let you major
42:25in something unless they felt like they
42:27were you were going to pay back the debt
42:28so I think you're gonna you know there's
42:30a reckoning there the university costs
42:32can't get bloated forever I think some
42:34of what's going on around admissions the
42:36scrutiny I think is healthy and you know
42:38I'm a big advocate of a competency-based
42:40world where it's not how long you sat in
42:42a chair or you know how long you threw
42:45Frisbees and went to frat parties it's
42:46it's what do you know and what do you
42:48not know and so we are working and I
42:51know others are working to create
42:52competency signals credentials that if
42:55you do it employers will recognize it
42:57and it doesn't matter if you did it when
42:58you're 16 or when you were 36 you're now
43:00qualified for that job I think that's
43:03where employers want to be and I think
43:04it's not gonna quote disrupt college but
43:08it's going to give College an
43:10alternative which I think is very
43:11healthy I think one of the big shifts
43:13that's coming as well is just the type
43:15of skills that are going to be relevant
43:16in a more and more AI related world so
43:20if you think I had 10 years 20 years 30
43:22years Etc Are there specific things that
43:25for example you're encouraging your kids
43:26to learn or that you think that people
43:28should really be focused on as they
43:29think ahead for their own children you
43:31know in terms of the skill sets that
43:32will be relevant in a generative AI
43:34World a lot and I both have Like
43:36Preschool kindergarten aged kids so
43:38these are active decisions no it's a
43:41good good question it is something I
43:43think a lot about I you know I think
43:44there's two there's at least two
43:46metaphors that can help us think about
43:48that problem one metaphor could be you
43:50know we can go back to the 19th century
43:52and I'm not just going to talk you know
43:53a lot of people made the metaphor of
43:54like steam engines and horses and all
43:56that but I actually think the camera is
43:58the best metaphor because being an
44:01artist was a real thing it was really a
44:03technical field you were a Portrait
44:05Painter and like the best artist you
44:07know you would study for years to be as
44:09accurate to reality as you could and
44:10they could like can look at how the
44:11light moves and all of that and all of a
44:13sudden the camera comes out and all of a
44:16sudden our artist starts saying okay
44:17that's the end of art like this thing
44:19can capture reality better than you know
44:22anyone can but then very quickly people
44:25realize that no in some ways this
44:26liberates the artist that this this
44:29allows us to think about I mean it's not
44:31a coincidence that the impressionist
44:32movement kind of coincided with the
44:34Advent of the camera people start saying
44:36no it's not about capturing reality
44:37that's art should be about the
44:39impression art should be about the
44:40feeling it conjures and it actually all
44:42of a sudden led to an explosion of like
44:44what art is it allowed art to come out
44:46of that trap of just painting nobility
44:48and like these Grand scenes and start
44:50getting into things that evoke and
44:52really challenge us I think the same
44:54amount you know people are saying oh
44:55crap this thing can write pretty well
44:57this thing can code pretty well this
44:59thing can create movies create images
45:03so what that tells me is it it kind of
45:05liberates the Creator to move beyond
45:07that someone who can Elevate and
45:11integrate and manage these tools and the
45:13other metaphor I would say is imagine if
45:15just as we're saying with conmigo every
45:17teacher now has a teaching assistant so
45:19now every coder is not going to have an
45:21army of coding assistance every writer
45:24has has a writing assistant so people
45:26are gonna have to move into the
45:27managerial ranks quite quickly and
45:29figure out okay instead of being a coder
45:31I'm now an architect instead of being a
45:33a writer I'm now an editor and in order
45:36to do that you have to know those skills
45:37really really well arguably better than
45:40previous generations so I think it's
45:43even more important that people get not
45:46just adequate but excellent at writing
45:49communicating coding I think creativity
45:52because there's so many outlets someone
45:55who's creative who leverages these tools
45:56is going to be like Unstoppable they're
45:58going to have like Godlike power and
46:00that's good I know there's you know
46:01there's these debates with the Screen
46:02Actors Guild and it's a very sensitive
46:04issue but the way I think it just is
46:06YouTube there's a lot of talent that
46:07would not have been discovered without
46:08YouTube you would not know about Justin
46:09Bieber you would not know about me yes I
46:12just put myself in the same category
46:13very self-aggrandizing but you know
46:15think about how many movies we've we've
46:17all seen that had like a 100 million
46:18dollar budget and we're horrible like
46:21horrible waste of time movies for 100
46:22million dollars you're now going to have
46:24a ton of people be able to create movies
46:26with similar special effects and and
46:28acting and and screenplays and
46:30everything and a lot of them are going
46:31to be horrible clock even more of them
46:33are going to be horrible but every now
46:34and then we're going to discover the
46:35Justin Bieber of movie making and and
46:37like that's an amazing movie and that's
46:39some kid he or she did it on a budget of
46:41a thousand dollars not a hundred million
46:43dollars so how do we Foster that how do
46:45we give more time and space for that but
46:47I think if you have kids it's kind of
46:49very traditional advice they should just
46:51get really good at reading writing math
46:53I I've never been a subscriber of like
46:55oh there's Google you can search stuff
46:57you don't have to learn anything anymore
46:58nope the more that you have rapid access
47:01to knowledge in your head the more that
47:03you're fluent with your mathematics that
47:05you have information and and Concepts at
47:08the tip of your fingers these tools will
47:10accelerate you more than anyone else and
47:12the kid that has to get the calculator
47:13for three times seven or has to go to
47:15Google to figure out when World War II
47:17was the world is is going to pass them
47:19by amazing I think that's a really great
47:21note in terms of an optimistic view of
47:24you know it's the same basic skills
47:25that'll be important 20 years from now
47:27and kids really just need to learn the
47:28basics and you've provided such a great
47:30platform to achieve that by so thank you
47:32for all the work you've been doing oh
47:33thanks for having me
