00:03all right I think we are now live
00:13um and I think it's it's maybe better to
00:15go or something horizontal
00:18uh we'll see you see more horizontal
00:28let's see and you can sort of read some
00:29of the comments there
00:32all right we're gonna
00:34let's see so here we are
00:39leaving Tesla headquarters and
00:42uh we just dropped a random pen at
00:44Stanford or whatever uh we can clear
00:46that though and I know we can pick it
00:53um let's see how it does
01:04I think well let's see um Okay so
01:11added to kind of random spot uh
01:15that's going to be pretty wobbly
01:19but it's it's really smooth sailing in
01:26and here we're encountering some
01:33driving around the construction so it
01:35has never seen this construction before
01:38uh it is near the headquarters but this
01:41construction is relatively new
01:51but now it's coming over to the right
01:59you can see the destination
02:02hopefully nobody meets us there
02:10let's see is this uh
02:15this live stream working hello
02:35I think it's working yeah
02:44I mean obviously it's a little boring
02:47because we're at a red light
02:51um and there's a lot of traffic
02:57I don't know bought people to death here
03:02sitting at a red lightning pole too
03:09yeah I think like these videos like this
03:11are maybe more interesting if um they're
03:23let's see I'll try to answer comments if
03:56man this is a long ride
04:00well the car is patiently waiting for
04:02the car light to change
04:24it's kind of hard to tell maybe from the
04:28the car is driving very smoothly
04:38I think I'll rely on others to take this
04:43edit out the boring bits and
04:52let's move right there
05:02entirely Ai and cameras just like our
05:07brain works which is neural Nets and
05:13yep I just slow down for Stevo which is
05:19just slow down for another speed bump
05:23and we did not program in if there's no
05:26part there's no line of code that says
05:27slow down for Speed bumps
05:29so it is doing this based entirely on
05:36yeah and there was about that we're just
05:37so uh there's a bicyclist
05:41uh again there is no line of code that
05:44uh give clearance to bicyclists it is
05:47just doing what people do
05:53can read science without ever
06:19once again there is no line of code that
06:23the sign or wait for another car
06:27uh okay well who came there's not like
06:30wait X number of seconds nothing like
06:32that nothing is that this is old Nets
06:37nothing but Nets yeah
06:52yeah and we're gonna get to our
06:53destination we'll change the pin as like
06:56the profit somewhere else right now
06:58maybe we'll run into Zuckerberg and uh
07:00we can we can challenge him to a fight
07:01that'd be fun spice it up
07:07somewhere to his backyard
07:18here we are at a roundabout
07:21so roundabouts were obviously pretty
07:24engine for those two cars yep it just
07:26waited for those two cars to go
07:28and then did the turn
07:32I gotta I think someone repetitive about
07:34this but we have never programmed in the
07:36concept of a roundabout
07:38we just showed a whole bunch of videos
07:42so a lot of big Fleet heads yeah I mean
07:46for you you definitely need a a lot of
07:49training data to a lot of video training
07:52data in order to make this work uh so
07:55it's and you need a yeah really
07:58um billions of dollars of training
08:01um and you need to have to run the
08:04neural net training Hardware
08:05so it's not like easy but
08:10the the mind-blowing thing is that there
08:12are no there there's no heuristics
08:17uh lines of code like there's a there's
08:19a guy in a scooter it's never
08:21it doesn't know what a scooter is it
08:22doesn't know what paddles are
08:25it's literally just been given a lot of
08:33and it's doing all of this on Hardware
08:36uh with about 100 watts of inference
08:39so it's not like obviously it's not some
08:41like massive Data Center and if if we
08:43were offline there would be no
08:45difference this is locally the old
08:47inference that's happening is local it
08:49does not need an internet connection
08:52and and obviously that's necessary
08:53because if you you know lost your
08:56cellular internet connection the car
08:58needs to drive safely
09:00um but we could be somewhere that where
09:02there is no internet connection
09:08and and it it's never seen the roads
09:10before it doesn't matter
09:16yes oh yeah we're running at the full
09:18frame rate so it's taking eight cameras
09:20uh at 36 frames per second
09:29the the pure AI version runs better than
09:32it runs faster than the version that is
09:35a mixture of normal software and AI
09:41in fact it would run it faster than 36
09:44uh frames per second
09:47um except the cameras are currently only
09:49capable of 36 FPS our current you know
09:56frame number is we think it could
09:58probably run at 50. uh frames a second
10:07reality over the roads are basically
10:09designed around 24 frames a second
10:14basically it's similar to I just I just
10:17we got to go to our destination so I'll
10:19just uh you know pick a new destination
10:25make some sort of random place here
10:30but yeah whatever somewhere
10:37I'm like hello assassins if you want to
10:40get me now is your chance you just need
10:45like this the Assassin count is low in
11:00dropped a sort of random and we don't
11:02know where we're going really
11:06just somewhere in Palo Alto
11:11but it's just also worth nothing I want
11:13to actually sort of provide some some
11:16um but we've got to a test drive
11:23FST 12 test drivers around the world so
11:26we've got we've got people in I think
11:28like New Zealand and like Thailand and
11:30Norway Japan every everywhere
11:34because nature generalized the concept
11:36of driving is General you don't need to
11:38like just do it in the US you could do
11:39it everywhere simultaneously right
11:42exactly just like a human
11:45um can go travel to a country they've
11:47never been to before and rent a car and
11:49drive around it'd be you know maybe not
11:51quite as as good as someone local but
11:54you can still rent a car in in a foreign
11:56country and drive around and
11:59we just have like some students over
12:02that test one of them stumbled briefly
12:04into the road and the car uh
12:12a lovely day in California
12:15the beautiful Stanford campus
12:31we'll change the 37 United also like you
12:33know instructions following so you could
12:35like say uh Lane change to the leftmost
12:38Lane yeah pull over here or something
12:40like that in the car showed you and
12:41respect those kinds of commands yeah
12:46yeah so here we had a roundabout of car
12:55I've never I never been to this
12:56roundabout before and the car is not
12:58specifically trained on this roundabout
12:59yeah and then once it's our turn and
13:01just proceeds yeah exactly so
13:05it it waited correctly for the correct
13:08amount of time drove smoothly about
13:09around the roundabouts
13:13again I won't be someone repetitive that
13:15there yet there is no line of code that
13:18says whatever this is a roundabout there
13:20is not nothing that says
13:23you know X number of seconds which is
13:27um explicit control stack that's
13:30the sort of version 11. uh there's over
13:34300 000 Lanes of C plus plus in the the
13:37explicit controls control stack of
13:40um and there's basically none of that in
13:42version 12. yeah and just because
13:44there's no lines of code doesn't mean
13:46that it's uncontrollable it's still like
13:47quite controllable on what you want by
13:50just adding data now you have to program
13:51the data instead of programming the call
14:00and then whenever we find that there's
14:02something say If a card doesn't perform
14:06we give it more examples of what it
14:09should do in that situation and
14:13you know it updates the the weights and
14:18didn't work yeah and also we have
14:19liberals cleaning and making sure the
14:21operator that goes in is like a good
14:22driving it and not like the bad drivers
14:25there are bad drivers yeah yeah
14:27absolutely it's very important the
14:29quality of the data is very important so
14:32um large amounts of mediocre
14:36data do not automate improved driving
14:39yeah it's quite the opposite actually
14:42yeah it makes it worse
14:45um that's why the data curation is
14:49quite difficult and I should say there
14:51is quite a bit of software around
14:54what data you know so selecting what
14:56data to train the system yeah so
14:58software that runs in the car is minimal
15:00but the software in the back end to
15:02train is like much larger and much more
15:04sophisticated yeah exactly so we do we
15:10you know C plus plus basically for
15:12python for deciding what data to select
15:19figuring out what what is the high
15:24the pretty good data
15:27and once we have a model we have to ship
15:29those models and Shadow mode to the cars
15:31and then every time it disagrees with
15:32what the user did yeah it's not going to
15:34get the data back and then you know kind
15:37that is more valuable than just
15:39collecting you know random data yeah
15:49yeah so we feel good about actually
15:53a very rapid virtuous cycle
15:56where when there is an intervention in
16:02um that with that intervention
16:04being uploaded uh to it to training
16:08um being integrated with training and
16:11then updating really just the the
16:19it's not the binary that's that's
16:20changing it's the weight not the
16:22execution binary it's the just really
16:52so I have not intervened uh once
16:56and the drive has been water smooth
17:08you know again being being smart
17:09repetitive uh we're both repetitive
17:11about being repetitive in fact
17:13um but we have not programmed in the
17:16concept of traffic lights
17:18there's not like uh this is a red light
17:22this is a green light and this is the
17:24traffic light position we have that in
17:27the normal stack but we do not have that
17:31this is just video video training
17:36like I said nothing but uh neural Nets
17:41and yet it knows which light
17:46and it stops at a red light
17:48accelerates the green light
17:50um now one of the sort of very slightly
17:53funny challenges we've had is that
17:57since the car is being trained on what
17:59humans do humans almost never stop fully
18:02at a stop stop treat if so when they get
18:05to a stop sign humans actually almost
18:08never go to zero miles an hour they they
18:11may think they did but usually they
18:13they're doing at least
18:15um a few miles an hour
18:17at a stop coming up to a stop Street
18:22sometimes you know people go faster than
18:25The Regulators or somewhat they're
18:28really quite insistent that we
18:30we go to uh come to a complete stop at
18:36and when we looked at the data uh only
18:39five percent of the time do humans
18:41actually stop fully even lower than that
18:45point five percent wow okay so basically
18:48people almost never fully stop at stop
18:55um even even if we look at those data
18:57it's a lot of them come to a stop and
18:58I'll get on their phone or something you
19:00know yeah it's not just regular people
19:02doing the stops yeah they might like
19:04Sammy stop and then move a little bit
19:06and that kind of thing so so we had to
19:09like uh pull the fleet for rare examples
19:13less than one percent of the time when
19:16people actually come to a full stop and
19:18artificially train the system to stop at
19:21stop signs um at the insistence of The
19:45like I said this is it's a little slow
19:49we're driving around and basically Rush
19:51Hour oh intervention sorry
19:55okay so that's our first intervention
20:00because the car should be going straight
20:04yeah so this model has a small
20:05regression and there's nothing lights
20:10but you know that's why we've got our
20:12releases the public yet
20:15so an intervention at this traffic light
20:18that's the first interaction in the
20:42yeah so just it just did a merge traffic
21:10so for that intervention that we just
21:12had the solution is essentially
21:18feed the network a bunch more video of
21:24so that was a that was a controlled left
21:25inner controlled uh left turn
21:29where there was green light for the left
21:31hand but not a green light to go
21:33um and so we'll featured a bunch of
21:36the control left tones and they don't
21:51in the next two weeks we're going to
21:53leave the shadow mode release where
21:54we're gonna run this network in the
21:56background and then check when for
21:58example in this case we would have
22:00wanted to go but then the driver would
22:02so we can just like check that okay we
22:04like and then we can get the data back
22:07into the neighbors and the neighbors say
22:08who was correct yeah yeah and that that
22:14but like we should not have gone so you
22:15don't have to even get the information
22:16you can just like passively observe what
22:19they want to do and this is what
22:42another controlled left uh traffic
22:46intersection although it's given both
22:48greens it's not kind of an easy one
23:00this the smoothness of the control
23:02like the car though just how smoothly
23:05the car is behaving is it's hard to
23:06convey I think on camera but I think
23:10super smooth yeah you have to feel it
23:14starts making a left turn
23:20so it's it's gotten itself for getting
23:22to the left turn lane
23:24again we've never programmed in the
23:26notion of a tone lane or anything like
23:28that or even a lean yeah it doesn't even
23:31know what we've never there's no line
23:33that says uh I think that it had there's
23:36no line of code about traffic Lanes at
23:45internal to its mind it might know all
23:47these Concepts you know how we think
23:52because it's not explicitly asked for it
23:55just like humans yeah you know
24:07Aaron we've arrived at this random PIN
24:09we might try to pull over let's see if
24:14sometimes it does blower
24:17yeah so it kind of so this is yes this
24:19is pretty cool so the car just just
24:21pulled over to the side of the road and
24:27so it would it knows at the end of its
24:31destination based on the video that's
24:33received that at the end at the end of
24:35the destination you pull over to the
24:37side and park yeah so it gets the exact
24:39pin location in addition to the
24:42you know just been as close is a good
24:46here it puts over here but in partly
24:48parking lots but that might not be any
24:49map yeah you should just go as close to
24:52the pin as possible Right
24:55or anything like that yeah in fact uh in
24:59it's a robot actually world it would
25:03you know probably perhaps know what you
25:05look like and say and just literally
25:07look for you yeah yeah if you have a
25:09picture or something you can just yeah
25:10exactly it's like if you sign up and
25:12it's like you know just said you don't
25:15have to but if you wanted the car to
25:17literally find you yeah so get you just
25:20have to send it a picture of you and I
25:23it will look for you and put and and
25:25wait for you yeah exactly and then you
25:27can also say drop me off at the
25:28Starbucks or something yeah you should
25:30drop you off at the building's entrance
25:31yes the Starbucks as possible as opposed
25:34to somewhere random you know yeah
25:37I guess let's see we'll
25:40just probably head back to HQ
25:44or it tells you you're under the fight
25:49wait where does he live
26:02I mean you know we can knock in the door
26:06of whatever Google says they say hi I
26:09guess you'll say I will say hi we'll be
26:13well it's a supplied inquiry as to
26:16whether you would like to engage in
26:23yeah you know not inconvenient
26:27perhaps you would like to
26:33okay so this is we have no this is
26:36literally we just Googled it we don't
26:38know if this is where he actually lives
26:39or not but we'll just go there
26:57we're at least going to where Google
26:58says uh you know Zuckerberg lives
27:02um you know I don't think this is really
27:04you know it can't be really considered
27:05doxing if we just Googled it as a
27:13so we'll see if it's the drive correctly
27:16to where Google thinks you live
27:56so doing a pretty good job of driving
27:57through the Palo Alto
28:10even this beat is just everything is
28:11automatic right it's gonna stop yeah
28:15it's off to the red line yeah
28:20okay pretty much what a person would do
28:26some cyclists over there
28:41and possible really is a
28:43lovely town it's it feels like um
28:48it's best for families yeah
28:52it's like everything's perfect
29:09I'll turn it to let's try and see in the
29:13yeah there's no Lane drifting it's super
29:22and it doesn't confuse a bike lane with
29:24a real lane or anything like that
29:30yeah this is the rider super comfortable
29:32as I see messages popping up I'll try to
29:34answer them that was a lot of messages
29:45so now it's gone into the
30:00headed to like Edgewood Drive I mean we
30:02don't know if this is actually realized
30:08I'm probably not because it's like I
30:10would expect it to be like a lot of
30:15um we are at the spot that's roughly
30:20but I don't think it just doesn't seem
30:22like probably realized because
30:25it'll probably be the security and stuff
30:34full of nice driving around Palo Alto
31:08I mean Zuckerberg did say like
31:10name the date in bold letters or
31:15you know whatever platform he's on and
31:18so I'm like hey right about right now so
31:23it's just does now work
31:29all right well we couldn't find him so
31:30we are heading back to Tesla
31:34the pedestrians or something
31:38really cautious about the pedestrians
31:47might be a little weird
31:58cars is very very polite with
32:04stuff to eat like for the couple to pass
32:06by and now it's continuing to drive
32:19so here we are in Palo Alto driving on
32:42it's headed back to Tesla Global
32:45Engineering headquarters at Palo Alto
33:17let's see how does it react to low
33:18visibility conditions was one of the
33:29mug says we were expected to like slows
33:32down and drain and then rides at lower
33:33speed and for our total intersections it
33:36kind of like cross forward and then
33:41that's one of these areas that we're
33:42trying to you know improve even more
33:47yeah yeah and one of the reasons we we
33:50training from all around the world is
33:52that the the weather in California is
33:54amazing and it's a you know
33:58song goes like basically never raised in
34:00California it's like sunny and nice
34:05um the divers are pretty nice too yeah
34:06right did you drive us here are very
34:11we need some aggressive drivers and also
34:16yeah it's like like we need uh you know
34:19situations where like there's a parade
34:20or a crowded situation or whatever a lot
34:24of pedestrians for whatever reason
34:27you're gonna be you know safe but also
34:29confident and be complete and enjoy we
34:33doing like hit the brakes all the time
34:38yeah but like like so
34:42like it's winter basically in New
34:44Zealand and so we have like this so we
34:47can conditions there that we can train
34:52there's a bicycle it's a little tight
35:20it is very conservative with bicyclists
35:23and pedestrians so generally yeah
35:44interesting this is like an unprotected
35:46done for us yeah okay so this is a
35:49tricky one so this is turning left onto
35:52Middlefield and Palo Alto with where
35:55visibility is not great
36:01the chords come from both sides at
36:07yeah no problem great
36:11so I'm unprotected left onto a
36:13high-speed Road fairly high speed Road
36:32yeah so V12 will be I would say actually
36:41end someone almost because you will make
36:43someone to be like so much like the
36:45first summer was okay but you know
36:46someone is gonna be like
36:56it's like dude don't you want some asaps
37:13speed up is quite nice too yeah exactly
37:17very very intuitive smooth speed up and
37:21acceleration in turns
37:30set speed is just enough max value yeah
37:33exactly it's like it currently set to 85
37:35but it's the it's it's ignoring the set
37:38speed it's driving at what would be
37:40intuitively the right speed for people
37:44the master Lane exactly so
37:47there's two lanes here uh there's a lot
37:50more cars in that lane fewer cars in
37:52this Lane and it's going straight so it
37:54picked Lane with the fewest cars
38:54in the vehicle following it today you
38:56know the distance and the velocity and
38:58everything acceleration that's nothing
39:01yep exactly it's not no explicit
39:03distance that's programmed in for how
39:06close you should be behind a car it's
39:09just a video training yeah
39:14and the right following speed like what
39:16would humans generally do uh
39:19and it picks like a reasonable follow
39:21distance and does that yeah the nice
39:23thing is you know for bad weather
39:24conditions for example it will
39:25automatically increase the speed yep
39:28yeah increase the distance yeah distance
39:43this traffic like takes a while oh yeah
39:48yeah El Camino and Page will
39:51a classic uh Silicon Valley intersection
39:54I've seen this intersection of
40:02these are red quarters is the HP it used
40:04to be the HP yeah yeah exactly but Tesla
40:08Global Engineering headquarters in Palo
40:10Alto are the former head of
40:13Hewlett-Packard headquarters
40:18it's an honor to be for our Global
40:22Engineering headquarters to be the you
40:24know kind of where the good place where
40:30so as you can see lovely place
41:11well actually we'll we'll see how it
41:13does in the parking lot so because
41:15parking lots are complicated especially
41:17the Tesla parking lot which is
41:20um so probably pretty full even on a
41:24at seven o'clock on Friday
41:33yeah this is a fairly tricky flight so
41:39you know controlled lefts and into
41:43straight basically two two turn lanes
41:46and two straight Lanes
42:01and it's almost like merge after this
42:02done which is also interesting I'm
42:04guessing it's like Dylan and Marshall
42:05yeah it's gonna be exactly it's gonna
42:07turn and merge ABS simultaneously which
42:54now one of the interesting things about
42:56it's a pure AI driving is it actually
42:59doesn't need a map at all so we could
43:02delete the navigation system simply give
43:05it a GPS point and say get to this GPS
43:09we're not going to tell you how it's you
43:11can tell like you see that building in
43:13the distance go there
43:15and it would it would do that even with
43:17no it would just it might make some you
43:23go down the road that's a dead end and
43:24then have to reverse out but you're
43:26basically able to do what a human can do
43:27where if you said please go to you know
43:34go there so that's going into the
43:39there is no explicit map of the parking
43:41lot parking lot so now it is just
43:44just trying to get to a GPS point
44:09yeah they're putting new superchargers
44:34all right so that was the
44:41um one Intervention which will fix with
44:43a bit more training data
44:46um and um otherwise uh
44:49I really have to like you know if this
44:51was a Uber driver pretty much apart from
44:54that one intervention five star yep so
44:56all right thanks everyone for tuning in
