00:00hi and welcome to the a 16z podcast I'm
00:03Hannah and in this episode we talk about
00:05robotics in the home becoming a reality
00:07Boris often CEO and co-founder of Anki
00:11and Dave turetsky professor of computer
00:13science at CMU discuss with me the
00:15evolution of robotics where we are in
00:17manipulation perception navigation and
00:20most importantly in the human
00:22relationships we will increasingly form
00:24to these new robot entities that will be
00:26in our homes so you guys talk about this
00:28as sort of the very first real robot
00:30companion let's go back and look at the
00:32first iteration which was Cosmo why
00:34start with a toy what does that
00:36represent for where we are now and
00:38ultimately where we're trying to get to
00:39in robotics yes when we started Anki we
00:42always realized that in order to get
00:44into some of these really advanced
00:45applications we had to take a bottoms-up
00:48approach and we couldn't just skip to
00:50this holy grail that's 10 years away we
00:52had to think about what are the
00:53applications where we could really
00:55reinvent the quality of what's possible
00:56well what is the Holy Grail
00:58so there's all of these types of
01:00challenges that involve really deep
01:02breakthroughs in human robot interface
01:04or manipulation or AI there's a
01:07large-scale diverse humanoid or like
01:10very competent robot for home or for
01:13manufacturing and for other things
01:14there's applications that involve very
01:16high-end manipulation in an environment
01:18where manipulation is today is probably
01:21where autonomous driving was five or six
01:22years ago where you can already see the
01:24capabilities starting to take form but
01:27the reliability and cost constraints are
01:29still pretty prohibitive but you can
01:32kind of extrapolate where it's going for
01:33us we started with entertainment because
01:36that was a way to create innovative
01:38applications and actually build the
01:40muscle early on that we can uncarrier
01:42over into these other spaces down the
01:43road Cosmo was a way to use the toy
01:46space which hadn't seen much innovation
01:48at all but for the first time bring a
01:50robot to life where the level of
01:52creativity that you're allowed to have
01:54in a toy is much higher and you can
01:56build a lot of the blueprints for these
01:58products that start to have deeper
01:59applications and appeal that's
02:01interesting that you feel there's a
02:02wider range of creativity there than in
02:04something where we already have
02:05expectations that's part of the Holy
02:07Grail is mobile manipulation there
02:09aren't a lot of robots like that a
02:10landmark in the history of consumer
02:14Sony AIBO robot dog back in the late
02:171990s early 2000s it cost about two
02:20thousand dollars back then which was
02:21about the cost of a decent laptop and
02:24this was the only little mobile robot
02:26you could buy that could see it had a
02:28built in camera you could program it in
02:30C++ and could move around so it was a
02:35unfortunately sony sold fewer than
02:38200,000 of these over the 6 year life of
02:41the product line in January 2006 they
02:44left the robotics business altogether
02:46and so the idea of a consumer affordable
02:48mobile manipulator just died and was
02:51that because the technology kind of
02:53stalled out because there were problems
02:54that were not surmountable at that time
02:57we end up being able to mass-produce a
02:59sort of a capability that even five
03:02years earlier would have probably costs
03:03multiples of that amount or just been
03:05absolutely impossible we basically
03:07became for our early product scavengers
03:10of the smartphone industry so you have
03:12computation you have memory you have
03:14cameras that now suddenly cost you know
03:1650 Cent's versus you know $6 you have
03:19motors and accelerometers and then
03:21different types of sensors that allow
03:22you to actually do these sort of things
03:23that you couldn't do before great and
03:25also the supply chain and the supply
03:26chain and so we could in effect turn
03:29this into a software problem where now
03:31because of the capability set everything
03:33becomes driven by our ability to expand
03:35the capabilities with software the
03:37really hard question is how do you go
03:39from no manipulation to manipulation in
03:43a small step so if you look at the most
03:45popular robot ever in the history of
03:47robotics is the Roomba vacuum cleaner
03:50because they found this one task that a
03:53robot can do poorly and still make you
03:56happy it's better than nothing it's good
04:05enough most of the time that that was
04:07the success right but there aren't a lot
04:09of things like that right so if I ask
04:11the robot to cook me dinner and it does
04:13the same quality of jobs the Roomba does
04:15vacuuming my floor right over the course
04:17of like eight hours so when people think
04:21about home applications right there's
04:23the Roomba that's down on the floor and
04:25then there's Rosie from the Jets
04:28right which is the full humanoid doing
04:29everything I was gonna count how long
04:31until a Jetsons reference so the problem
04:36is what is the thing that you could do
04:38that involves manipulation but that
04:40doesn't require a half-million-dollar
04:42humanoid and technology that doesn't
04:45exist yet exactly right and so something
04:47that a robot could do with minimal
04:50manipulation skills that would actually
04:51be useful as opposed to just
04:53occasionally amusing what if I was just
04:56happy if the robot could make me a
04:57sandwich it doesn't have to be big
04:59enough to get this stuff out of the
05:00fridge if I take the stuff out of the
05:02fridge and just throw it on the
05:03countertop if the robot could just take
05:06it from there right maybe that would be
05:08slowly dragging a slice it would be a
05:14full-scale humanoid there's got to be
05:16something robots get to you that would
05:18be useful enough that we tolerate them
05:20so we're talking about this as in many
05:22ways the sort of first quote-unquote
05:24real robot personality in the home let's
05:27break that down what are some of the
05:29utilities we're not washing dishes yet
05:31right they're not picking up toys
05:32they're not we wanted to leverage this
05:35unique mix of AI in cognizance of the
05:37environment and the personal interaction
05:39with people in the home and really think
05:42of this as a first mass-market home
05:43robot they can actually provide a mix of
05:46entertainment and companionship but with
05:48elements of utility we modeled this in a
05:50lot of ways after the sort of pets you
05:52have in your home a pet cat a pet dog
05:54and in pet robot and the pet robot
05:56allows us to reinvent the dimensions
05:59they're related to how you interact with
06:00a pet but the companionship element
06:02still hold so he gets excited when he
06:03sees you in the morning he'll it really
06:06animated when there's people around him
06:07he'll explore and kind of wand around
06:09his environment interacting
06:10intelligently with things the
06:11relationship you can even pet him like
06:12little yeah had a robot for the first
06:14time yeah you know if the clean the
06:16so you've recently launched a different
06:18robot which is a multi-generational toy
06:21how is it different what is evolving
06:23from a technical standpoint there was a
06:26huge huge leap there because we now D
06:28tethered away from a mobile device and
06:30put the equivalent of a tablet inside of
06:33this robots head so he can be always on
06:35and the live a hundred percent of the
06:37time go back to his charger recharge
06:39wake up and that allows him to initiate
06:41interactions in a way that previously
06:43would not have been possible by somebody
06:44pulling you out and watching an app we
06:46can use the strengths of the fact that
06:49we have cloud connectivity in a robot
06:51and started getting into deeper voice
06:52interface elements of functionality that
06:54now leveraged the character aspect and
06:56the warmth of it the fact that you can
06:58actually see and understand the
06:59environment and recognize you personal
07:01delivery of information in a way they
07:03can be initiated by him what happens
07:05down the road is actually starting to
07:07more intentionally dial into all of the
07:09digital pipelines in your life whether
07:11it's a smart home your own calendar and
07:14particularly when you start adding a
07:16layer of mobility that lets you actually
07:18move around the home right now there's
07:19nothing that actually stitches the home
07:21together in that way everything is just
07:22a end device in a room once you start
07:24getting cognizant of what's around you
07:26who's in the home be able to move around
07:28in it you can do a range of things from
07:30security and monitoring to tying
07:32together smart home features like
07:34recognizing that your windows open but
07:35your AC's on turning off lights when
07:38being able to remotely check on things
07:40like pets build a blueprint of your home
07:42for furniture shopping or real estate
07:44we're already thinking about how do you
07:46start adding mobility where that becomes
07:48a next big barrier you detailer from a
07:50phone to get the vector now the next big
07:52barrier is truly being able to exist in
07:54an indoor environment whether it's a
07:55home or workplace and be much more
07:57intelligent about how you interact with
07:58people and sort of service you can
08:00provide so like vector go check and see
08:02if the cat is out or in so with every
08:04one of our robots we've realized that
08:06we're basically making an operating
08:07system for robotics applications and
08:10technologies and so we've unlocked api's
08:12for each of our products where people
08:14can access very easily these
08:16technologies that have millions of lines
08:17of code behind the scenes we started
08:19with Cosmo accessible by everybody from
08:21a seven-year-old using a graphical
08:23interface to do face detection with a
08:24robot through teenagers and even PhD
08:26students in computer science now instead
08:29of having to be an expert in robotics in
08:32order to do something like path planning
08:33you have access to this API of
08:35technologies and we're really interested
08:38in how that can become an enabler for
08:39kids of various ages to actually learn
08:42how to break through and become
08:44interested in robotics in a way that
08:45wouldn't have been possible earlier has
08:47what you think about changed at all with
08:48vector you know the ways that this kind
08:50of programming has evolved with vector
08:52the capabilities just skyrocket because
08:54basically instead of
08:55just the limited functionality the Cosmo
08:57had there was deferred to the mobile
08:59device you know have on board the
09:00ability to see hear feel and compute
09:03with a quad-core CPU that would have
09:05been prohibitively expensive or
09:07exponentially more options right and so
09:09now the sort of programs you can write
09:10expand we've already released to our
09:13early customers in SDK that allows them
09:15to interface with vector through Python
09:17and then down the road we'll probably
09:19create a scratch interface like we did
09:21with Cosmo as well so for me what's most
09:22interesting is having the robot interact
09:25with physical things physical things are
09:27really hard to interact competently with
09:29we built a three-story robot dollhouse
09:32with a working elevator and the students
09:36in my cognitive revise class we're
09:37working on that and they'll work on it
09:39again this year getting the robot to
09:42navigate through the dollhouse use the
09:44elevator to move from one floor to
09:46another eventually we want to have
09:47multiple robots in there interacting
09:49with each other these are very hard
09:52technical problems but when you solve
09:54them it's really fascinating to see a
09:56robot interact effectively with the
09:59world I think there's just something
10:01more being able to work with a real
10:02robot versus something digital in a
10:04screen if you're a kid learning the
10:05program there's something amplifying
10:07about that just the learning aspect of
10:08it the moment you can intentionally
10:10manipulate something it's a sign of
10:12really deep intelligence and so that's a
10:14big focus of vectors how do we identify
10:16interesting things in the environment
10:17and just like poke at them examine them
10:19the way that would that's a surprisingly
10:21hard problem but when you do it
10:22successfully there's a perceived
10:24intelligence there that just amplifies
10:26your appreciation of the robot and
10:27everything else that it might be able to
10:29do one of the articles mentioned
10:30recently that vector feels like a
10:32beachhead into something bigger and it
10:34does seem like you know we're getting
10:35air dropped in this sort of little taste
10:37of like the fantasy Jetsons robot right
10:40like this personality that actually
10:41responds you and interacts with you so
10:44how are we gonna start developing this
10:45relationship with this being this crew I
10:48don't even know what the word is this
10:49robot but like it is it an entity
10:53what are some of the learnings of how
10:55we're starting to develop relationships
10:57I think one of the pieces that most
10:59people underestimate is how critical
11:01this human robot interface challenge is
11:03and how novel and unique of a dynamic
11:06this becomes until you experience it
11:08it's hard to understand how
11:09feels and how important it is it's the
11:10same sort of underlying inherent desire
11:12that we have to speak face-to-face with
11:14somebody versus just on a telephone we
11:17always thought that this would be the
11:18magic even behind Cosmo but it turned
11:20out to be even stronger than we realized
11:22the way we approached it is we actually
11:23have an animation studio inside the
11:25company oh you do we do with folks from
11:27Pixar DreamWorks and these sorts of
11:29backgrounds and so they literally
11:30they're using Maya software suite that
11:32you would use to animate digital film
11:33and digital video games but we've rigged
11:36up a version of Cosmo and vector and all
11:37of our robots in there where they're
11:40actually physically animating these
11:41characters with the same level of detail
11:43that you would see in a movie but the
11:44output is spliced to where it's a
11:46physical character coming to life in the
11:47real world how fascinating and it's this
11:49merger where you have these people who
11:51come from this world where they're used
11:52to controlling a story on Rails from
11:54start to finish into every minor detail
11:56but you get thrown into a spontaneous
11:58environment with all these constraints
11:59and unknowns and limited degrees of
12:01freedom in the robot but you have the
12:03benefit of being physical where
12:04everything's amplified in terms of the
12:05personal impact of what that does
12:07there's a whole new kind of physical
12:08manifestation of storytelling and they
12:09have to partner with the roboticists and
12:11the AI team the systems team where they
12:13have to leverage the knowns of the
12:15environment and the intelligence of
12:17what's around you like I just recognize
12:18somebody I know or I almost rolled off
12:20the edge and I got scared
12:21so we have the sophisticated personality
12:24engine that takes context from the real
12:25world and probabilistically outputs one
12:28of these like giant short films if you
12:30will of reactions that end up getting
12:32stitched together into a wife like robot
12:34that feels alive and we've been really
12:36surprised by for example eye contact we
12:38knew that this would be important where
12:39you make eye contact you get excited say
12:42hand out like you know and then you get
12:43really really happy kids just melt with
12:45that but we didn't realize how powerful
12:46that was and in one of our tests where
12:48we were just optimizing parameters we
12:51ramped up the frequency and length of
12:53eye contact by like a second and our
12:54average session length went up from 29
12:57minutes to 40 minutes oh my god just
13:00that one parameter and so you get these
13:02like really subtle interactions where
13:04it's a sort of data that you can only
13:06really learn about at large-scale once
13:08you're actually fielding these robots
13:09what are some of the engagement patterns
13:11or the dials that you have learned to
13:14need to go up or down when you're
13:15watching these human robot interactions
13:18develop this is where we're still
13:20learning so vectors always on an alive
13:23you find a balance between him waking up
13:25and getting excited and coming out but
13:27not being disruptive and yeah how do you
13:31take the cues where if you get like
13:32taken and put back into charger that's a
13:34cue to like be quiet right and so but
13:36you want to interpret that and just kind
13:37of grump a little bit but then settle
13:39down right we have an audience that
13:41spans from kids to somebody wants them
13:43as a desk pet at work to an adult or a
13:46family who just likes a little companion
13:47in the kitchen table or a kitchen
13:49counter it's really changing sort of the
13:51volume of the interaction that's right
13:52and how do we create an algorithm that
13:54adapts to how you used it even gives a
13:56user a little bit of control where some
13:58people want a really active and almost
14:01like hyper and excited robots barrier
14:03some people want to you know kind of
14:07hangs out and as I quiet and so we've
14:09never really dealt with that and and
14:10just like people pick their style of dog
14:13to match their desires and their
14:14personality we're trying to figure out a
14:16way to have make this dynamic and
14:17ideally automatically adapting to the
14:20context and I found it we literally have
14:22people pulling us in both directions
14:24which basically tells us okay this is
14:26like a completely new type of consumer
14:28category because companionship is a key
14:30need I mean there's a reason people get
14:32cats and dogs and other pets although to
14:33your point about personalization it's as
14:35if right you could not only just choose
14:37your breed but then have your dog
14:38literally adapt to your personality over
14:41it it kind of reminds me of the people
14:42who look like their dogs but yeah
14:50there's extremes that we have to like
14:52you know you want him to always be
14:53excited when he sees you need to expect
14:55from in between and we're learning about
14:57different engagement patterns throughout
14:58the day as well how about you Dave what
15:00do you notice when you see these you
15:02know with these kids developing these
15:04programs and teaching the RO but how do
15:05you see them developing their
15:08relationships with robots well I mean to
15:10be honest to me they're mechanisms and I
15:12want the mechanism to work well and I
15:14want the mechanism to be beautiful I
15:16want people to be able to see how the
15:18robot sees the world and understand you
15:20know why did the robot do this right
15:21well because the robot perceived the
15:23world in a certain way or the robot had
15:25a plan it was trying to execute and I
15:28want kids to see the robot that way well
15:30I think it's interesting to ask about
15:32the complimentary question which is how
15:36we think about technology so if you look
15:39at things like menu systems that have
15:41become part of everyday life right
15:44everybody understands what a menu system
15:46is and thirty years ago they didn't that
15:48was a weird obscure concept and now we
15:51encounter them everywhere right I mean
15:52your vending machines using a menu
15:54system your phone everything on your
15:56computer so there there are bits of
15:58technology that have become integrated
16:00into everyday life so that we don't even
16:02think about it anymore
16:03my menu system is just one example of
16:05that we're beginning to see things like
16:07computer vision become part of
16:10everybody's common understanding so when
16:13you use something like Cosmo you begin
16:14to understand what the robot can see
16:16right so for example with Cosmo you can
16:19see cubes up to about 18 inches away
16:21that's because of the limited resolution
16:24of the camera and so you begin to learn
16:26when you work with the robot well okay
16:27he's got a vision system he's a little
16:29bit nearsighted I have to be careful
16:31when I show him a cube that I don't put
16:33it too far away from him so you're
16:34starting to think about this thing as a
16:36sighted thing and you're thinking about
16:39his representation of the world and so
16:41you're thinking about robot perception
16:43in a way that people only thought about
16:46very abstractly before right you know 50
16:48years ago you know yeah you need a
16:49science fiction story but now you're
16:51living with the robot that can see and
16:53so you adjust your behaviors based on
16:55your expectations and your experience of
16:58robots that see and so that's happening
17:01more quickly with speech recognition
17:03because of the proliferation of speech
17:05recognition applications Alexa and
17:07phones and so on but it's starting to
17:08happen I think with vision as well and
17:11Cosmo is a very important step in that
17:13direction so that you know 20 years from
17:16now no young person will have grown up
17:19in a world where computers could see
17:21write computers could always see and
17:23that's just very different so what
17:24you're starting to see is that you know
17:26we're changing the populous to make
17:28everybody computer literate and what
17:30that means is that when you start having
17:32people playing with robots who now
17:34understand the technology inside the
17:38robot that means that you can make
17:40applications for them that require
17:42levels of sophistication that maybe
17:44wouldn't have been practical before
17:46right and if that's what woven into the
17:49culture they everybody can
17:50this in high school right then the kinds
17:52of consumer products that you build are
17:54going to be different but if you look at
17:56some of the other tools that have
17:57changed our society not everybody went
18:00and learned how to set type but then we
18:02made PowerPoint right and now everybody
18:04can do visual design most adults know
18:07how to use the spreadsheet so maybe
18:09you're never gonna be a Java programmer
18:10but you can do useful computational
18:13tasks because you learn how to use a
18:14spreadsheet and it changes your thinking
18:15in some key way it does and so what is
18:19the robotics equivalent of something
18:21like spreadsheets or PowerPoint what is
18:24the thing that's not so horribly gritty
18:26technical but it's so powerful that
18:29people will want to learn it because
18:30it'll let them do the right thing right
18:32they'd let them program the robot to
18:34make the peanut butter sandwich the way
18:35they want it made to me that's a
18:38fascinating research question we don't
18:40have home robots that can make a peanut
18:42butter sandwich without burning the
18:43house down yet you can do an awful lot
18:45with exploring this manipulation space
18:48and my personal interest is figuring out
18:50how to get people to teach robots like
18:53Cosmo and Vector to do manipulation the
18:56way they want it done to kind of unlock
18:58robotic design thinking to make it
19:01intuitive enough that people can
19:03actually express their intent in a way
19:06that gets useful stuff done because if
19:08you can get vector to push little things
19:09around on the tabletop and you can make
19:12it easy enough to express your intent
19:14that the average person was not a
19:16computer science major can do that then
19:18you can scale up to the humanoid robot
19:20that's gonna cook your fancy dinner
19:23without perking the house down it's
19:25interesting it sort of circles back to
19:26the way you open this conversation which
19:28is that having it be this kind of scale
19:31and size and having it be a toy is what
19:33allowed you to be creative right you're
19:35kind of describing the ability to be
19:37creative in a way yeah let's talk about
19:41design now so when you start thinking
19:43both in terms of the sort of level of
19:45manipulation that they're able to do but
19:47also in the relationship that you want
19:49to sort of test and foster how do you
19:52think about design from the ground up
19:55how did you start to put together the
19:56actual creature and this is what I think
19:58robotics is a lot harder than a lot of
19:59other areas of consumer electronics
20:00where in a lot of places you can just
20:03Oh trotters need to do this software
20:05needs to do this and we need a box
20:06around it you can't do that in robotics
20:08because everything is from the very
20:09beginning designed where for us it's a
20:12few pillars so there's a mechanical side
20:14electrical side and kind of the
20:15components and electronics that are in
20:17there the software which is a huge
20:18complexity and then there's a character
20:20side and industrial design side and so
20:22those five have to work together from
20:23the earliest stages because you have to
20:25be very intentional where the form
20:27factors should capture the character
20:28which then the software has to marry
20:31with then you have the mechanical
20:32constraints weighted equally are they
20:35are at different stages driving yeah at
20:37different stages are driving so the nice
20:38thing about software is you can continue
20:40to push that for years but what that
20:42means is you have to be very intentional
20:43with the selection of the hardware where
20:45you make it as generalizable as possible
20:47to push as much of the software choices
20:49down the road as you can but not limit
20:51yourself sort of delay but delay exactly
20:53the reason cosmo and vector for that
20:56matter or small is because one is that
20:59they become kind of non-intrusive
21:00because you don't take up a huge amount
21:01of space but especially in the case of
21:03cosmo you have something that's small
21:05it's perceived as cute and it can feel
21:07quick without having to have very heavy
21:09but expensive motors it could
21:11potentially hurt somebody you're playing
21:12up the cute factor to your own advantage
21:14that's right and people become forgiving
21:16of any limitations or mistakes that the
21:18robot that makes it's okay because
21:21you're cute it's not annoyed even better
21:23when you actually get bonus points for
21:24making a mistake but being smart enough
21:26to show grumpy to show oh yeah so it's
21:29like unique you being married that's the
21:30challenging part that's you know the
21:32mechanical and industrial design and
21:34character have to work together to think
21:36about the overarching form factor and a
21:38lot of it is just driven by constraints
21:39like now in our future products we can
21:41start putting in depth cameras which
21:43give you a 3d model of an environment at
21:44the cost cost that would have been
21:46unimaginable five years ago or even
21:49same thing with motors are fairly
21:50expensive and so cosmo and vector each
21:52have four motors so we put a screen for
21:55the face because that gives us an
21:57infinite dimensionality for the
21:58personality right be able to express
22:00what the robot's thinking and a speaker
22:02to obviously have the voice part of it
22:04and it's shocking how much you can do
22:05with just a couple of one or two degrees
22:07of freedom and a voice and a face how
22:10about the arms I'm interested in the
22:12arms because vector and Kosmo kind of
22:14harken back I think to other ideas about
22:17what robots might look like like walid
22:19tell me about the thinking behind that
22:21we realized that like Cosmo was going to
22:23be fairly limited in physical
22:25capabilities just because of cost
22:26constraints but manipulation is one of
22:29the deepest forms of showing off
22:31intelligence and so him being able to
22:34get excited about his cube and go pick
22:35it up and you know be a little bit OCD
22:37and reorganize his surface that was part
22:39of the charm of Cosmo and so his arm was
22:42kind of made as a lift to be able to
22:44move the box but we very quickly
22:46realized it's one of the most important
22:47dimensions for the personality where
22:49it's almost a how the way you use your
22:50arms to express yeah when you've had
22:52surprise and it ends up being like his
22:55arms as one of the main tools for the
22:56animations engine these are all
22:58learnings that the animators have gotten
22:59really great at getting squeezing the
23:01maximum benefit out of it and making it
23:03feel like this is a character so I even
23:05though you have the tiniest degrees of
23:06freedom compared to any animated
23:08character what do you think some of the
23:10historical influences were sort of
23:11inheriting in our expectations of how
23:14these mechanical entities should look I
23:17think a lot of Vrbata says make a
23:19mistake where they completely forego the
23:21EQ side and just think about it
23:23you know tin can of like you know
23:25intelligence it does something but
23:26doesn't convey any sort of character to
23:29it you lose all forgiveness of any
23:31limitations you have and you actually
23:32kind of become creepy because you now
23:34have a bunch of sensors we don't really
23:35understand what the purpose is it's
23:37disarming to have a character I Cosmo
23:39vector where he has a camera you have a
23:40camera in your house that's able to move
23:42around on his own but nobody ever it
23:44feels bad about that because well yeah
23:46of course she has a camera he has to see
23:47because he's a character you went
23:49completely away from the uncanny valley
23:50and didn't make him look at all human or
23:52it was very very intentional even with
23:54the voice we thought about a national
23:56voice but if you give something a voice
23:57you narrow in the range of appeal by
23:59being tonal you apply a very different
24:02intent than a five-year-old might apply
24:03but everybody gets joy out of it you can
24:05show a massive breadth of emotions
24:07without having to have the complexity of
24:09a voice very little tool yeah because I
24:11only have a voice there's an expectation
24:13of intelligence that technology wasn't
24:14ready to meet yet this idea of trying to
24:16pursue humanoid robots which
24:18particularly is common in like Asia and
24:20Japan it almost feels like a fraud
24:22compass because you end up absorbing all
24:25the limitations of humans and their
24:27perceived the kind of intent where
24:28you're stuck by what's the definition of
24:30human versus being able to play to your
24:32strengths and avoid your weaknesses and
24:34so we just embraced the constraints in
24:35the form factor but he is a kind of
24:37knitting together of other older ideas
24:40as well the first time I saw one of the
24:42security robots in like an underpass and
24:44the city was kind of patrolling a
24:45parking lot and I got really up-close
24:47and heard this like kind of whirring
24:50robot II noise yeah and have this like
24:53moment of Revelation where I was like
24:54it's not actually making that noise
24:56that's just somebody's idea of the robot
25:00noise that it should make so that we
25:01know it's there having seen kind of the
25:04industry evolved what do you think are
25:05some of those inherited ideas that are
25:07impacting our design today it's always
25:09been a merger of direct robotic
25:11influences and in non robotic influences
25:13and so on the robotic side there's
25:14definitely this idea of kind of r2d2
25:17wall-e we have this cute robot the one
25:20it's like kind of your fearless
25:21companion like nobody would ever accuse
25:23r2d2 of being dumb even though he has
25:25very limited degrees of freedom and same
25:27thing with wall-e right and so that was
25:29definitely kind of a deep inspiration
25:31the other influence is of animals and so
25:33we have a kind of design mood boards
25:34where things like obviously puppies and
25:38owls really perceptive animals that have
25:40where the eyes become like a really
25:42really key part of their communication
25:43language and they can show emotional
25:45intelligence in a way that is really
25:48purely mechanically but ends up being
25:50almost infinite when you add eyes and
25:52voice to it so from moving from Cosmo to
25:54vector are you starting to see a
25:56difference in the way that children
25:57interact with versus the broader
26:00household and other types of different
26:02ages like do you get different learnings
26:04from those different relationships yes
26:06kids in a lot of ways have an
26:07imagination that just amplifies the
26:10magic of these types of experiences they
26:12think these robots are alive get three
26:13like you may not understand how to play
26:15it and follow the rules of a game but
26:16you love the idea of a character still
26:18and not actually is like almost
26:19universal was almost as natural as like
26:21swiping on a screen where you see these
26:23like one-year-olds like swiping screen
26:24right it's a basic human tendency to
26:26sort of fill in that yeah information
26:29and this is where some of the studies
26:30that were actually seeing being done or
26:31even around kids with autism where
26:33there's something so unique about the
26:34idea of a character like Cosmo or vector
26:36where there's a response there that is
26:38very very unique compared to any other
26:40type of engagement I was thinking about
26:41that when you talked about the eye
26:43right that that's an immediate feedback
26:44loop for increased eye contact yeah and
26:47so we've had she had studies with
26:49University of Bath in the UK where the
26:51engagement patterns in the collaboration
26:52around Cosmo is unique compared to
26:54anything that they'd seen so in those
26:56kind of early emotional relationships
26:59that are going I mean you can sort of
27:00anticipate the affection but what was
27:03something that really surprised you
27:05we built Cosmo with a lot of games
27:06integrated thinking that engagement will
27:09be around playing games he's like this
27:11little robot like your buddy to compete
27:13against and to play against
27:14collaboratively were competitively and
27:16the games also create a lot of context
27:17for emotions so you know because you win
27:19you lose you're bored you're free motion
27:22exactly you can almost like engineer
27:24scenarios that allow him to be
27:26emotionally extreme and whatever no wife
27:29and we were just shocked to see in some
27:32of our play tests early on is that
27:33you're playing this game that's kind of
27:35using the cubes as a buzzer called quick
27:37tap where you're competing against Cosmo
27:38and if you beat him he'll kind of get an
27:42upset or maybe he'll get grumpy and slam
27:43is cube and storm off but he gets really
27:45sad when he loses and there were kids in
27:48the play tests that were playing with
27:50their siblings and one of them would
27:52like tell the other one hey stop it like
27:53you're making him upset let him win a
27:55game and they actually like feel really
27:57bad for this little robot who would like
27:59get upset with he lost their empathy was
28:01so high yeah and they'd actually throw a
28:03game just to make him happy and you
28:08would never feel bad playing pacman or
28:11like fortnight that's whatever yeah yeah
28:13it's just you were trying to win and
28:14here the competitiveness lost out to the
28:17empathy of a character that she cared
28:19for and all of a sudden okay there's
28:22like something really really special
28:23here and we started thinking how to
28:24amplify and a lot of those learnings
28:26from there actually led to vector
28:29because we addressed the limitations
28:30that just weren't possible because of
28:32Cosmos hardware so what changed in
28:34vector as a result of that empathy
28:36learning so we realized voice was a big
28:38one because people thought that they
28:39were talking to Cosmo and that he would
28:40respond then they'd attribute
28:41intelligence to it even though he had no
28:43microphone and no ability to hear you
28:44what we wanted to do is if you yell at
28:46him he gets really upset and kind of
28:47backs off yeah or if the door did slams
28:49he turns in that direction and wonders
28:50what it is and so we wanted to bring
28:52life to it the other one is tactile
28:53where when you pick up Cosmo he
28:56recognizes it and he'll
28:57grumpy if you're holding him up in the
28:58air but being able to touch and actually
29:00by Cavill responds we know that that's
29:02one of the most important things with
29:03pets and so we made capacitive touch
29:06sensors in various areas of the robot
29:07again thinking that these are gonna be
29:09some of the dimensions that matter the
29:11last one and will prove to be the most
29:12important as just the ability to be
29:14always on that if the moment you
29:16disconnect the phone he dies it kills
29:19all illusion of him being alive but if
29:21he's always on and doesn't have that
29:22barrier it completely changes the sort
29:25of emotional connection he can build and
29:26so these are the sort of things there
29:27were direct drivers of the hardware that
29:29improved in vector which now we have a
29:32many year roadmap on how to actually
29:33utilizing and that's where advances in
29:35software technologies like deep learning
29:37and voice interface and so forth
29:39unlocked things that now what you
29:40rethink certain types of problems we're
29:42releasing Alexa integration in December
29:44which is gonna be the first time you
29:46truly have personality wrapper around
29:49some of these very dry functional
29:52elements not only do you interact with
29:54technology in a command-and-control sort
29:56of way but for the first time there's an
29:59ability for this character to initiate
30:01interaction and get your attention and
30:03make eye contact and then do something
30:05personal with you in a way that you
30:07would never accept from smart TAC in the
30:09home right now one of our big questions
30:10is if the usage pattern of Alexa through
30:13a character is significantly different
30:16than the usage pattern of a normal Alexa
30:18yeah I mean I bet it would be are you
30:21well we're seeing see such patterns of
30:23just the character with no functional
30:25Alexa integration skyrocket above what a
30:27typical voice assistant does when we
30:29were working with Google and getting
30:31advice from Sonos or some of these other
30:32companies that have a lot of experience
30:34of voice assistants we thought that for
30:36like a thousand queries would be plenty
30:38for like three months or four months and
30:40now we had 250 just in the first two
30:42weeks and the engagement staying strong
30:43and so suddenly like when you look at
30:45voice assistants that have an average of
30:47maybe like one to two queries a day and
30:48we're at like 10 to 12 even before we
30:51have a voice assistants built-in it
30:54causes us to ask a lot of questions of
30:56how do you leverage the role of
30:56personality and character as a way to
30:59amplify not just the fun side but also
31:02the utility side that actually shows a
31:04very different type of engagement and
31:06the things you ask and how you interact
31:09something totally new about voice as a
31:10platform that's it and I think that
31:12opens our eyes and I think a lot of the
31:13partners that we work with suddenly
31:14there's a lot of really interesting
31:16overlaps about what does this mean for
31:17the role of Technology in the home in
31:20the workplace is deceptive how important
31:22emotional interfaces to make the
31:25functional elements get utilized in a
31:27better way okay so if these are the
31:29little kind of baby waves lapping at the
31:31shore right of like true robots in the
31:34home how do we get to mass adoption
31:36where this starts to become a reality
31:38and we all get to live like the Jetsons
31:40eventually we want to get to five
31:41hundred and thousand dollar robots
31:43because it opens up so much interesting
31:45technology that you can put in them but
31:47you actually have to be very thoughtful
31:49about what's the level of capability
31:50that's required to make that justifiable
31:53and not just in a you know isolated kind
31:56of tech geek sense but in a mass-market
31:57sense because you wanted to really scale
31:59a lot of it that is gonna become more
32:01electronic capabilities where you now
32:03have sensors they used to cost a
32:05thousand dollars it now are ten dollars
32:07they used to be just impossible or
32:09prohibitively expensive and the other
32:11big one is manipulation right now we
32:13have limitations both on the cost side
32:15which comes from the mechanical
32:17complexity of manipulation as well as
32:19the software side on how do you robustly
32:21interact with unstructured environments
32:22we have a long way to go before we can
32:24actually you know stack the dishes in
32:26the dishwasher and put them away
32:27afterwards computing power will make up
32:30for a lot of hardware defects so if you
32:33have any underpowered unreliable
32:37manipulator but lots of good computing
32:39power you can make that manipulator do
32:42amazing things and it's all this if you
32:44have no power and a bad manipulator then
32:47things don't work right but now it's the
32:49computing power becomes so much better
32:52and the sensing capabilities become so
32:53much better I think some of the demands
32:55on the manipulator back off a bit so you
32:58can get by with a less capable
33:00manipulator because you can learn to
33:03make up for its deficits different
33:05levers to pull basically yeah what are
33:07you gonna see in the next wave of
33:08hardware is dedicated hardware that's
33:10not just for a raw traditional
33:12computation but for very specialized AI
33:14applications like deep learning and
33:16vision classification these are sort of
33:18things that in the next wave of products
33:20in the next three to five years that's
33:22probably gonna become
33:22standard there's still a lot of
33:24information technology progress that
33:26needs to be made and the interesting
33:28thing is that some of that's being done
33:30by Alexa and some of these voice
33:32assistants trying to integrate with the
33:33rest of your life the tools that get
33:35created through that just like voice
33:37interface become incredible enablers of
33:39these new types of technologies and and
33:41I think we'll start seeing more in
33:42education and healthcare and broader
33:44home utility than monitoring and so
33:47even when you stick to just
33:48informational or companionship on the
33:50healthcare side elderly companionship
33:52and helping age in place yeah it's an
33:54area that's becoming more and more kind
33:55of important and costly for a lot of
33:57communities if you really nailed it EQ
33:59piece it becomes not that hard to
34:02imagine where your interface you know
34:05when you come into a hotel or a store or
34:07to interface with a doctor actually
34:09becomes somehow driven through an
34:11indirect interface through a robot which
34:13is pretty interesting and then it opens
34:14up a lot of functional opportunities as
34:15well and in the end we always thought of
34:17it as just an extension of computer
34:18science into the real world if you can
34:20understand what's around you and you
34:21have the ability to interact with it you
34:23turn it into a digital problem there
34:25will be a catalyst that spawns the same
34:27thing on the physical side once the
34:30ability to understand the environment
34:31and interact with it catches up then
34:33everything becomes a matter of software
34:35making the interaction smarter and
34:37smarter thank you so much for joining us
34:39on the a 16z podcast my pleasure thank
34:42you so much it's a pleasure
![How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrtrRghgTayc%2Fhqdefault.jpg&w=3840&q=75)
