00:00hi this is the a 16z podcast on Kyle
00:02Russel a deal partner here at the firm
00:04looking at virtual reality and augmented
00:05reality among a few other things I'm
00:07joined today by two people who know a
00:09lot about VR I'll let them introduce
00:10themselves so you can put a name to a
00:12voice so if you'd like to start off hey
00:15i'm sakunthala partner I'm a research
00:18I'm gonna kill me search team here and
00:20Jason Horowitz I focus on VR and other
00:22amongst other emerging technologies
00:24before this I was an intern at oculus
00:27and hi my name is beau Cronin I have a
00:30background in computer science and
00:32perceptual and computational
00:33neuroscience and I've become
00:34particularly interested in VR recently
00:36because it allows us to in many ways
00:39expand our perception of both the real
00:41world and worlds that we get to make up
00:43from scratch so thanks you both for
00:47joining today I'm really excited to talk
00:49about you know kind of where VR is at
00:51and you know what aspects of it I think
00:54or maybe under considered by people who
00:56just kind of look at it as a raw set of
00:59technologies of things that track motion
01:00and our screens attached to someone's
01:03face I think that it's a lot of aspects
01:05where you know if you look at mobile or
01:08desktop computing in other ways that we
01:10use technologies aspects that are
01:12commonly discussed aren't even brought
01:14up sometimes when you looking at v8
01:15when you're looking at VR so one of the
01:19things I like to like kind of just check
01:20in on first is where VR is at today in
01:23terms of the technology and in the
01:25commercial space so recently Samsung and
01:29oculus released the gear VR Saku you
01:31actually used to work at oculus so maybe
01:33it'd be fun for you to talk about where
01:34the consumer platform is that today
01:37right so the big thing with oculus is
01:39that the VR headsets which had been
01:42around for maybe 10 15 years before that
01:44finally became affordable so and also
01:47usable so we got the right ratio of
01:49being we are high-quality enough and
01:51also it's reasonable for the average
01:53person to buy one so the oculus
01:56Kickstarter came out in 2012 was a
01:58massive success and then we've had two
02:01releases so far the dk1 which is a
02:04Kickstarter the dk2 which is like a
02:06second developer kit that oculus
02:07released and we're still waiting for the
02:09the fight of the consumer which is like
02:12the super high quality version
02:13just coming out next year and then to
02:16kind of give an idea of the evolution of
02:18this tag so dk1 basically is screen with
02:21motion sensors that you strap to your
02:22face and I gave you know something that
02:24felt like presence then with dk2 they
02:27took that same kind of core concept and
02:29added a camera that track your position
02:30that's not quite true
02:32so DK one in like what we say with
02:35presence is it's like a buzzword in VR
02:37to describe the visceral feeling of
02:39feeling you're actually in the place
02:40that the the screen says you are so the
02:44way I explain to people is if you're in
02:46a in the VR universe you're standing on
02:48the ledge you feel kind of scared that
02:49you're gonna fall off right and it's not
02:50like a psychological thing it's like you
02:52know physical reaction to the distance
02:55right your lizard brain reacted to input
02:57exactly and dk1 was not quite there I
03:01don't know look at all the videos of
03:03people food like go on a roller coaster
03:04and fall out of their seat okay maybe
03:08you know it's it was it was it
03:11definitely did hate your lizard brain
03:12but maybe didn't quite click presence
03:14but with dk2 and cby we definitely get
03:17to that point and I think one of the
03:19interesting things about each system
03:20that's come out DK 1 DK - now the gear
03:23VR which we'll probably get to in a
03:24second and then these prototype systems
03:26that we will also talk about that are
03:27going to be commercially available is
03:29that each one has kind of proven
03:31something additional about what this
03:33technology can do so I think the dk1 for
03:36a lot of people I've actually never used
03:37one but many people who have was the
03:40first time that they went oh my gosh you
03:42can do really high-quality tracking you
03:44know didn't have positional tracking but
03:45you can do really high quality low
03:47latency tracking the screen is good
03:48enough that I can imagine where it's
03:50going to go and it allowed the right
03:51kind of people to sort of have that leap
03:53where they said oh wow this is this is
03:56going to be real it's worth me devoting
03:58my time and energy to it DK - was beyond
04:00that now Sam with the gear VR you have
04:03something that's available for anyone to
04:05buy for $100 on top of the price of a
04:07Samsung phone that proves that this is
04:09something that people can enjoy in their
04:12own homes and so it's just each of these
04:13systems sort of taking it one step
04:15beyond bringing this to a broader and
04:17broader audience and making them realize
04:19this really is for real I think one
04:21thing to add is that the different types
04:23of tracking we have in VR so the dk1 has
04:26rotational tracking so
04:27you know if she turned your head it
04:29would it would be able to tell which
04:31directions it had turned but the dk2 had
04:35positional tracking which was like an
04:36outside camera so even if you were
04:38walking around and it was no longer
04:41anything to do with you something stuck
04:44to your head they could actually tell I
04:45think it was like a more complex
04:47measurement of movement right the extent
04:49to which dk1 felt accurate was to the
04:52point where oculus could make an
04:53accurate neck model essentially when you
04:55were moving your head around it kind of
04:57feel like he was following your head in
04:58a natural way then with dk2 you could
05:00actually lean around and look at
05:01something from different angles and it
05:03was like you were that's what kind of
05:05what gave the I'm here feeling of I see
05:07this like really cool out I would say
05:08you know the demos we see are cars or
05:10you're in a video game that's an arcade
05:12style shooter or something and you're in
05:14a cockpit and you can go up and read the
05:16dials on the like mech you're in where
05:19it's I feel like I'm there because I can
05:21move around and this is actually a
05:22pretty good time for me to give my my
05:23pitch for the power of motion parallax
05:25right because the ability to do that
05:28dosnt I mean there's a reason that that
05:30is so visceral makes things feel so real
05:33is because that turns out to be an
05:34incredibly powerful cue for our brains
05:36to understand the structure of the
05:38environment that we have around us so
05:40when you look at something like the gear
05:42vr which does not have positional
05:44tracking and positional tracking is not
05:46available in any purely mobile system it
05:49doesn't have an external camera it's
05:51taken a step back there it doesn't have
05:52that positional tracking and so what you
05:54go and you look at it you'll find that a
05:55lot of the content on there much of
05:57which is awesome and great and totally
05:59worth the price of admission is designed
06:01around having experiences that are far
06:03enough away from you that that that lack
06:07of positional tracking isn't as obvious
06:08basically because the angles involved
06:10are so much smaller that it's not as
06:12important to have motion parallax which
06:13is really important for what we call
06:15sort of near-field VR things that are in
06:17your personal zone and just really to my
06:20mind kind of ratchets up the level of
06:21experience you can provide a whole you
06:23know a whole nother level right because
06:25it's interesting because I think that a
06:26lot of people who are real the are
06:28fanatics think positional tracking is
06:30yesmy know exactly so but when I played
06:33with the gear VR I did notice that so
06:36many of the other aspects and improved
06:37the resolution to the point where I
06:39could see pixels but it wasn't
06:40particularly distract
06:41ding frame rates were kept high enough
06:43Plus and this is kind of nerdy but
06:45Samsung uses low persistence display so
06:47pixels are lit up for shorter amounts of
06:50time so you have like less blur as you
06:52move your eyes across the screen all
06:53these little things added up to an
06:54experience that really does feel like
06:55worth than $99 plus maybe the phone if
06:58you're like me and refuse to give up
06:59your iPhone but at the same time I then
07:03go and do and maybe now we can talk
07:05about the future a little bit a oculus
07:07see v1 which is the upcoming consumer
07:10model or the HTC vive dev kit which is
07:12from HTC and valve you go and try those
07:15and you can walk around and you have
07:17controllers that track your hands and
07:19that's an even it feels like a whole
07:21other generational shift where it's how
07:23could we go back to the previous way of
07:24doing things yeah that's right and so I
07:26think it's worth saying there is an
07:27incredible amount that they got right
07:29with the gear VR it's awesome I mean so
07:32the standard line that I've seen online
07:33which I agree with is if you already
07:35have one of those phones just go buy it
07:37don't think about it go get it it's it's
07:39like and I totally agree then thinking
07:42about the systems like that you just
07:45mentioned the vive or sort of the
07:47upcoming version of the oculus yeah
07:50there is absolutely a whole other level
07:52so there's two pieces of that that I
07:54think are really important one is as you
07:55just mentioned the ability to move
07:57around in larger volumes and that's
07:58great the other one of course is input
08:01and being able to use for both of those
08:03systems and addition also the the Sony
08:06system that's coming up as well your
08:07hands in a very natural way without
08:10really having to think about it and have
08:11one-to-one highly accurate hand tracking
08:15using handheld controllers in those
08:16environments and that opens up not just
08:19a whole lot of awesome game mechanics
08:21and so on and so forth but it really
08:23starts the race I think for people who
08:26want to develop other kinds of
08:27applications in VR and want to think
08:29about this as an overall computing
08:31platform and think you know what kinds
08:33of experience can be provide there in
08:35terms of productivity apps just as one
08:38existing that the original idea or VR
08:41was not just a headset was a headset and
08:43gloves so if you guys remember like
08:45Nintendo Virtual Boy which was very 80
08:47is very go keen retro it wasn't just
08:49something you put in your face but it
08:51also came with gloves so it's always
08:52been part of the idea of VR it's you
08:55not just something you know just
08:57something that you see but something
08:59that you interact with don't you miss
09:00when gadgets came with names like power
09:02glove I really miss those days words
09:04like no now you're playing with power
09:06okay cool but no it is interesting to
09:09think that you know in a few years were
09:12you know people like us who are really
09:14into the cutting edge are going to have
09:15what gloves plus hand controllers Plus
09:18maybe cameras tracking our hands we're
09:20gonna have all of these different
09:21constraints that are gonna limit what
09:23kinds of software is made and also
09:25introducing other opportunities so I
09:26think maybe now as we're thinking about
09:28the future it's a good time to think
09:29about what does design look like for VR
09:31in terms of these two mobile based
09:34headsets where you're plugging in your
09:35phone you know they don't have
09:36positional tracking but you know they
09:38can render a 3d world with crazy high
09:40frame rates and it looks great all the
09:42way up to these controllers where it
09:43feels like mate you know sometimes you
09:44forget that you're in a virtual reality
09:46because moving your hands feels so
09:48natural interacting with the world feels
09:50so natural especially as we add things
09:51like haptic feedback but thinking about
09:54those constraints and the spectrum of
09:56what VR looks like what what is
10:00development for that going to look like
10:01I mean are people going to is it going
10:03to be very like differentiated based on
10:05different categories and how does that
10:07affect like what gets developed first
10:09you know yeah absolutely so I'm not a
10:12designer but I am following closely a
10:14number of very very smart designers who
10:17are trying to figure out exactly these
10:18questions what does interaction look
10:20like what are the affordances that we
10:22want to offer how can we make these
10:26interfaces at the same time you know at
10:30once powerful and very natural so I
10:33think in VR design there's the split
10:34between like realistic design and low
10:37skeuomorphic design and then something
10:39which is a little bit more abstract and
10:40a little more symbolic so you know with
10:43the phone you know our buttons like like
10:47everything seems to kind of how to make
10:48an attempt to resemble a real-world
10:49object in the skeuomorphic design with
10:52VR maybe a general purpose out you know
10:54it might be you know for your writing
10:55out there might be a virtual desk and
10:57you're sitting there writing that might
10:59be the way they do it or it might be
11:00that there's some kind of more abstract
11:02way of interacting with words in VR that
11:04we just haven't thought of yet and it's
11:06it's worth remembering that these things
11:09time to develop like with film you know
11:12it took like 30 years before the jump
11:13cut was invented so that's like you know
11:16you're cutting from you know it used to
11:19be just like filming people on stage
11:21like like it's like if they could
11:23recorded play and they took a while for
11:25all those tropes of editing to develop
11:29so film is what is one great analogy
11:31that we can use another one that's more
11:32recent is the transition to you know
11:35multi-touch displays on on mobile so if
11:38you go back to you know the initial
11:40iPhone release in 2007 and sort of you
11:43know where we where we started there and
11:44where we ended up now we as you know
11:47number one as a design community but
11:49really as a larger society have absorbed
11:51a set of conventions that you know back
11:54in 2007 no one not even the designers
11:57knew really that pinch and zoom and you
11:59know various kinds of swiping we're
12:00going to become these both dominant and
12:03natural and you know almost totally
12:05subconscious at this point you know
12:07components of those interfaces so right
12:10now there are a lot of very smart young
12:12designers who are trying to explore the
12:14space and figure out you know what are
12:16the equivalents in VR I mean one of the
12:18challenges that they face is that the
12:20major systems that are coming out aren't
12:22exactly equivalent so they don't really
12:24know what they're going to be able to
12:26count on between sort of the oculus
12:27touch controllers the HTC valve vive
12:32lighthouse controllers which is a
12:33wonderful mouthful and then the
12:35PlayStation the PlayStation Move
12:37controllers they all have you know some
12:38some pretty important differences
12:40between them I also think it's
12:42interesting to look at the way that the
12:46development of the software and then how
12:48its received is going to shape the
12:49development of the underlying hardware
12:50and technology in terms of right now its
12:52slot your phone into something and maybe
12:56have like the ability to tap and use
12:58gaze based controls and then you have
13:00the more advanced headsets which are
13:01basically just screen and motion sensors
13:04and maybe extra controllers that you
13:06plug in to something with a lot of raw
13:08CPU and GPU horsepower like a gaming
13:10console or a gaming PC but as these
13:13things evolve its you can see how it
13:16could evolve to be the phone still slots
13:19into a headset but then when you're at
13:21home you plug then that
13:23set into something with all that
13:24horsepower and it's kind of a almost
13:26like surface like it the device changes
13:29to meet the needs or you can also see a
13:31standalone headset that has its own
13:33brains that's separate from your phone
13:34because it turns out people maybe also
13:36want to use the phone for some kind of
13:38input and then again that hooks up to
13:40real horsepower or you know that model
13:43kind of goes away and the mobile based
13:45approach just takes off even further and
13:47you know once they're as powerful as a
13:49gaming PC of today you don't really mind
13:52the jump in graphics compared to like
13:54what your desktop could and there are
13:55certainly people in the field who say
13:57look the future of VR is definitely
13:59mobile it's just a matter of and I pull
14:01more lucky may even be on record saying
14:02as I'm not sure that the tethered
14:04desktop version is a very important but
14:07it's a it's a stepping stone really
14:08until we can get to that fully mobile
14:10version I think one other related point
14:12I would I would make is right now these
14:15systems and these hardware platforms are
14:17all coming out and it's wonderful that
14:19we have gaming and entertainment as sort
14:21of this this wedge application that is
14:23driving the initial investment that sort
14:26of guarantees there there will be some
14:28minimum viable market for these things
14:30because there's a certain subset of
14:31people who on day one will you know will
14:33shell out and are waiting for this to
14:35happen but then of course there's this
14:36question which is what does the larger
14:39market want once you get these out here
14:41you know what are people really going to
14:42find interesting and so once you get
14:44these feedback loops between you know
14:45what the market is really responding to
14:47I think that the set of applications
14:48that get a lot of attention I mean I
14:51have my guesses I have some things that
14:53I think are going to be really
14:55compelling in VR but you know at the end
14:57of the day most people I know in this
14:59space have their own set of guesses and
15:01we have fun you know debating you know
15:03what direction that's gonna go but it's
15:05it's a really big open question I think
15:07even before we get to that point you
15:08have to ask like how is like like our
15:11thesis is that VR is not just gonna be
15:14for gaming it's going to be it's a you
15:16know it's like the next platform I think
15:17this is what we believe the oculus as
15:20well it's that this kind of like VR is
15:23an enabling technology which makes you
15:25feel like you can be anywhere so that
15:28doesn't mean doc I mean obviously that's
15:30great for entertainment you can think
15:31you know wow I'm in this like super cool
15:34gaming world you know so much fun but
15:36like both applications but beyond that
15:39it's just a question of like figuring
15:42out how and when that will happen so
15:44what does it mean for me BA for me to be
15:47able to move into any space at will and
15:51especially when those spaces can be
15:52populated by other people I know and
15:55love and want to interact with even if
15:57they're not around or people that I've
15:59just met so it shouldn't be forgotten
16:01that oculus just released their their
16:04social alpha right which is the ability
16:05to go in and watch watch a movie or
16:08video in a fake movie theater with other
16:11people who can be from anywhere around
16:13the world and it's very early and the
16:15level of interaction is very crude but
16:17the fact that they thought this was you
16:19know a very early thing to push out
16:21there you know very quickly says
16:23something real which is that I you know
16:24I think that for a long time you know
16:29one of the knocks against VR from
16:30various quarters has been that it's
16:32going to be this very isolating
16:33experience there has long been a threat
16:35I think going if you go back to you know
16:36jaron lanier you know back in the day
16:38and then certainly continuing to today
16:39of people saying you know wait a minute
16:42really what's interesting about VR one
16:44of the most important things interesting
16:46about VR is that it allows us to share
16:48experiences with other people the
16:50technology is gonna take a little while
16:51for it to catch up but you know I'm one
16:53believer and I know that there are
16:54others who think that you know social is
16:56not just something that's a nice to have
16:58in VR but for many application domains a
17:01core and fundamental capability that is
17:05going to be built into everything and
17:06you know we don't even really even know
17:08how powerful it's gonna be but it's it's
17:09absolutely essential I think a lot of
17:11people when we talk about capturing
17:14experiences and sharing them think about
17:16using 360 degree video or in the future
17:19light fields to capture what's happening
17:21around you in terms of big events or
17:23concerts and I think that that misses
17:26kind of how important in our lives that
17:30the boring times are I think that we're
17:33gonna see a lot of social usage of
17:34people getting in their VR headsets and
17:38quickly scanning in the room they're in
17:40and then hanging out with their mom
17:4220 minutes instead of doing the phone
17:44call playing poker I think is a great
17:45example of people of use I think there
17:47are some people who I you know in or
17:49interact with you say look the
17:50day that I can sit around a poker table
17:51with a few friends and have the kind of
17:53eye contact and social experience that a
17:55good poker game has that's when I know
17:57this technology will have arrived on the
17:59one hand it's very mundane on the other
18:01hand when you break down what's
18:02necessary to have a good poker
18:04experience it's pretty non-trivial there
18:06are a lot of things you have to be able
18:07to get right that just aren't available
18:08today one thing that was really popular
18:10on like I think there's this VR company
18:14called space one thing that they found
18:15was really popular was people hanging
18:18out and watching YouTube videos together
18:19and it's actually it's actually really
18:21interesting when you realize that the
18:23average person I think over 40 spends
18:25like seven hours a day watching TV or
18:27something like that imagine if they
18:28spent that time you know hanging out
18:30with virtual friends instead I think
18:31that's just kind of a nice experience
18:33for everyone one of the most compelling
18:35experiences that I've had in VR was when
18:38next VR at the very beginning of the
18:40basketball season live-streamed
18:42the first Warriors game this is before
18:44of course we knew the Warriors we're
18:45gonna go on and be the awesome team that
18:47they are this season but so this was
18:48this was great and the the camera view
18:50is right at the half-court line you're
18:53on the court and you are kind of like if
18:55you were live at a tennis match you know
18:56your head is going from side to side as
18:58you're watching this game you're that
19:00close and basketball is a phenomenal
19:01game for this because the ball was big
19:03enough that it worked with the current
19:04resolution and and they just in a lot of
19:05ways totally nailed it and I was hoping
19:07it was gonna be awesome and you know I
19:09have been pretty unimpressed with a lot
19:11of things I've seen in VR this was the
19:13future and I want to be able to pay
19:14money to do this on a regular basis but
19:17I was at that game alone right and
19:20there's no way for me now I mean my wife
19:22was you know sitting next to me on the
19:24couch she had no idea what I was doing
19:25and it was you know on some level
19:27incredibly compelling but also very
19:30isolating things so I think a huge you
19:32know nut to crack is how can I share
19:35those entertainment experiences with
19:37those with people who might be in the
19:39same room with me or might be across the
19:41country and I think once you've once
19:42you've got that there's just a really
19:44compelling combination so one additional
19:47point about presence even if you're so
19:50some people have said that even if
19:51you're with like a very crude model of
19:53another person it's even because if you
19:56have VR presence it's kind of still
19:57feels like you're with people I think
20:00anyone who's tried the oculus toy box
20:02demo can probably relate to that
20:04in that seeing a floating head and hands
20:06with no emotions whatsoever no textures
20:10to convey gender or race or anything
20:13you still feel like you're in the room
20:14with that person even no matter what the
20:16distance is between you and just seeing
20:18how people just a stick you late while
20:20they're making a point and knowing that
20:23like oh yeah this isn't just it feels so
20:26completely removed from these green
20:28stuff like a Skype call we're sure
20:29you're see you might even see the actual
20:31person doing that but it doesn't feel
20:32like they're there so there's an
20:33interesting concept from from
20:35neuroscience called filling in which has
20:37been around in neuroscience for a long
20:38time and now I see a lot of people in
20:40the VR community using it because it's
20:41very useful which is that once and it's
20:44a little it's a little complicated in
20:45the sense that it's hard to predict sort
20:48of AA priori or from first principles
20:49exactly when the brain is going to sort
20:51of kick in and start filling things in
20:53but you know one example that people
20:54will often cite in VR is that if you get
20:56a certain amount of for example visual
20:58cues right then even very incomplete or
21:02crude haptic cues will feel very
21:05complete because your brain is expecting
21:08them and it's going to basically fill in
21:11all those details that you know may not
21:14actually be there in the real stimulus
21:16so you know the most important thing is
21:17don't have your cues be dis consonant
21:20don't have them contradict one another
21:21so make sure you don't do that
21:23the next level is you could have those
21:26haptic cues missing completely but then
21:28if you can go one level beyond that and
21:30just have that mirror suggestion of
21:32haptics or you know but whether it's you
21:34know the audio or whatever you can get a
21:36lot of bang for your buck with even
21:39relatively you know low fidelity and low
21:41resolution cues the flipside though is
21:44that it's not always obvious as I was
21:46just saying what counts as low fidelity
21:49and so there's a lot of experimentation
21:50that's taking place to figure out sort
21:52of what is the minimal sort of
21:53additional cues that we can provide but
21:56once you hit that minimum you can have
21:57these incredibly rich multi-sensory
22:00experiences that really kick you into a
22:03new level of presence toybox is one
22:05great example of that and and there are
22:06a number of others that people are
22:07discovering that's one who thinks about
22:09kind of the public perception of VR
22:11though I will say though that it's a
22:13little disconcerting that people are
22:17people see a computer strapped to your
22:18face and they go oh that you know that's
22:20what makes people think it's isolating I
22:22like the term face toy so but over time
22:26you as you add things like haptic
22:28feedback and you know making it feel
22:29like your arm actually faces resistance
22:31as you do something in an experience you
22:33know are we all gonna be strapping on
22:34like full-on bodysuit so that we can
22:36feel when our arm is nicked in the game
22:38oh I bumped into that wall are we gonna
22:40go full ready player one right where
22:41they're sitting in a room by themselves
22:43yeah in these in these full body suits
22:45probably catheterized and you know not
22:47leaving for for quite some time yeah I
22:50think there are there are big questions
22:52here about what is going to be socially
22:55acceptable and what's gonna be socially
22:57acceptable in the home what's gonna be
22:58you know when you get into a are you
23:00know what about when you're out in in
23:02the real world obviously there was a
23:03whole set of issues around you know
23:05acceptability of Google glass which is I
23:07think the example that most people have
23:09in their mind when they think about it
23:11you know I don't know I think one of the
23:14advantages of VR is that because it's
23:17fully immersive and you're not expected
23:18to interact with other people face to
23:20face and you're doing it to a large
23:22degree and the privacy of your own home
23:23or you know or who knows it kind of gets
23:27a pass on that at the beginning but
23:30these issues are going to be very very
23:32important going forward and it's really
23:34hard to predict how social acceptance is
23:36going to evolve right yeah some people
23:38are saying that the whole football ec
23:40thing might be like a you know a
23:42comeback for the arcade so maybe not
23:45everyone will you know buy the like
23:48camera kit and bodysuit and like eye
23:50tracker and every single input device
23:52but you might have some places which
23:53have all of those things and then
23:54whoever's user might just use mobile
23:56they are there it's an exceptionally
23:58standout experience where I have you
24:00feel presents unlike anything you could
24:02do at your own home for you know tens of
24:05that each the setup would be tens of
24:06thousands of dollars and things like
24:07in a very meaningful way the virtual
24:11reality works better when it's tricking
24:12your brain along its various senses you
24:15know you're seeing things that look
24:16realistic you're hearing things that
24:17make you feel like you're there maybe
24:19even you feel something that makes you
24:21like you touch something that virtual
24:22space so I'm gonna using that framework
24:25I'm going to pivot to an interesting
24:26application set that I think is going to
24:29be very important for virtual reality
24:31you know we talk about kids learn
24:32somewhere better visual learners some or
24:34better audio learners some are better
24:35when they write something down or
24:36experience the actual task of doing that
24:39so with virtual reality kind of making
24:42your brain see these things at a very
24:44like intimate we working with your
24:46wiring level what do you think that
24:48means for the future of Education you
24:49know Google is talking about putting
24:51shipping cardboard headsets to schools
24:53but if you know people are actually able
24:56to experience something where they're
24:57walking around seeing the Constitution
24:59be signed for the first time or you know
25:01what what does that mean yeah well that
25:05the obvious first application for VR and
25:08education is things like you know flight
25:10simulators learning to drive things
25:12which have like a very visceral like
25:13physical hand-eye coordination component
25:16so think of it like in the matrix when
25:19canneries like puts on the head sense
25:21like oh I know kung fu I mean maybe
25:23that's kind of it will be like for a VR
25:25when it comes to more abstract things it
25:27can also really be really helpful
25:30because you know if you have the more
25:32intense and varied your memory like
25:35helps with remembering facts and you
25:38know like seeing something on screen you
25:41know 16 your mind more than bathing on
25:43the book seeing in VR even more so so I
25:47think there are a few things and as I
25:49have a three-year-old daughter I've been
25:50thinking a lot about you know how she's
25:51going to experience school and you know
25:53if and when she'll ever even go to
25:54college and all these things and so I
25:56think that for VR education I think that
25:59it's fairly obvious that both history
26:02and science education should just be in
26:04VR like why on earth would I want to you
26:07know read about a you know these
26:09important historical events rather than
26:11be present within them I mean so that to
26:13me is just a matter of producing the
26:15experiences there are a couple of
26:22studios that are that are doing this so
26:24the one experience that my
26:25three-year-old daughter has watched more
26:27than any other is this Apollo 11
26:28experience which if you haven't seen it
26:30super awesome they've done great assets
26:33for the Saturn 5 rocket and then inside
26:35the capsule and she's actually been on
26:37the moon and the whole thing it's not
26:38even in full release yet but I have like
26:40the pre-release version it's great so
26:41for that kind of thing like duh go do it
26:45this out there I mean that that's really
26:46obvious going back to what sake was
26:49saying where I've personally been pretty
26:51interested is how do we translate
26:53abstract concepts into VR so for example
26:56you know CS concepts so I don't really
26:59get distributed systems it's never
27:00anything I really studied you know
27:02intensely I've never had to implement
27:04them but I know that they're really
27:06important so what does it mean for a lot
27:09of different you know machines and
27:11computers to have to reach consensus you
27:14know what what how are those messages
27:16getting passed around I haven't seen the
27:19VR version of the thing that teaches me
27:20that but I bet when I do see it I'm
27:23gonna understand those concepts much
27:24better than if I read about them and
27:27maybe even better than if I implemented
27:30them you know maybe maybe not I'm not
27:31sure but I think that I'd really like to
27:33see those kinds of abstract concepts be
27:35rendered you know in these environments
27:37and I want to sort of see what that
27:38means I mean if you guys have heard of
27:40Burt Victor's kill math project and so
27:42his idea was to take I'm sure concepts
27:45like math and and translate it into just
27:48like symbols which you can kind of which
27:50kind of translate the ideas but in a
27:52kind of visual and indirect way like one
27:56thing you might do with VR is for to
27:57teach chemistry would be like have a
27:58virtual Commission set you could just
28:00move things around in the experiment
28:01with them and then you have you know it
28:03you have to get one yourself you can do
28:05dangerous things and you can see things
28:08like in the gaming space traditional
28:10gaming space you have things like Kerbal
28:12space program which is kind of an
28:13animated silly take on building a rocket
28:17and trying to send it on missions to the
28:18moon or wherever else and a lot of it is
28:20just trying to build it oh no I failed
28:22let me tweak this one thing Oh again uh
28:24and you do that a hundred times then
28:26finally you understand some of the basic
28:28concepts of physics and like how these
28:30things actually work and you look in
28:31NASA's actually now trying to make a
28:33kid's version of this game that's maybe
28:35a little bit more simplified but still
28:37gets the same fun aspects you know it
28:39still captures that and there's a few
28:42areas you know you brought up science
28:43education like that looking at cells
28:46splitting which do students engage with
28:48more a piece of paper you know printed
28:51on an 8 by 11 sheet where you see cells
28:53split in eight different frames or an
28:56animation that shows it in distinct
28:58in a 3d model that shows what's
29:00happening inside the cell at a very
29:02detailed level that's right and so high
29:03fidelity one of the philip rosedale's
29:05company is using cell biology in
29:07particular as I think they're they're
29:09sort of initial application for for
29:12exactly this reason and I've haven't
29:14been I haven't seen it in VR but I've
29:15seen it projected on a screen but it's
29:17awesome I mean it's great and I think
29:19that that kind of topic is is just is
29:22perfect for it well the use case I hear
29:24talks about a lot is a VR a dev
29:27nearly every developer I talk to about
29:29VR wishes that they could have a headset
29:32that they could put on and have you know
29:3310 or 15 screens to code on my neighbor
29:37you've heard right and one of the
29:38complaints right now is that people who
29:40are developing for VR particularly are
29:43taking on and putting off or they have
29:47to take their hmd on and off hundreds of
29:49times throughout the day so just at a
29:50minimum you want to be able to create VR
29:52within VR and then beyond that space
29:55right there's all the whole tool set so
29:57there's creative tools there's the
29:59equivalent of the Adobe Creative Suite
30:01there are development environments of
30:03other of other types you know what will
30:05an office environment within VR look
30:08like how will that work that that's
30:10clearly a big area we know we need a lot
30:13of innovation there going back to the
30:15design interact interaction issues we
30:17were talking about about before so a ton
30:20of interesting stuff to be done there
30:21but then beyond that you know sort of
30:23the tool space which it has to happen or
30:25else you know we're not going to get
30:26anywhere there are there are a number of
30:28others which we can at least touch on
30:30touch on briefly so one that I'm
30:32personally very excited about is you
30:34know roughly speaking how do you kill
30:36videoconferencing right so you know
30:38everybody I don't know if you guys have
30:41an investment in any video conferencing
30:42companies but I was go ahead and say it
30:43okay then this is my opinion only
30:45everyone hates videoconferencing I mean
30:47it's just it has a lot of real real
30:50drawbacks and you know you can't really
30:53make eye contact with people you don't
30:54really know who's paying attention it's
30:55really you know it's really difficult to
30:57have a natural and free-flowing
30:59conversation just you know a lot of real
31:01real issues there so if you could give
31:03people the ability to you know be in a
31:05room together and have a productive
31:07business conversation together there's a
31:09ton of value there and I think a
31:12I think that at the firm we definitely
31:14see instances of that not quite through
31:17implemented through VR but we use being
31:19robots where you know it's the head on a
31:21tablet driven around by wheels and you
31:24can actually yeah it's you're engaged
31:26with this person it feels like you're
31:27talking directly to them because it's
31:29just their head looking back at you you
31:31can tell when someone's distracted like
31:33no we're actually engaging eye-to-eye
31:34and when you see that it's so much
31:37better than having to do a insert video
31:40conferencing application here it just
31:42feels like it's more productive because
31:45they're there with you there's even if
31:47there's no delay whatsoever with a video
31:49chat for some reason there's it feels
31:51like there's this distance right and
31:53it's important to point out that within
31:55you know business communication there
31:56are probably at least dozens of
31:58individual use cases some will be better
32:00with telepresence robot some are
32:02probably really well served right now
32:03others you know others much less so I
32:05was really excited to see that alt space
32:08and of course they're early adopters
32:09they do their stand-up meetings in VR
32:11right now and I think that's awesome I
32:13mean it's obviously weird and awkward to
32:16a certain degree right now but that is
32:17that that's definitely the future so you
32:19know then there's there's business
32:20communication but then you can also
32:21imagine you know personal communication
32:23so I think you know FaceTime has been
32:24wonderful for grandparents everywhere
32:27and so the question is you know what is
32:29the equivalent that we can provide in VR
32:32and you know my personal opinion is that
32:34that's probably a little bit further
32:35away because you know you're dealing
32:38with normals roughly speaking and so how
32:40do you get that you know how do you how
32:42do you get that level of adoption but I
32:44think there's also you know a big big
32:46opportunity there and there are issues
32:47just you know what kind of avatars will
32:50people have do you just go with the
32:51untextured blank face of toy box do you
32:55try to scan people's faces in do you let
32:56them create their own avatars like a
32:58video game what works what's broadly
33:00acceptable yeah absolutely I think that
33:02the choice of avatar both resolution and
33:05you know realism is incredibly
33:08I can make cases for any of the
33:10possibilities that you just laid out and
33:11I feel like I have basically no idea
33:13which way it's gonna go because this
33:15gets into the social acceptance issues
33:17and they're really dynamic yeah I think
33:19one interesting other point is that if
33:21you if you have really really good
33:22telepresence that affects like how we
33:24behave in other ways like think
33:26about how many miles people travel every
33:28year for business you know how how far
33:30people go to visit their friends you
33:31know driving like all those kinds of
33:34things I mean yeah and that starts to
33:36point out that VR like pretty much any
33:39very general purpose technology and
33:41computing platform is going to have some
33:44really interesting second and third
33:45order effects and it's we can sit here
33:49and make guesses and a lot at the end of
33:50the day a lot of its going to come down
33:52to some pretty fine-grained issues of
33:54economics and the quality that's
33:56provided and a lot of things and so yeah
33:58I think absolutely you know one of the
34:00possibilities is that self-driving cars
34:04may be less important than we think
34:07they're going to be because people find
34:09it less important to travel for the same
34:12reasons that they do right now I'm not
34:14saying that's going to happen I'm saying
34:15it's it's entirely possible and that VR
34:18at least you know plausibly has the
34:20power you know to have an effect on that
34:22level and you know could you know just
34:25as self-driving cars are probably going
34:27to have a pretty big effect on you know
34:29how real estate value is distributed
34:31because certain areas will be easier and
34:33harder to get to I think you know VR - I
34:36mean if we have to travel less if
34:37commuting daily for a job becomes less
34:40important because you know the our
34:42telepresence you know really delivers on
34:43its promise then in 10 or 15 years you
34:46know what effect is that going to have
34:48on you know where people choose to live
34:50relative to where they work and you know
34:51and all those issues I think you know
34:52there are a lot of big possibilities
34:54there at this point we're just guessing
34:56about exactly how it's all going to knit
34:57out just so that we kind of covered the
34:59spectrum of what you know the Nerds are
35:01will really appreciate you are VR
35:03enabled future and what people in the
35:05real world are going to want to
35:06experience people who aren't making it
35:08up as they go in their garage I think an
35:10interesting aspect of VR and you know
35:12kind of related category augmented
35:14reality is fashion you know the idea
35:16that once you have this avatar that
35:18represents you in all of these spaces
35:20could you try use that avatar to try on
35:23virtual goods that then correlated the
35:25purchases you'll make in the real world
35:27I think shopping is interesting because
35:28ecommerce is has been growing steadily
35:30over the last 10 or 15 years but it's
35:32still only like seven to ten percent of
35:34all the things that we buy and given
35:36that you know people are willing to like
35:38instacart like groceries there
35:40by pretty much everything online and and
35:42closed so like the things that we buy
35:44the second most often and yet it's still
35:47quite hasn't made the link to e-commerce
35:48yet and part of the reason with that is
35:50you know it's you don't want to buy
35:52something which doesn't fit and then you
35:54have like you know some kind of wonky
35:55thing that you have to return later and
35:58so I think VR is interesting in that way
36:00it's worth noting that like online
36:03retailers have been trying this problem
36:04for a long time they've been they have
36:06these like virtual models they're like
36:07oh maybe people can try things on this
36:09way but none of them have really caught
36:10on so it might be VR that gives it the
36:12final push right so if you look at the
36:15argument for the continuing relevance of
36:17you know brick-and-mortar retail you
36:19know 10 15 20 years from now it's that
36:21there are certain physical experiences
36:24that you need to have in the context of
36:26that store that you're not going to be
36:28able to get at home and so you know the
36:30idea behind behind VR and also and also
36:32a art which we'll get to in a second is
36:34that it really eliminates that that
36:36final reason that you actually would
36:38ever have to go to a store so this is
36:41you know far more the argument is far
36:44easier to make in some areas in other
36:45words fashion is a great one
36:47another one that I saw pointed out
36:49recently online is you know furniture
36:51and sort of you know large goods and
36:53this sort of gets to the whole area that
36:55I think is really interesting of you
36:56know architecture and interior design in
36:58general which is that if I can you know
37:00envision a space and put new things in
37:03it and try them out holy crap do I want
37:06to do that right that is so obviously at
37:08least you know at a conceptual level
37:10better than the alternatives as someone
37:12who recently you know bought and
37:14furnished a house oh my gosh would I
37:17have preferred you know a when I was
37:18shopping for houses to be able to do
37:20virtual walkthroughs and that's you know
37:22it seems to me like a slam-dunk case
37:23that you know will be out there very
37:26soon and there there are companies that
37:27are going after that opportunity but
37:28then once I had that house you know
37:30being able to virtually furnish it
37:32before before actually getting these
37:34very exciting I still have a set of
37:37closet doors sitting in my garage that
37:40are probably never gonna leave my garage
37:42because they didn't turn out to be the
37:43right ones and I you know just couldn't
37:46know that based on what I saw online if
37:48I could have hung those doors virtually
37:50I wouldn't have those doors sitting in
37:53ten years and that's you know that
37:54that's pretty compelling and with
37:55something as intimate as fashion you
37:57know uh immense small medium large
37:59woman's zero six and twelve those all
38:01mean very different things at different
38:03retailers and but at the same time they
38:05all have the absolute data of how much
38:07fabric is in every different size and
38:09every different garment so why not in
38:12put your exact measurements into a
38:14virtual reality viewing application and
38:15see exactly how that piece of clothing
38:17drapes over your body size I mean what
38:20that seems like you know we talk about
38:23Amazon having two-day shipping or even
38:25same-day shipping and oh that lets you
38:26return things easier and you'll try more
38:28things if you can know exactly how this
38:31shirt or dress that's right I shopped
38:33for a lot of clothes online and I
38:34basically accept that only about half of
38:36them are gonna fit and right now for me
38:38it's still so much more convenient as me
38:41to do it that way I just sort of I eat
38:43that but that's clearly you know
38:45suboptimal there's a lot of room for
38:47improvement along exactly those lines it
38:49was nothing that fashions are very low
38:50tech industry in general
38:52only if you guys have heard of Zara in
38:54the whole fast fashion movement that was
38:55applying a very small amount of
38:57Technology you just like a little bit of
38:59analytics a little bit prediction and
39:01then they just became really huge just
39:03from you know measuring what people were
39:05buying and so you know I think like you
39:07know the fashion is she could really get
39:08a huge boost from something like this if
39:12you used the dk2 and gone on oculus
39:15share or if you now have the new gear VR
39:17and goes you know check out the
39:19experiences section of the oculus store
39:21one of the things that was pretty quick
39:24to arrive on those platforms was oh here
39:26live on a fantasy Beach for ten minutes
39:28and I think that that kind of that's a
39:31very early example of something that
39:32also could be big in the future which is
39:34the idea of you know virtual tourism or
39:37virtual travel the idea of if you can
39:39simulate presence and being somewhere
39:41and capture environments and your
39:43photorealistic detail why not just go
39:46there in a headset versus making the
39:48real trip and spending maybe thousands
39:49of dollars to go see something where you
39:51know frankly you can't actually go up to
39:53that piece of artwork touch it and feel
39:55it so what's the difference
39:56yeah actually and the company John VR
39:59was they started it because one of the
40:00founders went on a trip to like Monument
40:04Valley in Utah and he
40:06oh my god this is so amazing I wouldn't
40:07look at it anytime I want and so after
40:10that they made this 360 camera recording
40:13break so what it does is it it's like a
40:14ring of cameras in a row and it records
40:17video and 360 degrees so then you can go
40:20go to some like really cool location
40:23record what it's like and then replay it
40:24back to yourself on your headset the
40:28other way people are doing capture is to
40:30doing like feel tech photography so that
40:32means it record like how the light is
40:35falling everywhere in so it's another
40:38camera rig but it kind of reconstructs
40:39the scene in a slightly different way -
40:41360 video and it includes lots more
40:44information about how the lighting falls
40:46on all the objects in the scene and so
40:48that's just another way of playing back
40:50the experience of being in some cool
40:52place it can visit for listeners who
40:55maybe haven't tried kind of the scope of
40:57these experiences from 360 video to
40:59light fields I think it's important
41:01maybe to differentiate in practical
41:02terms what those technologies provide in
41:05terms of ease of capturing content and
41:07then what the experience will actually
41:09be like for the viewer so with 360
41:12degree video it's actually using
41:13traditional camera sensors and optics
41:16and then you're stitching together that
41:17footage so that it's 360 degrees around
41:19you and maybe there's even a 3d effect
41:22so there's some depth to what you're
41:24looking at with light fields on the
41:27other hand you're actually capturing
41:30this as like 3d points so when you're
41:33viewing it it's not just something
41:35that's a video playing at a fixed
41:36distance from your head but there's even
41:38the possibility of you know as we talked
41:41about earlier with the dk2 and having
41:43camera and beyond to look in and what
41:44that means with these life fields you
41:46can actually do that and see what you
41:47captured from different angles and you
41:50know again it contributes to the feeling
41:51of being there but at the same time
41:54there are a couple of downsides aku
41:55could you kind of detail maybe what the
41:58downsides are to life fields given they
41:59sound so appealing yeah life fields
42:01serve problem is to just say it's a long
42:03time to capture you know you need to
42:06take a scene and then take everything in
42:08it and everything about the lighting
42:11information then you know replay it back
42:12like that's a lot of computational work
42:14but on the plus side that's a very
42:16parallelizable problem so we might see
42:18more of it in the future
42:20so and one way of thinking about that is
42:22in when Litro Lytro when they recently
42:26announced their they're like field
42:28yeah they announced a camera but
42:30alongside it as part of the system is
42:32the giant rack of basically disk and
42:35compute that you need just to you know
42:37capture and then preview the content
42:39that you're capturing it so it's just
42:41you know at a computational level it's
42:43it's it's a massive problem the amount
42:45of data that you're capturing on a you
42:47know bits per second basis is is just
42:49truly truly massive it's worth
42:51mentioning that most VR isn't done using
42:53either 360 or light field it's done in
42:55the same ways games are made using 3d
42:57modeling and rendering and like the same
42:59techniques people use to do special
43:01effects in movies ya know in in the
43:03medium / long term these also it seems
43:06like there's potential films to kind of
43:07blend together you're capturing light
43:09fields but then you do some rough
43:11computation on it to build you know a
43:13underlying 3d model and then you slap
43:16the light field on top of it as a
43:17texture essentially right and then you
43:19have a light field that you can also
43:21maybe interact with in a game-like way
43:23and it's fascinating to think about what
43:25that would enable that where you capture
43:26roam but also you can go into the shop
43:29or like open the door or you know in
43:31maybe something less you know light
43:35touch you know something like if a video
43:37game like an Theft Auto where you use
43:38like fields to capture a real city and
43:40then you play around with it right like
43:42one of the cool thing about unreal is
43:45that they've been making this big push
43:46to include like Hollywood quality assets
43:49in their game making so like a lot of
43:51our developers have been using like
43:53these these assets which have been
43:55captured from like motion motion capture
43:57studios or like you know sometimes even
44:00like field stuff they incorporate it
44:02into that games and so you have this
44:03kind of blend between the traditional
44:05video game way of doing things and then
44:07the new light field or motion capture
44:11way of doing things one more thing on
44:13light field so light fields are really
44:15hard to wrap your mind around and a
44:18podcast might be the worst possible
44:20medium to explain why what light fields
44:22so I just want to put in a plug maybe
44:24we'll put it in the show notes for a
44:26great set of talks that came out of the
44:27USC ICT lab that explain it and
44:32they're awesome and if you want to
44:33understand this topic it's foreign to it
44:35explained by the absolute best people in
44:37the world to be doing this explanation
44:38and that is what you should watch in
44:41order to sort of get a handle on the
44:44so I'm really incredibly excited about
44:47the possibilities for for travel and VR
44:50I think it's it's worth pointing out
44:51that this is an area where the quality
44:54of the experience matters a ton and so I
44:56think you know there's the great pulse
44:58fo line which is never mistake a clear
45:01view for a short path or something along
45:03those lines so so clearly being able to
45:06transport yourself to somewhere that you
45:08can't actually physically visit or that
45:10would be too expensive is just a huge
45:13and I strongly believe in this
45:15application but for it to be more than a
45:18novelty and as someone who's tried these
45:19out I think that you know whether it's
45:22light field or even more exotic
45:27additions like redirected walking in a
45:30room scale environment so really quickly
45:32redirected walking is where you give
45:34someone the impression that they are in
45:36this sort of infinitely large
45:37environment by subtly changing the way
45:40in which the virtual scene is rendered
45:42so that they are basically prevented
45:45from you know walking into a wall but
45:47your walk up you like it you walk a
45:48little bit to the left and but what you
45:49think you're going in a straight line
45:50yes that's right and there all kinds of
45:51tricks where you know as people are
45:53either you know going through doors or
45:54turning by sixty degrees you can turn
45:56them by maybe eighty degrees and they
45:57won't notice and it's it's it's super
45:59cool so you know there's gonna be quite
46:01a bit that's necessary for me to really
46:03feel like I'm at the Pyramids of Giza to
46:06the level where I don't actually need to
46:08go to the Pyramids of Giza for this to
46:10you know to have an experience that I
46:13would consider to be to be comparable
46:14but once you've gotten to that point I
46:17think there's just an incredible amount
46:20that that is unlocked I think that you
46:22know one of the you know one of the
46:24great potentials for VR is just to give
46:28a give far more people in the world the
46:32ability to have the awesome experiences
46:35that currently only a relatively very
46:38small number of people can can afford or
46:41in the right place at the right time to
46:43experience so you know being courtside
46:46at a basketball game is one kind of
46:48prosaic example of that I mean you're
46:49basically removing the scarcity of
46:52courtside seats you are you are
46:53transitioning that scarcity from the
46:56world of atoms and you
46:57it's now you know zero marginal cost
46:59world of bits right it's sort of that
47:01that standard transition but applied to
47:03what up until now were you know we're
47:05very physical embodied experiences and I
47:07don't want to claim that this is going
47:09to be a quick transition or that people
47:11are going to consider this to be you
47:14know a fair trade anytime soon but I do
47:17think that is to my mind sort of one of
47:19the most exciting sort of long-term
47:21trends that we have here I'm actually
47:24surprised we managed to cover a lot of
47:25bases in VR we talked about the
47:27technology and the medium itself but
47:29also kind of design paradigms and things
47:31to consider so I really appreciate you
47:33guys you know joining and sitting here
47:35for an hour plus to really go and dig
47:37into all these aspects thanks for having
47:40us yeah it's been great thanks a lot
![How To Succeed As A NEW & YOUNG Realtor [Deals Every Month + Luxury Listings]](/_next/image?url=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrtrRghgTayc%2Fhqdefault.jpg&w=3840&q=75)
