00:08hi i'm john wiley and I work at Google
00:13I've been a go go for about nine years
00:15this is a this is a talk about what
00:18keeps me up at night this is a all the
00:20cards that have everything that keeps me
00:21up at night on the cards and I'm a
00:25director of immersive design which is
00:28actually a title I came up with this
00:29week because I was trying to fig out
00:30what's called myself I work on virtual
00:33reality at Google but I'm also going to
00:35be working on augmented reality at
00:37Google's some of the stuff I'm working
00:39on right now and and sort of volumetric
00:42design and three-dimensional computing
00:45and fun stuff so I wasn't quite sure
00:47what's called myself I think it's called
00:48the mercy of computing immersive design
00:50it's a new frontier and so we've got to
00:53figure out kind of what all these names
00:55are so yeah so this is going to be about
00:59what keeps me up at night good and bad
01:01so I think this phrase is often used to
01:03sort of a negative connotation it's like
01:05what keeps you up at nine psycho that's
01:06that's like worrisome but there's things
01:08in keeping you up at night to that are
01:09exciting and are interesting and I it
01:11really excited about and want to write
01:12down a bunch of stuff about and someone
01:14talked a little bit about that too so a
01:24bit of history I as I mentioned a minute
01:27Google for a little while and last year
01:29I went to Google i/o so Google i/o is
01:32Google's kind of annual it's mostly a
01:34developer conference but it also tends
01:37to be someplace where we occasionally
01:38make product announcements and last year
01:41Google i/o 2014 was actually the first
01:44time that I attended I oh I usually just
01:46watch it from my desk and the reason i
01:49was at i was at Google i/o was because I
01:52was there to present material design
01:53it's kind of her to believe actually
01:54that it's it's it was last year that we
01:57presented that so much has happened in
01:58that time but I was I was actually part
02:00of the material design team had been
02:02working on that for two years prior to
02:04presenting it last year do and also at
02:08i/o at the same event we announced
02:10google cardboard so hopeful
02:13most of you have had a chance during the
02:15breaks or whatever to just kind of go
02:17back there in the back we have a kind of
02:19a google cardboard setup hopefully if
02:20you've picked up your google cardboard
02:21this is it right here this is google
02:25cardboard and and in 2014 last year i
02:28heard about this at the same time
02:29everybody else too did when we announced
02:32it on stage and so I was coming what
02:34what is that that's kind of and we at we
02:36now sit with no fanfare right it's like
02:37hey everybody gets one of these peace
02:39out right you know and and so everybody
02:41kinda like puzzled about what it was
02:43this is it this is actually v2 of
02:45cardboard and if you haven't gotten
02:48yours yet it's pretty simple this is the
02:50sleeve just take the sleeve up and you
02:55just fold it like this and put it around
02:58and it's got these it's got these lenses
03:01in it and do that and then you're ready
03:04to go with a phone so you pair cardboard
03:07with a smartphone can work on iOS
03:10devices or Android devices and you run
03:12software on it that projects two points
03:16of view into both of these lenses and
03:18that gives you stereo but not only
03:20stereo so it's like the old viewmaster
03:22right you know the master you like and
03:24you get to see like i'll let you know
03:25pictures in stereo chichen itza or you
03:28know all these the pyramids in egypt or
03:29whatever but but one of the cool things
03:31about smart phone technology is that
03:34they have sensors in them that allow you
03:35to detect orientation on the phone and
03:36so now with something like this very
03:38very simple technology you can actually
03:40just look around and you're immersed
03:41inside of this world so as you turn your
03:43head and as you look and you see things
03:45you can see these things in 3d and
03:47stereo near things in ear far things are
03:49far and I when I tried it the first time
03:56I tried it I said this is one of the
03:59most googly things we've made in a long
04:01time and the reason I said that was
04:04because it's really simple technology
04:07right it's really i mean like this it's
04:10you could barely call this technology
04:12it's a little bit of optics a little bit
04:13of cardboard some plastic and that
04:17simplicity sits on top of something
04:19that's very powerful which is the phone
04:20and this is this is actually pretty
04:23similar to how if we think about sort of
04:25the iconic google experience the google
04:27page Google search so I worked on google
04:29search for five years prior to working
04:30on virtual reality and that is a great
04:33example of very simple intuitive
04:35technology in fact when when Google the
04:37first google homepage came into
04:38existence it was competing with a bunch
04:40of things were really noisy and the
04:42Google homepage was actually really
04:43simple it's just a white page with you
04:45know a box and some buttons but it's sat
04:48on top of some incredible computer
04:50science some amazing technology in fact
04:52Google search was probably one of the
04:53first cloud applications that you could
04:56open up something in your browser and
04:57that was actually it was actually being
05:00powered by these massive data centers
05:02right and so all of that technology sort
05:04of finding an endpoint into something
05:06like this is it was really fascinating
05:09commune and what's also cool about
05:11cardboard is it you know just given the
05:12materials it sets expectations way down
05:14here right you just like what but the
05:17experience of cardboard and the
05:18experience of virtual reality and being
05:19immersed that way that experience is
05:21much better than the expectations and in
05:23that gap is delight and something very
05:26interesting and this kept me up at night
05:32I was at i/o 2014 i was looking at
05:35cardboard i was playing with it and i
05:36kept thinking about it i kept thinking
05:37about it I kept counted all these ideas
05:39about like what I would do in VR and all
05:41these kind of products it'd be really
05:42awesome to make I kept turning it over
05:44my head and you know when you do that
05:47when you're staying up a lot and you're
05:48writing down these ideas and the ideas
05:50don't stop you know that's that's
05:53something you want to pay attention to
05:55so I made a decision last year to leave
06:00my comfortable post working on design
06:03for search it spent many years I knew
06:05that I knew the area inside it out and I
06:07sort of up ended my career and said I'm
06:09gonna go work on VR whoo hey Anna join
06:11their via our efforts full-time so
06:16virtual reality it's it's going through
06:20a lot of cycles I think if you've you
06:22know if you've paid attention to virtual
06:24reality technology going back to the 80s
06:25and the 90s it's gone through lots of
06:28these what we call you know the hype
06:29cycle right it's the it's the you know
06:31this peak where it goes through this leg
06:33oh my gosh it's gonna be amazing and
06:34then sort of you know the trough of of
06:36disappointment to disillusionment and
06:39and vr has actually gone through several
06:40of these and and part of the problem is
06:44that the idea of virtual reality is
06:46powerful and it's been possible to
06:48create virtual experiences with really
06:50expensive technology like military
06:52technology or or you know university
06:55technology I am a graduate of the
06:57University of Texas and we used to just
06:59hand like like all these like weird
07:00roboticists and people made strap like a
07:02big CRT on your head from the ceiling
07:04and they do all this stuff I was really
07:06expensive not very accessible mobile
07:09technology though has has made it
07:12possible to take all the things that you
07:13need for VR which is good tracking and
07:16good sensors good displays good cpu so
07:19that you can do all the processing
07:19necessary to render this technology in
07:23real time mobile technology has been a
07:25driver of this and all these things have
07:26come to a head at this point and so now
07:29I think VR is probably here to stay I'm
07:31going to proceed on that assumption in
07:35in the next year to 18 months you're
07:40going to see a lot of VR devices
07:42becoming available to consumers most the
07:45most popular of which is is oculus I'm
07:46sure all of you have how many of you
07:48actually tried oculus rift all right ah
07:51my people vr people in the house
07:54fantastic and how about the HTC valve
07:58the vive headset got a couple of people
08:00right so vibe the vibe will be out next
08:04year oculus will be out next year
08:05sounding Morpheus to probably be out
08:07next year and these are all very
08:09high-end devices via our devices and
08:13I'll talk a little bit about what that
08:14means in a bit but over the course in
08:17the next you know couple years you're
08:18going to see a lot of it's gonna it's
08:20like anything that is new it's this new
08:21technology be like the internet and 90s
08:23there's gonna be a ton of press about
08:24it's like oh my gosh vr is the most
08:26amazing thing ever and and then there's
08:28you know there's gonna be people
08:29enthralled by it they're gonna be
08:30talking about what's your VR strategy
08:32are you free our first and you know
08:40there's just going to be a lot of hype
08:42in and we'll there'll be a lot of noise
08:44and we'll have to sift through the noise
08:45to find the signal I think there's going
08:50to be a lot of cynical reporting right
08:51it's gonna be it's a doomed
08:52no one's going to use this why would you
08:54stop a strap computer to your face right
08:55and at the and and and somewhere in
08:58between lies the truth right on the one
08:59end it it's very promising interesting
09:02technology they probably will we think
09:04have value on the other side of it you
09:06know there's there's definitely some
09:07complexities involved and there's
09:08definitely muser experience of all in
09:10terms of the cost of getting into a VR
09:13experience so as I said when I when I
09:19first saw cardboard and when I was able
09:21to try lots of these VR devices and I
09:23started thinking about VR and it was
09:24keeping me up I was mostly thinking
09:26about the product experiences so I was
09:28really excited about sort of the product
09:30potential V are as a designer I hadn't
09:33given a lot of thought I was excited
09:35about like you know spending time in VR
09:37as a consumer and being able to like you
09:39know I I'm a I'm an astronomy geek so
09:41like having like a virtual planetarium
09:43that I can step into and point at the
09:45stars and do fun stuff I was really
09:46excited about the product ideas in terms
09:48of thinking about the design I didn't
09:51give it a lot of thought in in order for
09:52me to actually make VR a career choice
09:56as an interaction designer to actually
09:59turn that into occur to make a choice to
10:00go in that direction I think VR has to
10:03be interesting and exciting from the
10:04from the perspective of an interaction
10:06designer so thinking about design you
10:10know if you set out to be a designer
10:12perhaps you go to school for it you
10:14might get a degree in you know HCI
10:16human-computer interaction and I think a
10:19lot of what we do is designers in that
10:21field is like humans on one side
10:23computers on the other and there's a
10:24chasm and we're trying to bridge that
10:25chasm and sometimes if we're lucky the
10:29computer is the one that moves across
10:31the chasm towards us towards us humans
10:33are humans are adaptable and that's
10:35great but we're not we're not that
10:37adaptable we've had we've had you know
10:39we're kind of we're kind of stuck in our
10:40ways we still we still have big hands
10:41and whatnot so you start with computers
10:44and you know the language of computers
10:46is like punch cards and things like that
10:47and that's really the human learning the
10:49computer language right it's like okay
10:50punch these cards or do something
10:52eventually moved on to like switches and
10:54dials and lights and things like that
10:55and I was still mostly the human trying
10:57to understand a computer we got the
10:58command line interface that one's
11:00actually a little bit of the human and
11:02the computer coming over right so now
11:03it's the computer starting to understand
11:05program and assembly the computer
11:07actually starting to have some language
11:08uses letters right we get a keyboard out
11:11of it that's pretty exciting and in each
11:13of these advances you get more people
11:14using computers there was a quote
11:19earlier actually I saw a Jenna tweeted
11:22it that was said at the conference
11:24earlier today you can either improve the
11:27users or you can improve the technology
11:28but the benefit you get typically out of
11:31user experiences when you improve the
11:33technology humans have certain
11:37expectations about how the world works
11:38they have certain evolution right in
11:40terms of how they approach things and
11:42you want the computer to fulfill that
11:44not the other way around so the you know
11:53and then we had the graphical user
11:55interface and then you have a ton more
11:57people using computers at that point and
11:58then with smartphones and touch screens
12:00we actually that's that's even the
12:02computer giving even closer right to the
12:04human because now I can directly
12:06interact with software I can directly
12:08manipulate software with my hands so
12:10I've got millions of years of evolution
12:11of like manipulating objects and picking
12:13them up doing things with them excuse me
12:15and and now we're getting apart you can
12:19actually touch software and so this is a
12:21new moment something you get an
12:22explosion of usage of computers now it's
12:24not all due to the fact that it's that's
12:27touch screens right it'sit's
12:28miniaturization it's very small but you
12:31know if you've shipped the world a bunch
12:32of smartphones and then you just
12:33included a keyboard with it right it get
12:35less usage so but humans have these
12:40physical expectations right that they
12:42come to it with with their bodies and
12:44how they work and actually this is a
12:48this is a lot of the foundation behind
12:49the thinking in material design material
12:52design is about sort of coming to the
12:55table with with the recognition of the
12:57of the capabilities that the human has
12:58and the assumptions of the human has
13:00right when we interact with physical
13:02objects we expect them to have mass and
13:05momentum and they have friction in
13:06physics and so when you're trying to
13:09create an authentic software interface
13:12then the complexity of that goes way up
13:14right suddenly it's not just like the
13:19one web web designer you know just
13:21making like content pages suddenly you
13:23have like motion graphics and you have
13:24to make it performant you have to take
13:26care of the frames because reality
13:27doesn't drop frames and you have to take
13:31care of the latency it's this fluidity
13:33this continuity of state right it's not
13:35one state a and B it's about going from
13:37one place to the next but it's a that
13:38continuity of states right so playing
13:41into the expectations that humans have
13:42about how things work particularly when
13:44they're physically interacting with them
13:45was a huge part of material design so VR
13:51virtual reality is another one of these
13:54transitions it's another one of these
13:56moments in which computers are getting
13:57closer to the humans in terms of how
14:00humans interact and perceive the world
14:02but it's turned up to 11 because with
14:06phones you know I'm designing for us I'm
14:08still designing for a square it's not
14:10all that different you know in many ways
14:11from from the design that came before it
14:13I'm physically interacting with it but
14:15I'm mostly just using my hands and it's
14:17generally in a single dimension right
14:19I'm it's 2d still whereas if you think
14:23about virtual reality it becomes
14:25volumetric it's like depth becomes into
14:28play but it's not only that it's about
14:29the turning of my head so I actually now
14:31have head motion I have body motion
14:33everything the first time most people
14:35get into VR a lot of times they do this
14:37where's my hands right so they're
14:40they're starting to have the expectation
14:42again the physical expectations that
14:44they have in the real world they're
14:45bringing them with the bring that into
14:48virtual reality and virtual reality
14:50fails to meet or exceed those
14:51expectations people have a bedtime so
14:56this is actually what drove me to to
14:58start working on which realities because
14:59I saw that like there's a lot of
15:01potential here for the computer to get
15:03closer to the human and therefore the
15:04humans going to get much more powerful
15:05in the use of computers I didn't really
15:08know anything about VR when I joined i
15:10mean i used it i didn't really know
15:11about it as a designer I actually
15:13assumed that the transition in terms of
15:15a career is my career as a designer I
15:17assume the transition to VR would be
15:19similar in complexity to my transition
15:21to mobile design but having worked on it
15:24for about a year now it's going to be
15:26more complex and it's going to be more
15:28complicated than then the mobile
15:35so what makes it complicated well it's
15:39sufficiently complex and different from
15:40existing design that you know the folks
15:43have devoted many hours to it earlier
15:45today I tweeted a link to bunch of other
15:48links a page that just has long link of
15:50resources for designing in VR there's
15:52lots of videos lots of people thinking
15:53about it clearly by no means salt
15:55because we we barely have any consumers
15:58using VR at this point but but people
16:00starting to think about and work on it
16:01right now one of the things that we
16:04created at Google was the cardboard
16:07design lab which you can download from
16:08the Play Store if you have an Android
16:10device and put onto onto your cardboard
16:12device and what it does it just gives
16:14you kind of like a we thought what
16:16better way to talk about VR design then
16:18actually putting it into virtual reality
16:20it's part of the reason I'm having
16:22slides today is because I started it's
16:25all 2d stuff and it just doesn't convey
16:27the real dimensionality of anything I
16:29think for us to actually have coherent
16:31presentations about VR we're out we're
16:34all gonna have to have VR be like step
16:37into a virtual reality in order to
16:38understand it so what makes it complex
16:41well I'm gonna throw out a few things
16:43real quick and this is all stuff that I
16:45just got bombarded with when I started
16:47designing virtual reality so one and
16:49there's gonna be a lot of jargon here
16:50right so every discipline has its
16:52charging so three da versus sixth off
16:53three degrees of freedom versus six
16:55degrees of freedom so cardboard based on
16:57the sensors in your phone is a three-d--
16:59off device it's three degrees of freedom
17:00so it's basically like on a on a tripod
17:02it's like pivot so you can rotate around
17:04the axis rotating around the X the Y Z
17:08sixth off is rotation plus translation
17:12so you can move in those axes and that
17:15gives you the six degrees of freedom and
17:18to something like the the valve HTC vive
17:21headset they have what they call room
17:23scale vr where you can actually be in a
17:26virtual environment and walk around in a
17:28full size room another issue is
17:33proprioception this is a word I did not
17:36know or joining virtual reality
17:39proprioception is your body's ability to
17:41sort of understand where the different
17:43parts of your body are
17:45it's what enables you to close your eyes
17:46and if you're sober do that and touch
17:50your nose you know where your hand as
17:52you know when your nose is this comes
17:53into play into virtual reality because
17:55there's we you could set up weird
17:56conflicts between what's real and what
17:58isn't again this is the thing where it's
18:00like oh I've got hands I feel my hands
18:01but suddenly my hand is passing through
18:03a solid object or it seems solid but
18:05it's not there and so that you get these
18:09kind of weird conflicts between you know
18:11what you feel like ought to be there and
18:12what isn't there's another big issue a
18:18big challenge with the technology side
18:20of virtual reality and that's vergence
18:22accommodation conflict so I'll show you
18:24what that is it has to do with depth
18:28perception focus of your eyes if you get
18:30into virtual reality design you end up
18:32learning a lot about about optics and
18:34how the eyes work so one quick way to do
18:37this is if you if you hold your if you
18:39close one eye and you hold your thumb up
18:40like this and you do like the camera one
18:42camera two thing right you'll see your
18:43thumb pot back and forth and that is
18:46basically how your body and how your
18:48brain determines depth that's the stereo
18:50images coming into your eyes it's just
18:52like changing of the background but one
18:55of the things that happens in reality is
18:56that the actual distance to the object
18:59the Virgin's right so your eyes are like
19:01focusing in so that angle as they as
19:04they turn into each other to focus on
19:05that object that the Virgin's distance
19:07that where they converge that distance
19:09is equivalent to the focal distance so
19:11what's happening is your eyes converge
19:13and it says ah that's how that far away
19:14that thing is based on that angle but
19:16then your body also focuses your eyes by
19:18changing the shape of your eyes and that
19:20distance is the same but if we are
19:22something can appear far away but is
19:27actually being rendered much closer to
19:29you because the actual screen that it's
19:31rendering on is right in front of your
19:33face and so your eyes have to focus on
19:35something that's much closer than what
19:38it actually appears to be and this can
19:39lead to eye fatigue and with VR devices
19:43what you get and I'm hoping more VR
19:45devices start publishing this data with
19:47VR devices you get actually what's
19:48called a virtual screen so it's that
19:50it's because of the optics it's not it's
19:52not not everything is here it's actually
19:53being projected at what's a virtual
19:55screen which is probably depends on the
19:57device it's on the order of like one and
19:582 meters and that's kind of like the
20:00safety zone so if you put things in that
20:02space it's less fatiguing because you're
20:04there's no conflict your eyes are
20:05focusing at the same spot that they're
20:07converging and probably a big one oh
20:12yeah and so the reason I was putting
20:13this up here is because if it's like the
20:15virtual screen distance is here so
20:17that's a good spot but if I'm over here
20:19virtual screen distance is much closer
20:21so the other part of this too is what's
20:24called sim sickness and sim sickness is
20:28what happens it's basically like its
20:30motion sickness it's the same thing you
20:31might get if you're in a car in your
20:32reading a book or a boat I get seasick
20:34and it has to do with a vestibular
20:36system in humans the part that tells us
20:38what how we're moving versus what we see
20:40and a challenge in VR is that you don't
20:42feel acceleration you're not actually
20:44moving and so if you move somebody in VR
20:46or you move objects in VR that are very
20:48large you can actually make people ill
20:51the stakes are pretty high for design
20:54when if you do something wrong you can
20:57actually make your users physically sick
20:59and and so you and and I'm probably one
21:04of the worst people to be working on VR
21:05frankly because I don't get motion
21:07sickness which is a blessing in many
21:09ways but if you're actually working on
21:10things that you hope don't make people
21:12sick you know I basically end up saying
21:14here try this you can get a little bit a
21:18problem and and and one of the things
21:22that can also cause sim sicknesses frame
21:24dropping so performance turns into a
21:26really big deal for VR if you're if
21:29you're looking around in a virtual world
21:30and tracking stops even for a moment
21:32it's basically like imagine walking
21:34around the street and the entire planet
21:35just goes boom like that right if it
21:38does that your body just goes boom it
21:40just it just it doesn't know what to do
21:41we actually had somebody who was working
21:43on one of our applications in a VR
21:46environment he was walking around and
21:47there was a bug in the system and what
21:49the bug did was it instantaneously
21:51turned his entire environment 180
21:53degrees so the floor became the ceiling
21:55and did that instantaneously and his
21:58body didn't know how to respond it
21:59actually just went into a panic mode he
22:01fell down to the floor immediately just
22:03instantly he was fine
22:07but but you know it's like this is where
22:10a bug can actually cause some real
22:12issues and so we have to be very careful
22:14and we have to be very cognizant of what
22:17we're doing in terms of the design of
22:18these applications input is another
22:21issue we haven't really solved the input
22:23problem in VR yet as i mentioned
22:25everybody wants to do this they all have
22:26their hands and that's natural where
22:28humans we that's how we interact with
22:29the actual environment typically we have
22:31our hands turns out hand tracking by
22:34computers is really hard it's um it's
22:37it's it's getting there it'll happen
22:39eventually but it's really hard it's a
22:41little bit like voice recognition in
22:42computers so you know voice recognition
22:44for many many many years was just bad
22:46and even if you get to something we have
22:49an input mechanism whether it's voice or
22:50hands you think talk about boys if it
22:53gets to the ninety percent ninety
22:56percent accuracy right that's failing
22:58ten percent of the time and the thing
23:01about humans right is that we we
23:02remember all of the failures we don't
23:04remember the successes so if you have an
23:06input mechanism the fails ten percent of
23:07the time you're just going to just going
23:08to give up on it and you're going to try
23:10to use the thing that works 100 of time
23:12which is why voice search is difficult
23:13because typing is easier voice search
23:15has gotten much better now by the way
23:16and that'll happen with hands too but
23:18right now what we stuck with for the
23:20most part is what the industry seems to
23:22be converging on his controllers so you
23:24get these track controllers so the vibe
23:25has them oculus will have them Sony
23:27Morpheus has the the ps4 controllers and
23:30those are that's another that's another
23:33you know the human is really coming
23:34across the bridge not the computer
23:36that's kind of a crotch you have to hold
23:38these things not entirely it is useful
23:41to actually have haptic feedback because
23:43another issue with virtual reality is if
23:45you press on things they don't exist and
23:47in that that feels odd so having a
23:50button you can actually press as helpful
23:52other issues with with thinking about
23:55designing in VR is seated versus
23:57standing I think VR is going to sell a
23:59whole lot of swivel chairs it just you
24:04know it's like this and then you oh I
24:05can see this new 360 but probably the
24:10biggest challenge I've run into as a
24:12designer is that it doesn't look like
24:13what I think it's going to look like so
24:16years and years of intuition and
24:18experience of working with 2d things
24:19even if you were a 3d
24:21designer what you see when you're
24:25designing over the last you know many
24:27years is it you you see everything as 2d
24:30and so you think oh well I can I can
24:32make something i mock it up it looks
24:34beautiful and I'll just pop that over
24:35here into VR and it's horrible and it
24:38doesn't work at all and our intuitions
24:39about what looks good represented in
24:42three dimensions just don't match with
24:43reality and that's actually something
24:45that you really as far as i can tell you
24:46can only learn from practice if your
24:49design discipline is to actually
24:50translate you know ideas into
24:52three-dimensional objects if you're
24:53working and you know if your artist
24:54you're working in sculpting or perhaps
24:55architecture you might have this skill
24:58already but but for me it's been
25:01difficult and i'm having to dig up some
25:03old knowledge so I I did not set out to
25:08become a designer I so why I didn't
25:11study a design and instead of
25:12human-computer interaction I actually
25:14have a theater degree I have a degree in
25:17acting mm-hmm and but it so for a long
25:21time that was just a piece of paper on
25:23the wall good job John yeah in which
25:25college but now as a designer working in
25:30immersive technologies in VR and AR I'm
25:32having to dig up my theatre history my
25:35study of staging and lighting theatre is
25:38actually a really interesting medium for
25:39VR and thinking about VR because unlike
25:43say film or television in terms the
25:44narrative in film and television the
25:46narrative is controlled by the director
25:48they get to control the shot they
25:49control the framing they can show where
25:50the camera goes the control the editing
25:52and theater there's no control over the
25:54camera y'all each have your individual
25:57control of your own camera you get to
25:58look wherever you want and and there's a
26:01certain amount you can do in terms of
26:02staging it and crafting it and putting
26:05the scene but you know and maybe you
26:06have a theater in the round maybe you
26:08have a proscenium etc but there's lots
26:11of things that go into actual theatrical
26:12design that I'm finding to be very
26:14useful to be our design set design
26:15lighting as i mentioned architecture is
26:18clearly you're building virtual
26:20environments and architecture is about
26:22building things function that function
26:24for humans and now you're having to
26:25build virtual environments that function
26:26for humans ergonomics as i mentioned you
26:30know knowing like how far humans can
26:34their torsos how far they can turn their
26:35necks being able to turn your neck this
26:38way is different than if you're holding
26:41cardboard right your arms up and so
26:43you're actually restrained a little bit
26:45so if you're actually designing an
26:46interface that you expect to fit within
26:48a certain field of view or a certain
26:50radius you actually have to take into
26:51account how flexible humans are and
26:53whether or not they look up like this
26:54they look up like this as you turn your
26:57head this way you're you're really
26:58increasing the amount of pressure on
27:00your neck muscles the amount of weight
27:01it's actually hitting you as you're
27:03looking down so we have to be really
27:04careful about where we put things in VR
27:07so that we're not stressing our humans
27:14and you know I as I was thinking about
27:18theater and kind of going back to my my
27:21studies in theater history I remembered
27:22a word which I will probably slaughter
27:25because it's German uh guess Thompson
27:28guess Tompkins Burke did I say there it
27:30is anyway no no cuz Sam sconce work
27:32which is a work of art that makes use of
27:34many art forms and strives to do so this
27:36was actually popularized by Richard
27:38Wagner as part of his his theory about
27:41drama and the Opera and the idea is
27:44you're taking lots of different art
27:45forms lots of different things and
27:47composing an experience together in
27:49harmony with each other and I think
27:51about that a bit when designing for br
27:54so yeah so it's complex and complicated
27:57and there aren't any good tools yet if
28:00you want to get into BR design you want
28:02to prototype quickly in VR design you're
28:04probably your best bet is to learn unity
28:06a lot of the skills today for VR exist
28:09and a lot of the tool sets come out of
28:11real-time 3d game design so unreal Unity
28:15game engines etc but it really is that
28:18there's no tool yet for creating VR
28:19experiences I think those tools will
28:21come but presently it's pretty difficult
28:24the other thing too is it's gonna I
28:26think be it's going to require more
28:29specialization in roles so I think we
28:31saw a pretty big transition from from
28:34desktop web design days into mobile
28:36software application design we had a
28:38fair amount of specialization in roles
28:40you started getting you know Google we
28:42started hiring motion designers right
28:44you know more visual designers
28:45typographers because it is
28:47let's get better and and you know
28:50prototyper engineers who can really you
28:53know squeeze out that every last bit of
28:55frame and performance and all of this is
28:57a consequence of sort of the complexity
28:59of delivering a high quality X program
29:03high quality experience on mobile well I
29:05think the same is going to happen in VR
29:07except you're going to end up with more
29:08specialization in a 3d generalist you're
29:10gonna get modelers you're gonna get
29:11light people it's again it's like the
29:13theater you know people who focus an
29:15audio design sound design environment
29:17design lighting design right and they're
29:19all going to come together to try to
29:20create these experiences for people in
29:21VR if you intend to be working as a
29:26designer over the next 10 or 20 years I
29:31think this level of complexity is headed
29:33your way and I think that design for
29:36virtual environments and designing for
29:37augmented reality environments and the
29:39Internet of Things and whatnot I think
29:41that's all good I think that's all going
29:43to land in your lap if you're working in
29:44design think of it about cardboard right
29:47cardboard is just this you know natural
29:50end points to a lot of the trends that
29:53we see in terms of miniaturization of
29:55compute and access to bandwidth the
29:58distribute distribution of these things
30:00right it's just going to get is just
30:02gonna be more and more of it and as I
30:05said it's largely driven by mobile right
30:07now two to three billion people on the
30:09planet have access to a network
30:10supercomputer in their pocket and by
30:152020 that number is going to be about
30:17six billion people so on the order of
30:19you know most of humanity will have
30:21access to a network super computer
30:22smartphone technology concurrent with
30:26that will be an explosion of other
30:29lesser devices that still have computer
30:32and and wireless radios in them we're
30:35starting to see things now we call it
30:36the Internet of Things home automation
30:38systems but there's also my friend Scott
30:40Jensen works on a project called the
30:42physical web and it's about just
30:44embedding almost anything with just
30:46enough compute to broadcast a web
30:48address that just connects you to some
30:50service in the cloud right and then
30:51suddenly anything can have anything to
30:55have a website but if it has a website
30:57that I can have a nap and have an
30:58interface and how much of other stuff
31:00always networked always updating n
31:03points for VR if we get to a point where
31:05we completely solve all technological
31:08problems with Virtual Reality you get to
31:11replace reality that's what you get you
31:13get to teleportation if you're capturing
31:16reality and so we're working very hard
31:18on capture technologies that's one of
31:19the things you can see over in the
31:20cardboard booth area is the jump camera
31:23and that is the ability to capture 360
31:25you know fully immersed stereo video and
31:29so it's it's one of the best of its kind
31:31in the world you think about 10 or 20
31:34years in the future you basically be
31:36able to fully capture an environment
31:38like this completely totally in 2020
31:42resolution potentially in real-time and
31:44rebroadcast that back to someone else
31:46right if its recorded that means we
31:49built a time machine because you can
31:52just go through any captured environment
31:55right and just really play it back in
31:56time we can also with virtual reality
31:59create in hot you know entirely imagined
32:01realities so we can we can bring p if
32:04you want to go study at hogwarts that's
32:06fine we'll just make a hog works for you
32:07can go do that augmented reality is very
32:10similar anything that you know can have
32:13an interface will you can just broadcast
32:17an interface to everything you won't
32:18need phones anymore at least you won't
32:20need all the black mirrors you won't
32:22need the phone you won't need the tablet
32:23you won't need the the display you won't
32:25need this right I mean again this is
32:27about fully realized augmented reality
32:29it's why distribution and I don't claim
32:31that AR and VR Tech is going to
32:33accelerate in the same way than mobile
32:35phones do I think mobile phones occupy a
32:36special niche but you know but in
32:40augmented reality there's no need for
32:41sort of these these these physical
32:43manifestations of displays and
32:45interfaces but that's all just raw
32:49technology right that's just pure
32:52potential and you know those of us you
32:54know we've worked in technology for a
32:55long time you know the technology can be
32:57very positive we know it can be negative
32:59you know that all this complexity can
33:00have unintended side it Quint side
33:07often like vision videos will paint a
33:10picture of you know of the optimistic
33:13side of potential so if everybody has a
33:15supercomputer and we have total
33:17virtuality we have total total augmented
33:19reality then you know we're going to get
33:21all this like amazing stuff it's gonna
33:22be great it's going to be this positive
33:25future of jet packs and if we do nothing
33:30if we do nothing about technology if we
33:33don't design it right no design is still
33:34bad design then we get all you know just
33:37just sheer entropy means that all of the
33:40complexity of the technology that we're
33:41generating and giving to humans is going
33:44to generate these horrible unintended
33:46sign side effects you don't need just
33:48you don't need malice or abuse right to
33:51create you know bad effects of
33:52technology simply simply complexity and
33:54error will take care of it so why does
33:57this matter why does this matter for
33:58design so I think mostly design is
34:03reactive to technology not entirely but
34:07I think a lot of design is is reacting
34:09to technology I think we're you know
34:12creating solutions within the
34:13constraints of the technological system
34:15that we inhabit or we develop
34:18opportunities as they manifest but i
34:21think the discipline of design needs to
34:24get better about getting ahead of the
34:26power and the complexity and the entropy
34:28that lies in our future because i
34:30actually think it's arriving pretty fast
34:32and i don't think that we're going to
34:34have time to be reactive i think we're
34:36going to have to be proactive if we want
34:37to create a positive future and i think
34:40we need to anticipate these trends in
34:41technology and we need to understand
34:43their potential and the pitfall and we
34:46need to design a future that magnifies
34:48the potential while heading off all the
34:50negative outcomes or at least minimizing
34:52them i think product design I think
34:56designing of products within that
34:59context is actually insufficient I think
35:01we need to design systems and platforms
35:03and tools invest in meta design
35:07investing in our own tools as designers
35:10as a design community our own approaches
35:11getting those better it's conferences
35:13get start an understanding you know the
35:17order of things right understanding in
35:19order to create solutions which
35:21which withstand the onslaught of the
35:23complexity that our own technology is
35:25creating so the way we do that is is
35:29listening carefully which designers are
35:33generally good at paying attention to
35:34developments in the in in the fields at
35:36the edge of technology you know be
35:38looking at those things that maybe
35:40aren't in your area of expertise that
35:42might be on that on the very edge of
35:43tech getting outside of your domain get
35:48close to this new technology try it out
35:50very pleased to see that there's a lot
35:52of hands for people who've tried virtual
35:54reality even though it doesn't really
35:55exist as a consumer technology yet try
35:58out home automation even if it's kind of
36:00broken and clunky I think consumer
36:02technology also has a bit of a glitzy
36:04component and so don't forget enterprise
36:07tech don't forget accessibility
36:10technology in many ways a lot of
36:12technology developments are on that are
36:14on the forefront that are on the on the
36:16bleeding edge are to assist people who
36:19have disabilities if you think about
36:21like you know right now it's possible
36:24for people to be fitted with with Bionic
36:27arms that that you know read brainwaves
36:29and allow them to actually to manipulate
36:31their to manipulate their their Bionic
36:34arms you know almost like telepathy
36:36right and you think about that
36:37technology and you built it's like oh
36:39they're interfaces they think about it
36:42what might else we apply that capability
36:45to but those kinds of texts enterprise
36:48and accessibility tech is often ahead of
36:49the curb curve so it's very useful to
36:51pay attention to those if you can I do
36:54suggest you know trying to devote some
36:56or all of your energy to working on
36:57platforms to working on systems of
36:59designs working out tools of design this
37:01is one of the reasons that I worked on
37:03material design what what drew me to it
37:05was because it was an abler for a
37:08positive outcome hopefully to create
37:10more good design in the world mm-hmm
37:13anything that can help us make sense of
37:16the complexity in which we work right
37:18anything that can help us as designers
37:20manage our own complexity so that we're
37:23creating better software or better tools
37:25for others and we need to you know tools
37:28that provide enough abstraction so that
37:30we can we can create you know positive
37:32solutions and minimize unintended
37:34I think the best outcome is that we you
37:38know we tell a story about the kind of
37:40world that we want to live in and that
37:41the story that we tell is believable and
37:43it's detailed enough that collectively
37:46you know the engineers of the world and
37:48the product managers of the world and
37:49designers the world the builders
37:50everybody together can work together to
37:52create that future so we have to be
37:53thoughtful about it and it has to be
37:55true to what technology can do but also
37:59aspirational of that technology we have
38:01the ability to shape where the tech goes
38:04we have an ability now today to tell a
38:07story that would direct these
38:09foundational technologies into a good
38:10place and not to a place that might
38:12cause problems and if we've anticipated
38:16and we've acted ahead of the onslaught
38:18of complexity if we've prepared
38:19carefully for it then by building these
38:23tools and working on these systems the
38:24network effects of this will be
38:26amplified and will create positive
38:28outcomes and positive systems and the
38:30world will be much better for it so get
38:34ready I'm work on those things and make
