00:08hi everyone thank you for joining us
00:10today I'm Anthony buona fede and I'm
00:13really excited to have David Burns here
00:16today in Karen Wong this event is being
00:19brought to you by NYU X in collaboration
00:23with the UX community in culture group
00:26here at Google and why UX is an
00:28organization that was started not too
00:30long ago to bring the UX community
00:33together here in New York to to learn
00:36and be inspired I do want to invite you
00:40all to a reception we're having
00:43afterwards we'll be here for about a
00:44half hour after the event there'll be
00:46some drinks and there's some food with
00:48no further ado I'm gonna hand it over to
00:50Karen and David welcome thank you so
00:59much for coming out it's great to be
01:02back today we're going to talk about a
01:05topic that has confounded that we're
01:10both passionate about which is what is
01:12the immersive so I was trying to think
01:17about what was the first moment one
01:20might experience immersion Wow okay here
01:31and and so you know just this notion of
01:34baptism and being being dumped but what
01:39I thought we would start here which is
01:43it's called the elemental theory of
01:45presence and can by who is a VR blogger
01:49made up this chart I think immersive has
01:53been so tied with the virtual reality
01:55tribes and certainly I think in today's
01:58presentation were interested in a
02:00broader spectrum of what immersive can
02:03be but this chart really deals with a
02:07lot of the basic I think components of
02:12so what he's done is he split it up into
02:14this kind of earth fire water scenario
02:17but if we just look at the words it's
02:20this notion of an embodied presence
02:22somehow that you are transformed or that
02:25you feel taken somewhere else the notion
02:30of an active presence that you have some
02:32kind of agency emotional presence that
02:36you feeling something maybe it's empathy
02:39distraught and then a social and mental
02:42presence meaning there's some type of
02:43cognition that you're hopefully gaining
02:46some knowledge so before I asked David
02:50what is his most interesting immersive
02:52experience of late so he can kind of
02:54think about it because he didn't know I
02:56was gonna ask him that I was going to
02:58share with you what my most interesting
03:01experience has been in the past couple
03:03of years which is Walter de Maria's
03:06lightning fields what it is is it's
03:09simply a project that this land artist
03:13worked on for almost 15 years before it
03:16was even ready to be presented it's a
03:19field of lightning rods and the notion
03:23is that at some point there'll be a
03:25rainstorm and then these rods light up
03:27for the most part people who go visit
03:30this they never see the lightning but
03:32what you see are these stunning steel
03:35poles that you'd literally have to spend
03:37a full day and evening and then you wake
03:40up at dawn and you look at something
03:43that is in this majestic field with the
03:45mountains behind it was the first time I
03:47had actually seen the milky way so
03:49present in the evening and so for me
03:52that was truly in an immersive
03:55experience David what was your favorite
03:58immersive experience of late I think
04:01thank you for giving me time to think
04:03about the last thing I did was the what
04:08is it the the Met workout do you know
04:10about this I do I'm a suspect of it go
04:14ahead and tell us about it so these
04:18choreographers Monica bill Barnes and
04:21company there's a very small
04:23choreographic company
04:25and a Manilla straighter and writer
04:28Myrick Hallman and some others they did
04:30I might still be running we might have
04:33got an extended they do a thing at the
04:35Met Museum where in the mornings at 8:30
04:38in the morning you do a kind of work
04:39they lead you in a workout and they
04:42should but I may have a picture of it
04:44somewhere where they you meet and
04:48they're in like sparkly dresses there's
04:50nobody in the museum except you and of
04:51some of the guards and it's a group of
04:53like 20 people and it basically just say
04:57follow us do whatever we do and then you
05:00know they're doing jumping jacks and
05:02doing stretches they're doing this and
05:03they're running through the galleries
05:05and then they'll stop in front of some
05:07work of art and do some kind of stretchy
05:10thing and then run to another work of
05:13art and you go why do we stop in front
05:14of this one why are we looking at this
05:17but it just keeps going they have a guy
05:19that goes with them who has a backpack
05:21with a little uh in a laptop where he's
05:25playing kind of 80s might even be before
05:2880s kind of dance music like you would
05:29get in an exercise class
05:31so there's Bee Gees and stuff like that
05:34naturally yeah I talked to them after he
05:37said he said you were a full-on
05:39participant because I'm sitting I'm
05:40singing along with the exercise it's a
05:46terran might have some some thoughts but
05:50it's their ideas it's a very very
05:52different way of experiencing a museum
05:55besides being a lot of fun and and I
05:58kind of I did sweat but it's a very very
06:02different way of experiencing a museum
06:05going through that way and running and
06:08jumping music pumping and all that kind
06:10of stuff so that was the last thing and
06:14and then the way we made this PowerPoint
06:17is we used a Google shared folder and so
06:21both of us were throwing a bunch of
06:23images actually dozens and dozens and
06:25they quickly fell into four areas so
06:29we're going to quickly run through them
06:31starting with outdoors
06:33okay I'm gonna jump into outdoors stuff
06:39definition of immersive includes kind of
06:43social gatherings outdoors indoors
06:46wherever this is something this is a
06:49picture that the the urban planner Yan
06:52Gale based in Copenhagen but he's been
06:55an advisor to New York South Paulo
06:58Melbourne all cities all over the world
07:01his intervention in this place this this
07:04place where all these people are
07:06relaxing and meeting and talking and
07:08socialising used to be a parking lot
07:12that used to be a parking area and
07:15simply by removing saying no no more
07:19it's not no more parking area the shops
07:21started putting out tables and it turns
07:24into a gathering place simple as that
07:26and people love it this is the March
07:31women's March that was a nice sign about
07:33the Trump loves Nickelback I went there
07:37and and it was a fully immersive
07:40experience you it was a very very
07:43different experience being there amongst
07:47hundreds of thousands of people and
07:49watching it on TV or on your device or
07:52whatever it's kind of the thing where
07:54the people who went there had this
07:57ecstatic transcendent experience that
08:01lasted them very hopeful that lasted
08:03them four days this ole ARP who know
08:08anybody know about LARPing or
08:10participated in LARPing yeah but somehow
08:13I thought you might have a few people
08:16this live-action role-playing kind of
08:24it's described as an extension or
08:27whatever of like Dungeons & Dragons and
08:29other kinds of games like that but
08:30played with live people real people
08:33people kind of be the carrot you are the
08:37character your role is defined and you
08:40have days or whatever to kind of reenact
08:42what it is a lot of times no surprise it
08:45the visual kind of is an extension of
08:47Dungeons and Dragons as well but it's
08:52there's a lot of people here there's
08:54things the groups can get very large
08:56they don't always have to do with
08:59medieval warfare this one is a
09:05role-playing they're playing the role of
09:07Stanford professors and actually I made
09:13that up but but this it's the UK LARP
09:19Society and you can see they're not all
09:21there the age range is all over the map
09:23this is one I think is another do we
09:25have another one LARP yes this is one so
09:28it's not all medieval this is Vietnam
09:30you can reenact with the what the
09:34Vietnamese call the American War and now
09:39we're going to move so that is a fully
09:42immersive thing you're you play the
09:44thing you you act it out you have to
09:46stay in character for the whole thing
09:51it's very popular very very popular in
09:55Scandinavia hasn't fully gotten into
10:01American culture but who knows it's
10:04coming it's coming then we saw we
10:07noticed that a lot of our images were in
10:10the realm of exhibition making and so
10:14David threw in this image which is Doug
10:16wheeler at David's Werner and he and
10:19James Terrell were part of the
10:21California light and space movement in
10:26the 70s and what I thought was so
10:30interesting as I went back and read some
10:33essays that were written about them at
10:35the time and you know what they talked
10:39about was how I'm gonna get the quote
10:42right so that um their medium is not
10:47light or new materials or technology but
10:50perception and with wheeler he wanted to
10:54reshape and change perception while
10:56tural often used the words that he
10:59wanted to extend an enhance perception
11:02and that it was this notion of
11:05the space between the architecture that
11:07they were most interested in so I
11:09thought that was very poetic they're
11:11really those guys are really interested
11:13in science science neuroscience I've
11:16talked to some of them they're really
11:17interested in how the brain perceives
11:18things especially when it miss perceives
11:22things and then you know with this
11:25notion of K all right some artists are
11:27dealing with the space in between the
11:29architecture obviously there are a lot
11:32of folks who are using the actual
11:34architecture to create some kind of
11:36immersion and this was something you
11:38went to david the museum a Museum of Art
11:43and Design at man yeah this is so what
11:46you stuck your head we show what's
11:48called Center I keep learning to call it
11:49smell but one of the thing was you stick
11:55your head into these kind of yes and you
12:00get in a smell comes up through one on
12:04one thing relationship with the smell
12:07this one yes this was at the new museum
12:11pippilotta wrist she built this kind of
12:15hat structure so that way you could be
12:18immersed in the single channeled videos
12:20and so you were right up there in front
12:23of the screen as well as that there was
12:26two speakers behind you I thought it was
12:29such a successful way to do it because
12:32often times when you're in museums you
12:34know the video pieces they're kind of in
12:35the corner sometimes on the floor and
12:37there's so much ambient noise that
12:39there's no way to pay attention to it I
12:41think I mean I think it's brilliant
12:42because it when you stick your head in
12:43there you've made a commitment you made
12:47a commitment to watch that for a certain
12:48amount of time instead of just glance at
12:50it and walk on and here is an exhibition
12:54at the Jewish Museum of Mazzone Devere
12:56which is a very famous modernists home
13:00in Paris which you know you can't really
13:04get into so what they did is they
13:05constructed some of the rooms in VR and
13:09it was one of the most elegant
13:10presentations I had seen in a while in
13:12terms of here is this simple stool it
13:15swivels and you have the
13:17goggles you know right next to you and
13:19there's no straps so I thought that was
13:22really well done and then expanding on
13:26this notion of architecture this was the
13:28teddy bear project built in the new
13:30museum an exhibition in artwork of 3,000
13:34images of people and their teddy bears
13:36and folks would just meander through
13:40this set design essentially and it in
13:44many ways it was a very powerful notion
13:47about memory the draftsman Congress at
13:52the museum I mean I think four years ago
13:55about Powell Alzheimer and in this case
13:58the artists built this architecture
14:00which originally was blank and then he
14:03invited the public to leave their mark
14:06until at the very end it became this
14:09cacophony of noise of visual noise and
14:13the show finished by the artists
14:17actually cutting up all this all of the
14:21work up and then giving it back to the
14:26we saw manifesto what did you think I've
14:28manifested at the park Armory David I
14:30was surprised how much I liked it he's
14:33really beautifully done this was Kate
14:35Blanchet playing multiple roles in
14:37characters sort of heard during Cindy
14:39Sherman and but but acting actually
14:41acting it out and the text that you were
14:44reading she performed were all kind of
14:46art manifestos throughout the say 20th
14:49century and she would do like one as a
14:51schoolteacher doing it one was like a
14:53newscaster doing it that was a really
14:55good one some of the locations shot
14:58around Berlin I think we're really
14:59extraordinary so again you know I'm
15:02using the space of the park armory to
15:04really move the crowds through so you
15:07would see different gatherings and and
15:10fascinatingly this artist Julian
15:14has also made this into a 90 minute
15:19normal film which actually premiered at
15:21Sundance a month ago and then right now
15:25at the here Shorin we have the renowned
15:29Japanese artist yeah
15:30okusama and what's very exciting for
15:34this artist is that again that this is a
15:36museum who's probably going to break all
15:39attendance records with this show she
15:42made most of these works she constructed
15:45what she called environmental
15:46installations again in the 60s and 70s
15:49and they were called infinity rooms so
15:52here just showing you a number of them I
15:56think will be well worth it to to go to
15:59the here short and right now I just read
16:03an article that their membership is up
16:05two thousand percent so that's that's
16:09what immersive environments are doing
16:11for the museum world and then we're
16:14gonna play this video I hope a lot of
16:18you guys came and saw the pippilotta
16:20risk which was our exhibition over the
16:24winter time it also broke all records in
16:26our 40-year history so female artists at
16:30the Museum so we were very proud of this
16:33show and again one of the reasons we
16:37understood it to be so popular besides
16:39having furniture and it being an
16:41Instagram phenomena was this notion that
16:44you know people really came in there and
16:48found their own ways of how to
16:50experience this work sometimes you know
16:54one might call it superficial but on the
16:56other hand pippilotta specifically said
16:59she was creating spaces so that the
17:01community could gather and that maybe
17:04there would be new connections made and
17:06and I was quite suspect of her of what
17:13she was thinking her work would do and
17:15then completely of course mesmerized
17:18when it actually started to happen and
17:20then this was on the fourth floor where
17:22she encouraged everyone to lie on a bed
17:24and look at the screens that were
17:26attached to the ceiling the next section
17:30is installations rain room at MoMA I
17:35think we all know what this was about
17:39here it's a backdrop for a proposal so
17:44which made me think of you know it
17:46happens at big sports stadiums so that's
17:49that's a Knicks game
17:50anyways and then in this case you
17:54actually have done this experiment this
17:56is called famous deaths what is this
17:58about a container like this and there's
18:08sound audio smells all that kind of
18:12stuff inside there that have to do with
18:15oh it's it's it's kind of sick Kennedy
18:19Kennedy assassination princess died
18:24quite a number but you're kind of in it
18:28in a black box immersed in that kind
18:30that those moments and I mean and did
18:35you actually have that experience of
18:38like I mean what was that experience
18:41like that just looks very weird to me
18:43and by the Dutch you're not coming out
18:49completely traumatized as you probably
18:51should thank you yeah here's another
18:54picture I mean it looks there's the
18:56scent this can say the Google canisters
18:59with scent so I mean and that's another
19:01interesting thing that keeps popping up
19:03this notion of I think scent has always
19:06been derided as something fairly
19:08insignificant within our five senses but
19:10certainly in installations it keeps
19:13okay I mean I'm turning this one / -
19:15okay over the holidays I went to see
19:19some friends in Santa Fe and I know some
19:21of you probably know but this
19:23installation that George Martin helped
19:25support called mal wolf it's a it's he
19:30took over a bowling alley and built all
19:33the stuff in there and invited mainly
19:34local artists to take over different to
19:37take over different rooms and the public
19:40comes in your pay and you can't find
19:43your way through so it's a little bit of
19:44a maze if there's a story there it
19:46wasn't apparent to me it's a really fun
19:49way to wander this is I think a little
19:51video this is like how you get from one
19:53room to another you crawl with some
19:57little video action here we go this is
20:01how you get from one room to another you
20:03crawl through this fire you crawl
20:05through a fireplace that was kind of one
20:08of the things that people say oh yeah
20:10it's really cool you crawl through a
20:11fireplace and then in another room I
20:13think is gonna whoops in another room
20:15you crawl through a refrigerator again
20:20yeah you know we call that just
20:25renovation very fun ways I so with
20:32supporting local artists making this
20:33very immersive thing hugely popular a
20:37lot of people carry a lot of whatever
20:40various raises a lot of questions but a
20:43lot of fun people really enjoyed it and
20:45they're gonna they're gonna franchise it
20:47and do it in other cities I'm gonna just
20:51quickly run through I want to get to
20:55this one which was this was one of the
20:59more successful things I saw at Sundance
21:01new frontiers with all the virtual
21:03reality installations that were going on
21:05- labs neuro speculative afro feminism
21:09essentially don't let me break it down
21:12black female identity you basically come
21:16into a beauty parlor
21:19you sit down in a fabulous chair that
21:22swivels and then with your headset you
21:26are transformed into a virtual beauty
21:30parlor where you become a character who
21:34is spoken to by a guru but again
21:39something where what I was fascinated by
21:43was this notion of being able to a
21:46combine both the physical and digital
21:47world so felt more seamless stage stage
21:53is really big for immersive I think
21:55magicians have been doing immersive for
21:57a long time they just didn't call it
22:00immersive who's this okay so visual
22:07quote from The Economist
22:11it's about mixed martial arts which were
22:15just recently allowed at Madison Square
22:18it was outlawed there until recently
22:21fans played 17 million a record gate for
22:24any event at the garden
22:27anybody who knows these events like
22:29there's a bad guy there's a good guy
22:31there's a kind of narrative it's there
22:33were women and women more than others if
22:39people watch it on TV but it kind of
22:42people the performers the wrestlers
22:45whatever jump into the audience people
22:46watch it on TV but really really the
22:49experience is to be had live like
22:53gladiators where we have not moved much
22:57further than Rome music festivals are as
23:03some of you know are really really
23:07musicians are making money on live
23:10performances live concert tickets in
23:12America more than doubled in the decade
23:14to 2015 theme parks etc have all those
23:22things are kind of about physical
23:25immersion actually this is really an
23:35immersive experience of about Imelda
23:44Marcos it's gonna be restaged in Seattle
23:48next month and it takes place in a kind
23:52of a fake disco so when you as an
23:54audience member come in you're standing
23:56the disco music is playing and the
23:57lights are turning and all that kind all
23:59that kind of thing and in for most of
24:02the show that the actors perform on
24:04platforms around you on the periphery
24:07and then sometimes these kind of ramps
24:09come out and they kind of move through
24:11the center's and then it by the end the
24:13performers are right there mixed with
24:15mixed in with you singing with you and
24:17people loved it there's a real story
24:20there's a this is the Marcos's who were
24:25they won the election in the Philippines
24:26went back in the 60s and they became
24:28dictators and they did horrible horrible
24:30things but at this point they're running
24:32for election and the audience is fully
24:34buys into them and loves them
24:35knowing fully well that these people are
24:38going to turn into you know this is
24:40going to be a dictator and his wife the
24:42shoes her famous shoes are never
24:44mentioned because everybody knows that
24:46it's this is another thing I went to at
24:49the public recently I had nothing to do
24:51with it the it's called the fever and as
24:56you saw the audience sits around the
24:57periphery and one of the ringers in the
25:00crowd somebody starts a little movement
25:02song and then someone and and then
25:05somebody else next to them does it and
25:06pretty soon they're all everybody's
25:07doing this movement there's no
25:09instructions like do this do this it
25:12just happens and then it goes it evolves
25:15until people are be following other
25:18people doing things like the next slide
25:20where you're in the middle I think I got
25:23asked to do this all this movement stuff
25:25in the middle you'd think it'd be
25:28embarrassing and make you very queasy
25:30they do it really really well it's a
25:33real skill to be able to involve people
25:36there's asleep no more which probably a
25:38lot of you know it's around the corner
25:39and people do that this is secret cinema
25:44it's a British thing where they present
25:48well-known movies like Star Wars or I
25:53think this is Shawshank Redemption and
25:55they you enter an environment that is
26:00like you've entered the world of that
26:02movie and I think there's probably
26:06drinks involved too and and there they
26:09pick all with really interesting spaces
26:11they charge top dollar this is one this
26:15is like you're you're in a scene in the
26:16movie kind of and it's kind of amazing
26:21and then I show you a movie that you
26:24could well you could probably watch on
26:26streaming for almost nothing but people
26:30really prefer doing it this way this is
26:33oh this is amazing amazing thing you
26:36should check it out online
26:39it's a Russian series of films I think
26:42it's 13 films and some TV shows that
26:44none of which have come out yet the way
26:46they were created was that they
26:48basically built a city a science
26:51institute populate and shot stuff over
26:55different that was to take place in
26:57different errors in like the 50s 60s 70s
27:00in so during Soviet times and in each
27:03one they would populate these this
27:06Institute and the the city with actors
27:09who had to basically live fully live on
27:13the set like thousands of actors living
27:16on the set and it was so it was a little
27:19bit like Big Brother they were picked
27:20some who were like the story but then
27:24everybody around it was just going about
27:26their business none of these have come
27:29out yet but I've heard talk to people
27:32seem some of them said some of them are
27:34really good this is give me bum bum
27:41train you me bum bum train thing where
27:46it's a theatre performance I would say
27:50immersive theater performance where
27:52there are hundreds and hundreds of
27:54performers and one audience member you
27:57were the audience member so you go in in
28:00that one you're on the wheelchair you're
28:02on the dole and you have to go you have
28:05to go and apply I think was the image
28:07before it was based no one mentioned
28:10before was secret cinema probably where
28:13you're here's a you me bum bum train I
28:16think there's another picture yeah yes
28:18you're that woman is suddenly thrown
28:21into a room with a bunch of football
28:23players she said she's in the locker
28:25room but there's other ones where you
28:28her throne yeah you're thrown into an
28:32operating theatre and you are the doctor
28:34all around you and go you have to do
28:37this now it's the financial model is
28:48all what happens is people participate
28:52as the audience member and they love it
28:54so much they and they ask them do you
28:57want to be one of the volunteers who are
28:58the performers next time and they get
29:01hundreds and hundreds of people to go
29:02yeah I'll do it I'll do it I'll be one
29:05of the football players or whatever so
29:08it goes goes all on like that and it's
29:10yeah so one of the reasons why we've had
29:15that what I would call the first chapter
29:17and now we're going into the second
29:18chapter of this talk neuro Society is
29:21I've been following this project now
29:25probably for a year it's very
29:27fascinating and so we wanted to
29:29contextualise something that David's
29:33been working on for several years which
29:35finally is being prototype tanned
29:37exhibited out in Menlo Park but I think
29:42you're gonna really run us through this
29:44project and and explain something that I
29:48would clearly say fits all those four
29:50categories of what is truly an immersive
29:53experience so I'm gonna turn it over to
29:55you okay and there's the clicker
29:57I even get get the clicker I press the
30:04press the green button some years ago I
30:08was reading about some it would seem to
30:11me very interesting neuroscience
30:13experiments that were taking place
30:14around the world and some of them I
30:18thought that sounds so cool
30:19which I would love to do that and I'll
30:22bet other people would love to be the
30:24guinea pig as well and see what that
30:25feels like this one is I did in
30:28Barcelona you're put on you know a
30:30motion capture suit you're an avatar and
30:32I became a small Spanish girl luckily I
30:39have a little know a little bit of
30:40Spanish so there's and I'd look down and
30:44see my dress and my little Mary James
30:46and then in then in the virtual world in
30:48comes mom mom of course is up here too a
30:53little girl as a giant and they have two
30:55different moms a nice mom and a bad mom
30:57so I'm pleading with the mom in Spanish
31:00please be the nice mom
31:01and it's it's pretty great so I've read
31:06about a number of these things and I
31:09thought very ambitious first the first
31:13idea was can we put a couple of a few of
31:16them in an art gallery and I got kind of
31:18more ambitious and said can we do a
31:20bunch of them like maybe ten of them so
31:22groups of maybe ten people go through
31:25from one room to another experiencing
31:29it's a enjoy kind of crazy experiences
31:34and you learn something about yourself
31:37your your brain and ultimately we can we
31:41kind of have a narrative you start with
31:42perception you go through cognition and
31:44you end up with social you end up how
31:46all your perception and your cognition
31:48affects how we interact with one another
31:50socially it's very ambitious but that's
31:54but and still a work in progress as
31:56research so the research was I went to
31:58there's a place in New Jersey where
32:00police training happens and you're in
32:02with you again you don't wear VR goggles
32:05but you walk into a projection of an
32:09incident and this is me and you're
32:14supposed to you're interacting with
32:15people who come out and they're kind of
32:16behaving erratically and somebody pulls
32:19a gun and I got shot I have a strap a
32:23taser to your to your butt to the back
32:27of smaller your back so that when you
32:28get shot that's me getting shot I
32:30basically fell to the ground I did not
32:37do the right thing but it's pretty great
32:43also a one-on-one experience but there's
32:47another one-on-one this is a group out
32:48of Barcelona called B another lab and
32:51they've there they do a thing with you
32:53and VR where you start to see me that's
32:59me in the white white of course and but
33:02I'm seeing through her eyes I'm seeing
33:05if I start seeing her body she's really
33:08good at mimicking my movements so if I
33:10just move my hands a little bit she does
33:12it too and really you really get the
33:15that's me in the black dress and and
33:18tights that experience is the what first
33:22one I read about it takes place it's in
33:24this was done by the Earth's in lab in
33:27Stockholm it was they published it in
33:30the scientific publication under the
33:32name being Barbie now they didn't really
33:34use a Barbie doll but it's a good title
33:36and here's how it works we do it we
33:39ended up doing it and it's running it in
33:41Menlo Park for the people really like it
33:46you're sitting on one kind of couch
33:48thing and the dollars on the other the
33:51doll instead of having a head it has two
33:55video cameras so when you put on the VR
33:57goggles it's live VR and you're seeing
34:00the doll's body instead of your own body
34:03and so when the the scientist touches
34:07your foot and the doll's foot at the
34:10same time you're seeing the dials foot
34:13being touched but you feel your own foot
34:15being touched by the wand think go back
34:18maybe I think we've is that a video no
34:22Amanda beam video it doesn't matter so
34:25we okay I'm gonna move ahead this is a
34:29former car showroom on El Camino in
34:32Menlo Park and that's what we did it it
34:34looks like some creepy cult but that's
34:37kind of the idea this here's a classroom
34:41where we worked up a version of an
34:46experiment that Alexander Todorov at
34:48Princeton did where you're shown real
34:51political candidates and Senate Senate
34:52races around the United States
34:54you don't ones you don't know that you
34:56wouldn't recognize and you just shown
34:58and you know two different parties and
35:01you're asked which one is going to win
35:03you see them for under three seconds and
35:05you vote on your tablet's almost seventy
35:09percent of the visitors get it right
35:11they don't know anything about them they
35:13don't know what parties they are they
35:14don't know their platforms they don't
35:15know anything we don't know exactly just
35:19talking to somebody earlier it'd be a
35:21little bit presumptuous to say well they
35:24vote it because they're more handsome or
35:25they're more pretty or they're women or
35:27a lot of factors come in but the what
35:31they voted based purely on appearance so
35:37the yes and you made a decision that
35:41affects your life based on three seconds
35:45of appearance and it matches they met
35:47basically you're told okay you met this
35:49is these are the people that actually
35:50won we did a room that was set up like a
35:55game show you you walk in you take a
35:59podium and we do two different things
36:01then we do kind of moral dilemmas where
36:03we show people clips of we shall miss
36:06scene from Star Trek where earth Kirk
36:11has to shoot McCoy to stop the Nazis
36:13takin over the world and we ask him how
36:16far would you go how much damage would
36:19you do to McCoy would you kill McCoy
36:21your fellow crew member and thespian or
36:28the answers are kind of interesting well
36:33obviously it's entertaining too but they
36:34are real moral dilemmas and we ask
36:37people to vote and then we show them a
36:38curve of where they kind of went off
36:40that their their moral sense is well in
36:44a sense relative depend it's context
36:46dependent we tried doing economic games
36:49in there you know where you kind of
36:51offer people money and you trade off we
36:53did it with these coins and tokens and
36:55stuff it to be honest it didn't work so
36:59we cut it we cut it when we were in like
37:01testing the testing phase which was we
37:05lost some money there but that's what
37:07happens this is the room where we
37:08reproduce the Barbie experiment you go
37:11in and you see it looks like you've gone
37:12into some weird cult II spa which is we
37:17like to think of it that way the doll on
37:19the chair is hidden behind the curtain
37:21yeah and there's obviously we have
37:25guides to talk you through it and yeah
37:28there's another room which you can't see
37:30the stuff here but if this is a
37:32phenomenon called equal luminance where
37:35if you have equal color intensities and
37:41lots of other black black and white
37:44information you if you're only you're
37:48only seeing color you don't see motion
37:51and you don't see depth we didn't we
37:54weren't able to stop depth but we were
37:56able to stop motion so we had like this
37:58little diorama in there with windmills
37:59turning and stuff and when you saw it
38:02with just under this single color the
38:04motion of the windmill stops then you
38:06put on these goggles it takes away that
38:08the color thing and you can see that
38:10windmills are moving moving moving and
38:11then you take them off in the bring we
38:13also stop and it's kind of like no this
38:16is not possible but this is how your
38:18eyes your eye brain works
38:22another room which we tested on
38:26there was an experiment that a scientist
38:29in London did called
38:31she called happy feet you you put on
38:34these slippers that have microphones in
38:36them and then you put on headphones so
38:38you hear the sound of your own footsteps
38:40but then the sound is manipulated we can
38:44make it sound big and deep and heavy or
38:47we can make it sound like tippy tappy
38:49and people the way they move we as we
38:53make a fight fake dance studio we ask
38:55her walk around in the dance studio the
38:57way they move changes instantly they
39:00don't realize it but their friends do
39:02you their friends laugh at them because
39:04their friends see how their their
39:06movement changes and and they're like
39:08what what the scientists wonders if
39:14there's a psychological change too if if
39:17making the the sound making you feel
39:20could walk like a troll if that makes
39:22you feel heavier more a little depressed
39:26a little sad or anything like that
39:28hard to tell this was a based on one
39:33thing that a Japanese scientist so she
39:35mojo did called disco disappear and well
39:39it's like a disco where you went to a
39:41disco room and two people stand in the
39:43corners and there's an object or focal
39:47point in the middle and the people
39:51disappear and they pop in and out
39:54nothing it's because your your I will
39:58follow the motion of the the light
40:00things and it's just a regular disco
40:02ball thing and you can also do it with
40:05objects like cell phones and things like
40:09that on a spinning kind of lazy susan
40:12thing with also with a pattern on it
40:16this is one way we're doing a CCTV
40:19surveillance room we're gonna
40:21demonstrate change blindness that you if
40:26you are watching a scene like a street
40:31scene and there are things that distract
40:35your vision momentarily you don't notice
40:38incredibly significant changes there's a
40:40film that's online that you can probably
40:43see that was done with it's the same guy
40:45we're working with we there's flashes
40:47every few seconds and something changes
40:50a car it's a street scene fixed camera
40:52the car changes the trashcan changes the
40:55awnings and the buildings change and you
40:56notice maybe 10% of it and then we
40:59rewind and go this is everything to
41:01change then people kind of go oh geez
41:05okay so this one this is a little video
41:07right you can run the video we wanted
41:11this is kind of the social part we want
41:13to try and figure out and we have not
41:16succeeded yet these are like folks from
41:18Kickstarter who helped us out because
41:20the Kickstarter people are really good
41:22at games and they're also really
41:26interested in this topic
41:27how do people you get how do you get
41:29people to cooperate and and with the
41:32common goal manage a resource a commons
41:36in a way that is sustainable there are
41:40places in the world where little
41:42villages around the lake have managed by
41:44themselves without regulation or
41:46regulations that they come up with
41:48themselves to manage the fish population
41:52of the lake because it sustains
41:53everybody so we thought let's see if we
41:57can do that so we thought we'd have you
41:59use ping-pong balls for fish and we
42:01haven't figured it out yet but getting
42:03there our idea is to do it in inside a
42:08box of some sort where the that is
42:11represents say the lake with the fish in
42:14it and is a drawing I don't think know
42:17if you can see it very well all right
42:18didn't sort of see it yes so the
42:19participants kind of fish by reaching in
42:22or using gloves and things to get in
42:24there and I fish and then it goes into
42:25their own tanks and their own villages
42:27and they get scores and they is you can
42:30either over if they over fish that bad
42:35the idea is to survive for a certain
42:39number of years a certain number of Rao
42:40H round would be like a year and if you
42:43over fish obviously there's a good
42:45chance that you will not survive so it's
42:46a little bit like there's board games
42:48like Catan and pandemic and stuff like
42:51that that have elements of that but
42:53those take hours to play we we need to
42:57have this happen in ten minutes which we
42:59haven't figured out yet so that's that's
43:03where this thing is it's in there's in
43:06Menlo Park we did three and a half rooms
43:08the idea is to do ten rooms and have it
43:12and yeah a massive massive thing but a
43:16lot of fun and you can yeah you can tell
43:20you kind of gets the experience of
43:25realizing these things about our
43:27perception our social tendencies etc is
43:31on a different level than reading about
43:34it in books not only is it enjoyable and
43:39sometimes good for a laugh but you it
43:42kind of you internalize that knowledge
43:45in a very different way than if you just
43:47read about it so I'm gonna go this is a
43:50little video we can play silently in the
43:52back so that that leads me to really
43:55kind of breakdown in terms of why the
43:59immersive for you in particular I feel
44:02with this topic you're coming from the
44:05angle of cultural production I'm
44:07probably coming in from it more as
44:09cultural critique I've been fascinated
44:12by the immersive because I see it as a
44:16fairly recent phenomena in terms of the
44:21marketed and branded and I'm trying to
44:23understand if it's a generational thing
44:26I'm trying to understand if this is a
44:30more potent experience than what we
44:35would call traditional cultural viewing
44:38which is tends to be one way I flew up
44:42to Boston yesterday to go see William
44:45Forsythe with the Boston Ballet David's
44:47like oh you're just seeing a lot of
44:48pretty people jump and but for me early
44:53on first sight you know a an avant-garde
44:56contemporary choreographer you know I
44:58felt that I had covered all of those
45:01areas where I felt transcended there was
45:06definitely emotion there was for sure
45:12intellectual nourishment the only thing
45:16I wasn't doing was being active I was I
45:19was sitting passively and I really liked
45:22sitting passively and being presented
45:25something that I could never in a
45:27million years do or or to see that kind
45:31of talent so throwing it back to you why
45:36why do you think we are so captivated by
45:39immersive experiences right now and why
45:42are you spending this project now
45:46probably in its third or fourth year why
45:49are you so committed to making this
45:56as you said was we both said the these
46:01immersive things have been around for a
46:04very very long time there's nothing that
46:08new about it but it has kind of
46:09broadened out so it's happening in art
46:12museums in theater whatever in places
46:17where it wasn't as common in the past
46:19it's kind of and well I think some of
46:24that is because we we now have access to
46:30everything on our screens instant access
46:34to everything on our
46:35but it's a different order of experience
46:38and so it actually increases the value
46:43of a unique experience that you can only
46:45have once of an experience that you can
46:48only have with other people
46:51collective experience with their people
46:53those things I have other kind of quotes
46:56and stuff about the value of experience
46:59as opposed to things then kind of in the
47:03luxury goods and restaurants and
47:06vacations and all those things are kind
47:08of booming as opposed to people are
47:11buying less and less luxury goods and I
47:15think that's yeah because it's a more
47:17it's they're thinking this is something
47:21that means something in my life it's not
47:23something to have it's something to
47:25experience and it extends into museums
47:28and theatres and everywhere else
47:31yeah it's it's a very rich experience
47:34that they can also then talk about so I
47:38did want to open it up and have everyone
47:43join this conversation so please someone
47:47wants to shout out with a question or a
47:48comment and there we go right there
48:18not yet but I'm trying to figure it out
48:21it's a little tricky and figure out
48:24there are people who have done
48:26collective musical things for a whole
48:29group of people like yourselves to
48:32collectively do have a musical
48:34experience that they participate in and
48:36I thought that I want to see how that if
49:21there's gonna be a holdout I think I
49:24think museums work quite slowly to
49:28decide if something is going to be
49:31deemed as part of the canyon and and I
49:35think virtual reality is very much still
49:39in a nascent stage if if it will be
49:43accepted as quote-unquote an art form at
49:47the Museum we just commissioned six
49:50artists to make VR works and the way
49:53we're presenting it is actually through
49:55a specialized app that you can download
49:57for free on your phone but but it is I I
50:02think one of the the big questions is
50:04okay if all these music festivals if
50:07Sundance are all looking at these
50:10immersive digital experiences how are we
50:15going to design for them because no one
50:17has figured that out yet
50:19Sundance Tribeca our experiences have
50:23been incredibly lacking almost shoddy
50:27and I feel there's a lack of respect for
50:30the people who are working in this area
50:32and so hopefully folks like you guys who
50:36are UX designers are you know will
50:39really start to think about what is
50:41going to be the proper environments to
50:43make these works more respectful I have
50:46a question like people out here economic
50:50question people are arrest wrists who
50:52had the shot in the museum okay it's two
50:57questions how does she make money on her
51:00on her work which is this question if we
51:05go who pays for it who takes into these
51:08are collectors who take things home and
51:10all that kind of stuff and then I like
51:15to try to keep that all veiled David I
51:17know I know I know I'm not asking for
51:18all the details unless what are the
51:20details but I'm wondering if that
51:22hinders people trying these things out
51:26because it's a it's a difficult
51:27financial proposition absolutely
51:31I mean museums or nonprofits do not pay
51:40participation fees the way obviously a
51:44museum is show is made is that there are
51:46a lot of philanthropists the galleries
51:50we make sure that there is a very
51:52healthy budget for those installations
51:55there's presentations to happen usually
51:59it's the gallery's problem to figure out
52:02with an artist like Pippa Lottie how do
52:05you generate income for the artists in
52:08this case there is a lot of videos that
52:13can be collected there are large scale
52:17photographs that can be collected and in
52:20some cases we do believe that certain
52:23installations now will be bought and
52:27then either represented in private
52:29foundations or in museums but yeah it's
52:32um it's not easy for people who are
52:36working in immersive or in video we're
52:41trying to help folks figure that out
52:43right now what is that economic model
52:46for for artists I think I mean virtual
52:48reality is a good case in point no one's
52:50figured that out yet how to make money
52:52with the thing in Menlo Park we charge I
52:54forget so it's not quite a my it's not
53:00quite a enough to pay for it
53:03but that's the idea that yeah rather
53:06than having objects that are for sale
53:09you pay as if you're going to the
53:10theater and that's actually running
53:12through the end of April I think so
53:16so if you're out there if you guys get
53:18called out to headquarters please go see
53:21it any other questions
53:33oh that's an art-gallery pace yeah pace
53:38is an art gallery mainly based here they
53:40happen to be doing these pop-up things
53:44they did one next door to this with a
53:46Tokyo group called team lab there was a
53:48lot of kind of computerized visuals that
53:51you walk through and so this kind of fit
53:54in and they said yeah let's try it out
53:56and they thought people in the valley
53:58are going to be receptive to the idea of
54:01art as experience as opposed to art as
54:04object yes so I mean already you can see
54:07it trickling if a gallery a blue-chip
54:10gallery in New York like pace is
54:12interested in art in technology and have
54:14invested in a space on the west coast I
54:17mean it's it's coming it's coming
54:23sorry I know we're supposed to
54:25microphone thing um I kind of have a
54:28question that goes back to what you were
54:29saying right before which is um I'm a
54:32writer here and I also write books and
54:34video games on the side and it's it's
54:37really amazing to be here and like write
54:38a few words and then millions of people
54:40read it but um I'm struggling with like
54:43I guess doing my own thing that where I
54:46can be a lot more experimental and do
54:49whatever the hell I want to and and
54:51coming here and doing things which are
54:54also really amazing but they're what
54:56Sergey and Larry want me to UM thank you
54:59sir um but I get I guess my question is
55:03you have you have this amazing musical
55:06career and you've done awesome things
55:07and you've like because of you my
55:09two-year-old knows all the words to
55:10Louis let me go sing fusee plays which
55:12is incredible um what you're also doing
55:15these things which are also incredible
55:17and also a lot more bendy but are
55:20getting exposed to a lot fewer kids in
55:23like ghettos where I grew up um so how
55:26do you balance or how do you decide like
55:29this is my crazy idea for today this is
55:31like who I'm going to try to bend it
55:33toward sorry that was yeah it's a good
55:38question it's kind of hard for me to
55:42in my career I've been very lucky I've
55:47also I think very early on I think I
55:50sort of positioned myself that I'm not
55:52gonna get pigeonholed I'm gonna allow
55:55myself some wiggle room and I started
55:58started doing music I was already
56:00interested in alright stuff but I would
56:01started doing music so I started working
56:03with choreographers with theater
56:05directors and film directors and stuff
56:08so I thought all was was very early on
56:10trying different things musically and
56:13then could build on that and I realized
56:19there's yes the lucky part is that some
56:23of those things I think what you're
56:25implying some of those things bring in
56:27an income and some of them not so much
56:37at the at the moment I have doing that
56:40the thing in Menlo Park is running I'm
56:42probably not gonna make anything from
56:43that I have a theater piece running at
56:46the Public Theater now and get Union
56:49wages for that which is respectable but
56:54it runs it's they stop as soon as the
56:57show ends in a month or so yeah so
57:02there's finding that balance of yeah
57:05income is not it's a puzzle it's a
57:08puzzle I've been very lucky but for but
57:13for emerging artists and creators it's
57:19well it's our capitalistic value system
57:22where popularity seems to trump
57:24everything I didn't want to use that
57:26word but anyways it you know it's
57:29lopsided because you know well what one
57:33could argue this this notion of you know
57:38when you're looking at immersive just as
57:42you said it's it's the focus on is on
57:45fewer people but the experience is
57:48perhaps so much richer so you know we
57:50have to figure out how to like balance
57:52these things out and
57:54what kind of cultural and social impact
57:56they'll have on us we'll take one last
57:59question before we break yes so a CT in
58:07San Francisco had to cancel their cure
58:10lies love staging and I read that the
58:13Seattle one they're trying to do it in a
58:15more like traditional theater how are
58:17you handling like the keeping the soul
58:19of the show there well still like
58:22staging it at a more it's easy that
58:25that's there's a slight misperception
58:27they are going to keep the show exactly
58:30the same with it a dance floor and
58:32they're moving platforms and walkways
58:34and all that kind of stuff where they're
58:36going to deck over all the orchestra
58:40seats in a conventional theater so that
58:43the stage and the orchestra becomes one
58:46big dance floor so they keep the
58:49integrity of the piece but the idea is
58:52they can then install it in the shell of
58:55a regular theater and so if they can
58:58figure that out and make it work then
59:01other regular theaters could do because
59:02we had a really hard time we couldn't
59:03find like a warehouse space or whatever
59:06in New York to move it into so that's a
59:10great place to end since that kind of
59:11bridges this notion about the idea of
59:14being active and and and not as well
59:19from passive to active in terms of
59:21what's the difference of immersion so
59:23thank you so much thank you David for
59:25coming here and being part of this talk
59:29and thank you everyone for coming I
59:31think I turn it back over to Anthony
59:41thank you David and Karen that was
59:44wonderful what a great way to start the
59:45week thank you all for joining us and I
59:49do want to extend the invite to join us
59:51for some wine some hors d'oeuvres and
59:56yeah and any question additional
59:58questions you have it sweetie so thanks
