Gadgetory

welcome to top shelf my name is David Pierce and this week we're looking at two were the old technologies that have both found new life again virtual reality and 3d have both been tried before and they both kind of failed and they were relegated to theme park rides and weird B movies you probably don't want to see but both are back both are being called the future and both are being pushed by the movie industry the video game industry and everybody else as the thing you're going to have to have next but are they we've set out to find out 3d movies in virtual reality seemingly have nothing to do with each other but they followed surprisingly similar paths both have been mostly confined to novelty users like the snake that jumps out of the screen at you at honey i shrunk the kids at Disney World and virtual reality has basically just been for video games and sex scenes and sci-fi movies more recently though both made a move toward the mainstream and they've been tapped to do roughly the same thing even make our movies games and the whole digital world much more immersive both are even designed to replicate how it is we see the world take 3d for instance when you look at the world through your actual eyes they see two slightly different pictures because your eyes are set apart your brain meshes them into one which is how you perceive depth accurately that's also why catching a ball with only one eye open is really hard 3d cameras do the same thing they shoot with two lenses and essentially capture two images that they've been projected to your separate eyes you use either anaglyph glasses where one eye sees a red image in the other sees blue or polarized glasses which pick up light being projected in two different directions and pipe them into each eye there's also active shutter 3d auto stereoscopic 3d which is just a fancy way for saying the screen does all the work and you don't need glasses and plenty of other options they all rely on your brain to take two distinct images and turn them into one 3d hasn't really caught on though at least not among consumers and everyday people it's difficult and expensive to do right it still requires that you sit with your head very still and at a very particular angle and there are a huge number of people for whom it either doesn't work at all or actually makes them sick the technology is still changing and it's going to continue to improve but here's the thing virtual reality might eat 3ds lunch before it ever gets a chance virtual reality is also been around forever it's also been gimmicky for most of time and is only recently threatening to become something we use in our day-to-day lives on one hand it has a lot of the same limitations as 3d you have to wear a thing connected to another thing and it can be really disorienting for some people but when done right virtual reality brings the immersion to an entirely new level it's all about exploration good virtual reality creates a three-dimensional world that doesn't just appear to have debt it has debt an object will look different depending on how you look at it and you can even see it from the back you can't do that with a 3d camera basically virtual reality works by tracking what are called place cells in our brand these stills are what orient us in a new environment there how we figure out where we parked or even get around our house and surprisingly once we're in a new environment these cells make it really easy to figure out where we are even if we found our just a short time but virtual reality has its own complications and challenges for this reality to feel real there has to be zero latency when you turn your head you need to be looking to the side exactly as fast as you expect any lag takes you out of the experience and the effect as it catches up can be so disorienting that you can actually get sick and these place cells you need information like smells and temperature to really give you a full sense of where you are both 3d and virtual reality are still maturing but they're both moving towards the same goal we want our movies to feel like video games and our video games to feel like real life converts will reality bring the change 3d promise and just how far into the matrix can we really go joining me now to answer all of my many questions about all of this is Frank rose the author of the book the art of immersion and a long time contributing editor at Wired he's also fresh off his first experience at the oculus rift so Frank thank you for being here really appreciate it right there so real quick just like 30 seconds oculus rift when you think is it the greatest thing you've ever seen I'm still gripping the table yeah yeah you did the coaster and you were like yeah go over the ledge it's like oh god I'm gonna die yeah yeah it's very intense ok so I want to start with I don't know 60 years ago really so ah 3d movies are not new at this point they're old they went on for forever and I always thought they were sort of you know kitschy things that you know disney world but they weren't they were movies uh so what what happened years ago and why didn't it work well basically it didn't work because it was all analog and you know the thing about digital one thing about digits very precise you can get things to line up very precisely when you're you know 3d is basically just you're looking at two images at once and they have to line up if they don't line up you're going to get a serious headache at best and it's even like the the slightest bit off yeah totally throws everything yeah exactly in fact almost the closer the closer it is if it's not exactly on the worse it gets yeah so along comes jinx Cameron you know 10 years or so ago you know fresh off Titanic and he decides he wants to reinvent 3d for the digital age and at this point it's much easier to do because you know with digital technology you can line things up exactly it's still however a very it's it's very difficult in that you have to go to incredible lengths to reproduce an effect that we actually see quite naturally you mean just like a thing and then another right behind it yeah doing that were difficult than it seems right yeah I mean seeing you know actual objects in you're in a depth field is quite easy for about ninety percent of people but but mimicking that experience on film is very very hard and you know remains so to this day I think Cameron obviously very serious about it and an extremely brilliant director and kind of had you know all the money in the world to do yeah whatever he wanted to this plan you know I tan I could never like all right to do whatever exactly exactly so he goes off with with a sort of camera expert named ninja in space and they spend several years they developed a stereoscopic camera and I actually met him when he was in I think 2060 was at us on a soundstage in Montreal where somebody else was shooting the first feature film to be shot you know using this camera his journey uh yes journey 3d and um was really actually called journey 3d i think so that's when i get to 3d mystic easy didn't know right it's in 3d exactly so you know so i get a chance to talk with cameron about all this and what he said basically was that the whole point of 3d for him is to remove what he calls the screen plane which in a normal movie the screen plane is just the screen but it actually becomes sort of a barrier between the audience and the experience and especially in 3d so that's what he sets out to do and the point of it is to create an experience of depth of something that can you know really draw you into it and you know he knew how to do this very very well for most directors the the easy you know sort of cheap shot thing to do is to have something come out at you you know for the snake you know whatever yeah and there's that the thing at Disney World arm is like a snake that came out and bit you and terrified small children yeah right right or a hand or you know or anything like that but but you know so it really comes down to a question of what are you going to do with it and how sophisticated is your understanding of how to make it work because again it's it's it's really small and we've seen this even in the sort of digital era as we've gone to it is some people do 3d really big and that seems cause problems and some people do it really small and that seems cause problems and there's like this sweet spot that you know a couple of movies i think avatar got a lot of credit for doing it right and life of pi got a lot of credit for doing it right right but a lot of other movies people still seem to think 3d is like it's either it doesn't work or it's not really that useful but that's really the question like is it I guess done well is it useful there seems to still be this huge debate like what's this in service of other than charging me five dollars more the movie theater yeah well don't under uh I think you know done well but that's that's a very big caveat yeah you know done well it can create an incredibly immersive experience but the thing is that you know very few people do it well so is it but is it immersive just in that it looks more like real life it looks more like you know if I'm looking at the world of wherever Pandora and after our it looks more like it would if I were actually there is that the the effect here that's what we're going for that's that's certainly the goal that's certainly go you know a lot of what happens to is that people try to you know retrofit 2d movies and make them 3d and that's never going to work well so that's that's actually interesting question it's like I think 3d has it was the future of everything yeah you know 3d TVs ESPN was going to do sports we were gonna do movies everything right and it seems like that's been pulled way back is that because the technology is not there is it because people don't care is it because enough people are doing it badly that everybody has a bad taste in their mouth like why is this it seems like the it's possible to do well right so why isn't it taking over the world it's possible to do well but it's very difficult and also i think the glasses remain a barrier you know the fact that you have to wear these glasses even though they're you know they're certainly far better than that last as you were in the 50s you know it's it's it's not good it cuts down on light it makes it a you know just sort of cuts into the experience and again I think that the idea that you have to go to all of this you know sort of extreme difficulty meant to mimic something that's actually quite natural suggests that maybe it's not all that good an idea you know I don't know it's certainly it can be done but the question is you know can it be done well by a lot of people on a mass scale you know that remains to be seen right but so it's it's I mean it's interesting that I think people do want something like this and you found this in your book the art of immersion it was it was really it seemed to me that their work there were sort of two branches to the book one was like how people want to interact with stories and it's like you had the madman example where people just started tweeting as madmen characters right and it became this whole crazy phenomenon and people like they got shut down and then they brought it back and it was great marketing for the show right but then on the other hand it's it's not only do we want to you know expand sort of horizontally integrate these stories with all these other pieces of the world we want the stories themselves to be better roar is that bro is that a fair way of categorizing it Yura Yura which I mean we've clearly veered toward d you know interacting with stories I feel like that's that's happened that we we won that battle against people who didn't want us to do it Europe and people are tweeting about chose now and every you know every movie has a video game and all this stuff but so how what kind of progress have you seen towards this let's fundamentally change how we tell the story not a great deal I mean you're right in that we're we're certainly moving toward a kind of a participatory experience but you know one of the things I realized when I was when I was working on the book was that every time we come up with a new medium a new form of a new medium for techno type one of the things I realized when I was working on the book was that every time we come up with a new technology a new medium for telling stories it takes people at least 20 or 30 years to figure out what to do listen Nina and list it with some movies motion picture camera was invented around eighteen ninety it was nineteen sixteen before you had a film that really took all of what we now take for granted at the grammar of cinema you know all the sort of tricks of the trade you know cups pans fades point of view shots you know all of these things had to be invented because before there was a camera there was no way to even imagine share and it was not a ten to twelve years after that before you had sound so with television television was invented around 1925 it was the early 50s before you had something that that was not just sort of mimicking radio or mimicking stage play and I think the internet is no different I mean you know we've we've had the internet now or we've had the web for you know 22 years at this point but we're still figuring out what to do with it and it's also changing you know its advancing pretty rapidly you know at the same time you know just as movie technology was advancing rapidly so it's not really surprising that we should still kind of be in an experimental phase and I think that's where we are you know you see you see things like lost for example which was a TV show that didn't really have all that many aspects that were obviously interactive but it was so complicated that it was a it was a show that's almost impossible to imagine before the web you know I watched that show religiously and I remember reading lostpedia between episodes and it was like I'd watch an hour of TV go what just happened and then spend the next six days they were reading through what everybody else was trying to piece together and like how the characters and it's kind of true i can't imagine watching that show your without having access to all that stuff and all these other people who are watching as well yeah exactly you know I interviewed the guy who created lostpedia and you know completely fascinating I mean this was a Silicon Valley guy he you know new technology very well he you know binge don the first season and then the second season came along he you know it was like whoa what was that just about and so he did sort of you know what comes naturally to people now which is you know start start a wiki as a way of you know developing developing the story the the showrunners you know they they had I mean I interviewed them as well they had somebody on the production whose job it was sole job was to keep track of everything keep track of all the characters keep track of what they were doing what they had done where they came from where they were going on days when like a like a Bible of it all there's like mr. Eko is you know X Y and not see yeah exactly and and in the you know they have a big writers room so in the writers room there were you know sort of giant post-its that covered three walls all color-coded yeah but on so on days when this guy wasn't wasn't there when he was off or something the the producers would turn to lostpedia to figure out yeah yeah that's great but so all this seems to be is this just like a stopgap until we can actually be part of the story it seems like that's that's what you know George Lucas in Joburg we're talking about this as like the next great step is like that are our movies feel like video games in the sense that I can be part of it and I can control it and that's something else that's been tried and failed and you you wrote a little bit about this was the I forget what the movie was called but it was Roger ever reviewed it and Roger said it was the worst thing he'd ever seen and it was like a movie where you could control the plot right is that exactly the this was in the this was in the early 90s and so there was a guy who decided that this was going to be the future of movies and it was a variant on the children in the scenario which which i think is a kind of a primitive example of interactivity and almost inevitably like a wrong path yeah but but especially a wrong path in the context of a movie theater so so he created these you know not very great movies on it you know to start with and then at certain points in the you know in the movie the audience was asked to choose you know a B or C which way with the movie guts and they had taken a couple of theaters in New York and LA and so to check them out replace the cup holders with with with buttons so you know green yellow and red and you could press a button and decide where the movie was going to go except the problem was of course you weren't watching it by yourself people watching it in a movie theater full of other people or in this case actually not full of other people looked like two other people yeah and so so led to this you know scenes of pandemonium where people would start leaping over the states to get to you know they can't control them and you know it was a complete mess but the whole point of immersion we've we've always had this impulse to immerse ourselves and in stories and in you know fictional stories in particular if we see a story or read a story and it doesn't really matter what medium it's in the something that we really identify with we want to be in that world you know in our at least in our imagination and that's been a universal impulse it has you know it's something that has not changed you know you can lose yourselves in eat you can lose yourself in books and you know people still do that Don Quixote the reason he went tilting at windmills and in 1605 and servant story was because he had lost his mind from reading two months which was you know then a pretty new activity and actually is probably completely possible yeah I'm still doable but it seems like that's you know there's this interesting convergence between those two things or it's like we still want to lose ourselves in these stories and we also want to participate in them right and that's amazing or something like like virtual reality becomes really interesting and I feel like in some sense I don't know if you agree with this but I think 3d is sort of uh it's you know virtual reality is partly 3d if that's part of the technology but it seems like it's everything 3d is plus all of the other things that needed to be really be successful yeah in the sense that not only does it look deep and it sort of looks like it would be as if I'm standing in the world mm-hmm but it feels that way too yeah um and that's I mean we've had this around for not this particular thing the oculus rift but we've had virtual reality around for forever to write so what when you were doing the book and when you were working on all this of what diverse reality come up at all it came up a little bit but at the point when I was working on the book you know frankly it was which was in 2010-2011 that's not that long ago Yeah right right exactly but you know even then it was not really on the radar it was something that had been on the radar a long time ago and and and was not now I mean the people behind the oculus rift have done something that's pretty remarkable which is to you know to bring it back in a in a credible version you know obviously technology has moved ahead tremendously since the late 80s Early 90s which was when you know you first had all of the all of the hyper but you know they are right they were like the big headsets you had to like lower down onto your head run yeah sorry but now you know you still have a little headset so you know but the thing about it is it's a very personal experience you know it's just you in this world and so I think there's a lot of appeal for something like that and it's certainly immersive in the sense that you know once you're there you know if it's if the production is well done you're really in it so yeah yeah I mean and I think I mean do you think it's true that this is I mean is this where we're going next are we all gonna buy like throw away our 3d glasses and buy virtual reality glasses cuz you mentioned the big problem 3d still has is they have to wear glasses rug and this is not only glasses it's big fat goggles you have to put it on attached to a pc I mean I do think that's problematic I think that certainly you know this will attract a following how big a falling I you know I really don't know you know will it become one of the next big you know interfaces for gaming that's entirely possible um gaming seems like the easy one yeah where it's you know just the ability to stand there because I'm already sitting on my couch I already have a controller in my hand and just to be able to go like this was enough of a selling point that I'm probably gonna your get into this idea right but so what about things like we were talking earlier about Sleep No More and so many shows that you've been to where and in a way you'd sort of said in the New York Times a couple weeks ago that this is it may be a hint at what the future of movies and video games are like what do you mean by that well it's really intriguing so sleep no more of course is a immersive theater experience where instead of sitting down in the seat and steering of the stage you know behind the proscenium arch you are in the middle of the experience in the case of Sleep No More there's a five story loft building that you are free to explore and action takes place all around you sometimes and sometimes nothing is happening at all but there's always you know that sort of like and you basically I haven't been asleep nor basically they give you three hours and just started open the door and turn you loose do have your own experience however you want exactly and you know I've been twice people you know many people have gone a number of times and you never quite have the same experience twice because part of it is just serendipity at you know what where you happen to be at any particular time so and it's its interactive in the sense that you know in some cases you just sort of watching what's going on around you in other cases you actually kind of brought into the action of them but and that's kind of the dream right for all four four movies for games that's as what we want we want to be part of the action right I mean what it really comes down to is the holodeck right which is which which you know Gene Roddenberry invented in the so to speak in Star Trek The Next Generation and the 80s and um you know kind of remains the ultimate holy grail now it's that in the minority report interface see me do things everybody talks about it like that I want in life right now the minority in the minority report interface is actually fairly simple but but is a lot more complicated and it's the holy grail in the sense that you know the Holy Grail was of course something that you were never able to actually attain and so far that's what the holodeck is but what really intrigued me about things like sleep them more is in a way it sort of like the holodeck except in real life it's not a virtual experience it's a it's a real experience it's a physical experience and it's you mean in the same way as the holodeck it's it's very much sort of what you want it to be and you can create it differently every time you go in exotic was I mean III it's sort of you know even a level beyond some of this other stuff for like if you wanted a chair it could make you a chair right and I guess I don't expect you know sleep no more virtual reality to do that for me but it did sort of put you in whatever environment you wanted to be in and it would always feel different yeah exactly so in the holodeck you could you could ride the Orient Express you could have a conversation with Einstein you could engage in a Victorian Gothic romance you could do any number of different things like this and in fact there's a wonderful book called Hamlet on the holodeck that was written by Janet Marie who was done at MIT and is now at Georgia Tech in the late 90s that looks at you know digital storytelling in a very prescient way quite quite interesting for people who are want to explore that further but so with the with the holodeck you could really experienced Saenz but in a completely virtual environment obviously but a virtual environment that was completely convincing you had life-sized 3d holograms that had had you know physical presence you get that you could touch them you know that sort of thing so that's pretty fanciful stuff we haven't quite gotten there yet now right now for that but something like Sleep No More you actually are in a room with you know 3d people and so is there I mean is there a way to combine any of that stuff like are we sort of our wheel well I guess we are a long way away from being able to be convinced by something like the oculus rift that we're in a real world because it's you know it's partly about touching it's probably about being able to smell things and hear things and walk around in a more convincing way but I mean is that that we were going are we going back to adding smell-o-vision on all of this stuff so we get to the point where it all works well you can certainly had smell-o-vision to the oculus rift I suppose maybe that hard yeah yeah right but again is that is that what people is that what we want is that what we're working towards that can be James Cameron's next big project is is let's figure out how to get people even more involved in the story I think I think that's definitely where we're headed I think that's the kind of thing we want we don't quite know how to get there yet and the technology isn't quite there but but but you know that's what things like the oculus rift or four is to show us where we can go so I think my other big question here is it's it seems like our attention spans are getting shorter this is the big complaint everybody you know every movie maker seems to have is people don't pay attention anymore right but then we're making these things that literally force you to pay attention where it's like I can't wear this and do other things mm-hm I'm just like physically that's not hot I don't know see people wearing this and driving I don't want to be on the road I'm sure will happen a 3d glasses and driving now they're a good look but so is that are are we going to come back to this place where this sort of deeply immersive storytelling becomes so good and so engrossing that it takes over again or are we are we lost forever to Twitter and everything else while we watch TV well it's funny because they are kind of contradictory impulses and yet they're clearly both happening at the same time you know if you I mean look at lostpedia or you know the wiki for any TV show there's not a TV show in existence these days it doesn't have a wiki about it you know look at something like The Walking Dead which is the most popular show on TV has you know thousands tens of thousands of tweets about it every time it's on the air I have to turn off Twitter if I'm an episode behind it's terrible right right and you know has a talk show about it which is also incredibly possibly so there's clearly a a big appetite for for immersing yourself in zombie stories and in that zombie story in particular so you know at the same time that we have you know a tendency to skim the surface of things and to you know move quickly from one thing to another if there's something that we want to do a deep dive on it's there for is to do you know when I was talking with Cameron he you know we we talked for a bit about avatar which at that point was not yet greenlit it was fairly clear that that's what he was going to do next but he didn't want to talk about it too much because he didn't want to at that point giveaway the plot and so forth but he did say that he described it a sort of an edgar rice burroughs action adventure story and he said that for him the best way to tell a story like that was to create almost like a fractal experience where if you're a casual fan you can just watch the movie for two or three hours and that would be it but if you were more committed fan you can jump in almost in powers of 10 and the pattern would still hold and that's what's really happening you know with something with something like The Walking Dead you know the more committed you are the deeper you can go and I think we're seeing that you know across a whole range of TV shows and to a somewhat lesser extent movies at this point you know partly these are marketing gimmicks for the properties but they're also ways to reward fans and to get fans to engage more deeply and as television loses you know more and more of its audience in the sense of you know broadcasting is obviously not what it used to be it's becoming apparent that one way to make up for that is to bring people in deeper to get them more engaged than more committed to a show to a story and I think this is one reason you see all this happening awesome well Frank rose thank you very much ver she taking the time so whether or not virtual reality is the future of video games and movies is definitely still under debate but one thing it's already being used for is in the medical field and all sorts of other places that you may not have expected so we decided to explore some of the other places virtual reality has been used for a long time and might really be the future virtual reality's whole appeal is how immersive it is how real it feels that's great for playing games and watching movies but this crazy intense experience is already being used for much more than that the military is one of the biggest virtual reality is used to treat post-traumatic stress disorder for veterans allowing former soldiers to relive some of their most frightening experiences but in a safe environment it feels every bit is real but it's presented along with therapy and doctors and safety and the virtual Iraq program has been incredibly powerful virtual reality is also being used to train soldiers and pilots allowing them to go through their missions before there's real gunfire involved it's basically ender's game in real life doctors are using virtual reality to treat addiction even fear you strap on the virtual reality helmet then work up to seeing the exact thing you're afraid of it's simple worryfree immersion therapy and it really does seem to work we've seen firsthand how affecting virtual reality can be thanks to the hunger and Los Angeles experience at the University of Southern California verge editor brian bishop found himself wearing an oculus rift watching a diabetic man collapse on the streets of Los Angeles and he couldn't do anything about it even with primitive graphics and a big head set on his face Brian couldn't believe how caught up he was this stuff works being able to explore the world you're in has all sorts of benefits as well it lets architects walk through the buildings are creating let's fashion designers see how their outfits look or lets doctors explore your MRI in a much bigger and more useful way it's even letting doctors perform much better robotic surgery instead of just staring at a monitor it's like they took the magic school bus and went inside your body virtual reality is an incredibly powerful technology both for what it can do and for what it can make us think we're doing whether or not it's the future of games we don't know but there's no doubt it's part of the future in a number of other ways we never even imagined that's it for a show thanks so much for watching thanks to Frank rose for being here for a great interview we'll be back next Monday and every Monday with lots more to talk about there's lots more coming this summer and it's going to be awesome so stick around we'll see you next week you you