Gadgetory

when I was a kid I was obsessed with visual effects movie specials how they made the rancor monster or how that water tentacle knee of this worked that's the stuff that I wanted to know about and most of the time the company behind it was Industrial Light and Magic now that same company they're getting into virtual reality they're getting into augmented reality and interactive experiences and the group is called ILM xlab the goal is to create a sort of digital Disneyland an interconnected world of experiences that let people immerse themselves in their favorite stories to ever degree they want so this this lets the mocap system sort of see where you're on space and make this view custom to your location so the character that you're seeing holy ha don't-don't-don't I'm sorry to campaigning at the dinosaur what were you saying I'll MX lab came out of a collaboration between three companies really its Lucasfilm story group and ILM for the visuals and Skywalker Sound for the sound and all of us works boring on top of the advanced development groups technology the area VR and AR and immersive technologies like these so so basically right now what I'm seeing is a dinosaur right here like at my hand basically like you know resting at the chin on my hand right now in the three-dimensional space it's pretty what you're seeing actually is Chris over there performing as as the Raptor and so he's you know responding to you and his performance gets translated over to the Raptors emotion so and all in real-time so this is nuts yeah because it's ILM they've got 40 years of visual effects experience to work with and because it's Lucasfilm they've got some pretty big movies to play with you know the ones we have different experiments that we're running right now to see which ones are going to be the most successful but the amazing opportunity here is to work with the filmmakers and the production art department and all of creators of both the standalone films anthology films and the episode you know seven eight nine films those are all those creators actually work right here in the building and spend time here working hand in hand with us so we have the opportunity to show them the latest technology show them the latest experiments in terms of storytelling and riff off of each other try the next generation of things to to see you know what is successful what what isn't and what what do people want to experience see feel shorter than I thought he was gonna be ah just what everyone says this is another kind of toy sort of related to some of our experiments with doing you know set design stuff so you can go and do sort of like virtual kind of building block yes certain things you just rough layout I might have to imagine the implications here for I mean funneling back through - ILM like if a filmmaker wants to tour a virtual set before they shoot in it or even like change things have the DP come in it's got to be edible opportunity yeah and that's actually one of the main sort of real-world applications that we use for this now one of the tools we created for Gareth on his film was to help him pre visualize some of his sets he's talking about Gareth Edwards who's directing Star Wars rogue one and he was looking at one of the sets through a virtual reality headset like the oculus rift he was in virtual reality for the first time and this was a very rough set it wasn't a high degree of finish or anything and he was in there exploring and a couple minutes in he said this looks way better in real life so I leaned over to Doug Chang who's a production designer I said Doug can you built this yet is he talking about real life or is he talking about he's like no he's talking about the VR headset oh and when when he you know two minutes into having the headset on referred to it as real life I was like okay we're onto something here it's fun yeah now can we have the Velociraptor hangout with the Threepio in our to take about five minutes to do that are you you know my whole career has been about creative development has been about designing moments and perspectives upon moments and things like this by things like this he means things like this if you saw Speed Racer you saw John's work pioneering a hacker stylized visual aesthetic depth photo anime and then this scene from The Matrix Reloaded use a technique called volumetric cinematography also pioneered by John know yeah all the time that was John to the actual experience of the matrix itself started expanding my my thought on the very topic that the movie was about so you know I joined that effort with the Wachowskis partly because I thought it was so well written how they sort of considered virtual reality completely sublime and in the perfect state it's it's flawless it's a flawless simulation you know our approach is as each film went on and then the years after was to really trying to make it in the city in the same sort of way that it might actually be made some day right to try to find some techniques that were about capturing reality and then of course it would end up as cinema you know composed and all that but yeah that we went deep into a lot of exploration and a lot of things fell out of that that you know are super relevant to to me today and what we're doing here and a lot of those friends younger people then have found their way into some pretty interesting places you know throughout the community that we you know we are watching right now for a while he left film to start his own company and focus strictly on AR and VR that's him there covered an augmented reality tank coop and if it seems like John is trying to build a matrix for real that's because John is kind of trying to build the matrix for real to quote his LinkedIn page I'm trying to design and enable platforms that lead to the matrix - machine dominance so that's reassuring and he's the kind of guy that really might pull it off just listen to him talk about the future of filmmaking and this behind-the-scenes featurette from 1999 saw baby steps towards something much larger that won't be really commonplace for a few years we're talking about cameras that are now broken from the subject matter that are virtual 15 years later so this is a virtual camera it's basically a window into the world so as a director you can see what's going on in your peripheral vision as you're framing up on something you can sort of pan over to action that's happening off-camera without having to rehearse it because you just see it X Labs is also making stuff that you might actually get to play with soon so all this is actually generated in real time from the cloud and it's streamed to this iPad on my hand and that less it's deliver a level fidelity that's way beyond what this tablet could do approaches movies or TV but at a touch of a button we can let the user take the cinematic borders away and everything is fully interactive we think experimenting a lot with story experimenting a lot with how we get people to relate you know what what is their persona in this environment that is our focus is if we can if we can solve those creative problems or at least make progress against those challenges then I think the demand and the audience will find a way to experience our stories and at the moment time is frozen so if we use a slide over here we can start to add time back and now we'll we can actually watch the whole section unfold while still having full control of the camera as well as camera control we let you interact with environment and characters so if you tap a stormtrooper we'll go into an orbit around this character and we could actually follow him through the whole piece and see everything that he sees quite literally we can jump into his perspective and now everything would play out with us here we have this prototype device in the back there's a depth sensor it's kind of like Kinect it's actually the same type of sensor because we have depth we can actually let you walk into the scene mehran Bjorn cameraman until you run into a wall we do think that people want to go further right they want to get deeper so we you know revisit a gate in the movie and observe our story sculpture you know it's like a work of art it's like a sculpture this next question is for film nerds only so I'm seeing here down here it says that it's running about 24 frames a second why that versus thirty or six so as you know yo cinema is 24 frames a second if I we start we actually physically model the properties of the camera so when you run it at 24 you get just the right amount of motion blur that gives you that cinematic feel and when we had that we did a lot of a be testing with people internally like in which one you prefer and overwhelmingly everybody thought that 24 was more cinematic the whole experience is just under two minutes you can re-experience that an infinite number of times an infinite number of ways anything that you may see should be things you're allowed to experience right not necessarily you know I mean I'm sure there are versions of this that involve gaming in competition but really what I'm talking about is kind of being in the shoes of a character like a character you saw in the story for instance I could be watching the speeder bike scene and return to the Jedi hit a button and then suddenly oh whoa I'm like on a speeder bike there's buttons here and there's weird dials here I put in your hand and so that's your speed control slow dad and lean sideways yeah you it's like a motorbike so I'm king of me leaning it turns it got the phone this is fun this was very much like a skunkworks project like we were building this other experience and in the hours after work kind of I decided let's just focus up to the oculus and see what works and like the cinematic cuts for example that you've seen we still need to figure out what the real cinematic language needs to be like when shook you cut to the uppercut there's a lot of like new hundred years of cinema learning that needs to be relearned in this medium he brings up an interesting point it took us a really long time to develop the ideas of what would make cinema work early on people thought you had to show scenes in their entirety before they realize you could compress time with cuts and evoke emotion through the juxtaposition of different imagery this is called montage and also create moods with lighting and framing and a million other things that we don't even think about anymore like you inherently know if I use a transition like this now this is kind of a cheesy low-budget video understanding of film vocabulary is so advanced now you can watch a heavily edited it seemed like this with no problem but 120 years ago people were just excited to see moving pictures in fact according to legend people saw this train headed straight towards them and jumped right out of their seats sound familiar human beings in general are comfortable with passive media story to storytelling in general right it's been with us forever and if you want to draw the masses and you have to do it in the most you know you have to baby step them deeper and deeper into you know what will become you know some pretty sophisticated types of engagement okay yeah this concludes the VR portion at the heart of it of course you know is the story group and I'm lucky insofar as I work with them directly the story group is kind of like lucasfilm's creative brain trust it's the team responsible for shaping and shepherding every piece of Star Wars content out there so they're all part of the same big story no matter what the medium that goes for new movies TV video games the VR experiences and any future form of entertainment not yet discovered real transmedia stuff that we've never been able to do before because in the past we've had one camp that was making the films and another camp that was doing the publishing and the licensing in the published fiction so now by putting them together we're able to be a lot more coherent a lot more cohesive a lot more connected and our stories going forward that's the kind of future xlab is envisioning one with interconnected experiences that go from films to VR to your tablet and back where movies are shot on virtual locations that you can then go visit with your friends and hang out in and because it's all happening under one roof it's part of the same giant interactive Star Wars story we'll see how far you know far we can get but I think that's the new model deep media highly connected smashed silo structures you know and all tapped and hubbed off of the origin creators you need to be close to that to understand how you can get emotion so it's the it's the X Factor around here you know if AR ends up being huge and VR ends up being medium-sized that works out just fine for us we've got lots of plays in both you know if even if we were to only leverage this technology for the filmmakers to be immersed in their environments that's really useful too so they could just stay behind the scenes and that would work out fine but the team at X Lab clearly had bigger plans and they're taking their inspiration from somewhere that should seem obvious we really really like working with the Imagineers because they are aiming Walt Disney you know is the godfather of experience UX and all the parks in a way are the first prototype for immersive entertainment and so there's some real foundational stuff there and their method you know this is exactly the same program modernized this is like a 2.0 supercharged version of the same philosophy once the destination is built you can wander around in it you can be social in it you can do other things you can play in it you know the Metaverse will begin like one one room at a time and what you put in the room and what people do in the room you know can be the beginning of many things right it's just like in the crafting of what this room is going to be about so I'm hoping and assuming right that people are going to create brilliant rooms right where there'll be interactions and it could be like you know just hanging out with my friends but you know folks will be more creative than that they'll create mechanics that you know make people interact in new ways I mean like I'm not sure if you've tried any other social vr stuff but it's pretty unique to be all in a common virtual space together and you're all over the world right but you are together this is one of those virtual spaces he's talking about that's John it's it feels very interesting you quickly you quickly forget that you're talking to someone who is on the other side of the planet you know you're just like they're a lot of a lot of kids already really pretty much live a life adorned as an avatar already right so let's show up let them go nuts