Gadgetory

so what does it take to do virtual reality right as it turns out virtual reality doesn't have to replicate the real world in order to work it just has to provide the right input in order to satisfy the relevant sensors and drive drive the correct inference to paraphrase Morpheus some rules can be bent others can be broken for example many of you will get to experience the demo of the latest iteration of the rift Crescent Bay during f8 when you do I'm confident that most of you will feel like you've been teleported into a virtual world like you're literally there and yet when you're using Crescent Bay you're looking at photons coming from pixel strobed at ninety hurts rather than photons continuously arriving from real-world surfaces and lights that works because the period during which the receptive fields on the retina accumulate photons is about 20 milliseconds and the structure of those fields is such that they respond just as well to the photons arriving in several short bursts as they do to them arriving continuously from real-world surfaces drop the framerate to 60 Hertz however and those same receptive fields will start to detect flicker drop it to 10 Hertz and motion will cease to appear smooth in fact Crescent Bay's display system differs from the real world in many ways but by taking advantage of the physiology of the visual system it nonetheless produces the desire of signals to the brain so part of making virtual reality work is learning how our sensors can and can't be driven to send the right signals to the brain the other less obvious part is learning what it takes to get the brain to make a desired inference for example remember this we were able to get the brain to infer a face shape in the corresponding head movement by leveraging the convexity assumption the same sort of thing is required in a general sense in order for VR to work and the key to that is agreement between multiple senses and our internal model of reality especially when it involves feedback loops for many people the defining moment in the Crescent Bay demo is finding themselves on the edge of a long drop the response is often to grab a nearby virtual pipe which shows that enough unconscious inference is kicked in to trigger the automatic responses that mean the means of brain believes it's someplace real why does that happen while the scene where neo climbs out onto a ledge outside a building and looks down with a thousand feet never makes anyone grab for support the difference is that when you move your head and Crescent Bay both the motion your vestibular organs report and the parallax or visual system report match the motor signals you sent your neck muscles and your sense of proprioception that is your model of the position of your body and they do so with low enough latency so that your brain confuse all those data points into a coherent model of the world at the core of this is a feedback loop you move your head and what you see changes correctly and with an almost imperceptible delay allowing your brain to maintain the same kind of model of the virtual world as it does for the real world which then leads smoothly to further motion that's the fundamental difference between movies and VR movies provide similar images but with no feedback loop for head motion as a result we've perceived movies as moving pictures on a flat surface still firmly embedded in the real world good VR and contrast isn't perceived as pictures at all it replaces rather than augments the real world vr is about driving our perceptions the way they're built to be driven if we can do that it should be completely unsurprising that it enables us to create new realities if the technology becomes good enough vr should theoretically be able in the limit to create not only any experience that's possible in the real world but any experience that we're capable of having you