Gadgetory

my name is Jason Gerald and i am the founder and principal consultant have next-gen interactions we focus on emerging technologies such as virtual reality and that's what I'll be talking today applied towards gaming entertainment in general but also a little bit about how some of these technologies applied towards serious gaming as well before I get started I'm curious how many of you have tried to oculus rift Wow almost everyone alright and how many of you actually are how one that you own yourself a few of you okay well the new ones coming out of the summers of maybe Gaza waiting for that that's going to be going to be even a better version the Deaf tip number two and then next maybe in a year or so the actual consumer version will be coming out the Deaf get one that we certainly have is pretty cool but it's not quite there yet we're still or oculus i should say it's still developing that constantly improving upon down to add new features and such as very basic at this point but you can kind of get an idea when you use the populace rift that what's coming to the consumer virtual reality margin curious out of those that have the ribs are any of you actually developing your content creation or anything with it a couple people okay great so you're probably well aware of some of these things i'll try to you explain some concepts that i picked a report for development and for those of you kind of considering that we can give you some ideas when you do start creating and what to do with okay so I'll just start off with what is virtual reality well I kind of define virtual reality as replacing the real world with computer-generated cubes those q's might be visual cues of course of the primary thing but as you all know with games things like audio is very important as well what we have with virtual reality is that also that sense of touch that sense of proprioception that I can feel how my head moves instead of just keeping my head still and only having visual and audio auditory cues also in some cases will trip through full-body tracking for the ideal virtual reality experience as a simpler version of that might just be trapped in the hand so that you can interact the work now virtual reality is something that's very difficult to explain in words it's something you have to experience to really get what it's about knows idea to try to be probably understand that that's hard for me to explain so also a short video that first I want to show an example reaction of some a tremor to reality for the first time and some of you may have seen this video this is a 90 year old grandma trying to virtual reality with the oculus rift for the first time and so specifically of course you'll see her reaction that's pay attentions careful attention to her words of what she says how she describes the experience something where I want the different alright that's a short version of that she goes on to say other amazing things as well that me to that video has gotten over 2 million views on YouTube so the massive number of consumers are starting to understand what virtual reality is and starting to imagine what can actually be capable now the specific word she used one phrase was my friends aren't going to believe this when I tell them right again something you can describe but you have to experience now if this 90 year old grandma goes to one of her you good friends or a relative and says yeah I was in my living room and I was transported to this other place that didn't really exist if they'd never heard of virtuality what you think they're going to think during the fictions you know kind of losing it that they're not going to take her seriously I'm in one place but I'm actually in another complete other universe I just sounds crazy and so something you have to actually see to actually experience them she also asked the question am I still where I was that implies she actually felt that she was present in that other world that's what we really something we're trying to strive for her in virtual reality is a sense of presence the sense of being somewhere else other than where you're physically located she pays happy she was implying that she wasn't sure how in my in my living room or in am I in this alternate border and so that's pretty pretty compelling pretty convincing when you actually feel like you're somewhere else sure where you're located yeah this was a very simple demo this is just the oculus rift by itself there's no 7 no body tracking she's not really interacting with the world it's more of a passive experience and she still had that very surreal experience per person okay up next I want to actually show you an example of a virtual reality experience I work with the virtuix the company virtuix with their bought me directional treadmill of creating a game or their system and that was showing on Shark Tank ABC shark tank back in December of this year have you guys seen that you guys all seem shark tank people or at least know what it is essentially it's a panel of investors that come on board and people come up companies come up and pitch their product trying to get an estimate the virtuix Omni is an omnidirectional treadmill so imagine that you're in the gym and you're exercising like you normally do in some cases they actually have a game in front of you that the faster you run you can collect points you can compete with other players and such as sort of a traditional Russian said traditional over the last ten years or so help me set been developing these interactive feeling like treadmill machines for exercise this is a little bit different because it's an omnidirectional treadmill so you can physically turn around 180 degrees a huge so imagine that you know you're fighting the bad guys and he pulls out they pull out the big guns you first thing you might do just instinctively might even not not even think about it just turn around you know run as fast as you can you can actually do that with this treadmill it's pretty cool so with that I'll show a short clip of that show because gone the CEO of virtue X he really explains virtual reality well I think maybe perhaps better than I can scream so you'll show it and also explain it as well as showing what this work to walk degrees which are mine okay so that kind of gives you an idea of what it's like although not that obviously tried yourself to get a bowl experience the shark the Sharks love what they had but yon was asking for far too much money than what they're normally used to investing in it was actually he was founded the company at 20 million dollars and was asking for two million dollars which was more than anyone had ever asked for in the history of shark and so they gave him a really hard time did not invest in virtue aches and Janka to Madonna cars I just read this morning that Mark Cuban one of the Sharks was one of the investors that put three million dollars into the company so they're finally starting to believe that this is actually taking off and it's becoming real where is it you know in just a few months ago they didn't believe this would actually take off it may have something to do with the Facebook acquisition of oculus for two billion dollars i suspect that you've got very interested in that so some of I know some people have been kind of upset thinking about Facebook what do they have to do with virtual reality well it's now nothing other than they bought oculus it's actually I believe very strongly it's a very good thing even if it is facebook the reason for that is because it gets credibility to what we do with virtual reality and now other investors are going to be able to come in and provide funding so that we're able to really make these experiences the best we can be we can only do so much kind of in our spare time if we can actually you know get funding to you know build studios and such but you all have the capability to do it will be much easier to do that because facebook is basically validated parking as well as others such as Sony recently announced a head-mounted display that will be coming on the markets and plenty of other companies as well lots a head-mounted displays are coming out on the market a little bit of history here is what I call old school virtual reality this is a project that we work with general motors on about 15 years ago in something called it a Kate a cave is a very different type of display than a head-mounted display a cave is actually a physical room you could actually see the edges of the walls these are walls this is the front wall this is the stocking and this is the floor of the room you actually sit and the stereoscopic images are projected onto those walls you have head tracking so the system knows where you're looking and how you move your head and such it's a very compelling experience you actually feel like you're you're in that automobile here instead of in a room with images on the walls the walls actually start to disappear because you're seeing that in each I had seen a different image and so we have the sense of depth kind of like a 3d television except you're completely surrounded by now and I show this because this is an example of where virtual reality has been but it wasn't available in the living room right he couldn't put this display your living room the rather head-mounted displays they were sometimes a hundred thousand dollars at work so I wasn't going to be in your living room unless you were someone like Mark Cuban maybe he had in his liver but most of us didn't have access to it now we can have access to that it doesn't cost these systems started at somewhere around 500 million five hundred thousand dollars I 590 I'd be a bit expensive but in other cases they completely have designed the buildings to go around these caves very expensive but not all of us have access to so this is a very exciting time for us is that they're indie developers insights because anyone is now able to develop these virtual reality experiences now head-mounted displays that's exactly what an oculus rift is is this display put on your face is trapped and look around so you're completely surrounded by the virtual world that's not a new thing oculus they've done it very well I they're doing amazing things I was a little skeptical when they first came out because I knew how hard of a problem it was but they've proven over and over again that they're doing things right right and creating a very compelling hardware as well as a software and so I'm very happy that what they've done with virtual reality to bring it to the masses head-mounted displays actually have been around for quite a while I just saw Steve Alice of NASA Ames Research Center he had the slide at the I Triple E virtual reality conference a couple weeks ago and he claimed is its kind of commonly quoted that Ivan Sutherland at the university of utah back in the 60s created the first head-mounted display that's what 40 or 50 years ago but it actually goes back further than them according to Steve Alice he traced it back and he also claims that it goes back all the way back to Galileo he created the first head-mounted display in order to determine his location when you know her navigating ships and such and so this has been around why the woman and is continuing to bball oculus has just taken it to the next level brought into the masses and done it very well and also the technology is starting to catch up so it's at an affordable price now so we can all have access to it so these virtual realities have been around a while however we haven't all have access to it to a massive number of developers it's often consisted of programmer art or scientific visualization that doesn't have that aesthetic appeal that we have advised people in this room how has gained about this really is a new artistic medium that artists now are able to create experiences to be able to create this compelling content that just blows us away that we just didn't have access to you know 10 15 years ago because and you know nothing against programmers on the program or myself but you don't want to see my heart right it's a pretty I might have you know the bad guys might be you know consist of cubes that's my arm right and so we need artists we need designers we need psychologists is a very much multidisciplinary field that now that this is accessible to everyone we're able to access all those different fields all those different types of individuals that mean and why I say it's new is because this is completely different than traditional video games the screen completely disappears so you start you stock perceiving this display that's fixed in space out right at you with a wide field of view you kind of lose yourself and you don't even pay attention anymore to the edges of the screen once you get really engaged in the game you can you know because I mentioned earlier you can completely turn around and so that you're just in the world instead of looking into a window into the virtual work you're completely immersed and because of that a lot of concepts and traditional video game design no longer apply we have to rethink that entire process and so the best virtual reality games will be designed from the beginning or virtual real and so some companies have parted some of their existing games to the oculus rift for example which is good that's important because we got art somewhere but to really make those games compelling we have to start from them and part of the reason for that is something called simulator sickness that I'll be talking a little bit later similar to car sickness or going on a carnival ride some people are just going to get sick because it's a very intense experience and there's some physiological things going on in the human body that we can't solve technically and so we have to be very careful how we designed the games and keep those you know thoughts in our minds i know some games that have been ported to the oculus rift you know the character in the regular game it's okay if your character runs at 20 miles an hour but virtual reality if you run a 20 miles an hour something's not going to be writing or 2d text on a display screen now needs to be true 3d you can't have like occlusion issues of the text going through walls and such and so those are things we need anything now today oculus is doing a very good job of selling the technology as well as some of the other companies however one of the reasons why oculus is so successful is because they're selling the story there's something to excitement they're selling the motion of what virtual reality can bring us and now it's one of the reasons why they're doing so well other companies of you know how ill head-mounted displays an attempted to something I Charlie believe that's one of the reasons why I oculus is so successful that's sort of how they start they had to have 20 years of experience building these virtual reality experiences however because they were able to build out excitement they were of liquor you know hire some of the best game developers there aren't they got they got John Carter math they have Michael average some of these just by you know these gods in the computer game development community they've been able to attract them because they built that excitement and so as we continue into the future we need to be you know very careful that me as a technical programmer I sometimes forget about that and so we just all need to remember that it's the experience or selling not the technologist reasonable that would become more and work more and more important is different mountain Studios come in and compete in the space ok now I'm going to get into some details of some example work I've done ends of concepts that you could apply in your development for virtual reality I mentioned earlier that we kind of need to think from the beginning how the interface is different than it is in a traditional game and a traditional game you're not trapped in your hand so it makes sense to put up information on the display just take it down in the corner of the screen you're how far or whatever it might be the problem without it in a wide field of view head-mounted display can put it down in the corner of the screen it's like way down here it's on the edge of my vision where I don't have high visual acuity and so that may not be the right space so maybe it needs to be actually and physical space are attached to your body in some ways so when you want to see something you know kind of look down to the side the similar that maybe you know you look at a watch on your hand and so what I did here the original six cents entertainment with the razor hydros the razor hydros are these handheld traffic controllers that trap the orientation and position of the hands and so you can kind of you know move around and see your virtual hands look around they simply out there on the mapping of the buttons the instructions of what the controller does up in front which was useful but then it kind of got in the way it was a little bit distracting what happens you know if I don't want that they're figure out how to turn off so what I did here was just extremely simple I mean this literally look took 10 minutes to do so there's nothing complicated about these things but you know sometimes in almost all the time and design if you could take something complicated and simplify it to its simplest forms that's where the real power comes where anyone is able to use that making something complicated very simple so all I did was I took those mappings that normally came up in front of your screen split the image in half so I had an image of the mappings for the left hand right hand and then I attach them to the hand and now wherever I move the hands my hands here so it makes sense okay if I want to see what the mapping does for this button where my thumb approximately is then I just you know label where that thumb approximately is so I look at my hand and now I know what that done button does and so that way you can be able to just you know look down where the interface is instead of this an indirect now yeah this is something very simple and I'm sure it will evolve you know maybe some more complicated interfaces for example you could you know when you push about you could highlight the buccaneer and sort of thing so these simple concepts of design will be important as we move forward not necessarily complicated just we need to get creative and think outside of the traditional user interfaces you're getting to them Oh another thing I forgot to mention here that's very important is you can see the user sees some sort of body well it's not like that body's on its screen out in front of the user remember the screen kind of disappeared to your aversive you're surrounded by the game well my lower body my legs is part of that immersion I'm look down and I see my own feet in approximate location where my me know it turns out that's extremely compelling especially for something called presence that oculus and others often talk about that sense of being there well if you just have this viewpoint this camera floating through space not attached to a body it's very weird to look down not see anything it's like you're just kind of floating in space somewhere so if you just have even a rough approximation of the body is that's extremely compelling and then if you have the trap can such as in this case when you see this even those hands help me to be photorealistic it they don't need to be the same color of your skin you don't have to have the same color jeans on to be able to see your body move around as if it's that same location of your body it's a complete different experience and that causes people need to get that sense of presence that really feel like you're taking on being that character being that avatar ok we'll move on another to even add more on to that is to have these avatars if you're tracking the you know even if you're simply tracking the head the hands and crowds are tracking where the heads looking such as you can do with the oculus rift that can be extremely compelling as well because now you have this sort of natural communication be is a very subtle body language you don't have to think about oh I want to push you know what was the what was the button to surrender again you don't have to think about that maybe you just hold up your hands and so for example here this is I'm going to call the Paul he's the other character and when I shoot the gun out he's like whoa you know I surrender it just doesn't automatically might not even be thinking about them and for example head tracking is another very subtle cues that you don't even think about maybe in this regular game that sense of eye contact even though we're not trapped in the eyes not yet anyway that where is the head looking in the very compelling so if they're looking in your direction you know that looking in your direction it's a very one-on-one different the video conferencing one of the challenges with video conferencing is and you know the cameras over here in the screens here you lose that sense of icon here we do have that since I content because the character is mapped in physical space where it should be okay I mentioned earlier a little bit about simulator sickness that's one of the real challenges of virtual reality just like the carnival rides that I mentioned some people it's not appropriate and they know they can't go in carnival rides and maybe if they do it more and more they can become accustomed to that and such but what we can do is we can reduce that simulator sickness as much as possible we can do things smart we can be careful of the technology be careful of that interface all those things we need to be very aware if you for example if you have a low frame rate and a game is over the traditional game is a little bit annoying and virtual reality if you're only rendering a 30 frames per second it's not just annoying it can make you physically sick the reason for that is because the system thinks I'm looking over here but a split second later I'm looking over here but it's still displaying what I was looking over there your brain gets confused the reason why I get confused is because we have the sense of balance in our interview cognitivist if your system it's that sense of balance a sense of a deceleration acceleration if those kind of subtle cues a balance don't match what my eyeballs are seeing then the brain gets very confused and it says wait a second something's wrong here maybe I from an evolutionary point of view maybe I you know it's a bad fairies that were poisonous not I better you know we check that out of my system and you get sick that's one you know theory of why that's the case but it's really all it is is a century coupon plate that you're different sense is being out of sync with each other and now can cause similar your sickness so we understand the basics of simulator sickness but this is very much an open open research at this point of understanding and one of the challenges of understanding simulator sickness is the biggest after in simulator sickness is to individual some things that I do for example I can you know virtually back up with my in virtual reality and I feel fine or i can straight to the left or right I feel fine other people sometimes that makes them sit and maybe they can do some things that made me sit but don't make medicine and so that's one of the real challenges is they can very much be an individual into some individuals simulator sickness affects them in different ways now there's a blue arrow here that I want to talk about what is that blue an area I it's part of the interface but it's not so much part of the interfaces this is a maybe a bold claim but that blue era helps reduce simulator sickness what's that that's all it is is a blue arrow it's nothing complicated right there's no complicated algorithm here to understand that's just a blue hair of and yelling in the world what it is and I this was me playing around I didn't know if it would work but I kind of had the idea I realize that some virtual reality experiences are in a phase of driving game or you're in some sort of wow that cockpit in the game is a stable world reference q no matter which way I look or no matter how I move forward that cockpit is always going to be in the same position relative to my body right if I fly forward the world outside the window moves forward but the cockpit stays in place that cockpit is actually stabilizes a real world reference to you no matter what I do it's always physically in front of me it's the same location as a physical keyboard and that helps those are good games to develop poor because people don't get nearly as sick in those systems because it's just like the real world like I was might be sitting in the car and also the periphery that wide field of view we tend to have that sense action more when something was in the periphery versus our central part of our bill with you affection is if any of you have been sitting in a parking lot maybe you're kind of looking down or something then a bus or something moves in your periphery and you kind of you feel that sense what what what happened you actually feel that sense of self motion even though you didn't move that's an illusion the same thing out some condoms with a virtual reality something moves in your periphery or the entire build of you moves you get that same sense affection which kind of makes you feel a little funny in continue doing actually do simulators so this blue arrow I thought okay a first-person shooter is not appropriate to have a cockpit because it'd be weird that if this maybe if your mech pod or something it might make sense but just if you're walking around you can't have a cockpit room so I thought what if I could add some sort of artificial real-world reference cues that are stable in the physical space with a helper you simulator settings and for me it made much more a bigger impact than I thought because I could if I started feeling hydrated still in that sense affection I could focus on that arrow and realize okay let's kind of ignore everything that's you know in the world moving is i virtually walk forward and focus on the arrow and think about that as something that's visible in the real world and for me it felt like wow this feels a lot better and the few people that I have tested out they felt they kinda commented on that as well and so this I haven't done you can form of user studies or anything on this but this is an example of what we can experiment with and you know further investigate we don't have all the answers yet but this excitement times because why open any of us can tragic scenes now all we need is an oculus rift Rose we needed is you know the Saudi hmp coming out and all of us can start experimenting a lot of those experiments may go bad some of those things are going to be pretty exciting as far as yep what we can do and what helps create these more about virtual worlds okay another topic is something called coordinate systems and all of you that a developer you know traditional video games you understand and you have these different ordinances dragging out the camera coordinate system you have the world coordinate system virtual reality it's no different in this game i have i'm showing here this you can see several different things is a little confusing again without training this is something you have to actually experience experience and so while it's a little difficult to describe when you see it you kind of get it immediately what are these different visual cues that are just you know more than the word and so again you see the blue arrows you can see i added more blue arrows and that was us saying is that's what the former directions that is what that physical representation is so as I move around you have this sort of real world reference to you to reduce that simulator students add more of those blue arrows now this is just kind of a direction my physical torso is facing so I know which ways forward oh one thing I forgot to mention is some people mentioned that you know for example oculus they suggest that move the player in the direction you're looking so if i push the controller forward by facing this way then move this however if i turn my head this way and move this way that's fine but when you had an avatar where you're actually seeing your own body that becomes very weird because if i turn my head like this what happens to your torso your torso still facing this way but what does the avatar move this way when you only move your head it's sort of an unknown in my case I said let's always assume the torso is facing this way and I can look down and see my body my torch is still forward but I'm looking to the left now I can walk in this direction instead of looking in this and then there's no it's a little bit better as far as the representation of the your own body when you look down just like it's more like the real worth so these blue arrows are not actually looking down always looking down on your mind so instead I put another reason i put these blue areas so you always have an idea of which direction here torso is so now when i push the controller forward i realize okay even though i'm facing this way the blue arrows are in this direction so my mind just gets accustomed to saying to thinking okay forward is in the direction of the arrows not necessarily the direction on the that gives you more control so you can look all around as you're moving throughout the word so that's one example of a coordinate system which is just about what i call the body centered coordinate system which way is my choice of basically of course we also have the game coordinate system right now you all know that from the games which way is nori and if you look at the bottom of this image what I did here was I surrounded the user by a compass so if they get lost it's very easy to get lost in these virtual worlds because if you virtually rotate you forget in the real world it can be difficult enough to remember which way starting but in virtual world to become lost very easily especially in our complicated matters in this virtual world i have here I completely I forget where things were even though I created a map and so having that compass allows me to better be able to create that cognitive map of okay what is this what is this world like I'm trying to head in the northern direction but if I rotate you know all around personally I forget which way's fine and so you can kind of just look down and see the game world coordinate system and remember oh I'm heading north I just need always had to go in the direction of this compass that I'm looking down on surrounding another coordinate system of course is the hand coordinate system that we're doing and tracking such as the image I showed earlier that you might have a user interface attached to that coordinate system another coordinate system i have in this example because i'm not doing man tracking in this case i'm tracking the head which way am i looking well this is a head century coordinate system this kind of transparent target area so you know if you want to target here the laser shootout room as if you know your eyes are shooting out a laser from your own and to be able to you know target things in the world that way so that's yet another coordinate system that when we're designing we have to think about how these how all these coordinate systems work together however when the user gets in there they get it immediately because it's like a real world helmet on looking all around those just this crosshair and Freddie similar to you know wearing the football game and so these coordinate systems headed maybe are a little bit hard to think about as we're developing but when the user gets in there they just give it automatically ok this is the compasses the blue arrows on facing this way these are my hands it sort of this very natural sort of usernames okay being that this is a serious games track I had to put in this image here for a serious game that we're working on we're working with the National Institute of Health on a educational game for kids this is grades 3 through 5 actually putting the kids into the game visit and so imagine being able to physically crawl through a body to explore different you know different organs and such within the body and so now it now it's no longer just a game of these visual and auditory use we have that kinesthetic you that sense of your arms actually moving around because we're using the sixth sense razor hydros in order to track the hand so you can kind of yell crawl around physically with your hands and we believe that I actually putting the kids in the game now they're learning physically the same way that they you know play and this applied specifically to neuroscience education which can be kind of a very abstract thing especially for kids but we believe if we you know physically put them in something that you know remember much better after all we're competing with him all the great games you guys are creating so we have to really you know what is the next thing that really you know get these kids engaged in the games and the real idea behind this is this applied towards on as I mentioned neuroscience education and so it's not just necessarily you know remember what organ this is and what is it is called I personally interested in more higher level concepts versus just memorizing different parts of the body and so in this game we should teach kind of self awareness of our own box of our own actions so imagine indecision it'd be useful for us as adults as well imagine you know teaching kids that you know when something happens in the world so you get bullied or you get teased or something to that effect imagine kids being able to step back and realize oh you don't have an this audit and some stimulus happened in the world and I'm having this sort of automatic response then realizing oh that's just the you know by an egg divided us you know some part of my brain reacting in a certain way however then realizing oh what this is happening again and then realizing you know that is happening that's my automatic response because now I'm aware of it I can step back and say you know maybe that response isn't the most isn't appropriate in this situation maybe it would be more empowering to be able to choose something different to reactive you know the call to bullion on what he's doing or you know whatever that might be given this specific situation so being able to you know become help them become more aware their own thoughts of you know how their brains working such okay that kind of sums up what or this slide kind of sums up what we talked about that there's lots of things to think about in a design point of view from a technical implementation point of view you know creating that compelling art and how to actually it's very important to embed things in the world and think about things as if they were real world objects make it as real as possible but then continue beyond what we can do in reality you know reality we don't have this you know interface in the world that you can reach out to touch things or you know manipulate objects out of distances you know like magic but first we have to make it to be the real world so that it behaves like room wearing otherwise don't be challenges with a simulator sicknesses now the I Triple E virtual reality conference a couple weeks ago one of my one of my mentors he gave the keynote about virtual reality he was so excited about the facebook acquisition because I validated you know what he's been doing for she's forever since the early 80s working on virtual reality and he saw that as this validates what we've been doing for you know a very long time and now people are taking this seriously now there's going to be you know you guys out there also contributing to what we're doing and to be able to bring in funding to be able to get people excited to bring this to the deal worked where the entire world will be able to eventually experiences so he was super excited and what was really inspiring is what he said about few people whatever their career is over I'll tag if they're you know how the desk job whatever it might be they're doing sort of these iterative changes because you know they're improving things but it's a very slow pace because it's very complicated these technologies have been out in the market for a while I with virtual reality is something brand-new at least outside of the research view of the field it's open to anyone that we really do have the opportunity to change the world to how teach the serious game concepts for example but just you know entertainment purposes to have something that you know will remember like the grandma remembered that said you know what one thing she said is I'm going to remember this the rest of my life even though she was 19 years old that people will remember these experiences of offering these compelling sort of engaging stories maybe you know applying and how that applies to their real life it's just completely wide-open we have you know a major opportunity here and Palmer Luckey 20 years old Brent when you started oculus two years ago he didn't have 20 years of experience it didn't have you know a bunch of advanced degrees he was able to create that because he was able to access the technology that's available now at an affordable price and if in two years he can create what he's created what's stopping us from doing you know creating great things ourselves so if a 20 year old kid can do it what god s is a community all coming together we have that opportunity and what what dr. Fuchs said as well he said let's not blow it right let's try to do this right let's be passionate about what we're doing let's create these world some of us have been dreaming up 20 years okay with that I'd like to mention some upcoming events that i'll be participating in here about what is that about two weeks ago to its away I'm sorry we'll be on the narrow game conference in san francisco and the idea there is to take Nero's I ensconce of such as you know since an EEG sensors in such sensing to what the brain is actually doing in Cork what can we do to incorporate to integrate together virtual reality technology ish with neuroscience technologies that is a completely wide-open builders some people have been playing around with it but something since wide open i saw i talked to help weeks ago where it was a virtual mirror virtual reality augmented reality sort of experience you looked in the mirror the player had the sensors on its head he looked in the mirror saw a virtual representation of himself and the sensors read what you know what the different parts of his brain raveena activated you could look in the mirror and as he could watch actually watch his thoughts occur overlaid on his green in a mirror right that's like one of it again pretty simple but just one of the coolest things like wow I never thought of that you know any of us could've done pretty cool stuff at that conference will i also have a power lodging the founder of oculus and richard marx the developer of the PlayStation Move and also one of the key players and Sony head mounted display the Morpheus that some was just recently announced they'll be honest they'll be on the same man I don't know if those guys have ever been in the same room so we hope to get a little controversial boy I'm gonna pass it there let's see what happens the silicon back belly virtual reality conference if any of your out there are in California are might even be more traveling if you really want to get serious about this i don't have a good definitely going to be there we have some great speakers as well we have yon CEO of virtue it's that we showed the homie director Miller where again Richard Mars Tom you're lucky you're going to be there as well as some other pretty amazing speakers that you know really the visionaries in this deal and a little bit after that is the Augmented world expo again in the san jose area and that is something that deals more with augmented reality which is a little bit different than virtual reality but a lot of similarities the simpler concept here inside those blue arrows i talked about that the real world being represented by this cue the match of the real world augmented reality is the inverse of that it's taken the real world embedding computer-generated imagery with in that room world against something that you can't have to you see or experience that's difficult to say in words and lastly this is not normally been announced yet but at secret in vancouver this year we're actually home i'm working with the committee now putting together a panel of some of these leading experts in virtual reality to make virtual reality is on par with the keynote at the conference is going to be a primary theme of the conference this year that we're going to have some big speakers there and what may be even more interesting to this audience is that we're putting together a contest where different people can submit their material of virtual reality experiences whether they've been working on in the entire year and we still have some time left right this is until August and so if you believe you have some really compelling ideas for virtual reality you'll be able to submit to that and will be you know have a panel the judges looking at those submissions and the best experiences or be showcasing those that see grandpa making a big yellow so guess you're an opportunity to show some you know cool ideas that you have maybe already in developing or virtual reality gives you chance they've you know show to the world the show to a large audience okay with that open the floor for questions also i mentioned of course contact me here at my email address and also my much work and contact me there okay we have a question a lot about Bonnie presence in virtual reality wondering how you see out haptics getting into that and how do you see that being implemented as well absolutely so haptics is a big deal i talked about that sense of kind of proprioception is self motion but that's all that's you know have the controllers well some sense that's passive seat now because I I feel at least I have something physical in my hand so by the sword or a gun or whatever my am least I feel something which is very different than Microsoft Kinect was there you know there's advantage of the beach it depends the network's great for some games and so there's not like one's better than the other but if you have that if you're holding a physical gun you expect haven't gone in that set to the button of actually physically building something so in some cases that's better now that's all passive happenings because the thing isn't actually by the way there's other controllers out there that will but you know what tumblr right that you know actually physically shakes so that's one very simple case there's some company called tactical happenings which is using the six centuries your Hydra controllers that I talked about they added a really compelling sense of touch and how they do is very interesting how they do not by itself it's like okay this is kind of what I feel like this the controller essentially moves it's got some things that you know push on your skin you can still move it's really cool but by them it's pretty cool by himself and when you put in a virtual environment now suddenly you're seeing it and you're filling it even though it's not going away perfect like the real world combining those together starts creating this or this illusion because if generally vision dominates the sense of touch nothing alive cases and so now you feel like wow that was really like this rude coin that might not feel like a recoil if you didn't have the visual the visual you know compelling images embedded within the virtual world not just on the screen there's other things to do I know on back in the day we used to buy when I was at UNC we would physically model the room and so you could go in you know visually you be like in a living room sort of environment and you go up and you could physically touch something because the virtual world match the physical all they were retiring on box you go up and like okay this is pretty cool and it was attractive so you can physically walk around and it like a 20 by tempe area and then you like okay this is pretty cool and we say okay well you know reach out and touch the table they do that then it kind of freaked out paper book this really is real and then they started to believe that they touch other things and then what we did is we said okay now let's walk into the next room so they physically walk into the next room and they look down and there be a big hole in the floor and now instinctively they learn this is the real world what I see as a physical presence and freaked out so we measured like for example heartbreak we measured palm swag yeah it would spike when they went in the next year and then what we did we added one more thing is we said okay but then if they walk out there the illusion is going to be rooming we did you know dig a hole in the floor we didn't go that far what we did do is we put an inch and a half apply with an inch and a half then we say okay you don't believe this go up walk to the edge and put your toe over the edge of that we call it the diving board over the flora and then they get even more less fighting that those are passive happens active haptics is something that you know physically my you reach out and touch something and you have others is famine devices are you familiar with those there's different types of devices that you actually model just as you model the root of the visually geometry this model is a physical like a robotic arm inverse of the robot it's able to on control where new touching noses after where we are in space that applies physical forces so I've the spiritual object in the sense that you know I intersected that object and apply a horse back to keep even moving through now and that's even more compelling because that you can dynamically change the model it's such a good dog for artists it's great because it can go in there and you know actually do you know sculpting and fill the surgeons and specifically what I was wondering about the Amazons Asia and handing up around yeah because that's actually I'm going to be very hard but you can't really just add weight to something on your arm yep and that's so the tactical haptics guiding guys they kind of do that but that's one of the big challenges right how do you change the waiters I don't know maybe there's like some crazy electromagnetics thing you pick it up and some of the you know turns on the magnet or something there's also other what's up or something with water programs yeah yeah I mean it's wide open and that's why it's such an exciting you know I can't think of it right now but I'm sure some of you maybe have some ideas another challenge is when you physically you know if I have this virtual wall here and I virtually walk through it there's nothing stopping so what happens do I go through the wall because I physically walk forward or do I stop the virtual work and say no you can't move it it's like wait a sec I kept walking I walked another five feet why is the world updating the grown and so they're gamblers these cute conflicts that creates this big challenge these virtuous doesn't matter that much they've salt something that is that and people have double treadmills for example they've had these crazy treadmills to take up like an entire room that are sort of this on active treadmills that you start walking and then the treadmill starts booting just like a regular treadmill except art an escalator for example or a moving sidewalk but you kind of be almost tripped when you'll be back because I love something starting to move but i think you know physically move start walking and there's all these challenges but what gone did with virtuix is he was able to you know totally rethink and said what if we make a passive trail or basically you know it's not the actual surface moving let's put it down to a simplest form we don't have moving parts and it's almost sort of like on roller skates this kind of it feels like not quite like real walking but maybe that's okay maybe that's better than means you know huge expensive mechanical systems any other questions there we go so you don't use I've been seriously if you're trying to achieve like resumption of what a user we are wrong it's how how would you college how would you do that life here trying to meet reception how to use but my map of my fav visible from my flight they're trying to do to an axe you are pulling for example so I'm sorry if I was out last part you brought a bully for example bullying okobogee mmm you bought that for example how how would you do that room in my leg up demo for support examples and because they were like if it's not realistic if you give it's like your seven-year-old they're just like always but without the actual reception am I admire or how I do it won't really change or maybe I'm just saying but how so by Paul Anka tabatha pet at Duke University she's got some amazing things studying for example gender differences are racial differences what if you put someone in the avatar of a female even though your nail and how can you make them feel real and there's different people have done studies like that they'd actually found a difference that people start having and after the oh I understand how it is to do you have you no obvious exit be discriminated against truth serum for me it's very compelling even if they're just cartoons it's not you know we all my photo realism those important for some things but it doesn't necessarily need to be photorealistic I think there's more important things like a wide field of view is very important in the sense of motion is more important in some cases than the visual appeal you remember graduates squad colleague Paul Simmons that was looking at your sense of presence and virtual environments need bike model this really nice room and in compared it to this very gridlock room where you could see you know this grid with going off in the distance and so you know the further off something is you know this very nice sense of visual cues that you might not have as much what you know a plastered wall it doesn't happen and he found at least the early preliminary findings were some people actually thought more present in that grid like because they have more of a sense but hypothesized it was because you got more a sense of depth you actually have enhanced visual cues and so it doesn't necessarily need to be you know photo realistic if I look down and it's a different color shirt than what I'm wearing I'm not going to remember what sure does i'm just i'm going to be wow my hands man i'm going to look like my hands just to have something like that another way to do that is this is where it is very near sm from like a southern psychological point of view it's important to have psychological cyclone just type of blood psychologists everybody involved with our work so what they've done they put them in the habit are they going okay this is cool yeah I'm good at this kind of my body I'm part of that is not always look at your body right so they'll have like virtual mirrors didn't see yourself as well we look down to hear what they also did as I say okay here's a virtual feather now we're going to you know touch your arm of this better get back to your question about haptics not a very enough technically advanced Harmonix but the research would then come up and have a real feather and rub their own with the real feather as a virtual feather did the same thing I was great without bond I don't assume you mean like from the physical or social equipment like him when kids seizes room and okay since dawn because that means an equally room is and you're having like you're trying to get them to even stand up for himself or to get a reaction oh that's really what I mean I said so from the point of view of the holier the person being bullied I try to get them to change their mental their young fit mentally well maybe related and give this I don't know the answers to all these things yeah I bet it's a good question and sometimes when you don't know the answer that probably means it's a good question right but going back of each other differences they actually found that that would change how people thought of you know oh I'm making me and you know these sexist remarks or something don't think nothing of them maybe as a guy because you never thought about it but if you put them in the from a female perspective then suddenly they start thinking all about we can do you can have empathy with it with female females now realize up reorg or you know changing their sense of height this chimney and how people realize up you know people experience the world differently if we're a different night another example of that is public speak there's no studies where they put someone in a virtual room with that virtual avatar audience and the psychology that just is pretty cool stuff what they've done studies like you know big audience like you guys are paying attention to me right so it's easy for what happens if the audience is nodding off or people are you know not paying attention they're looking at their cell phones what happens and they found them it's a huge difference when they put people in these virtual environments even though they're just cartoon characters they're so obviously not real their behavior actually suggests that they do believe these characters are real and an extreme case what they do they add up one of the one of the 10 10 G's of the practicing of the speech get up and scream and run out of the room and so that was a good example how you can you can play yes over yeah there's been a lot of research over glass like 30 years applied towards on jet fighter pilots that you know and what's interesting about that is there's a lot of data the problem is it's not arguably not validated the reason why is because if you're a jet fighter pilot and your report simulator sickness is probably going to be a while before you fly again so they drastically underestimated how many people get simulator sickness the general finding was there yeah don't do you know stupid things don't do crazy things but it's not that bad and now you know we hopefully will get more of an honest feedback on that there's some academic research on that but it's not really brought together trying to think offhand a website or something for that it's something personally I want to do more research on and you know test those blue arrows for example like that's my hypothesis i tested i'm obviously violence maybe my users advice because i say hey try this is cool right unbiased true formal studies that's I think for you know this sort of virtual reality that are not you know these high-end simulators is completely home so sorry i can answer bedroom that and it would Roger up up saw answer this real quick then we can talk afterwards which is kind of a situation since signifie house series of anyone thought testing out by trim or a judgment for avoiding or reducing knowledgeable young so when I was working with Val that was kind of a joke like I came in one day because you know i was doing crazy hi he's crazy cat head mounted displays before Dracula's came out and you know someone had taped some some of that to my guests right so that was kind of a joke but somehow I've never tried it but others have reported that they have and that works again that's bias because you know maybe it's the placebo effect we don't know but then again my argument of the placebo effect is psychologically that's what's interesting is buying it's a placebo effect repairs it works right but for no studies of that would be very interesting I think it's one of those things that probably works for some people and doesn't wear other it's just like being a real room when you go on the boat it works for some people are not others either placebo or not will need to do you no harm studies good okay with that we'll move on to our next speaker and again I invite you to come down to the next gen interactions food court will be showing the game with shark tank we don't have the virtuix Omni unfortunately I try to get here won't be able to get it till later this summer so maybe next year I'll be able to show that whole system here and so come on down I highly recommend to stay for richard boyd speech coming up next he's has been doing for