Gadgetory

if you ever find yourself in Pittsburgh and want to catch a glimpse of the future Carnegie Mellon University is probably a good place to start to get inside the robotics lab you'll need to walk past tank he's a Robo ception a-- stand well actually tank is really important to this story but he isn't what I want to show you right now this is looking at this large geodesic dome covered in wires you can't help but feel like you're staring into a time machine and in a way you kind of are but rather than traveling through time this fin optics to do is capturing it using an array of cameras researchers here are able to capture and then replicate any action into a 3d model from a dance to a cello performance it can perfectly reconstruct these moments in time and while that's pretty impressive by itself it's just the starting point in a quest to decipher an age-old code so basically you can see that this is a kind of geodesic dome looks like a sphere we specifically designed this architecture so that we can put cameras as uniform as possible and you can see also that we specially mobilize the system so that we only design one panel and we can duplicate the panel's one panel is composed of 24 small cameras and one high resolution cameras and they are saved in the local hard drives and in the end all the local data are there transmitted to the NAS and so that sounds like a ton of data like how much data is in one minute of capture so one minute of data is about 600 petabytes that's huge data that's a big job in fact to this point the Panoptix to do has captured over 1 petabyte of data that's about a million gigabytes and the latest is well me yeah so while I'm dancing those 500 cameras are capturing every movement I make and after a couple of weeks of processing it looks something like this this point cloud view is not only showing a bunch of tracking points its tracing the past they could take while I'm moving around all right got a stretch before that now motion capture by itself isn't anything new similar technology has been used to bring digital characters to life and films for years now motion capture technology has gotten so good that an actor like Andy Serkis can really inject life into a fully digital character but to do so requires a full motion capture suit with tracking markers placed all over your body even paint it on your face now this works well enough for trying to bring a dragon to life but it's hard to feel natural in that kind of equipment the Panoptix to do in contrast allows for similar tracking but without any of the physical markers you can just freely move your body in a way that feels natural and unrestricted which is a necessity for them to capture what they are really looking for really good quote from Edward Sapir who's a linguist he says there's an elaborate and secret code that is known to no one but understood by all and he's referring to to all the gestures that we use to communicate and that's kind of the code that we're trying to sort of crack or understanding our goal essentially is to endow machines with the capacity to understand in social interactions so that's where tank comes back into play you remember tank the Robo ception esteem e via camera and react to my presence but he can't understand more than what I typed to him so when I start to get frustrated that he isn't helping me find the panopticon or react in a way that helped ease my tension which kind of makes him bad at his job machines have a very poor understanding of what all those things are so for example if if I'm not paying attention to you or or if I'm looking at you paying attention those are the kinds of things and machines just don't respond to right now and that's kind of a big deal studies have suggested that when we communicate with each other more than half of the message we're sending is based on the way we move our posture facial expressions and gestures can at times convey more than our words alone here let me show you so now you can see me and you can read what my body is telling you even if my words are saying something different maybe for my posture you can tell them in a cramped little sound booth or you can tell I'm excited about this story that I'm nervous about reading from a script that I'm hopelessly improvising into a camera to try and give you a sense of how expressive body language can be when we communicate each other we use many interesting hand gestures which is really important to communicate each other but it's very hard to understand why we are doing this motion when we are doing this motion right I don't know why I'm sitting I'm not trying to send you I mean I guess I'm sending you a signal but I don't know what it means exactly usually we are using our gestures but it's very hard to define why we are using it's not we are not doing this consciously right this is number adjusters I mean I'm just like trying to sit in a way that looks good okay I like your socks by the way thank you I'm trying to show off my socks that's why I'm sitting like this and that's the really weird thing about body language it's something we all do unconsciously and understand innately but if I were to stop and try to tell you why I'm holding my shoulders a certain way or moving my hands I wouldn't understand it and to be able to explain it so if we can't explain it to each other how can we possibly teach it to machines well it turns out we can teach them the same way we've been teaching a lot of other artificial intelligence systems machine learning specifically a technique called deep learning has produced a number of huge breakthroughs in the last few years a very simplistic view of it is you feed machines a lot of data about a certain subject like say driving a car feed in an update about how we drive and we'll be able to start understanding the system at play and react in a similar fashion the way we do most recently google's deepmind watched a few years worth of TV shows understand how to read human lips now it can do that better than most experts the panopticon enough visual data showing how humans interact it's deep learning system we'll be able to tease out the structure of that elaborate code making it possible for computers to read and write our body language possibly understanding a facet of our communication better than we do ourselves so if all this sounds familiar you are probably watching the new HBO series Westworld it's about a park filled with robots that can so vividly replicate human behavior it becomes impossible to distinguish who's a real human and who's not so Westworld is science fiction sure but to a quick youtube search for lifelike robots and you'll find we've actually made some incredible progress in the last few decades and some stuff to fill your nightmares find you and me be good friends but the benefits of these social robots could be immense look at Milo he's a robot that's being used to help kids with autism he could help these children identify social cues and emotional reactions that they can sometimes struggle to express and interpret themselves and robots will also help us care for the elderly so less Westworld and more Franken the robot beyond helping accomplish physical tasks future robots could become vital emotional supporters as well imagine instead of Robo ception estanque it helps someone suffering from Alzheimer's socialize on a more regular basis identifying their mood and reacting appropriately there's many components that need to happen before this works an AI in particular needs to get better right but I think that there will be a big advantage in terms of how we will treat machines once they start respecting at least for example whether we are paying attention to them or whether we are open to an interaction and other sort of social signals that that we use in our daily conversations but if all this talk of lifelike robots makes you nervous about the coming of some Skynet that'll wipe out humanity consider this teaching robots this social behavior might be the best way for us to foster a better understanding between humans and their creations so one way of putting it is that we use machines and even robots as tools currently but well what we will achieve when they interact with us as almost equals as we really find collaborators not so we won't start using machines as tools but rather as as press agents that we can collaborate with so I think in those terms that that will be very important so if we want to integrate machines and it's so eventually going to happen they should at least respond to us in a way that we appreciate this research is already being used by a company you probably know well Facebook the social networking giant and its oculus division are working with researchers from the Panoptix studio to find ways to make avatars in VR more accurate and social this could make communication with a friend more intimate or it could give Siri a body that can walk and talk just like a real person