CNRS-AIST JRL (Joint Robotics Laboratory), UMI3218/CRT is a joint laboratory established between a French public organization CNRS (Centre National de la Recherche Scientifique) and AIST and located at Intelligent Systems Research Institute of the AIST at Tsukuba. The researchers from both countries are closely collaborating to pursue the means of increasing robot's functional autonomy, using a humanoid robot as a main platform.
It takes a little while for the robot to register the thought. “There is a small delay between the start of the neural activity and when we can optimally classify a volunteer’s intentions,” says Cohen. But he says that subjects can adjust for this by thinking of the intended movement ahead of time.
Shapira took part in three trials, including one in which he was able to move the robot around freely and another where he was instructed to follow a person around a room at the French lab. In the third trial he successfully piloted his avatar to locate a teapot placed somewhere in the room. To test the extent of his feelings of embodiment, the researchers also surprised him with a mirror. “I really felt like I was there,” Shapira says. “At one point the connection failed. One of the researchers picked the robot up to see what the problem was and I was like, ‘Oi, put me down!’”
The brain is very easily fooled into incorporating an external entity as its own. Over a decade ago, psychologists discovered that they could convince people that a rubber hand was their own just by putting it on a table in front of them and stroking it in the same way as their real hand. “We’re looking at what kinds of sensory illusions we can incorporate at the next stage to increase this sense of embodiment,” says Kheddar. One such illusion might involve stimulating muscles to create the sensation of movement.
The next step is to improve the surrogate. Replacing the current robot with the HRP-4, made by Kawada Industries in Japan, will increase the feeling of embodiment as it is roughly the height of an adult human and has a more stable and dynamic walk, says Kheddar.
The VERE project is concerned with embodiment of people in surrogate bodies so that they have the illusion that the surrogate body is their own body – and that they can move and control it as if it were their own. There are two types of embodiment considered - robotic embodiment, where the person is embodied in a remote physical robotic device, and which they control through a brain-computer interface.
Credit:
Joint Robotics Laboratory
The main research subjects include: task and motion planning and control, reactive behavior control, and human-robot cooperation through multimodal interface integrating brain-computer interface (BCI), vision and haptics. We are actively conducting collaborative research with external research institutes within Japanese and European projects.
The framework is identified as the Unité Mixte Internationale (UMI, International Joint Research Unit) for CNRS and concurrently as a Collaborative Research Team (CRT) for AIST, and defined as UMI/CRT.
“The ultimate goal is to create a surrogate, like in Avatar, although that’s a long way off yet,” says Abderrahmane Kheddar, director of the CNRS-AIST joint robotics laboratory at the National Institute of Advanced Industrial Science and Technology in Tsukuba, Japan. He is part of an international team that hopes to use this kind of technology to give healthy people and those who are “locked in” – unable to move but fully conscious – the ability to interact with the world using a surrogate body.
The framework is identified as the Unité Mixte Internationale (UMI, International Joint Research Unit) for CNRS and concurrently as a Collaborative Research Team (CRT) for AIST, and defined as UMI/CRT.
“The ultimate goal is to create a surrogate, like in Avatar, although that’s a long way off yet,” says Abderrahmane Kheddar, director of the CNRS-AIST joint robotics laboratory at the National Institute of Advanced Industrial Science and Technology in Tsukuba, Japan. He is part of an international team that hopes to use this kind of technology to give healthy people and those who are “locked in” – unable to move but fully conscious – the ability to interact with the world using a surrogate body.
Credit: Joint Robotics Laboratory
Teleoperated robots, those that can be remotely controlled by a human, have been around for decades. Kheddar and his colleagues are going a step further. “True embodiment goes far beyond classical telepresence, by making you feel that the thing you are embodying is part of you,” says Kheddar. “This is the feeling we want to reach.”
To attempt this feat, researchers with the international Virtual Embodiment and Robotic Re-embodiment project used fMRI to scan the brain of university student Tirosh Shapira as he imagined moving different parts of his body.
Ori Cohen and Doron Friedman from the Advanced Virtuality Lab at the Interdisciplinary Center in Herzliya, Israel, and colleagues first took Shapira through several training stages in which he attempted to direct a virtual avatar by thinking of moving his left or right hand or his legs. The scanner works by measuring changes in blood flow to the brain’s primary motor cortex, and using this the team was able to create an algorithm that could distinguish between each thought of movement. The commands were then sent via an internet connection to a small robot at the Béziers Technology Institute in France.
The set-up allowed Shapira to control the robot in near real time with his thoughts, while a camera on the robot’s head allowed him to see from the robot’s perspective. When he thought of moving his left or right hand, the robot moved 30 degrees to the left or right. Imagining moving his legs made the robot walk forward.
To attempt this feat, researchers with the international Virtual Embodiment and Robotic Re-embodiment project used fMRI to scan the brain of university student Tirosh Shapira as he imagined moving different parts of his body.
Ori Cohen and Doron Friedman from the Advanced Virtuality Lab at the Interdisciplinary Center in Herzliya, Israel, and colleagues first took Shapira through several training stages in which he attempted to direct a virtual avatar by thinking of moving his left or right hand or his legs. The scanner works by measuring changes in blood flow to the brain’s primary motor cortex, and using this the team was able to create an algorithm that could distinguish between each thought of movement. The commands were then sent via an internet connection to a small robot at the Béziers Technology Institute in France.
The set-up allowed Shapira to control the robot in near real time with his thoughts, while a camera on the robot’s head allowed him to see from the robot’s perspective. When he thought of moving his left or right hand, the robot moved 30 degrees to the left or right. Imagining moving his legs made the robot walk forward.
The humanoid robot in France (controlled by thought from Israel)
Credit: The Advanced Virtuality lab (AVL)
It takes a little while for the robot to register the thought. “There is a small delay between the start of the neural activity and when we can optimally classify a volunteer’s intentions,” says Cohen. But he says that subjects can adjust for this by thinking of the intended movement ahead of time.
Shapira took part in three trials, including one in which he was able to move the robot around freely and another where he was instructed to follow a person around a room at the French lab. In the third trial he successfully piloted his avatar to locate a teapot placed somewhere in the room. To test the extent of his feelings of embodiment, the researchers also surprised him with a mirror. “I really felt like I was there,” Shapira says. “At one point the connection failed. One of the researchers picked the robot up to see what the problem was and I was like, ‘Oi, put me down!’”
Robot Avatar Control with fMRI machine
Credit: The Advanced Virtuality lab (AVL)
The patient, in the fMRI machine, watches the environment through the eyes of the robot. He can go forward, turn left or right just by thinking. The robot is located in France ( IUT Béziers ) and the fMRI machine is in Israel ( Weizmann Institute )
The brain is very easily fooled into incorporating an external entity as its own. Over a decade ago, psychologists discovered that they could convince people that a rubber hand was their own just by putting it on a table in front of them and stroking it in the same way as their real hand. “We’re looking at what kinds of sensory illusions we can incorporate at the next stage to increase this sense of embodiment,” says Kheddar. One such illusion might involve stimulating muscles to create the sensation of movement.
The next step is to improve the surrogate. Replacing the current robot with the HRP-4, made by Kawada Industries in Japan, will increase the feeling of embodiment as it is roughly the height of an adult human and has a more stable and dynamic walk, says Kheddar.
Credit: Béziers Technology Institute
The researchers are also fine-tuning their algorithm to look for patterns of brain activity, rather than simply areas that are active. This will allow each thought process to control a greater range of movements. “For example, you could think of moving your fingers at different speeds and we could correspond that with different speeds of walking or turning,” says Cohen, who presented the results of the embodiment trials at BioRob 2012 in Rome, Italy, last week.
So far, only healthy people have embodied the surrogate. Next, the researchers, along with Rafael Malach’s group at the Weizmann Institute of Science, in Rehovot, Israel, hope to collaborate with groups such as Adrian Owen’s at the University of Western Ontario in Canada to test their surrogate on people who are paralysed or locked in.
The researchers are also fine-tuning their algorithm to look for patterns of brain activity, rather than simply areas that are active. This will allow each thought process to control a greater range of movements. “For example, you could think of moving your fingers at different speeds and we could correspond that with different speeds of walking or turning,” says Cohen, who presented the results of the embodiment trials at BioRob 2012 in Rome, Italy, last week.
So far, only healthy people have embodied the surrogate. Next, the researchers, along with Rafael Malach’s group at the Weizmann Institute of Science, in Rehovot, Israel, hope to collaborate with groups such as Adrian Owen’s at the University of Western Ontario in Canada to test their surrogate on people who are paralysed or locked in.
The VERE project is concerned with embodiment of people in surrogate bodies so that they have the illusion that the surrogate body is their own body – and that they can move and control it as if it were their own. There are two types of embodiment considered - robotic embodiment, where the person is embodied in a remote physical robotic device, and which they control through a brain-computer interface.
For example, a patient confined to a wheelchair or bed, and who is unable to physically move, may nevertheless re-enter the world actively and physically through such remote embodiment. The second type of embodiment is virtual where participants enter into a virtual reality with a virtual body representation. The basic and practical goal of this type of embodiment is to explore its use in the context of rehabilitation settings.
Underlying these practical considerations is fundamental scientific research across a range of topics that are core to the progress of the practical goals: the neuroscience of body ownership, the construction of a machine to accomplish embodiment that reads signals from the participant and delivers virtual sensory data to the participant, the recognition of intentions through the monitoring of brain and physiological signals, the embodiment of these intentions into actions by a physical robot and the representation of participants by such a robot, the technology underlying virtual embodiment, and an overall software platform in which the different streams of work can be developed.
There are two applications, one for immobile patients based on physical embodiment, and another for rehabilitation and training based on virtual embodiment. A critical part of the research is the consideration of the philosophical and ethical principles underlying embodiment, and consequently there is an entire stream of work that acts as the ethical and philosophical conscience of the project.
During the first year a number of goals of the project have been realised - studies involving the brain representation of embodiment at various levels, the design of an 'embodiment station', the first results on intention recognition using a variety of techniques based on both EEG and fMRI signals, the implementation of robots controlled through a brain-computer interface, and technical advances in the virtual reality representation of individuals. There has also been progress in research leading to the applications - including results showing that embodiment influences social attitudes, and progress on each of the fronts mentioned above.
A major and prestigious seminar was held in Centro Stefano Franscini, Monte Verità, Ascona, Switzerland, 26th September to 1st October, 2010, called 'Body Representation in Physical and Virtual Reality with Application to Rehabilitation' which was organised by the VERE coordinator, another PI of VERE together with another expert on rehabilitation, at which many of the VERE PIs presented their work. Another event was organised by a PI of VERE at the American Association of the Advancement of Science which was held in February 2011: "From Artificial Limbs to Virtual Reality: How the Brain Represents the Body". As well as this meeting being organised by one of the VERE PIs who also spoke at this meeting, the VERE Coordinator presented at this meeting.
The consortium has arranged a PhD Symposium, that is being organised and will be run by the students of the VERE partners, part of which will be open to the public.
The consortium has organised an Open Competition for groups or individuals to compete for a prize that would help them towards the exploitation of the VERE technology in a novel application.
During the first year a number of goals of the project have been realised - studies involving the brain representation of embodiment at various levels, the design of an 'embodiment station', the first results on intention recognition using a variety of techniques based on both EEG and fMRI signals, the implementation of robots controlled through a brain-computer interface, and technical advances in the virtual reality representation of individuals. There has also been progress in research leading to the applications - including results showing that embodiment influences social attitudes, and progress on each of the fronts mentioned above.
A major and prestigious seminar was held in Centro Stefano Franscini, Monte Verità, Ascona, Switzerland, 26th September to 1st October, 2010, called 'Body Representation in Physical and Virtual Reality with Application to Rehabilitation' which was organised by the VERE coordinator, another PI of VERE together with another expert on rehabilitation, at which many of the VERE PIs presented their work. Another event was organised by a PI of VERE at the American Association of the Advancement of Science which was held in February 2011: "From Artificial Limbs to Virtual Reality: How the Brain Represents the Body". As well as this meeting being organised by one of the VERE PIs who also spoke at this meeting, the VERE Coordinator presented at this meeting.
The consortium has arranged a PhD Symposium, that is being organised and will be run by the students of the VERE partners, part of which will be open to the public.
The consortium has organised an Open Competition for groups or individuals to compete for a prize that would help them towards the exploitation of the VERE technology in a novel application.
Contacts and sources:
CNRS-AIST JRL (Joint Robotics Laboratory)
VERE project
No Camels
Béziers Technology Institute
CNRS-AIST JRL (Joint Robotics Laboratory)
VERE project
No Camels
Béziers Technology Institute
If you want make controlled by the humanoid robots, then you need first very thoroughly study the classic work on this subject: http://deepdivertech.narod.ru :-)
ReplyDelete