Objective: The aim of my current research project is to develop emotion-sensitive robots that can interact naturally and intuitively with humans. This requires a two-pronged approach: first- real-time detection and recognition of human emotions and second-integration of this capability in robot architecture such that a robot can perform its routine tasks while being responsive to the emotions of the user.

Current and Future Work: Emotion recognition and detection using physiological signals is a complicated as well as interesting process that requires an insight into psychophysiology, biomedical signal processing and pattern recognition. It is important to note here that the phenomena of person stereotypy (different people expressing same emotion differently) and context stereotypy (same person expressing a single emotion differently under different circumstances) make it difficult to derive universal physiological signatures for any given emotion. We have therefore adopted a person specific approach in emotion detection and recognition. In the past, we have conducted experiments in our laboratory to elicit emotional responses from Subjects doing cognitive tasks on computer. These tasks include solving anagrams, math problems and performing auditory discrimination. While the Subjects are engaged in these tasks, biofeedback sensors record their physiological signals indicating their cardiac activity (Electrocardiogram-ECG and Blood Volume Pulse -BVP), electrodermal activity (Galvanic Skin Response-GSR), muscle activity (Electromyogram-EMG), and Skin temperature. The Subjects give a self-report periodically during the tasks that indicates how they felt during a particular session, i.e., whether they found the task engaging, boring, frustrating or stressful. This subjective measure along with the physiological data is used to extract patterns corresponding to various emotions for each subject. The physiological signals are analyzed and an array of indices is derived from these signals using Fourier Transform, Wavelet Transform and some customized algorithms. I have been actively involved in all the stages of Subject testing and data analysis. I have designed an adaptive neuro-fuzzy inference system to predict emotional state from a set of given physiological indices. I have also designed a Regression tree based person-specific affect predication and classification system.

The other important focus of my research is developing a robotic architecture that accommodates emotion-sensing capability. I have designed and implemented a human-robot cooperation framework based on hybrid Subsumption architecture that has an affective behavior module. The affective behavior module consists of both deliberative and reactive responses as the robot should not only be able to quickly and correctly detect emotions but also be able to perform context based reasoning to identify the best counteraction. This experiment demonstrated human-robot interaction in an exploration setting where a mobile robot was engaged in exploring a   workspace while also being responsive to the anxiety of the human operator. The robot could successfully detect the operator’s anxiety and modify its behavior in order to help the operator. 

The initial results were encouraging and I am now in the second phase of the research wherein I am going to conduct elaborate subject testing, to identify fast and reliable learning techniques for person specific emotion recognition. The methods under investigation are Bayesian Learning, Support Vector Machines and Neural Networks. I have also done a detailed study of mixed-initiative architecture and am currently working on developing a similar framework that allows implicit communication between human and robot based on physiological sensing.