Multi-modal stress recognition

Author:  Mr Martin Gjoreski STSM Period: 2018-04-01 – 2018-04-30
Hosting institution: Electronic Imaging Department, Fraunhofer IIS, Erlangen, Germany.
From ITC: Yes

Most healthcare research focuses on disease rather than health. However, people are interested predominantly in health and well-being. According to the 1948’s Constitution of the WHO, health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity [1]. Mental and social well-being are closely related to stress and emotions. Stress is a natural occurring process in our body, but when present continuously it can trigger chronical stress. Chronical stress has negative health consequences, such as raised blood pressure, bad sleep, decreased performance and slower body recovery processes [2]. Stress and emotions are tightly connected.

More specifically, change in the emotions is just one of the 3 components that constitute the stress response, with physiological response and behavioral response being the other two components. In 2013, the cost in Europe of depression was estimated to be €617 billion annually. The total was made up of costs to employers resulting from absenteeism and presenteeism (€272 billion), loss of productivity (€242 billion), health care costs of €63 billion and social welfare costs in the form of disability benefit payments (€39 billion) [3]. A system for stress and emotions monitoring can help to fight the costs of stress and emotional disorders, and can be of benefit for improving the overall mental and social well-being. In our research we will identify the state of the art in the field of stress and emotions recognition in smart environments. This includes, but it is not limited to: – smart furniture (e.g., furniture equipped with video and sound analysis), – wearable sensors (e.g., wristbands equipped with physiological sensors), – and smart environments (e.g., keyboard and mouse equipped with pressure sensors.) After the identification of the current state of the art, we will develop novel signal processing, feature selection and machine learning approaches with the aim to improve the current state of the art in the field. We plan to research combinations of Deep Learning and machine learning methods in order utilize the benefits of both of the techniques, i.e., Deep Learning to learn from large amounts of data, and machine learning to utilize the human expert knowledge encoded through the feature extraction process and the tuning/learning of the machine learning models. Finally, a big part of the development process of the novel solutions for stress and emotion recognition would be the possibility of integrating the solutions into everyday habitats. Such enhanced habitats, may significantly contribute to happier, healthier, more productive aging.


[1] “Constitution of the World Health Organization,” International Health Conference in New York, 1948.

[2] S.C. Segerstrom, G.E. Miller, “Psychological stress and the human immune system: a meta-analytic study of 30 years of inquiry,” Psychological Bulletin 130, 601. 2004.

[3], /,
[Accessed 27.03.2017].