Author: Dr Angelica Poli
STSM Period:2020-03-01 – 2020-03-31
Hosting institution: University of Alicante
From ITC: No
PURPOSE OF THE STSM
Literature suggests that older people prefer to live independently at home as long as possible. This requires providing care at home, and Ambient Assisted Living (AAL) solutions can represent a way to do so. Among other applications, AAL solutions can be used for monitoring and improving the emotional state and the wellbeing of an individual at home. For example, certain living environments can counteract negative psychological states (e.g., anxiety) and thus have an influence on the psychological and physical well-being. The main goal of the STSM was to investigate whether and how living environment designs affect the individual emotive capacity. To do this, we, first, evaluated physiological metrics (e.g., Galvanic Skin Response, Heart Rate) which were measured by means of the Empatica E4 wristband device and, second,
we extracted relevant information from the signal to recognise individuals’ emotions.
According to WG 4.1. of the Sheldon, the specific objectives of the STSM were:
1. To conceptualise the methodological approach for evaluating elderly’s experience with different living environment designs;
2. To develop a system which is able to process and interpret emotional state of elderly at home;
3. To draft one or more outlines for peer-reviewed publication(s) that would highlight the findings.
DESCRIPTION OF WORK CARRIED OUT DURING THE STSMS
During the STSM period 1st March – 13th March 2020, the collaboration with Dr. Florez-Revuelta was structured in three main tasks as detailed below.
Task 1: Conceptualisation. This task aimed at developing a Conceptual Framework by assessing the emotions measured from the exposure to different living environment designs, using non-intrusive devices. Dr. Florez-Revuelta and I initially planned face-to-face meetings to kick-off the collaboration and to present each other’s works, background and expectations for the STSM. Additionally we carried out individual
research for preparing draft materials to be used for the conceptualisation of the methodological approach.
Task 2: Experiment. This task investigates whether and how living environment designs affect the emotive capacity of the elderly. Based on the monitoring and the analyses of the emotional state and its changes, it can be possible to customize living environment designs to make living places more comfortable. First, the state-of-art was reviewed. We focused on two main types of non-intrusive interventions to stimulate
and monitor different emotions:
1. Virtual experience through which the elderly experiences a virtual interior space (e.g., living room);
2. Real experience in which a set of images were displayed on a projector screen. Images were extracted
from the International Affective Picture System (IAPS).
In these two settings, individual physiological signals are collected through two wearable devices, namel the E4 and an ambulatory monitoring device (VU-AMS) provided by the Research Group on Psychology Applied to Health and Human Behaviour (University of Alicante). Individuals physiological signals are used to detect users’ emotions.
In addition, Task 2 also aimed at identifying existing secondary public datasets which could be used in combination with the physiological signals for training the data.
Due to the Coronavirus outbreak and the closure of Spanish schools and universities, I had to interrupt my research activities at the University of Alicante on the 14th of March. When we learned that some universities in Spain were closing and more universities would have followed, Dr. Florez-Revuelta and I rescheduled the pending workplan. Specifically, we decided to invest my time left in Spain on the validation of the two different and recently-released E4 devices we have at the University Politechnic of Marche and at the University of Alicante. This means that data was collected from both E4 wearable devices, and the physiological signals were compared in order to verify the correlation (i.e. visual check, and cross-correlation function) between the two devices.
Task 3: Outline of publications. This task aimed at outlining peer-reviewed publications based on the study. Dr. Florez-Revuelta and I conducted separately an individual brainstorming and preparation activity for drafting possible options of papers (including main goal and possible journals/publication opportunities), on the basis of the Conceptual framework (Outcome 1) and data available (Outcome 2). We both agreed on a Draft outlines of publications as final output (Outcome 3). After the STSM (which was quite short in duration), on the basis of the initiated collaboration, more options for future papers will be also explored with more time for reflection and discussion.
DESCRIPTION OF THE MAIN RESULTS OBTAINED
The main outputs achieved during the STSM are the following ones.
1. In accordance to a literature review and based on the discussions with the Research Group of the Department of Psychology, a conceptual framework was developed. From this, we can expect that the design of living environments and psychological states are associated. Their association is crucial to increase the psychological and physical well-being. Correlates of emotion can be measured through noninvasive methods, detecting physiological indicators (e.g., heart rate (HR) and galvanic skin response (GSR)) for arousal to stimuli.
In this respect, the underpinning pillar of the research will be the data collection, employing two wearable devices (Empatica E4 and VU-AMS) for detecting the user’s emotions by analyzing their physiological signals.
2. In order to develop a robust system which is able to interpret and associate the emotional state of older people with living environment design, Dr. Florez-Revuelta and I coded algorithms for processing the data. Due to the closure of universities, data was not collected. Among the existing secondary public datasets, we selected the following one:
– WESAD: a dataset for wearable stress and affect detection (neutral, stress, amusement). Such dataset features physiological and motion data, recorded from both a wrist-(Empatica E4) and a chest-worn (RespiBAN) device. Additionally, self-reports of the users from established questionnaires, are detailed.
WESAD dataset allowed us to visualize data collected from E4 and to test some different processing approaches, especially for what concerns the GSR signal which works as predictor of emotions. First, in order to eliminate artifacts, a pre-processing tool (i.e EDAExplorer) was used. Then, the signal was decomposed into phasic and tonic components using the cvxEDA tool. The most common features used in literature were selected and extracted from both the components. As the reliability of the device is essential for data quality, we decided to compare the two available E4s and check their performances in acquiring data. We collected a short session: 2 minutes resting and 10 minutes walking. The analyses included, first, a visual check on the GSR data to identify possible failed measurements, arrival signal delay and artifacts. Then, crosscorrelation and statistical parameters were computed to determine the validity of the parameters. The results of these analyses showed that the two E4 wearables were correlated. Hence, they can be used interchangeably and their data can be merged.
3. Based on the work done and expected results from future analyses, we agreed on the following publication plan with provisional titles:
– “Influence of affective environments in ageing at home”: the goal is to understand if the emotional response affects overall psychological wellbeing of elderly living at home.
– “Development of an emotion recognition system in response to visual stimuli, through the galvanic skin response and heart rate”: the goal is to understand if living environments designs influence emotional states, based on physiological data.
– “Impact of living environments designs on emotion: a comparison between virtual and real experiences”: the goal is to understand if and how the virtual environments change the perceived emotion.
Dr. Florez-Revuelta and I also discussed possible future collaborations in the field of emotion recognition in living environments. In particular, we want to focus more on improving the reliability of monitoring systems to detect changes in individual emotional states in the context of daily living. Based on this, we want to implement effective strategies for modulating negative emotions.
We decided to continue working together on the STSM project and further develop it. In particular, the University of Alicante will collect a dataset of physiological signals (as soon as the recruitment of participants could be conducted) based on the protocol (e.g., location in which the experiment is carried out, time duration for each session, baseline detection, self-report questionnaires) designed in collaboration with the Research Group on Psychology Applied to Health and Behaviour. Based on this dataset, which will be collected with the Empatica E4 and VU-AMS devices, we will investigate the validity of the E4 wearable respect to the VUAMS device. At the same time, I am currently working on improving the algorithms developed by Dr. FlorezRevuelta and myself, considering the findings in the review of the state of the art conducted in Spain.
Specifically, I am testing several system methodologies which include different processing approaches (e.g., filtering techniques, features selection, removal of artifacts). In this way, we will train the algorithms with data collected with the Empatica E4 to recognize the emotion in response to visual stimuli related to different indoor living space environments. Such finding will be compared with the existing literature, in order to assess the results, estimating points of successes and weaknesses.
We are also investigating if realistic virtual environments could be used to record a dataset without the need to move the users to a real environment, facilitating therefore the acquisition of the dataset and increasing the number of available environment designs.
Finally, we explored further future collaborations:
Finally, we explored further future collaborations:
– exchanges or visiting periods in Italy/Spain, where each other could have a residential stay, exchange knowledge on own research, and plan internal/external seminars;
– new publications (journal articles, book chapters, symposia at conferences etc.) of mutual interest and exploiting available data at hosting institutions;
– new research projects to be developed and submitted to national or European authorities for funding.