A Cooperation between Politecnico di Milano and Karlsruhe Institute of Technology

Starting in February, May, and November 2024

Master Level Practical Seminars (4,5 ECTS) or Theses (30 ECTS)

INTRODUCTION

Bio-signals are nowadays being studied more intensely than ever to understand the processes within the human body and brain, which can inform technologies that help us understand and support physical and mental activities. Thanks to the technological developments of sensors integrated into devices that we use every day (e.g. Smartwatches, Headphones, etc.), it is possible to study this field more widely, trying to add quantitative data to the observation of human behavior.

Emotions and the implicit cognitive process can be detected from the analysis of bio-signals as individuals typically have low control over these processes, so, physiological changes in our body can give researchers a new way to study cognitive and affective processes. In Consumer Neuroscience, for example, we can use this world of data as insight into the human perception and evaluation of a specific product or situation, to predict future purchase behavior.  In Neuro-Information Systems (NeuroIS) research, we can apply cognitive neuroscience and neurophysiological theories to develop adaptive and self-regulation support systems, with the use of live biofeedback or develop IS interfaces that automatically adapt to a user's current state.

The goal of this international seminar is to investigate the physiological side of human behavior, using electrophysiological measurements (ECG – heart rate, EMG – muscle activity, EDA – sweat gland activity, or EEG – brain activity) under the supervision of experts from the domains of information systems, marketing, and bio-medics. We aim to both analyze how physiological responses impact behavior and how – from a managerial point – this research could help practitioners in designing products, services, or interfaces. The specific projects (that you can apply for) are listed below.

STRUCTURE

How does it work?

Each project lasts between 4-6 months and can be started in February, May, or November. You can apply to the corresponding deadline (see below). While the projects start individually, all participants are expected to attend at least 2 of the 3 research meetings during 2024 (either in Karlsruhe, Milan or at other European institutions).

You can apply for any one of the topics listed below and complete the project either as a practical seminar (4,5 ECTS) or as a Master Thesis (30 ECTS). The following practical seminar courses are selectable for KIT students:

• T-WIWI-106207 Practical Seminar: Data-Driven Information Systems
• T-WIWI-108765 Practical Seminar: Advanced Analytics
• T-WIWI-109940 Spezialveranstaltung Wirtschaftsinformatik

You will work as a 2-person team or alone to carry out the project. By joining this project, you will get to network with students and researchers from both universities – either in person or virtually (remote attendance will be possible).

The work for each project should include: (i) a literature review (ii) the design of an empirical study to test the research hypotheses, (iii) results in analysis and interpretation (iv) a discussion of academic, managerial, and policy-makers implications (v) limitation of the study and future research. The extent of these five parts will be adjusted to suit each project.

To enable your bio-signals research project, we will teach about biosignal experimentation, data collection, and analysis in our meeting events, and we will provide you with Python code templates to make the start of bio-signal processing a piece of cake! Find out more about the dates, requirements, and application process below.

How can I apply?

You can apply if you have completed your Bachelor's degree, are fluent in English, and have previous experience with statistics and (ideally) with Python programming.

The application period is now open until January 22^nd, 2024. To submit your application, please write an e-mail to lorenzo puppo ∂does-not-exist.kit edu including the following documents:

• A letter of qualification (1p max.): This should indicate what qualifies you to conduct such a research project. We assume that you are motivated if you apply. So please focus on describing the skills and previous experiences that qualify you;
• Your current transcript of records;
• Your three most preferred topics (ranked – i.e. the first topic you name should be the one that you are interested in the most).

TOPICS

1.Topic: EEG Headphones: Going into the Field!

Main Supervisors: Michael Knierim (KIT) & Felix Putze (University Bremen)

How cool would it be if your headphones could actively improve your daily productivity and well-being? This reality might not be that far away! Wearable EEG systems are continuously being developed to monitor brain activity and support IT-users' workload, flow and fatigue. Prolonged or excessive mental workload are linked to stress and lower productivity in everyday worklife. Unfortunately though, mental workload is hard to assess objectively as it develops through the balance of a persons abilities and task demands. An innovative way for workload monitoring could be the use of brain activity monitoring using EEG headphones (see, e.g. https://github.com/MKnierim/openbci-headphones). In this project you can contribute to evaluating a new functionality of these headphones, collecting data to answer a new workload-related research question, or work with already available data sets by developing machine learning models to predict mental workload levels across various tasks and settings.

Sensing Modalities: EEG, ECG/PPG, EMG, and contextual sensor data

2.Topic: Golden Eye or Golden Zone: Assessing the impact of eye-level versus arm-level item placement

Main Supervisor: Marco Mandolfo (PoliMi)

Proposed topic: This work aims to explore the impact of item placement on customer and worker behaviour in retail and warehouse logistics environments, focusing on the distinction between eye-level (“golden eye”) and shoulder/waist-level placement (“golden zone”). It aims to understand the effects of these placement strategies on purchasing and picking decisions, with a particular emphasis on how eye-level placement (versus shoulder/waist-level) impacts item visibility, physical fatigue, picking speed, and overall picking volume in both settings from the marketing and logistics perspectives.

The research will involve laboratory experiments, which will simulate a retail and warehouse environment, allowing for controlled observation and analysis of participant behaviour in response to different item placements on shelves. The empirical activity involves a multi-method approach, which combines various techniques. This includes eye-tracking technology to measure visual attention, behavioural analysis to observe physical interactions with items, neurophysiological measurements (such as electromyography) to assess physical responses like fatigue, and self-reports to gather subjective perceptions.

Overall, the work includes: (i) a thorough literature review of the behavioural mechanisms of consumer and worker attention regarding item picking behaviour, (ii) the design of a laboratory experiment to test research hypotheses, and (iii) an empirical investigation adopting a multi-method approach. The work is expected to be structured as an empirical work with implications related to practitioners, policymakers, and academia. For practitioners, such as retailers or warehouse managers, the findings will offer practical guidance on shelf arrangement to enhance customer and worker engagement and operational efficiency. Policymakers will gain insights into the ergonomic aspects of retail environments in stores and warehouses, potentially informing regulations or guidelines. Theoretical contributions will extend to empirically-based knowledge for retail or operations management research, broadening the understanding of the interplay between physical environment and human behaviour from marketing and logistics management perspectives.

Sensing Modalities: Eye-tracking, electromyography, motion tracking

3.Topic: Fix it or buy it? Exploring the emotional drivers of consumer decisions to replace or repair a durable good

Main Supervisor: Debora Bettiga (Polimi)

Content / Work description: Consumers show increasing interest in product repair. Extending the lifetime of a product is beneficial for both financial reasons and environmental ones, in terms of the impact of product disposal (Scott, K. A., & Weaver, S. T, 2012). The buyer’s inclination toward repair than replacement is shaped by several motivations and barriers (Guiltinan, J., 2010). Time consumption has been identified as a barrier to repair practices, as consumers may value the ease and convenience of replacing a product compared with the repair alternative (Page, 2014). Cripps and Meyer (1994) indicate that consumers are more inclined to replace for technological obsolescence than for physical deterioration. Antonides (1991) pointed out that, while some replacements are driven by product failure or weakening performance, others are motivated by simply voluntary motives such as the desire for something “new,” or the anticipation of new benefits. In such cases of “unforced” replacement, consumers will be more excited about and interested in the decision to replace and, thus, more motivated toward it (Grewal et al., 2004). van Nes and Cramer (2008) identified a list of product characteristics that may induce replacement, by inducing arousal for upgrades: technological performance, hedonic value, features and technological advantages, psychological value, ergonomics, economic value, and ecological benefit. Research further confirms that product replacement is not only based on rational decision-making, but on emotional drivers. Among them, product attachment (the emotional bond a consumer experiences with a product) may induce consumers in repairing owned products. Memories, self-expression, group affiliation and pleasure have all been identified as drivers of such product attachment (van den Berge, R., Magnier, L., & Mugge, R., 2021).

Therefore, research highlight from one side the role of affective reactions, especially arousal, in pushing consumer toward product replacement. On the other side, product attachment may spur consumers toward product repair. This thesis wants to investigate such issues,  exploring the role of emotional drivers in the consumer decision to repair versus replace. Specifically, the role of arousal, pleasure and memory will be explored to understand their impact on consumer decision-making process.

The thesis work should include: (i) a thorough literature review of the affective mechanisms driving consumer decision to replacement versus repair (ii) the design of a laboratory experiment to test research hypotheses (multiple studies are suggested), (iii) an empirical investigation adopting a multimethod approach, which combines self-reports and biometric data (iv) results analysis and interpretation (v) discussion of academic, managerial and policy-makers implications (vi) limitation of the study and future research.

Good statistical knowledge are needed.

Sensing Modalities: EDA, EEG;

4.Topic: Decoding Flow: Predicting Task Absorption through Physiological Markers

Main Supervisors: Pierluigi Reali and Stefania Coelli (PoliMi)

The so-called flow state is a highly functional condition linked to increased motivation, effort, and perseverance. Experiencing flow involves immersing oneself in an activity that aligns closely with one's skill level, demanding just the right amount of effort. When this happens, the activity starts to feel like a positive and manageable challenge; it is carried out with total involvement – even losing track of time and self-awareness – and the resulting flow experience is often accompanied by positive emotions and a sense of accomplishment. Frequently experiencing a state of flow can also have surprising benefits in the long run, such as shielding against stress and burnout and enhancing job productivity. This is why health scientists, practitioners, and companies are actively seeking new objective and reliable methods to measure flow without interrupting the experience or interfering with its positive effects.

Biosignals, such as electrocardiography (ECG), respiration, electrodermal activity (EDA), and electroencephalography (EEG), can conveniently address this need. Various features in these signals have been linked to the perception of flow, yet their reliability across multiple studies remains debated. This adds a layer of complexity to the quest for robust biomarkers of flow, making it a challenging – but undeniably exciting – endeavor.

The present project aims to address this challenge by analyzing already available ECG and EEG data collected during an experimental protocol to induce flow in participants. Specifically, this work includes: i) performing a literature review of traditional and innovative ECG and EEG features to characterize flow; ii) applying signal processing methods to extract the features of interest; iii) choosing adequate statistical or machine learning strategies to select the most sensitive combinations of features to flow variations; iv) training and validating models to predict the perceived flow levels from such features.

Candidates are offered the opportunity to develop their skills in biomedical signal processing and machine learning. Knowledge of signal processing basics and programming (Python or MATLAB) is required.

Sensing Modalities: ECG and EEG

5.  Topic: Bioadaptive Music Discovery System

Main Supervisor: Fabio Stano (KIT)

How cool would it be if your headphones could automatically detect if you like a new song and save it to your playlist? In this seminar, we will focus on designing and conducting an experiment involving participants who will listen to various music excerpts while wearing EEG sensors. The goal is to collect EEG data that, when combined with (or compared to) song characteristic information (e.g., from the Spotify API), can be used to train a classifier. This classifier would predict whether a person likes a song based on their EEG response. Findings from this project could contribute towards developing a hands-free system that can automatically detect song preferences and build your playlist while you listen to new music, and have interesting implications for both consumers and platform providers for building novel recommendation systems. For students, this project offers a practical deep dive into human-computer interaction research. You can gain experience in developing and conducting comprehensive experiments, while also acquiring skills in bio-data processing and machine learning.

Sensing Modalities: EEG, maybe multimodal

6.  Topic: Predicting Investment Decisions through EEG

Main Supervisor: Lorenzo Del Puppo (KIT)

Decision-making plays an essential role in the interpersonal interactions and cognitive processing of individuals. The application of biometric data has been the focus of neuromarketing research. Understanding the mechanisms underlying consumer or investor buying behaviors and how emotions and cognitive processes affect such behaviors are questions often asked in the scientific literature but in need of further investigation. Several studies show how through the use of electroencephalography (EEG) we can pick up signals that can be interpreted as predicting choices.

In this project you can contribute to data collection and data analysis with EEG, to answer research questions regarding investment choices and brain signals during a trading task with high ecological validity in a laboratory setting.

Sensing Modalities: EEG and ECG

10. Topic:  SleePyLand: A python library to analyze the large amount of NSRR sleep data via deep learning algorithms.

Main Supervisors: Luigi Fiorillo (SUPSI) & Davide Marzorati (SUPSI)

Polysomnography (PSG) is used in sleep medicine as a diagnostic tool to objectively analyze sleep quality. Sleep scoring is the procedure of extracting sleep cycle information from whole-night electrophysiological signals. The sleep recordings are usually scored by human sleep experts according to the American Academy of Sleep Medicine (AASM) manual. The scoring procedure requires up to two hours of work per whole-night, and it is highly biased by the inter- and intra- scorer variability. 

A wide variety of machine learning based algorithms have been proposed to automatize the sleep scoring task, reaching very good results. However, none of these algorithms has been ever introduced into the daily clinical routine. One undeniable reason is that there is no tool, to date, to fairly compare and evaluate the available sleep scoring algorithms on the same datasets and metrics. 

The main goal of the project is to release SleePyland, a freely accessible Python based software, able to:

1. 1. Manage, pre-process, analyze and extract overall sleep parameters/statistics from the larger amount of sleep studies ever used in literature, i.e., tens of thousands of PSG recordings from open access data repository (i.e. NSRR).
2. 2. Provide for each sleep recording a structured report with a completely automated ensembled sleep scoring analysis. The idea is to include in a single open source repository the recent high-performing deep learning based sleep scoring algorithms. The algorithms will be trained and fairly evaluated on common-ground datasets, to then output the ensemble prediction.

Your contribution will be to provide the sleep community with a one-step more advanced software solution. On top of what has already been done in Luna by the NSRR community, SleePyLand's main focus is to provide for each PSG a structured report with a completely automated ensembled sleep scoring analysis. Differently from what Luna does, SleePyLand will exploit all the EEG, electrooculography (EOG) and electromyography (EMG) channel derivations.

The ultimate goal is to release the software on a freely accessible repository, to then share it with the NSRR ORD community, by integrating it into Luna.

The project will be in cooperation with NSRR and Purcel Lab from Harvard University.

Sensing Modalities: EEG, EOG, and EMG.

This project would benefit from another supervisor who:

• Is knowledgeable in field study research with biosensors.
• Is knowledgeable in machine learning, particularly in the context of sleep medicine.
• Aims publishing in digital medicine or specifically sleep medicine related outlets.

11. Topic: Assessing the relevance of contextual information for physiological data modeling.

Main Supervisors: Davide Marzorati (SUPSI) & Luigi Fiorillo (SUPSI)

Physiological data collected in the wild from wearable sensors (smartwatches, smart rings, …) are often accompanied by contextual information that can come in many forms: structured and standardized questionnaires, ecological momentary assessments (EMA), event loggers, and so on. Modeling physiological data together with this accompanying information may provide better insights and improved results.

Commercial products, such as the Whoop 4.0, already make use of this feature. Whoop allows users to log their behaviors and events occurring in their life, and it shows insights on the impact of these events and behaviors on the recovery of the users, thus allowing them to understand if that behavior has a positive or negative effect on them:

In this project you will:

1. Get acquainted with datasets containing both data from wearable sensors and contextual information, such as the Lifesnaps dataset, which is an open dataset composed of data from Fitbit devices, questionnaires, and EMA collected in the wild from ~70 people for one month.
2. Develop an automatic pipeline for the extraction of relevant features from such datasets.
3. Explore the modeling of physiological data taking into consideration the contextual information that is provided together with them.

The final aim is to build an algorithm that is able to determine the effect of the event/behavior on the recovery of the users, with recovery metrics assessed through physiological data.

Sensing Modalities: Wearable data, questionnaires.

This project would benefit from another supervisor who:

• Is knowledgeable in field study research with wearable sensors.
• Is knowledgeable in statistics and machine learning.