Hearing loss is a growing public health concern, with many individuals struggling to understand speech in noisy conditions. The standard approach to evaluate speech understanding in noise is to measure the Speech Reception Threshold (SRT) behaviorally, which requires an active response. This PhD project aims to investigate the estimation of SRT using electroencephalography (EEG), allowing for a more automatic approach. Such an approach could enable SRT measurements to be incorporated into hearing aids, optimizing noise reduction, continuously monitoring users’ speech intelligibility, or even allowing at-home SRT assessments.

To estimate the SRT value from EEG, two methods were explored: one using the envelope of the stimuli reconstructed from the EEG response to continuous speech (SRTneuro) and the other using a neural component (SRTP300). For the SRTneuro estimation a linear decoder was used to reconstruct the envelope at different signal-to-noise ratios. A sigmoid function was then fitted to the reconstruction accuracies and the midpoint of the function was used as the SRTneuro. This method was used in younger (YNH) and older (ONH) normal hearing individuals. Within these populations, both scalp and in-ear electrodes were used for SRTneuro estimation. The results showed that the SRTneuro estimation quality for the ONH in general was similar to the YNH population, despite an overall increase in reconstruction accuracy observed in the ONH population. Furthermore it was seen that SRTneuro can be estimated in a subgroup of the tested populations using only EEG in the ear and that the estimation quality improved when combining in-ear electrodes with electrodes around the ear. To better understand the increase of reconstruction accuracies, the temporal response functions were compared, revealing an enhancement of amplitudes in the ONH population.

A potential problem with the SRTneuro method is that the reconstruction accuracy of the envelope is not a direct measure of speech intelligibility. Therefore the second method estimating SRTP300 was explored. Pilot-work and initial results of developing a paradigm to evaluate speech intelligibility based on a cognitive component is included in the PhD thesis. The results were inconclusive and future investigations should be conducted to utilize the paradigm for a SRTP300 determination.

Hearing is fundamental to how humans interact with the world. Without hearing, we are significantly limited in our communication and interaction with the world. Hearing impairment may lead to depression and feelings of isolation. Unfortunately, a large proportion of the population has been diagnosed with hearing loss, which is a partial or total inability to hear. According to the World Health Organization, in 2021, 432 million adults and 34 million children are suffering from hearing loss. By 2050, it is estimated that over 10\% of the world’s population will have hearing loss. A common solution for individuals to deal with hearing loss is using a hearing aid which is designed to compensate for the lost components of the sound, making it audible to a person with hearing loss.

Despite tremendous advancements in hearing aid technology over the past decades, challenges remain. Particularly, in environments with multiple active speakers, current hearing aids offer limited assistance, as they lack information about which speaker to target. If the hearing aid could identify the speech source of interest, i.e., decoding auditory attention, it could enhance the listening experience by suppressing irrelevant speech and amplifying the desired signal. Auditory attention decoding (AAD) emerges as a promising technology in this context and is becoming a trending research area for the next generation of hearing aids. In this thesis, I present a novel approach to auditory attention detection: using the endogenous processing of auditory events. This will combine state-of-the-art machine learning methods with deep analysis of auditory event-related cognitive features of the scalp- and ear-EEG signal, using recordings from normal-hearing subjects.

In the first part of this thesis, I investigate the high-level cognitive responses to natural speech events that occur during auditory attention. Specifically, I present four experimental paradigms with increasing degrees of realism: a word category oddball paradigm, an oddball paradigm with competing speakers, and competing speech streams with and without specific targets. EEG data has been recorded from 24 participants using 32 scalp electrodes and 12 in-ear electrodes (ear-EEG) to conduct a study of cognitive components related to speech events. I find that cognitive processing of natural speech events is observable at parietal electrode sites, peaking around 620 ms, likely corresponding to the P3b component. Additionally, in multi-talker paradigms, this component remains detectable in the attended speech stream, and its amplitude is affected by the degree of attention engaged by the brain to the speech events. These findings suggest that this component is a potential feature for decoding auditory attention. Furthermore, by using the spatial filtering method, the component can be extracted from ear-EEG signals.

In the subsequent part of the thesis, I develop a deep learning model to detect this cognitive component within single EEG epochs associated with speech events. This addresses the single-word AAD problem, determining whether a single word is attended or not. The results demonstrate that the proposed model can exploit cognitive-related spatiotemporal EEG features, achieving promising results on of the most realistic competing paradigm for unseen subjects. Importantly, this result suggests a direction to solve the AAD problem by using extended deep learning models for a series of cognitive responses to auditory events in a speech stream.

Next, we focus on enhancing current state-of-the-art AAD models. Traditional AAD methods typically include two separate stages: reconstructing the envelope or a third-party representation of the attended speech and determining the attention by comparing the similarity between the reconstruction and the actual representation of all candidates. I introduce AADNet, an end-to-end deep learning model, which combines these two stages into a direct approach to address the AAD problem. The proposed AADNet outperforms the competing methods including linear stimulus reconstruction, canonical correlation analysis, and non-linear stimulus reconstruction for both subject-specific and subject-independent models on two different datasets. The results highlight improved generalization to data recorded on unseen subjects and demonstrate the promise of using a deep learning approach to enhance AAD performance.

Finally, I apply the developed AADNet and propose a new sparse representation of speech signals to adopt the cognitive response-based approach to address the AAD problem and validate the performance on both scalp and ear-EEG data. I find that the proposed sparse representation correlates with the neural responses and this correlation can be used to decode the auditory attention. Although the proposed method currently performs modestly compared to envelope-based approaches and is limited to only scalp EEG, these results demonstrate the potential of utilizing cognitive response features rather than the stimulus-driven response to address the AAD problem.

Individuals with hearing loss can benefit from the use of the hearing aids, which should be fitted in close accordance with the individual's hearing thresholds. Until recently, fitting of hearing aid has predominantly been performed in a clinic based on behavioral tests such as pure tone audiometry. Characterization of hearing loss can alternatively be performed based on electrophysiological tests such as the auditory steady-state response (ASSR), which can be recorded from electroencephalography (EEG) electrodes placed on the scalp. Traditionally, ASSR measurements are limited to laboratory settings and require trained personnel to perform the test. Ear-EEG, where EEG electrodes are placed in the ear, allows hearing threshold estimation to be performed in daily life. Integrated into a hearing device, this technology would enable both initial and recurrent fitting of the hearing aid to be performed automatically in the everyday life of the user.

The ASSR is small compared to the background EEG and various noise sources, which means the low signal-to-noise ratio (SNR) for ASSR. This makes estimation of the ASSR challenging. Several methods can be used to enhance the ASSR amplitude, thereby making the response more prominent. This PhD project has investigated the effect of the stimulus bandwidth on the ASSR. The results show that a small increase in stimulus bandwidth (from pure tone to 1/3 or 1/2 octave) improves the detectability of ASSR at low stimulation levels while only having a small impact on the frequency specificity of measured responses.

The low ASSR SNR becomes an even larger challenge when ASSRs are estimated from ear-EEG recordings. To address this, the project has investigated whether detection of ASSRs in ear-EEG can be improved by applying spatial filtering methods. Spatial filtering is a noise reduction method which utilizes multichannel recordings to suppress noise while maintaining the desired signal. We have designed an iterative gradient-based method which maximizes the ASSR SNR. The results show that the proposed spatial filtering method outperforms the conventional approach, where a pair of electrodes giving the largest SNR (Best Pair) is chosen. We have compared the proposed method with other spatial filtering methods and found no difference between spatial filtering methods, while all spatial filtering methods have shown significant improvement in performance compared to Best Pair approach.

Bracketing technique is usually used in traditional ASSR-based hearing threshold estimation. There, the same stimulus is presented at different levels of intensity and the lowest level at which ASSR is detected is considered as a physiological threshold. The conventional stimuli for ASSR-based hearing assessment - pure tones and chirps - are synthetic and monotonous, which makes them inconvenient for repeated use in daily life. The current PhD project has introduced an approach where the ASSR versus presentation-level relation is estimated using a sub-band amplitude modulated continuous speech signal. The results show that the ASSR can be estimated as a function of the level both in scalp- and ear-EEG with slopes comparable to those reported in the literature for the conventional ASSR stimuli. This allows translation of the relation between ASSR and presentation level to physiological threshold. If incorporated into a hearing device, this approach would enable continuously monitoring of hearing thresholds in everyday life without, or with minimal, inconvenience to the user.

During the last decade, overweight and obesity have become an increasing global issue. According to WHO, in 2008, around 1.4 billion people over the age of 20 were overweight, at least 500 million were obese and at least 40 million children under the age of five were overweight.

The Food Industry's response to the obesity epidemic has been to produce a number of low fat and sugar food products that enable the consumer to eat the same food while consuming fewer calories. However, an investigation conducted by the Food Administration shows that people tend to consume extra-large servings of the light products, negating any benefits the light products might offer. 

A solution to the above-mentioned obesity epidemic requires a more thorough understanding of the brain's response to varying salt, sugar and fat levels and subjective satiation. Traditionally, food ingredient selection is based on physical and sensory analysis methods. However, in connection with salt, sugar and fat substitution products, objective measurement methods lack the ability to describe what we can register with our senses. In this regard, brain recordings are particularly interesting.

The idea behind the project is to utilise EEG methods to screen salt, sugar and fat substituents when selecting new food ingredients. The goal is to compare EEG results with physical or sensory data for new food ingredients with the hope of supplementing selection criteria for new food ingredients with objective physiological EEG responses.

Ear-EEG is a novel EEG (electroencephalography) recording approach in which the EEG signal is recorded from electrodes embedded on an ear-piece placed in the ear canal. The ear-EEG has great potentials within continuous brain monitoring in everyday life and will have application within both medical and consumer electronics devices.

The integration of brain monitoring based on EEG into everyday life has been hindered by the limited portability and long set-up time of current wearable systems as well as by the invasiveness of implanted systems.

To address these issues, the ear-EEG has been introduced which is a discreet, unobtrusive and user-centred approach to brain monitoring. The ear-EEG recording concept has been tested by using several standard EEG paradigms and benchmarked against standard on-scalp EEG.

All ear-EEG recordings made so far have been based on wet-electrode technology. In order to improve the usability and user-friendliness, this project will exploit so-called dry-contact electrode technology. This has impact on the design of the electrode itself, the supporting mechanics and the electronic instrumentation for acquiring the EEG signal.