Clarity logo
The Clarity Workshop on Machine Learning Challenges for Hearing Aids (Clarity-2021)
ISCA logo

Results

Prizes


The prizes for the MBSTOI evaluation were awarded to the Brno University of Technology team (first place), the Music Tribe team (second place) and the ELO-SPHERES Consortium (third place). The University of Sheffield team was not eligible to win prizes due to connections with the organisers.

The overall prizes for the listening test evaluation were awarded to the University of Oldenburg team (first place) and the ELO-SPHERES Consortium (second place). In the noise interferer category, the University of Oldenburg and Googlears teams were the best performers, while in the speech interferer category, the University of Oldenburg team and ELO-SPHERES Consortium were the best performers. For a discussion of the differences in performance for the noise and speech interferers, see the [YouTube video].

The Amazon student prizes were awarded to Katerina Zmolikova from the Brno University of Technology team (first place), Tomas Gajecki from the Medical University Hannover team (second place), and Xi Chen from Shenzhen University, who was an intern at Tencent while working on this challenge (third place). Congratulations to the winners!

Results

The tables below present median results on the evaluation dataset for the initial MBSTOI evaluation for noise (eval-noise) and speech (eval-speech) interferers, and the Automatic Speech Recognition (ASR) derived listening test results for 27 hearing-impaired listeners for noise (eval-noise) and speech (eval-speech) interferers.

MBSTOI evaluation

Rank Team Paper Eval-noise Eval-speech
Univ. Sheffield (E009) [paper] 0.693 0.741
Brno Univ. Technology [paper] 0.678 0.715
Univ. Sheffield (E002) [paper] 0.672 0.704
Music Tribe [paper] 0.639 0.701
ELO-SPHERES Consortium [paper] 0.653 0.676
Googlears [paper] 0.632 0.670
Univ. Oldenburg (E021) [paper] 0.611 0.636
Univ. Oldenburg (E019) [paper] 0.607 0.634
Univ. Oldenburg (E016) [paper] 0.603 0.628
Tencent/Shenzhen Univ. (E004) [paper] 0.558 0.553
Hannover Medical School [paper] 0.481 0.549
Tencent/Shenzhen Univ. (E018) [paper] 0.524 0.521
Baseline [paper] 0.282 0.335

[TOP]



Listening test evaluation

Rank Team Paper Eval-noise Eval-speech
Univ. Sheffield (E009) [paper] 80.426 82.432
Brno Univ. Technology [paper] 75.424 81.498
Music Tribe [paper] 72.222 77.778
ELO-SPHERES Consortium [paper] 83.613 82.895
Googlears [paper] 85.532 4.444
Univ. Oldenburg (E021) [paper] 84.914 83.929
Univ. Oldenburg (E019) [paper] 86.726 86.885
Hannover Medical School* [paper]
Tencent/Shenzhen Univ. (E018) [paper] 60.593 44.681
Baseline [paper] 33.202 51.152
* The Hannover Medical School results are not valid due to a problem in the final submission, which involved the application of a suboptimal hearing loss compensation strategy.
[TOP]