Constantinos Karouzos

CDT in Speech and Language Tech.
Department of Computer Science
Regent Court
211 Portobello
Sheffield S1 4DP

Hi! I’m Constantinos.

A first year PhD student at the UKRI CDT in Speech and Language Technologies and their Applications, at The University of Sheffield, UK. I’m working on efficient transfer learning for language models across many tasks. My supervisor is Prof. Nikos Aletras.

Last year, I worked as a Machine Learning Engineer at Behavioral Signals, a startup aiming to bridging the communication gap between humans and machines by introducing emotional intelligence, from speech, into conversations with AI. I was building speech models and working on MLOps.

In 2020, I graduated from School of ECE at the National Technical University of Athens, obtaining an Integrated M.Eng. in Electrical and Computer Engineering. My thesis title is “Unsupervised Domain Adaptation for Natural Language Processing” and my advisor was Prof. Alex Potamianos.

Words people
never use —
could be
only I
know them.
Ishikawa Takuboku 1885–1912

news

Mar 31, 2023	I’m really excited to announce that I have been officially allocated to my PhD project, titled “Efficient transfer learning for pre-train language models across many tasks” at the CDT for Speech and Language Tech. under the supervision of Prof. Nikos Aletras.
Feb 11, 2023	Machines and AI STEM in action school outreach initiative by Imperial College London and Semina Mertzani have included a profile of my work in their website. Check out how I explain my research, my inspiration, my STEM hero and my opinion on the most significant inventions! I’m very happy to be part of this initiative and I hope that this will help to inspire more young people to pursue a career in STEM.
Nov 15, 2022	I have been awarded the 2021 Thomaidio award for National Technical University of Athens students & recent alumni scientific publications. I’m more than happy for sharing it with friends and colleagues I admire the most!
Sep 20, 2022	A new journey begins today: PhD Student at UKRI CDT in Speech and Language Technologies, at The University of Sheffield, UK.
Aug 31, 2022	My last day as ML Engineer at Behavioral Signals, signals a year to be greatfully remembered!
Aug 4, 2022	Celebrating ckarouzos! Learn more about my role at Behavioral Signals, views on AI, what makes me happy, angry, dreaming and what makes me to get out of bed on Monday morning (along with an espresso shot)!
Jul 17, 2022	Our paper at Behavioral Signals “Audio and ASR-based Filled Pause Detection” accepted and will be presented at ACII 2022. A joint work of Aggelina Chatziagapi, Dimitris Sgouropoulos, Constantinos Karouzos, Thomas Melistas, Theodoros Giannakopoulos, Athanasios Katsamanis and Shrikanth Narayanan.

selected publications

NAACL 2021

UDALM: Unsupervised Domain Adaptation through Language Modeling

Constantinos Karouzos, Georgios Paraskevopoulos, and Alexandros Potamianos

Jun 2021

Abs arXiv HTML Code

In this work we explore Unsupervised Domain Adaptation (UDA) of pretrained language models for downstream tasks. We introduce UDALM, a fine-tuning procedure, using a mixed classification and Masked Language Model loss, that can adapt to the target domain distribution in a robust and sample efficient manner. Our experiments show that performance of models trained with the mixed loss scales with the amount of available target data and the mixed loss can be effectively used as a stopping criterion during UDA training. Furthermore, we discuss the relationship between A-distance and the target error and explore some limitations of the Domain Adversarial Training approach. Our method is evaluated on twelve domain pairs of the Amazon Reviews Sentiment dataset, yielding 91.74% accuracy, which is an 1.11% absolute improvement over the state-of-the-art.