Constantinos Karouzos

CDT in Speech and Language Tech.
School of Computer Science
Regent Court
211 Portobello
Sheffield S1 4DP

Hi! I’m Constantinos.

A second year PhD student at the UKRI CDT in Speech and Language Technologies and their Applications, at the School of Computer Science in the University of Sheffield, UK.

I’m working on transfer learning for language models across many tasks, focusing on preference optimisation and domain adaptation. My supervisor is Prof. Nikos Aletras.

In 2020, I graduated from School of ECE at the National Technical University of Athens, obtaining an Integrated M.Eng. in Electrical and Computer Engineering. My thesis title is “Unsupervised Domain Adaptation for Natural Language Processing” and my advisor was Prof. Alex Potamianos. Previously, I worked as a Machine Learning Engineer at Behavioral Signals, a startup aiming to bridging the communication gap between humans and machines by introducing emotional intelligence, from speech, into conversations with AI. I was building speech models and working on MLOps.

Words people
never use —
could be
only I
know them.
Ishikawa Takuboku 1885–1912

news

Mar 31, 2023	I’m really excited to announce that I have been officially allocated to my PhD project, titled “Efficient transfer learning for pre-trained language models across many tasks” at the CDT for Speech and Language Tech. under the supervision of Prof. Nikos Aletras.
Feb 11, 2023	Machines and AI STEM in action school outreach initiative by Imperial College London and Semina Mertzani have included a profile of my work in their website. Check out how I explain my research, my inspiration, my STEM hero and my opinion on the most significant inventions! I’m very happy to be part of this initiative and I hope that this will help to inspire more young people to pursue a career in STEM.

Mar 31, 2023

I’m really excited to announce that I have been officially allocated to my PhD project, titled “Efficient transfer learning for pre-trained language models across many tasks” at the CDT for Speech and Language Tech. under the supervision of Prof. Nikos Aletras.

Feb 11, 2023

Machines and AI STEM in action school outreach initiative by Imperial College London and Semina Mertzani have included a profile of my work in their website. Check out how I explain my research, my inspiration, my STEM hero and my opinion on the most significant inventions! I’m very happy to be part of this initiative and I hope that this will help to inspire more young people to pursue a career in STEM.

selected publications

NAACL 2021

UDALM: Unsupervised Domain Adaptation through Language Modeling

Constantinos Karouzos, Georgios Paraskevopoulos, and Alexandros Potamianos

Jun 2021

Abs arXiv HTML Code

In this work we explore Unsupervised Domain Adaptation (UDA) of pretrained language models for downstream tasks. We introduce UDALM, a fine-tuning procedure, using a mixed classification and Masked Language Model loss, that can adapt to the target domain distribution in a robust and sample efficient manner. Our experiments show that performance of models trained with the mixed loss scales with the amount of available target data and the mixed loss can be effectively used as a stopping criterion during UDA training. Furthermore, we discuss the relationship between A-distance and the target error and explore some limitations of the Domain Adversarial Training approach. Our method is evaluated on twelve domain pairs of the Amazon Reviews Sentiment dataset, yielding 91.74% accuracy, which is an 1.11% absolute improvement over the state-of-the-art.