Benchmarking Sentence Embeddings in Textual Stream Clustering with Applications to Campaign Detection

Stampe, Lucas; Lütke-Stockdiek, Janina; Grimme, Britta; Grimme, Christian

Abstract

Motivated by the emergence of large language models, we conduct a benchmark of sentence embeddings used to represent short texts in textual stream clustering. We achieve comparable results by adapting a non-textual stream clustering algorithm to use sentence embeddings compared to textual stream clustering approaches that use other textual representation mechanisms. Benchmarking datasets with differing degrees of preprocessing are used. The results suggest that the chosen approach using sentence embeddings does not perform as well as previous approaches on preprocessed datasets but has more significant potential on less preprocessed datasets. This highlights the need for new and more application-oriented benchmarking datasets for stream clustering. Further, we conduct a case study in the context of social media campaign detection and show that the approaches are able to find traces of orchestrated activities. 

Keywords
stream clustering; embeddings; benchmark