Artificial Social Media Campaign Creation for Benchmarking and Challenging Detection Approaches

Pohl, Janina Susanne; Assenmacher, Dennis; Seiler, Moritz Vincent; Trautmann, Heike; Grimme, Christian


Zusammenfassung

Social media platforms are essential for information sharing and, thus, prone to coordinated dis- and misinformation campaigns. Nevertheless, research in this area is hampered by strict data sharing regulations imposed by the platforms, resulting in a lack of benchmark data. Previous work focused on circumventing these rules by either pseudonymizing the data or sharing fragments. In this work, we will address the benchmarking crisis by presenting a methodology that can be used to create artificial campaigns out of original campaign building blocks. We conduct a proof-of-concept study using the freely available generative language model \texttt{GPT-Neo} in this context and demonstrate that the campaign patterns can flexibly be adapted to an underlying social media stream and evade state-of-the-art campaign detection approaches based on stream clustering. Thus, we not only provide a framework for artificial benchmark generation but also demonstrate the possible adversarial nature of such benchmarks for challenging and advancing current campaign detection methods.

Schlüsselwörter
Social Media; Campaign; Benchmarking; Augmentation



Publikationstyp
Forschungsartikel in Sammelband (Konferenz)

Begutachtet
Ja

Publikationsstatus
Veröffentlicht

Jahr
2022

Konferenz
International Conference on Web and Social Media

Konferenzort
Atlanta

Buchtitel
Proceedings of the 16th International Conference on Web and Social Media – 1st Workshop on Novel Evaluation Approaches for Text Classification Systems on Social Media (NEATCLasS)

Herausgeber
Association for the Advancement of Artificial Intelligence (AAAI)

Erste Seite
1

Letzte Seite
10

Verlag
AAAI Press

Ort
Palo Alto, CA, USA

Sprache
Englisch

DOI

Gesamter Text