A Human-is-the-Loop Approach for Semi-Automated Content Moderation

Link Daniel, Hellingrath Bernd Ling Jie


Abstract
Online social media has been recognized as a valuable information source for disaster management whose volume, velocity and variety exceed manual processing capacity. Current machine learning systems that support the processing of such data generally follow a human-in-the-loop approach, which has several inherent limitations. This work applies the human-is-the-loop concept from visual analytics to semi-automate a manual content moderation workflow, wherein human moderators take the dominant role. The workflow is instantiated with a supervised machine learning system that supports moderators with suggestions regarding the relevance and categorization of content. The instantiated workflow has been evaluated using in-depth interviews with practitioners and serious games. which suggest that it offers good compatibility with work practices in humanitarian assessment as well as improved moderation quality and higher flexibility than common approaches.

Keywords
Disaster management; social media analysis; human-is-the-loop; content moderation; supervised machine learning; humanitarian logistics



Publication type
Conference Paper

Peer reviewed
Yes

Publication status
Published

Year
2016

Conference
ISCRAM 2016

Venue
Rio de Janeiro, Brazil

Book title
Proceedings of the 13th International Conference on Information Systems for Crisis Response and Management

Editor
Tapia AH, Antunes P, Bañuls VA, Moore K, Albuquerque JP

Full text