Comparing Simulated and Real Conversations for QoE Assessments: Insights from ARKit-Based Facial Configuration Analyses

Abstract

This manuscript investigates the suitability of video-based conversation simulations for studying human reactions to quality degradations, focusing on facial configurations as a proxy for QoE. We analyze data from two distinct studies: a video-simulated video-telephony scenario using the storytime dataset, where participants passively watched videos, and a second study involving real conversations between participants. In both studies, facial features were continuously recorded using Apple’s iOS ARKit API. We identify a factor structure of facial features that significantly relates to participants’ QoE ratings in the first study and validate its robustness by replicating it in the second, independent study. Our findings suggest statistically significant estimations of QoE ratings across both paradigms, demonstrating the suitability of passive conversation simulations for studying human reactions to quality degradation. We assess the value of the proposed approach at its present stage and conclude that it can be a valuable tool when used in conjunction with other methods, as its predictive capabilities are still not robust enough to rely solely on this analysis technique.