The Audience Effect: Do Observations Change Outcomes in HCI Studies?
IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC), July 2024
Huimin Tang, Boon Giin Lee, Dave Towey, Kaiyi Chen, Yichu Fang, Runzhou Zhang, Matthew Pike. 2024. The Audience Effect: Do Observations Change Outcomes in HCI Studies?. In IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC). DOI:https://doi.org/10.1109/COMPSAC61105.2024.00088
Huimin Tang and Boon Giin Lee and Dave Towey and Kaiyi Chen and Yichu Fang and Runzhou Zhang and Matthew Pike. (2024). The Audience Effect: Do Observations Change Outcomes in HCI Studies?. IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC). https://doi.org/10.1109/COMPSAC61105.2024.00088
Huimin Tang and Boon Giin Lee and Dave Towey and Kaiyi Chen and Yichu Fang and Runzhou Zhang and Matthew Pike. "The Audience Effect: Do Observations Change Outcomes in HCI Studies?." IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC), 2024. https://doi.org/10.1109/COMPSAC61105.2024.00088
Huimin Tang, Boon Giin Lee, Dave Towey, Kaiyi Chen, Yichu Fang, Runzhou Zhang, Matthew Pike. 2024. The Audience Effect: Do Observations Change Outcomes in HCI Studies?. IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC). doi:10.1109/COMPSAC61105.2024.00088
Huimin Tang and Boon Giin Lee and Dave Towey and Kaiyi Chen and Yichu Fang and Runzhou Zhang and Matthew Pike, "The Audience Effect: Do Observations Change Outcomes in HCI Studies?," IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC), 2024. doi: 10.1109/COMPSAC61105.2024.00088
@inproceedings{compsac-2024-2,
title={The Audience Effect: Do Observations Change Outcomes in HCI Studies?},
author={Huimin Tang and Boon Giin Lee and Dave Towey and Kaiyi Chen and Yichu Fang and Runzhou Zhang and Matthew Pike},
booktitle={IEEE 48th Annual Computers, Software, and Applications Conference (COMPSAC)},
year={2024},
doi={10.1109/COMPSAC61105.2024.00088}
}
Surveys, Sociology, Tablet computers, Observers, Particle measurements, User experience, Software, Human-Computer Interaction, Observation Effects, Task Performance, Mental Workload, Sudoku, Task Medium, Task Engagement
Abstract
Observational studies are widely used in Human-Computer Interaction (HCI) research to evaluate usability and user experience with technologies. However, the act of observation may influence participant behaviour and performance, threatening the validity of study findings. This paper investigates the impact of three observation types on participant outcomes in a simulated HCI study context. Participants completed Sudoku puzzles under baseline (no observation), human observation, sensor-based observation, and combined human/sensor conditions. Performance was assessed by puzzle completion rates. The mental workload was measured via NASA-TLX surveys, heart rate, galvanic skin response, and infrared thermal imaging. Results showed observations negatively impacted performance versus baseline, with human observers inducing the greatest distraction. Experienced participants were more influenced than novices. Task medium also affected engagement and observation reactivity. Findings demonstrate observations introduce bias in HCI research, emphasising careful consideration of observation methods to improve result validity.