Do you start shouting and waving if the quality of Zoom or Skype call starts to deteriorate halfway through your meeting? If the answer is yes, you’re not alone. Researchers from the Radboud University and the Max Planck Institute for Psycholinguistics found that the poorer the quality of the image or sound during a Skype or Zoom call, the louder participants talk. Most people also try to convey their message through gestures to compensate for the poor sound, according to a study published in the Royal Society Open Science journal.
This is not entirely surprising. When we try to talk to somebody in a busy and loud environment, we use some of the same tactics, explained James Trujillo, first author and a cognitive scientist at Radboud University and the Max Planck Institute for Psycholinguistics.
“If you’re talking to someone in a busy area with a lot of background noise, you typically use gestures to support your speech, and you start talking louder. When we talk in a video call, any issues that make it harder for people to understand each other usually come from technical difficulties. However, we still tend to use similar methods to compensate for those issues.”
For the study, the team set up video calls involving 20 pairs of participants. Participants were placed in separate rooms and told to have a casual conversation for 40 minutes. During the call, the quality of the video started to deteriorate from clear to extremely blurry. During the call, the researchers tracked the participant’s speech and gestures.
Initially, as the quality of the video call started to drop, participants stopped body and arm movements but then increased them significantly as the quality decreased further. At this stage, gestures grew quickly at first, and then a plateau. Participants also started shouting when the video quality dropped. Even when the image quality was so poor that they could barely see each other, people continued to use gestures and shout.
“What this shows is that speech and gestures are integrated. People are compensating for losing out on visuals by adapting with louder speech and bigger gestures. Even when the image can barely be seen, people don’t suddenly stop gesturing. It’s similar to how people talk to each other on the phone: we don’t see each other, and during an involved conversation, we still move and gesticulate,” said Trujillo.
This is the first time researchers have studied how what we see during the call influences our behaviour. “Previous research has shown that speech and gestures are linked, but ours is the first to look into how visuals impact our behaviour in those fields,” noted Trujillo. ‘It indicates that speech and gestures are dynamically adapted to our demands. What this study shows is that if you’re studying communicative behaviour, you have to look at the full picture”.
Previous research has suggested that gestures are only a minor addition to speech, and not an essential part. According to the authors, this study proves precisely the opposite: if gestures were not crucial, people wouldn’t use them to compensate for the loss of sound to continue with the conversation. Further research is needed to look at factors like loudness and tone of voice, as well as analyse how these gestures are connected to people’s behaviour.
Trujillo J, Levinson S and Holler J (2022) A multi-scale investigation of the human communication system’s response to visual disruption. Royal Society Open Science, https://doi.org/10.1098/rsos.211489