More and more companies are turning to artificial intelligence to help with hiring. AI doesn’t just scan resumes anymore, it can now conduct full job interviews, often through lifelike animated avatars, and even help decide who gets the job. One of the main selling points is that AI is supposed to be less biased than a human interviewer.
But a new study presented at the CHI Conference on Human Factors in Computing Systems in suggests that things aren’t quite that simple. Even when people know they’re talking to a machine, they still react to how that machine looks, almost as if the AI were a real person. According to Enkelejda Kasneci, professor of Human-Centered Technologies for Learning at the Technical University of Munich (TUM), this reaction is something that’s been largely overlooked so far. She explains that we all unconsciously respond to an avatar’s appearance, even knowing it isn’t human, and that the moment an AI starts behaving in a human-like way, the interaction becomes a social one, not just a technical exchange.
Testing How Appearance Shapes Trust
To explore this, researchers from TUM and Lund University ran an experiment with around 220 participants from Germany, the UK, and the US. Each person took part in a simulated job interview for a fictional customer support role. The interviewer was a photorealistic AI avatar that could respond naturally to answers and ask human-like follow-up questions. The researchers created four versions of this avatar, varying by gender (male or female) and skin tone (light or dark). While participants answered questions, the team tracked their eye movements, and afterward, everyone filled out a questionnaire about the experience.
Trust Holds, Until Rejection Enters the Picture
The eye-tracking data showed that people paid closer attention to the avatar’s face when its skin tone was different from their own. Interestingly, though, trust in the AI stayed high across the board at this stage, regardless of whether the avatar matched the participant’s gender or skin color.
That changed once participants were told they’d been rejected for the position and interviewed a second time. After the rejection, people became noticeably more likely to suspect they hadn’t been judged fairly, and how fair the decision felt depended heavily on the avatar’s appearance.
When the avatar’s skin color differed from the participant’s own, people were more likely to suspect bias played a role in their rejection. But the most surprising finding involved people who matched the avatar in just one characteristic, either gender or skin color, but not both. This group felt the most unfairly treated of everyone, judging the process more harshly than both those who matched the avatar completely and those who didn’t match it at all.
Rethinking Fairness in AI Design
Kasneci noted that conversations about fairness in AI tend to focus almost entirely on whether the underlying models were built and trained without bias. But she pointed out that even a genuinely unbiased AI system can still come across as unfair to the people interacting with it, and often for reasons that aren’t immediately obvious. She argued that if AI tools like this are going to work as intended, for example, producing a hiring process that everyone involved sees as legitimate, then designers need to pay much closer attention to how humans naturally behave in social situations, not just to the technical fairness of the algorithm itself.
Lau, K.H.C., & Kasneci, E. (2026). How fair does AI seem in job interviews? Perceptions of avatars depend on gender and skin color. Presented at the CHI Conference on Human Factors in Computing Systems. DOI: 10.1145/3772318.3790379
Leave a Reply