The rise of Artificial Intelligence (AI) has created great changes in our society and significantly altered the way we live, learn and act. The rapid deployment of AI technology has enabled AI agents to take on various roles as communicators, such as virtual assistants, robot journalists, and AI doctors. People even think that the technology could “overtake human thinking” in near future. As AI gets smarter and takes on a growing diversity of tasks, people are curious about the communication effectiveness of AI versus humans. It also attracts scholars to know how AI’s role of persuader influences the receivers’ responses in various domains of human life.
Professor HUANG Guanxiong, an Associate Professor of CityU’s Department of Media and Communication (COM) has recently published a research article entitled “Is artificial intelligence more persuasive than humans? A meta-analysis” in the Journal of Communication, trying to provide a fuller understanding of the relative effects of AI versus humans on persuasion and help uncover the circumstances under which these effects are stronger, weaker, or null. This article was co-authored by Dr Sai WANG, a recent CityU COM PhD alumna and now a Research Assistant Professor of Hong Kong Baptist University. The study meta-analysed 121 randomised experimental studies that compared the effects of AI and human agency on persuasion outcomes, including perceptions, attitudes, intentions, and behaviours.
Although many studies have examined how well AI agents perform as communicators and persuaders relative to human agents, the findings are mixed, and a large variance has been observed across different roles played by AI agents in communication processes. In this meta-analysis, the role of AI in communication is classified into four categories: converser (e.g., chatbot, virtual assistant, smart speaker), creator (e.g., robot journalist, virtual influencer, algorithmic artwork generator), curator (e.g., factchecker, recommendation system, content moderator), and contemplator (e.g., decision maker, AI doctor). The direction of human-AI communication was operationalised as a binary variable based on whether the communication was one-way or two-way. Two demographic factors gender and age are also examined as potential moderators.
The study results found that AI agents were as persuasive as human agents in terms of overall persuasion outcomes. The finding suggests that AI agents can serve as sources of communication or interaction that lead users to apply the social rules they develop when interacting with a person to their interactions with these nonhuman agents. However, the results diverged when it came to different types of persuasion outcomes. AI was less effective than humans at shaping behavioural intentions, but did not differ significantly from humans in eliciting perceptions, attitudes, or actual behaviours. This discrepancy underscores the complexity of AI-based persuasion, a topic marked by substantial heterogeneity in both theories and empirical evidence. It would be fruitful for future research to explore why AI is less persuasive than humans in the contemplator role and how to improve the persuasiveness of AI decision-making systems.
The direction of AI-human communication also makes a difference in actual behaviours. In one-way communication, AI exhibited a greater impact on actual behaviours than humans. However, this effect was not observed in two-way communication. This suggests that the mere presence of AI as the source of communication or interaction on media interface may serve as a cue that leads users to apply favourable machine attributes (e.g., machines are more accurate and objective than humans) to guide their actual behaviours. This tendency, however, seems to disappear when AI provides agency to users and allows them to interact with the system.
Furthermore, several interesting results were observed in the moderating effects of age and gender on the overall and specific persuasion outcomes. For instance, the relative advantage of humans over AI on perceptions and behavioural intentions were more pronounced in the sample between 25 and 40 years old than in other age groups. One possible explanation is that millennials have relatively low confidence in AI technology, as some qualitative evidence suggests that they tend to question the usefulness of AI and worry that their personal data will be misused by AI.
In short, the findings shed light on the conditions (source features, contextual factors, and audience characteristics) under which AI is more or less persuasive than humans and thus serve as an important step toward an integrated framework for persuasion in the era of AI.
Publication and achievements
Huang, G & Wang, S 2023, 'Is artificial intelligence more persuasive than humans? A meta-analysis', Journal of Communication, vol. 73, no. 6, jqad024, pp. 552–562. https://doi.org/10.1093/joc/jqad024
