Abstract
As generative AI systems are increasingly used to provide advice in complex situations, understanding how recipients evaluate such messages has become essential. The present study examined how perceived credibility and adherence depend on message source (AI vs. human), message quality (well-written vs. poorly written messages), and individual dispositions toward AI and intellectual humility.A sample of 175 participants evaluated ten scenarios describing challenging student situations, each followed by a response attributed either to a human expert or to ChatGPT and manipulated to display either high or low content quality. Data were analyzed using cumulative link mixed models (CLMM) to account for the ordinal nature of the dependent variables and the repeated-measures design.The results show no main effect of message source on credibility or adherence: responses attributed to AI are evaluated similarly to those attributed to a human expert. Message quality increases perceived credibility, with well-written messages judged as more credible, while no main effect of quality emerges for adherence. In contrast, attitudes toward AI modulate these evaluations: more favorable attitudes heighten sensitivity to message quality. Intellectual humility does not emerge as a significant main predictor, although exploratory analyses suggest a secondary role.Overall, these results indicate that the evaluation of advice relies primarily on message content and individual expectations rather than solely on source attribution. They suggest that fostering critical engagement with these systems is a key lever for their effective integration in advisory contexts.

This publication has 18 references indexed in Scilit: