What’s It Like to Trust an LLM: The Devolution of Trust Psychology?

Competence

While technologies such as smartphones and search engines offer individuals the opportunity to access vast amounts of information online, a distinction is made between physical access to the technology and “access” in terms of possessing the skills and knowledge to use the technology effectively. This is when skills also impact whether an individual can “access” technology, in addition to the more physical definition of access [7], [8]. Proficient use of technology here is not equated to proficiency in skills such as learning, comprehension, or critical thinking. The same argument can be made for the use of LLMs—being able to effectively use an LLM for a goal does not require or inherently lead to competency or the development of skills. However, these critical thinking skills are needed to make a trust decision about anything, given the inputs in the functionalist trust model described above.

Traditionally, effort has been a signal of competency—someone gains competency by having put effort into training, e.g., through working as an apprentice or studying at a university. However, in the age of LLMs, effort is no longer valued in that way—anyone can generate an essay, or computer code, without any investment in training. Indeed, investing effort to create an artifact now becomes seen as a lack of competence and a symptom of technology averseness. Thus, the approved signals of competency are at risk of becoming more and more hidden by the increasing use of generative AI: if anyone can generate a recommendation letter in 30 seconds using an LLM, then recommendation letters decline in value and eventually become irrelevant. Consequently, the human ability to make trust decisions based on perceived competency is undermined, not just in terms of whether to trust an LLM, but even in terms of trust decisions in interpersonal or business relationships.

Predictability

The essentially probabilistic nature of LLMs requires well-trained and skilled users who can competently prompt the model to produce a specific behavior and create output that does not consist of hallucinated text. Therefore, proper use of an LLM not only requires expertise in the domain of application but also in the emerging skill of prompt engineering.

A cautious user of LLMs, for example, in an application that is more safety-critical than a task with high requirements for creativity, is aware of the inherently probabilistic rather than deterministic nature of the output, plus the possibility of “hallucination,” bias, and potential model collapse. When LLMs are used in that way, users undergo a shift from just doing a job to needing to also consciously evaluate the quality of that part of the job effectively subcontracted out to the LLM. To do this successfully, they not only need to know how to do their own job but also how to “teach” the LLM (through appropriate prompting and correction) to produce consistent, reliable, and accurate results.

This neatly illustrates a new LLM-enabled style of automatic programming, in which the LLM is used as a mechanism to produce “low-level” programming code. Programmers are enabled to think (i.e., program) at a higher level of abstraction, such as providing glue code to interoperate with the LLM-generated code or components. While LLMs seem to make things easier at first sight by not requiring the specification of quality criteria, this evaluation becomes implicitly part of the duty of the human. However, passing on the unpredictability of LLM outputs as the responsibility of the human to validate those outputs—which, in the context of LLM-generated code, we might translate as an obligation of “caveat programmor”—diminishes the nature of trusting relationships in general.

Honesty and integrity

Reputation-based scoring systems have become a well-established feature of online interactions and commercial transactions. They are designed to assign a quantitative value to an individual’s or organization’s reputation based on a number of data points: a specific example is the AirBnB host–guest rating system, which is a key tool for hosts to build trust with potential guests, and for guests to be accepted by potential hosts. These systems offer a potentially rich source of third-party signals, which can be used to inform trustor decisions (and, indeed, provide constructive feedback for trustees to improve a product or service). However, while there is widespread evidence of systematic abuse of such scoring systems, for example, in fake reviews or blackmail threats to provide poor reviews, the more significant problem here is uniformity produced by the convergence of grade inflation (everyone has to get the highest score) with—when everyone has easy access to generative AI that is indifferent to the quality or accuracy of its output—prosaic, clichéd, and essentially indistinguishable reviews. Such standardization of reviews implies both thoughtlessness and a certain degree of dishonesty, and the resulting distrust is being observed in other domains of human activity and interaction. Anecdotally, for example, it is reported that teachers are inclined to automatically discount student dissertations at the first occurrence of certain notorious trigger words [9].

However, it is not that such reviews or references are syntactically identical, but that they look and sound sufficiently similar to give an unwarranted credence to their honesty or veracity. This is linked to the rhetorical quality of ethos, the character, or integrity of the speaker. Integrity is crucially important not only in the misuse of LLMs but also in the foundations on which they are built, i.e., in the training data itself. In the creative space, authors often remain uncompensated when their works are used to train LLMs, accessed through pirate websites or other illegally distributed content sources. While this could reasonably count as gross misconduct, it serves as a warning to what could happen if “faux science” (i.e., statements that give a vague appearance of scientific validity but are incompatible with the scientific method) is similarly distributed intentionally, without the regulation offered by the rigors of the scientific method, (for all its potential biases and idiosyncrasies) the peer review process, and (for all its perverse financial incentives for publishing houses) reliable gatekeepers of knowledge. It provides another route for the “Merchants of Doubt” [10] to “flood the zone” with fraudulent pseudoscience and conspiracy theories.

Willingness and benevolence

Willingness, as a trustworthiness feature, is concerned with the alignment of the potential trustee’s motivations with the trustor’s needs. In this context, willingness can be considered in terms of benevolence: the trustee acts in the trustor’s interest, not in their own. This is not necessarily the case with interaction with a commercial LLM, for example, the trustee (LLM) might be telling the user what the trustee wants the user to know (or not know, e.g., DeepSeek and Tiananmen Square [11]). Second, each interaction allows the collection of more data points about the user. Third, commercial LLMs are not a charity—venture capitalists will seek a return on investment. This return might be either directly in terms of revenue stream or indirectly in terms of social or political influence and domination. Services are offered for free until a critical mass is achieved, and users will then have to pay a subscription or “pay by ask.” Even financial loss can be acceptable for those who can write it off as overheads or a business expense if there is sufficient indirect recompense, such as increasing the dependence of users on the technology or reducing debate about complex and nuanced subjects to polarized slanging matches with armies of AIs distorting both balance and proportion.

Proxy trust

Lewis and Marsh say that through proxy trust, an object or technology can also act as a facade for other trust relationships. Thus, proxy trust can create trust in both the technological artifact itself and trust in the providing organization, and there is a potential duality: the LLM can be trusted because the providing organization is trustworthy, and the providing organization can be trusted because the LLM is trustworthy.

Here, however, there is a direct parallel with cryptocurrencies, where people are effectively asked to detach an existing and well-established trust relationship with an institution, in this context, a bank, or building society with a reputation for trustworthiness and reattach it to the owners and programmers of a blockchain and so by proxy in the blockchain application itself. However, there are already many examples of the abuse of blockchains in fraud, financial misconduct, pyramid schemes, and money laundering to suggest that reliance on trust decisions can be misguided because of these misconceptions about the transfer of trust-relation analogies. Similarly, there is a preexisting proxy trust relationship between people and the traditional nonprofit gatekeepers of knowledge that is being co-opted and reattached to the commercial organizations deploying LLMs, which equally should be reappraised from a first encounter trust perspective.

Situational disposition to trust

Humans are naturally context-aware, yet that feature can be a bug in the widespread adoption of LLMs. One might gravitate toward a simple Google search to fact-check or briefly research information and then be presented, unprompted, with an LLM-generated summary of the results. Search engines are one domain where we expect relevant and factual information to rise to the top but are, instead, being replaced by error/hallucination-prone summaries. This behavior can be reinforced since search engines are used by society to access information, while, simultaneously, this collective action of its users influences the relevance and order of displayed information [4]; this may, in turn, facilitate the propagation of errors or hallucinations (although the term “hallucination” is itself a misdirection: the statistical model produces a false exterior response rather than a false interior experience [12]). Trust in such contexts might be inaccurately biased, raising important questions about the deployment of LLMs in situationally sensitive domains, such as political opinion, medical advice, or even subjective reviewing.

In many domains of human endeavor, using an LLM generated output of a human–computer interaction as the indistinguishable input signal to a human– human interaction can be seen as “cheaply signaling”.

In summary, the analysis of the previous section identifies two features of evolutionary social behavior that has previously enabled human communities to maintain large-scale cooperation and trust, but those are now put at risk by current and future generative AI technologies. These two features are given as follows.

• The loss of costly signaling: for example, a well-crafted reference letter conveys evidence of its own reliability because of the cost to produce it, which is negated if such letters can be produced without cost; similarly, the reported surge in job applications is partly attributed to LLMs reducing barriers to entry, rendering the resume practically worthless as a document.
• The loss of indirect reciprocity: indirect reciprocity encourages social cooperation by third-party observation of benevolent interactions, which is how reputation and reviewing systems work; however, some restaurant owners have been blackmailed with fake negative reviews that are easy to generate in potentially convincing detail using the reviewing system data itself.

Evolutionary biology proposes that animals have evolved reliable signaling systems to ensure the coordination of the receiver and the speaker. This has been refined with the idea of costly signaling: since signals must be reliable and interaction starts with the receiver, the signaller must actually incur some resource cost that both prioritizes attention to the signal but crucially reinforces the interpretation that the receiver can trust the signaller. Evolutionarily, this affords a competitive edge compared to individuals with less ability who could not afford such a cost [13].

In many domains of human endeavor, using an LLM-generated output of a human–computer interaction as the indistinguishable input signal to a human–human interaction can be seen as “cheaply signaling”—it can be interpreted by the receiver as an indicator of a lack of care, respect, and thoughtfulness. However, if it becomes harder and harder to distinguish LLM-generated text from human-generated text, then the ability to recognize cheap signaling breaks down, and hence, the ability for costly signaling to support cooperation and trust decisions breaks down.

How could we prevent this? The loss of costly signaling affects our ability to judge competence. To prevent this, we need to change the signaling system, that is, change how competence is assessed. To come back to the reference letter example, reference letter requirements could be changed to require the attachment of evidence to support the claims being made or the format changed to a phone call conversation with the referee to assess their personal experience of working with the applicant. In the higher education system, university assessments could be changed to restore confidence in the signal, for example, through closed-book or oral examinations.

The second social support from our evolutionary past that is at risk is indirect reciprocity [14]. This is the ability to condition cooperation and trust on information from third parties, i.e., reputation. This risks being undermined by the ease with which reviews can now be generated.

How could we prevent this? Reputation systems need to be changed to prevent exploitation by automated generation. While there could be attempts to detect and prevent generated reviews, e.g., monitoring for spamming, a more fruitful avenue may be to weight reviews by the amount of objective evidence they provide. For example, hotel reviews that include proof that the reviewer stayed there during those dates may be weighted higher when calculating the average star rating compared to reviews without such evidence.

In conclusion, generative AI is conflicting with our psychology, which has evolved over millions of years to support trust decisions. There is, therefore, an urgent need to understand how the outsourcing of cognition to generative AI may lead to cognitive deskilling. This article has examined this in the context of trust decisions. Future studies should examine the risk of losing other defining features of our humanity.