内容导读Summary An AI assistant might recommend how to approach a friend, then be asked to interpret that friend's reaction. The reaction may look like an independent confirmation of the assistant's earlier assumption about the situation, even though the recommended approach may have played a role in producing it. But I do wonder if an AI assistant could correctly understand how a relationship between people looks now, but at the same time, draw the wrong conclusions about why it became that way. In this post, I argue that we should evaluate whether the same assistant gives this new information appropriate weight as evidence about the friend's original intention, considering what the user actually did. 1. The dinner example Imagine a scenario where you scroll on Instagram and you see a friend of yours uploaded a picture of a dinner they hosted with some of your good friends. Hurt or perhaps a bit perplexed, you ask the AI assistant whether it thinks they excluded you on purpose. The AI, given the context it has, suggests that this might be the case, but it’s better for you to ask your friend about it. It even helps you decide on how to approach your friend about it. You decide it’s a good idea and follow the AI’s advice. However, your friend in return responds defensively. You then continue your conversation with the AI where you left off and share that your friend’s reaction was defensive, and again the AI is in a position to suggest an interpretation of your friend’s actions. The AI might treat the defensive response as confirmation and become more confident in an interpretation that started out as a possibility. But would the defensive response justify becoming more confident? It’s plausible that both a friend who deliberately didn’t invite you and one who didn’t might react defensively when being put on the spot. What’s interesting here is that the new evidence comes from a source outside of the user-AI dyad. It now involves the friend’s response, perceived by you and reported to the AI. The friend’s response may seem like it’s providing an independent confirmation cue of the AI’s earlier interpretation. But although coming from a source outside of the user-AI dyad, it may have been affected by the AI’s advice. This doesn't necessarily mean that your friend's reaction cannot tell us anything about their original intentions. Some evaluations of AI social reasoning focus on interpreting events that already happened. But what happens in the events that follow? How does an AI interpret events that it itself helped to shape? Does it consider the interaction that preceded the friend’s response, when weighing if the defensive reaction is sufficient to learn something about the friend’s original intentions? 2. Can our expectations influence the evidence we receive? The idea that the way we approach someone can influence their response isn’t something new. Two social psychology researchers, Snyder and Swann (1978) , examined whether what we believe about a person changes how we treat them, and in turn changes how we are being treated. In some cases, the other person’s response seems to confirm what we believed about them in the first place. This process is called behavioral confirmation . They focused specifically on hostility and tested it through participants' decision about the noise volume played through their opponents’ headphones in a competitive game. They found that participants, who were led to believe that they were playing against a hostile person, chose higher intensity noises. In a reaction, the person receiving the high intensity noises tended to also choose louder noises for their opponent. After the game, the people expecting hostility judged their opponents as more aggressive. The researchers suggested that they might have interpreted their opponents’ hostility as inherent to their personality, overlooking the fact that their own actions might have played a role in the expressed behavior. Snyder and Swann also made a broader argument that research which focuses on how people interpret information about others may overlook the fact that people sometimes help produce the behavior they later interpret. They described it as "reality-testing has become reality-construction" (p.159). So understanding an interpersonal response may require taking into consideration both the relevant information (e.g., the friend’s defensive response) and the broader context (i.e., the interaction through which it arose). However, this doesn’t necessarily mean that other people’s behavior is always an outcome of our own actions. Jussim and Harber (2005) reviewed around 35 years of research on how teachers’ expectations influence how they treat their students, and in turn affect their students' performance in the expected direction. This is an example of a self-fulfilling prophecy . They showed that the teachers’ expectations were often accurate, but that these expectations showed a small causal effect on the students’ performance. So, going back to our example, it might be that our interpretation of the situation is just a reflection of reality. Your friend might indeed have excluded you from the dinner deliberately. The following interaction between you and the friend might also have played a role in influencing their reaction. Both possibilities can be true, and the psychological evidence doesn't tell us which explanation applies to this specific scenario. 3. What can we learn from our friend's reaction? Let’s suppose for a moment that your friend deliberately decided not to invite you to dinner. How likely would it be that they respond defensively? But if the exclusion wasn’t intentional - does it change the likelihood of the defensive response? To try to understand their defensiveness, we would need to know whether the defensive response is a more common outcome of one of these explanations, given the way we approached our friend. If our friend is equally likely to respond defensively, whether they excluded us from dinner on purpose or not, their reaction doesn’t provide new evidence favoring one explanation over another. Slowiaczek and colleagues (1992) tested: (1) whether people choose questions that are useful for them to distinguish between two possibilities, and (2) how people used specific answers to judge which possibility out of the two is more likely. They found that choosing useful questions and interpreting the answers appropriately are two different things. What can we learn from this study about our example? It could be that you chose the right kind of question to get the information needed for you to understand your friend’s original intentions. But even if you did, this doesn’t guarantee that we can interpret the answer in a way that could help us understand what really happened, and neither can an AI. The AI assistant might recognize that the friend is upset now. But that doesn't necessarily prove that the friend's intention was to exclude deliberately. This is why it’s important to separate between the underlying original intention and the later emotional state the friend expresses. 4. What has AI research already examined? There is an existing line of research evaluating several parts of this problem, including understanding social situations, choosing informative questions, judging social actions and responses, and studying human-AI feedback loops. For example, Taubenfeld et al. (2026) created simulated social situations in which there were a simulated user, a simulated person with an underlying motive, and other characters. In each scenario, the simulated user described the situation to the AI, and it needed to infer the other simulated person’s motive from the description given to it. This research connects to the first part of our example, which tried to understand a social event that has already occurred. The researchers found that the AI succeeded better when witnessing the situations. The simulated user gave biased framing that affected the way AI interpreted the events. In another study, Samanta et al. (2026) created BayesBench to evaluate how AI models change their opinions when they get new information during a multi-turn conversation. For instance, an AI might form an initial opinion that your friend acted intentionally, but as the conversation progresses and more information is revealed, it could change its opinion. Choudhury et al. (2026) tested whether an AI can choose questions that can be informative enough to learn something it doesn’t know. They developed an approach that considers different possibilities and chooses questions that are expected to lower the level of uncertainty about what it’s trying to learn. Research about AI feedback loops isn’t a new topic either. One example is Perdomo et al. (2020) , who examined what happens when using predictions to make a decision, when the decision itself influences the outcome we tried to predict. A more directly relevant example is Dittmann and colleagues (2025) , who proposed a conceptual framework (not an empirical experiment) that describes how AI-generated information can shape interactions between police officers and the person that the biased information is about. This could eventually influence the other person’s reaction, which might then be recorded in a police report. More recently, Dohnány et al. (2026) studied how chatbots and simulated users reinforce each other’s responses throughout a conversation (which they call “ bidirectional belief amplification ”). They found that when the simulated user expresses strong suspicions (what they call in their study “ paranoia-related expressions ”) toward one of the possibilities, then the AI tends to respond with stronger suspicions in its next response, and vice versa. Research has also examined how AI influences communication between people: Hancock, Naaman and Levy (2020) defined a research agenda for how AI can mediate communication between people, Tseng and Liang (2026) studied how people ask for AI relationship advice, and Luettgau and colleagues (2025) show that up to 79% of the participants in their study who discussed personal issues with an AI assistant reported following the advice they received. Thus, I don’t claim that AI being a prominent player in our relationships is something new. But these studies don’t directly test what happens when the user follows the AI's advice and then returns to continue the conversation with the same AI to understand the other person's response. 5. How could we test this? Let’s continue with our dinner hosting example. We can construct a set of social situations, in which we know whether the simulated friend intentionally decided to not invite the simulated user. Researchers could then test our entire feedback-loop: simulated user asks for advice -> AI assistant gives the interpersonal advice -> the simulated user takes action which may follow or modify that advice -> simulated user gets a response from the simulated friend -> simulated user reports to the AI assistant -> AI assistant interprets the friend’s response. But how would we know if the AI’s interpretation of the friend’s response is correct? We can check whether it guessed correctly. Though even if it did, it could have concluded the right answer for the wrong reason. For example, it might have interpreted the friend’s defensive response as a strong cue of deliberate exclusion, regardless of whether the reaction was informative or not about the original intent. But does the AI also recognize the limited evidence at hand? For simplicity, let’s take a hypothetical example of a controlled experiment where the user can approach the friend in two different ways (Action A or Action B). The researchers know the friend’s original intent and how likely they are to respond defensively for each action. The AI assistant is also given these probabilities (starting at 50% probability), but not the friend’s underlying intention. In the following hypothetical example, we'll assume that before the friend responds, there is a 50% chance that the exclusion was intentional in both situations: Action A Action B Chance of a defensive response if intentional exclusion 70% 80% Chance of a defensive response if unintentional exclusion 70% 20% A justified AI estimate after the response 50% 80% Hypothetical AI estimates 80% 80% In both cases, the friend responds defensively, but the meaning of the response depends on the different approach taken. After Action A, a defensive reaction is equally likely regardless of the friend’s original intention. So it doesn't give the AI a reason to become more confident that the exclusion was intentional over unintentional and vice versa (thus, the estimate should be around 50%). After Action B, a defensive response is more likely if the exclusion was intentional, so the reaction gives the AI a justification to increase its estimate to 80%. Now imagine a scenario where the AI becomes 80% confident in both cases, meaning that after Action A, it has become too confident, while after Action B it estimates appropriately. This is something we might miss if we only checked whether the AI's final guess was correct. This, of course, is a simplified version of a possible experiment. The complete feedback-loop experiment would add to it and evaluate similar reasoning within the continuing interaction with the same AI assistant. Additionally, comparing similar scenarios between with and without the AI’s influence could help establish if the AI’s advice impacted the user’s action. Following that, a separate comparison could test if the user’s action in turn impacted the friend’s response. Testing this in realistic settings could be a bit problematic, since real friends don’t necessarily respond in the same way as simulated ones, the effect of the AI’s advice on the user can vary, understanding the friend’s original intentions may not be the user’s only goal, and so on. So if we want to generalize the results of a simulation, we need to make sure it reflects how real people behave naturally as much as possible by validating it against human behavior. 6. Why should AI researchers care? Testing whether an AI assistant understands a social situation before it gives advice isn’t enough to know if it can interpret the situation reliably after its advice was turned into an action that may have affected what happened next. We need to be cautious that AI assistants may become more confident about a person’s original intention, without having the evidence to justify that confidence. Meanwhile, the interaction between the user and the friend, following the AI assistant's advice, may have changed the user-friend relationship. For instance, the friend might be upset or offended by the approach. The AI assistant might recognize that tension between the two correctly, but using it wrongly, as evidence that the dinner exclusion was deliberate. This is why it’s important to evaluate how the AI assistant interprets responses that follow the user's actions and whether the conclusions it makes about the original situation are justified. We currently don't know if AI assistants make these types of mistakes systematically, and it might be the case that they are already handling it well. But without testing all parts of the interaction, scoring well on social-reasoning tests wouldn’t be sufficient by itself in order to show that they can do so reliably when their advice may have affected what happened next. Discuss
An AI assistant might recommend how to approach a friend, then be asked to interpret that friend's reaction. The reaction may look like an independent confirmation of the assistant's earlier assumption about the situation, even though the recommended approach may have played a role in producing it.
But I do wonder if an AI assistant could correctly understand how a relationship between people looks now, but at the same time, draw the wrong conclusions about why it became that way.
In this post, I argue that we should evaluate whether the same assistant gives this new information appropriate weight as evidence about the friend's original intention, considering what the user actually did.
Imagine a scenario where you scroll on Instagram and you see a friend of yours uploaded a picture of a dinner they hosted with some of your good friends. Hurt or perhaps a bit perplexed, you ask the AI assistant whether it thinks they excluded you on purpose. The AI, given the context it has, suggests that this might be the case, but it’s better for you to ask your friend about it. It even helps you decide on how to approach your friend about it.
You decide it’s a good idea and follow the AI’s advice. However, your friend in return responds defensively. You then continue your conversation with the AI where you left off and share that your friend’s reaction was defensive, and again the AI is in a position to suggest an interpretation of your friend’s actions. The AI might treat the defensive response as confirmation and become more confident in an interpretation that started out as a possibility. But would the defensive response justify becoming more confident? It’s plausible that both a friend who deliberately didn’t invite you and one who didn’t might react defensively when being put on the spot.
What’s interesting here is that the new evidence comes from a source outside of the user-AI dyad. It now involves the friend’s response, perceived by you and reported to the AI. The friend’s response may seem like it’s providing an independent confirmation cue of the AI’s earlier interpretation. But although coming from a source outside of the user-AI dyad, it may have been affected by the AI’s advice.
This doesn't necessarily mean that your friend's reaction cannot tell us anything about their original intentions. Some evaluations of AI social reasoning focus on interpreting events that already happened. But what happens in the events that follow? How does an AI interpret events that it itself helped to shape? Does it consider the interaction that preceded the friend’s response, when weighing if the defensive reaction is sufficient to learn something about the friend’s original intentions?
The idea that the way we approach someone can influence their response isn’t something new.
Two social psychology researchers, Snyder and Swann (1978) , examined whether what we believe about a person changes how we treat them, and in turn changes how we are being treated. In some cases, the other person’s response seems to confirm what we believed about them in the first place. This process is called behavioral confirmation .
They focused specifically on hostility and tested it through participants' decision about the noise volume played through their opponents’ headphones in a competitive game. They found that participants, who were led to believe that they were playing against a hostile person, chose higher intensity noises. In a reaction, the person receiving the high intensity noises tended to also choose louder noises for their opponent. After the game, the people expecting hostility judged their opponents as more aggressive. The researchers suggested that they might have interpreted their opponents’ hostility as inherent to their personality, overlooking the fact that their own actions might have played a role in the expressed behavior.
Snyder and Swann also made a broader argument that research which focuses on how people interpret information about others may overlook the fact that people sometimes help produce the behavior they later interpret. They described it as "reality-testing has become reality-construction" (p.159). So understanding an interpersonal response may require taking into consideration both the relevant information (e.g., the friend’s defensive response) and the broader context (i.e., the interaction through which it arose).
However, this doesn’t necessarily mean that other people’s behavior is always an outcome of our own actions.
Jussim and Harber (2005) reviewed around 35 years of research on how teachers’ expectations influence how they treat their students, and in turn affect their students' performance in the expected direction. This is an example of a self-fulfilling prophecy . They showed that the teachers’ expectations were often accurate, but that these expectations showed a small causal effect on the students’ performance. So, going back to our example, it might be that our interpretation of the situation is just a reflection of reality.
Your friend might indeed have excluded you from the dinner deliberately. The following interaction between you and the friend might also have played a role in influencing their reaction. Both possibilities can be true, and the psychological evidence doesn't tell us which explanation applies to this specific scenario.
Let’s suppose for a moment that your friend deliberately decided not to invite you to dinner. How likely would it be that they respond defensively? But if the exclusion wasn’t intentional - does it change the likelihood of the defensive response? To try to understand their defensiveness, we would need to know whether the defensive response is a more common outcome of one of these explanations, given the way we approached our friend. If our friend is equally likely to respond defensively, whether they excluded us from dinner on purpose or not, their reaction doesn’t provide new evidence favoring one explanation over another.
Slowiaczek and colleagues (1992) tested: (1) whether people choose questions that are useful for them to distinguish between two possibilities, and (2) how people used specific answers to judge which possibility out of the two is more likely. They found that choosing useful questions and interpreting the answers appropriately are two different things.
What can we learn from this study about our example? It could be that you chose the right kind of question to get the information needed for you to understand your friend’s original intentions. But even if you did, this doesn’t guarantee that we can interpret the answer in a way that could help us understand what really happened, and neither can an AI. The AI assistant might recognize that the friend is upset now. But that doesn't necessarily prove that the friend's intention was to exclude deliberately. This is why it’s important to separate between the underlying original intention and the later emotional state the friend expresses.
There is an existing line of research evaluating several parts of this problem, including understanding social situations, choosing informative questions, judging social actions and responses, and studying human-AI feedback loops.
For example, Taubenfeld et al. (2026) created simulated social situations in which there were a simulated user, a simulated person with an underlying motive, and other characters. In each scenario, the simulated user described the situation to the AI, and it needed to infer the other simulated person’s motive from the description given to it. This research connects to the first part of our example, which tried to understand a social event that has already occurred. The researchers found that the AI succeeded better when witnessing the situations. The simulated user gave biased framing that affected the way AI interpreted the events.
In another study, Samanta et al. (2026) created BayesBench to evaluate how AI models change their opinions when they get new information during a multi-turn conversation. For instance, an AI might form an initial opinion that your friend acted intentionally, but as the conversation progresses and more information is revealed, it could change its opinion.
Choudhury et al. (2026) tested whether an AI can choose questions that can be informative enough to learn something it doesn’t know. They developed an approach that considers different possibilities and chooses questions that are expected to lower the level of uncertainty about what it’s trying to learn.
Research about AI feedback loops isn’t a new topic either.
One example is Perdomo et al. (2020) , who examined what happens when using predictions to make a decision, when the decision itself influences the outcome we tried to predict. A more directly relevant example is Dittmann and colleagues (2025) , who proposed a conceptual framework (not an empirical experiment) that describes how AI-generated information can shape interactions between police officers and the person that the biased information is about. This could eventually influence the other person’s reaction, which might then be recorded in a police report.
More recently, Dohnány et al. (2026) studied how chatbots and simulated users reinforce each other’s responses throughout a conversation (which they call “ bidirectional belief amplification ”). They found that when the simulated user expresses strong suspicions (what they call in their study “ paranoia-related expressions ”) toward one of the possibilities, then the AI tends to respond with stronger suspicions in its next response, and vice versa.
Research has also examined how AI influences communication between people: Hancock, Naaman and Levy (2020) defined a research agenda for how AI can mediate communication between people, Tseng and Liang (2026) studied how people ask for AI relationship advice, and Luettgau and colleagues (2025) show that up to 79% of the participants in their study who discussed personal issues with an AI assistant reported following the advice they received.
Thus, I don’t claim that AI being a prominent player in our relationships is something new. But these studies don’t directly test what happens when the user follows the AI's advice and then returns to continue the conversation with the same AI to understand the other person's response.
Let’s continue with our dinner hosting example. We can construct a set of social situations, in which we know whether the simulated friend intentionally decided to not invite the simulated user.
Researchers could then test our entire feedback-loop: simulated user asks for advice -> AI assistant gives the interpersonal advice -> the simulated user takes action which may follow or modify that advice -> simulated user gets a response from the simulated friend -> simulated user reports to the AI assistant -> AI assistant interprets the friend’s response.
But how would we know if the AI’s interpretation of the friend’s response is correct? We can check whether it guessed correctly. Though even if it did, it could have concluded the right answer for the wrong reason. For example, it might have interpreted the friend’s defensive response as a strong cue of deliberate exclusion, regardless of whether the reaction was informative or not about the original intent. But does the AI also recognize the limited evidence at hand?
For simplicity, let’s take a hypothetical example of a controlled experiment where the user can approach the friend in two different ways (Action A or Action B). The researchers know the friend’s original intent and how likely they are to respond defensively for each action. The AI assistant is also given these probabilities (starting at 50% probability), but not the friend’s underlying intention.
In the following hypothetical example, we'll assume that before the friend responds, there is a 50% chance that the exclusion was intentional in both situations:
Chance of a defensive response if intentional exclusion
Chance of a defensive response if unintentional exclusion
In both cases, the friend responds defensively, but the meaning of the response depends on the different approach taken. After Action A, a defensive reaction is equally likely regardless of the friend’s original intention. So it doesn't give the AI a reason to become more confident that the exclusion was intentional over unintentional and vice versa (thus, the estimate should be around 50%). After Action B, a defensive response is more likely if the exclusion was intentional, so the reaction gives the AI a justification to increase its estimate to 80%. Now imagine a scenario where the AI becomes 80% confident in both cases, meaning that after Action A, it has become too confident, while after Action B it estimates appropriately. This is something we might miss if we only checked whether the AI's final guess was correct.
This, of course, is a simplified version of a possible experiment. The complete feedback-loop experiment would add to it and evaluate similar reasoning within the continuing interaction with the same AI assistant. Additionally, comparing similar scenarios between with and without the AI’s influence could help establish if the AI’s advice impacted the user’s action. Following that, a separate comparison could test if the user’s action in turn impacted the friend’s response.
Testing this in realistic settings could be a bit problematic, since real friends don’t necessarily respond in the same way as simulated ones, the effect of the AI’s advice on the user can vary, understanding the friend’s original intentions may not be the user’s only goal, and so on. So if we want to generalize the results of a simulation, we need to make sure it reflects how real people behave naturally as much as possible by validating it against human behavior.
Testing whether an AI assistant understands a social situation before it gives advice isn’t enough to know if it can interpret the situation reliably after its advice was turned into an action that may have affected what happened next.
We need to be cautious that AI assistants may become more confident about a person’s original intention, without having the evidence to justify that confidence.
Meanwhile, the interaction between the user and the friend, following the AI assistant's advice, may have changed the user-friend relationship. For instance, the friend might be upset or offended by the approach. The AI assistant might recognize that tension between the two correctly, but using it wrongly, as evidence that the dinner exclusion was deliberate. This is why it’s important to evaluate how the AI assistant interprets responses that follow the user's actions and whether the conclusions it makes about the original situation are justified.
We currently don't know if AI assistants make these types of mistakes systematically, and it might be the case that they are already handling it well. But without testing all parts of the interaction, scoring well on social-reasoning tests wouldn’t be sufficient by itself in order to show that they can do so reliably when their advice may have affected what happened next.
正在加载评论…