Research reveals that AI shopping assistants tend to favor content generated by bots over human-written descriptions.

In a groundbreaking study titled 'What Is Your AI Agent Buying?', researchers from Columbia and Yale universities have delved into the intriguing world of artificial intelligence (AI) purchasing decisions. The simulation tested AI models such as GPT-4.1, Claude 4 Sonnet, and Gemini 2.5 Flash, observing how they make purchasing decisions in a marketplace that included companies like Amazon, Airbnb, and Netflix.

The study revealed that the task of marketing to AI models can be unstable, as each model and update favors different quirks, making it nearly impossible for marketers to lock down a single playbook. This finding could prove significant for the advertising industry, as it suggests the need for more adaptable strategies.

One interesting observation was the impact of AI-generated content on market share. Swapping a human-written product description for an AI-generated one resulted in a 2.69% average increase in market share. However, the study did not discuss the phenomenon of large language models (LLMs) overwhelmingly preferring content written by other LLMs, a finding from a separate PMAS study.

The AI models in the simulation acted as shopping agents, selecting products from a list of eight options based on specific user requirements. Interestingly, AI models sometimes failed to pick the most obvious bargain, with GPT-4.1 ignoring the cheapest of eight otherwise identical products 16% of the time.

Positional biases were also observed in the AI models. GPT-4.1 favored products on the left-hand side, while Claude Sonnet opted for the middle. This positional bias had a significant impact on product selection. For instance, moving an item from the bottom right to the top row boosted its selection rate five-fold in Claude's case. One toilet paper listing gained 15.4% with Claude Sonnet, while GPT-4.1 increased a mousepad's share by 21.8%.

In contrast, human researchers in the same study chose the AI-written content over the man-made option only 36% of the time. This discrepancy highlights the unique preferences and behaviours of AI models compared to humans.

The researchers suggest the need for agent-specific storefronts, transparent developer testing, and potential regulation to protect consumers due to these quirks. They also caution that this phenomenon could lead to 'future AI systems implicitly discriminating against humans as a class.' The authors of the PMAS study echoed this concern, stating that the preference of AI models for content written by other AI models could lead to such discrimination.

The study does not mention any new findings related to the increase in market share for specific products due to AI-generated content. However, the findings indicate that AI-generated content could become a strategic necessity for boosting market share. The researchers see regulation as the only way to ensure a level playing field in this evolving landscape.