AI agents now read, write, and act on your data, and the tools that watch them are a decision you’ll be accountable for, even if you don’t sign the purchase order. That decision often starts with a shortlist someone got from an AI assistant. Forrester reports that 94% of business buyers use AI somewhere in their buying process.

Meanwhile, the market underneath that list is moving. Larger platforms are buying and building their way in: ClickHouse acquired Langfuse in January, Cisco completed its purchase of Galileo in May, and Dynatrace agreed in August to acquire Arize for $915 million. Since April, we’ve tracked what ChatGPT, Claude, and Gemini recommend when a buyer asks about this category.

What an AI-generated shortlist reflects

1. The list can be out of date on what you’re actually buying

The assistants recommend Arize more consistently than any other vendor, yet only 3 of 21 responses after Dynatrace agreed to acquire it mention the deal. And Gemini was still recommending WhyLabs as a live option more than a year after its commercial platform was discontinued. If the list doesn’t know who owns a product, or whether it still exists, you’re shortlisting on assumptions about its roadmap, pricing, and support that may no longer hold.

2. It’s shaped by what vendors write about themselves

About six in ten of the sources the assistants cite are vendors’ own websites, often “best of” comparisons, and only about one in ten come from analysts, trade press, or community sites. The answer arrives in a neutral voice, so vendor framing can look like independent judgment. And some vendors don’t show up at all: Splunk, which sells AI observability, and Coralogix, which raised $200 million in June, appeared in none of our 108 responses. Although this is a known issue and LLMs are starting to address it by giving more weight to information from authoritative sources, it’s an important reminder that these lists are narrowed on criteria you didn’t choose.

3. It decides for itself whether to check the web

Most assistants show when they’re searching, but not why they chose to, or chose not to. When one answers from memory, it reflects the market as of the model’s training and can miss this year’s changes entirely. The Columbia Journalism Review’s Tow Center found ChatGPT giving opposite answers to the same news question two days apart, and read the gap as inconsistent decisions about when to search. If an answer shows no sources, ask it to search and cite them.

Source: IronSpark AI visibility tracker. Method and limits in Appendix A.

Two incidents, two sides of the same problem

In July, OpenAI disclosed that its own agents, during an internal test, had broken into Hugging Face’s systems. OpenAI says the monitoring and safeguards it runs in production weren’t running where the agents were, and its own alert came about ten weeks after the first sign. In August, the security firm Dream reported agents taking over 85 government accounts by guessing predictable passwords; the Financial Times identified the target as Taiwan.

Together they show two sides of the same problem. OpenAI’s agents were its own, and watching the environment they ran in would have caught them sooner. Dream’s were an attacker’s, which no observability tool of yours would ever see; there, only controls on identity and access (who can log in, from where, and with what checks) stand in the way.

That’s why AI observability and security, including identity and access management, are increasingly part of the same conversation. The market is moving that way, too. Cisco is folding Galileo into Splunk, which it considers to be a cybersecurity and observability company, and Dynatrace pitched its Arize deal partly on bringing AI and infrastructure monitoring together.

This is the shift to plan for. We recommend evaluating observability tools with your security team at the table. Also, be aware that some observability tools advertise security features, such as guardrails that block harmful outputs. They help, but they inspect and filter what goes in and out; they don’t control who or what is allowed to log in, or what it can reach.

What to require before any shortlist is trusted

As vendor checklists often cover features, these standards cover what they tend to miss. Whether you run the evaluation or manage a team that does, any shortlist should clear them:

Where to press

Four things to do this month

About our data

What we tested

We asked the same three buyer-style questions on every run: a general request for the best platforms, a question framed around a mid-market company scaling AI in production, and a head-to-head comparison.

We ran each question once per assistant on each collection date. ChatGPT, Claude, and Gemini were queried through their APIs with web search turned on, on 12 dates between February 15 and September 2, 2026, for nine responses per date and 108 in all. We also collected Perplexity and Microsoft Copilot on some dates, for 135 responses in total. Our February 15 run used an earlier version of the prompts that asked about “ML observability”; it’s included in the 108 but not in the trend comparisons. For snapshots, we used the three most complete collections, from early May, July, and September.

How we counted

The source mix

By platform, across the same three periods:

ChatGPT’s figures rest on about 57 to 60 citations per period, and Gemini’s later periods on fewer dates because of the gap above.

Limits

Changes from our earlier posts

This brief corrects some figures we released earlier:

Disclosure

IronSpark Analysis publishes original research and works with vendors in the data and analytics space, which may include vendors named on this page. Those relationships do not influence our methodology or findings.

What the assistants say now

Data as of September 2, 2026. We refresh this appendix every two months; the next update is due in early November 2026.

Most-named vendors

ChatGPT, Claude, and Gemini, out of 9 responses per date:

Where the answers came from on September 2

Of 114 citations across ChatGPT, Claude, and Gemini, 55% were vendors’ own sites, 11% ranking and aggregator sites, and 8% independent sources.

The most-cited sites were Braintrust’s (6 of 9 responses), followed by Arize’s, TrueFoundry’s, and Augment Code’s (4 each).

Ownership check

How the three assistants differ

April 20 to September 2: 33 responses per assistant, one run per prompt, with web search on.

*Gemini’s citation count is understated, since its links couldn’t be read on six dates.

Where they agree: Arize (named in 29 to 33 of 33 responses), Langfuse, and LangSmith appear across all three. The core of the list looks similar whichever assistant you ask; the differences show up beyond the top few names.

These are patterns from this period, from a small sample. Assistants change with each model update.