ChatGPT and Gemini have differing opinions on the top software in one third of the categories, and the review sites have not been included.

      GetIntel released its AI Software Index on Wednesday. The research posed approximately 80 buyer-phrased questions across 126 software categories, involving 1,825 brands. The inquiries were directed at the consumer apps of ChatGPT and Gemini rather than their developer APIs, resulting in 9,978 usable responses collected on August 6.

      The main takeaway reveals a scenario akin to a coin flip. In one out of every three categories, the two assistants identified different top choices. Which chatbot a buyer chooses to engage with determines the recommended company.

      Examining the sources cited, techradar.com emerged as the most frequently referenced domain throughout the study, with 1,114 mentions. Reddit ranked second with 807 mentions, followed by Zapier with 418. Combined, G2, Capterra, and TrustRadius only represented 5% of total citations.

      This indicates an industry undergoing significant shifts. Those three sites were intended to serve as the reference point for software purchasing. Instead, a consumer technology magazine and a forum have taken on that role. Gartner received 238 citations, which is fewer than Zapier.

      Reddit's second-place ranking poses its own challenges. The platform has begun to crack down on brands that generate fake reviews for chatbots to replicate, claiming it now detects 25,000 such posts daily. The assistants are relying most heavily on a source that is difficult to keep credible.

      ChatGPT generates traffic, whereas Gemini does not.

      The two behave distinctly after selecting a recommended product. ChatGPT cites the product's official website 53% of the time, while Gemini does this only 13% of the time, opting for third-party websites more frequently.

      Thus, the same recommendation carries different weight. Winning on ChatGPT provides a link, whereas success on Gemini results only in a mention. This distinction is crucial for those who continue to evaluate publisher traffic as the end result.

      The independent sources cited by Gemini are even stranger. Its most frequently referenced third-party pages consist of blog posts created by other software companies, often relating to categories unrelated to the publishing company.

      The response varies based on the identity of the inquirer and their location. Identifying oneself influences the outcome significantly. In some categories, a brand’s win rate can shift from 34% to 90%, solely based on whether the buyer identifies as "a freelancer" or "an enterprise." This variation occurs despite the same category and question.

      Geography plays a role as well, particularly for European readers. The top recommendation changes between US-phrased and UK-phrased versions of identical questions, happening in about half of the assessable categories. Regulators have debated fairly opting out of AI responses for two years, yet no one has questioned whether the same query receives consistent answers on either side of the Atlantic.

      Recognition and achievement are not synonymous.

      The leaderboard data is not favorable to the well-known names. HubSpot appears on 19 category leaderboards, leading three, while Salesforce appears on 13 with none. QuickBooks features on seven and wins five.

      Some categories are already determined. GitHub is the top recommendation in 91% of CI/CD responses and 72% of code review responses. Shopify claims 84% of e-commerce, while Miro captures 83% of whiteboards. Square leads with 78% in point-of-sale, and Loom with 76% in screen recording.

      Conversely, others remain competitive. In affiliate marketing software, the top choice garners only 5% of responses. Contract management sits at 13%, while invoicing, live chat, and order fulfillment are each at 19%. The leading option often varies with each rephrased question.

      Who conducted this measurement, and what is the purpose?

      The caveat lies in the underlying business model. GetIntel is a Bengaluru-based company selling AI visibility software. Its comprehensive index concludes that companies are often unaware if AI mentions them, which is also the crux of its sales pitch. Founder Tarang Agarwal states this almost directly.

      Agarwal describes the issue clearly. A company can accurately quote its Google ranking to the decimal but may not know if ChatGPT has ever mentioned it. "Closing that gap is the primary reason for the existence of an AI visibility tool."

      This is a crowded market. Peec AI from Berlin achieved a $200 million valuation by promising similar insights. The actual effort required to gain citations still resembles the process of achieving Google rankings.

      The method has real limitations that should be acknowledged. The data represents a single day’s worth from two engines, lacking error margins and peer review. "Top pick" merely refers to the first brand mentioned in a response, which is a simplistic measure of a more nuanced reply. A snapshot alone cannot determine if the findings are consistent.

      This raises a necessary test. GetIntel states it will update the Index quarterly. If the rate of disagreement remains around a third in November, it may indicate that buyers are indeed receiving different