ChatGPT and Gemini differ on the top software in approximately one-third of the categories, and the review websites have been overlooked.

      GetIntel released its AI Software Index on Wednesday. The analysis posed approximately 80 buyer-centric questions across 126 software categories, encompassing 1,825 brands. The queries were directed toward the consumer applications of ChatGPT and Gemini instead of the developer APIs, yielding 9,978 usable responses collected on August 6.

      The key takeaway is that results are largely a toss-up. In one out of every three categories, the two chatbots identify different top choices. The chatbot a buyer decides to use determines which company is recommended.

      Examining the references made by the assistants, the most frequently cited domain in the entire study was techradar.com, with 1,114 mentions. Reddit followed in second place with 807, while Zapier was third with 418 citations. Together, G2, Capterra, and TrustRadius represented 5% of the total citations.

      This indicates a shift in the industry dynamics. Those three sites were designed to serve as the reference points for software purchasing, but a consumer tech magazine and a forum are filling that role instead. Gartner received only 238 citations, which is even fewer than Zapier.

      Reddit's second-place ranking poses a separate issue, as the platform is currently tightening its policies against brands initiating fake reviews for chatbots to copy. It claims to now identify 25,000 such posts each day. The assistants are predominantly relying on a source that is difficult to keep reliable.

      ChatGPT generates traffic, while Gemini does not.

      The two behave very differently after selecting a recommended product. When ChatGPT suggests an item, it cites the product's own website 53% of the time, compared to only 13% for Gemini, which relies more on third-party sites.

      Thus, the same recommendation holds different value; winning in ChatGPT results in a link, while a win in Gemini results in just a mention. This distinction is significant for those still evaluating publisher traffic outcomes.

      The independent sources utilized by Gemini are even more peculiar, with its most-cited third-party pages being blog entries produced by other software companies, often concerning categories unrelated to those companies.

      The response varies based on who is asking and their location.

      Identifying oneself can alter the results significantly. In some categories, a brand's winning percentage can jump from 34% to 90% simply based on whether the buyer identifies as “a freelancer” or “an enterprise.” This occurs even with the same category and question from different customers.

      Geographical factors also play a role, which is a finding relevant for European readers. The top recommendation can flip between U.S.-phrased and U.K.-phrased versions of the same question in about half of the categories evaluated. Regulators have spent two years debating whether AI answers are equitably distributed, yet no one has scrutinized whether they provide the same responses for identical questions on both sides of the Atlantic.

      Winning does not equate to visibility.

      The leaderboard data provides an unflattering view of well-known brands. HubSpot appears on the most category leaderboards, with 19, and claims victory in three. Salesforce appears on 13 and wins none, while QuickBooks shows on seven and wins five.

      Certain categories are already decided. GitHub is the top recommendation in 91% of CI/CD queries and 72% of code review queries. Shopify leads with 84% in e-commerce, and Miro holds 83% in whiteboards. Square commands 78% in point of sale and Loom 76% in screen recording.

      Other categories remain competitive. In the affiliate marketing software segment, the top choice wins only 5% of the responses. Contract management stands at 13%, and invoicing, live chat, and order fulfillment all rest at 19%. The leading answer shifts almost every time the question is rephrased.

      Who conducted this measurement and why

      The caveat lies in GetIntel's business model. Based in Bengaluru, the company offers AI visibility software. Its comprehensive index concludes that many companies are unaware if AI recognizes them, which is also its primary sales pitch. Founder Tarang Agarwal states this directly.

      Agarwal clearly articulates the issue: a company can detail its Google ranking precisely, but it may lack knowledge of whether ChatGPT has ever mentioned it. "Closing that gap is the entire purpose of an AI visibility tool."

      The market for this solution is competitive. Berlin's Peec AI recently attained a $200 million valuation by promoting the same promise. The practical effort to build citations closely resembles improving rankings on Google.

      There are real limitations to this method that need to be acknowledged. This analysis represents a single day’s worth of data from two engines, lacking error bars or peer review. “Top pick” merely signifies the first brand mentioned in a response, which is a simplistic measure of a complex answer. A single snapshot does not provide insight into whether these responses are stable.

      This leads to a key query. GetIntel plans to update the