A new study reveals that ChatGPT recommends women to request lower salaries.

      A recent study has revealed that large language models (LLMs) like ChatGPT consistently suggest that women request lower salaries than men, even when both have the same qualifications. This research was conducted by Ivan Yamshchikov, a professor of AI and robotics at the Technical University of Würzburg-Schweinfurt (THWS) in Germany. Yamshchikov, who is also the founder of Pleias, a French-German startup focused on ethically trained language models for regulated industries, collaborated with his team to examine five well-known LLMs, including ChatGPT.

      They provided each model with user profiles that varied only by gender while maintaining identical education, experience, and job roles. Subsequently, they requested the models to recommend a target salary for an upcoming negotiation. In one instance, OpenAI’s ChatGPT O3 model was asked for advice regarding a female job candidate, and in another, the same request was made for a male candidate. “The difference in the prompts is two letters, the difference in the ‘advice’ is $120K a year,” noted Yamshchikov.

      The salary disparity was most significant in the fields of law and medicine, followed by business administration and engineering. The models offered nearly identical salary suggestions for men and women only in the social sciences. The researchers also evaluated how the models guided individuals on career choices, goal-setting, and behavioral advice. Consistently, the LLMs produced different responses based on the user’s gender, despite identical qualifications and prompts. Importantly, the models do not acknowledge their bias.

      This issue is not new; AI has previously demonstrated a tendency to reflect and reinforce systemic bias. For example, in 2018, Amazon scrapped an internal hiring tool after finding that it systematically downgraded female applicants. Additionally, last year, a machine learning model for diagnosing women's health issues was shown to underdiagnose women and Black patients due to being trained on biased datasets primarily featuring white men.

      The researchers of the THWS study contend that merely implementing technical solutions will not resolve this issue. They emphasize the need for clear ethical standards, independent review processes, and increased transparency in the development and deployment of these models.

      As generative AI increasingly serves as a primary source for a range of topics, from mental health guidance to career planning, the urgency of addressing these concerns grows. If left unregulated, the perception of objectivity could become one of the most harmful characteristics of AI.