Nvidia is establishing a team focused on AI safety, and there is a commercial motivation behind it.

      Nvidia's philosophy can be seen in a job advertisement. The chip manufacturer is forming a new team focused on AI safety and security engineering, as reported by Business Insider. A series of job postings from late last month reveals this information. Rather than expressing caution about AI's potential risks, the message is that trust drives consumer purchases.

      The team will be small but composed of experienced professionals. Nvidia is looking for a prominent engineer to serve as the "founding technical leader," along with a security research engineer, an evaluation engineer, and a senior manager. According to the listings, their responsibilities will include vetting AI agents prior to deployment and developing AI-driven tools to address software vulnerabilities.

      The argument for openness is clearly articulated in one of the listings. It describes open-weight models, transparency, and extensive scientific examination as "foundational to American AI leadership and cybersecurity defense." Open-weight models disclose the trained weights determining a model's behavior, while the training data and source code remain private. They contrast sharply with the closed systems developed by OpenAI and Anthropic.

      Nvidia has been vocal in advocating this perspective. In his first post on X last month, CEO Jensen Huang published a letter encouraging US policymakers to support open models, asserting that they "enhance safety and cybersecurity." Shortly afterward, Nvidia became a founding member of the Open Secure AI Alliance.

      This coalition includes 120 companies such as Microsoft, Palantir, and SpaceX, working together to create open-source security tools. The listings for safety positions emerged just ahead of this initiative and outline similar objectives.

      The rationale behind this philosophy is supported by financial considerations. Open models enable far more users to access AI compared to closed services, resulting in a greater demand for the silicon necessary to operate these models. This is why competitors like AMD are pursuing the same open-source opportunities. Nvidia aims to retain its advantage in this space, as increased adoption translates to higher chip sales.

      Safety is crucial for promoting this adoption. AI is evolving from simple chatbots to autonomous agents capable of accessing sensitive corporate information and taking action based on it. Therefore, the primary barrier shifts from mere capability to trust. Companies will not grant an autonomous agent access to their systems unless they feel assured of its safety.

      This is why organizations are now evaluating agents for unauthorized actions before activating them.

      The debate surrounding open models is ongoing. Critics contend that publishing weights provides valuable tools to malicious actors as easily as to defenders. Researchers have noted a safety gap, where open models may match cutting-edge capabilities but lag in terms of protective measures.

      Nvidia did not reply to Business Insider's request for comment, meaning the company's perspective is primarily conveyed through its job advertisements. It is also notable who stands to benefit if this argument succeeds. The organization advocating for the widespread adoption of safe, open AI also manufactures the hardware that supports it.