Not acquiring skills: research indicates that junior developers using AI fail to learn debugging.

      Research released this year has identified a phenomenon that employers have been contemplating for some time. Deskilling refers to the decline in ability that occurs when an expert ceases to practice. Conversely, never-skilling is when a beginner fails to gain proficiency from the outset, representing a more troubling issue since those affected are often the individuals companies are hiring less frequently.

      The most compelling evidence comes from a randomized controlled trial conducted by Anthropic researchers Judy Hanwen Shen and Alex Tamkin, published in January.

      They recruited 52 predominantly junior software engineers, provided AI assistants to half of them, and asked all participants to learn Trio, a Python library unfamiliar to them, before quizzing everyone on the concepts they had just encountered.

      The average score for the AI group was 50%, while the hand-coding group scored an average of 67%. Anthropic highlights this difference as equivalent to nearly two letter grades and noted its statistical significance with a p-value of 0.01.

      The anticipated speed advantage, which is typically why individuals seek assistance from AI, did not materialize to a meaningful extent. The AI group completed tasks about two minutes faster, a difference that was not statistically significant, partly because some participants spent as long as 11 minutes formulating their queries, consuming about a third of their allotted time.

      They learned less, did not finish more quickly, and performed worse on crucial tasks, particularly debugging—the area showing the largest gap between the groups. The control group, without an assistant, encountered errors and had to resolve them, which accurately reflects the learning process of debugging. In contrast, the AI group did not encounter any errors.

      The field of medicine has raised similar concerns from a different perspective. A Perspective piece published in Nature Medicine in May, led by Duke-NUS Medical School with co-authors from Harvard, UCL, and King’s College London, highlighted risks associated with trainees who rely on AI during their critical clinical training years without developing the reasoning skills necessary for safe, independent practice.

      This discussion introduces a third category that has received even less attention: mis-skilling, referring to trainees who accept AI-generated errors without critical assessment and internalize them as facts.

      The authors emphasize caution, noting that direct evidence from medical training is lacking. Their argument is based on learning theory and preliminary indications from non-clinical environments, such as studies like Anthropic’s.

      They propose a three-phase framework: initially build competency without AI, then teach individuals to adjust their skepticism, and finally introduce AI tools under supervision.

      The manner in which AI tools are utilized is more crucial than merely their use. In the Anthropic trial, high-performing participants asked conceptual questions or requested clarifications in addition to the code, whereas low-performing participants tended to offload responsibilities entirely or relied on the assistant to handle debugging.

      Employers are already taking these insights into account. Gartner predicts that critical-thinking decline will lead half of global organizations to mandate “AI-free” skills assessments by 2026, indicating that hiring managers are beginning to distrust portfolios.

      Ford has also been rehiring engineers to resolve errors created by its AI systems, which serves as an expensive example of what occurs when those who could have prevented mistakes are no longer employed.

      The trial does have limitations, as acknowledged by its authors. The sample size was small, the quiz assessed comprehension immediately instead of months later, and it employed a sidebar assistant rather than an autonomous coding agent. The researchers anticipate that the effects of these factors will be more pronounced, not less.

      Notably, the trial was conducted by Anthropic, which sells the assistant. They published a paper suggesting that careless use of the assistant can degrade job performance. This could be viewed as either unusual transparency or a strategic move to advocate for more effective learning methods, with both interpretations potentially being valid.

      The research does not advocate for juniors to code by hand. Instead, it highlights that the shortcut and the skill are not equivalent paths, challenging the industry's previous two-year assumption that they were.