Never-skilling: studies indicate that junior developers who rely on AI do not acquire debugging skills.

      Recent research published this year has articulated a concept that employers have been contemplating for some time. Deskilling occurs when an expert ceases to practice and consequently declines in skill. In contrast, never-skilling refers to a novice who fails to improve at all, presenting a more challenging issue, as this affects individuals that companies are already hiring less frequently.

      Strong evidence for this comes from a randomized controlled trial conducted by Anthropic researchers Judy Hanwen Shen and Alex Tamkin, published in January. They enrolled 52 predominantly junior software engineers, provided half of them with an AI assistant, and instructed all participants to learn Trio, a Python library new to them. Afterward, they quizzed everyone on concepts they had just encountered.

      The group with the AI assistant scored an average of 50%, while the hand-coding group achieved an average of 67%. Anthropic noted that this discrepancy corresponds to almost two letter grades and is statistically significant, with a p-value of 0.01. The anticipated speed advantage from using the assistant did not emerge as expected; the AI group completed their tasks about two minutes faster, a difference that lacked significance since some participants spent as long as 11 minutes crafting their queries, about one-third of their allotted time.

      They learned less, did not finish any quicker, and performed worse in the crucial area of debugging, where the disparity between the groups was most apparent. The control group, which did not use an assistant, encountered errors that they resolved, accurately reflecting how debugging skills are developed. Conversely, the AI group did not encounter as many errors.

      Similarly, concerns in medicine arise from a different perspective. A Nature Medicine Perspective published in May, led by Duke-NUS Medical School with collaborators from Harvard, UCL, and King’s College London, highlighted the risk for trainees relying on AI during their early clinical training, which may hinder the development of the reasoning necessary for safe, independent practice. This introduces a third category, mis-skilling, referring to trainees who uncritically accept AI errors and take them as fact.

      The authors stress that there is no direct evidence from medical training on this issue, focusing instead on learning theory and preliminary signals from non-clinical settings, such as studies like that conducted by Anthropic. They recommend a three-phase framework: first, develop competence without AI; second, teach individuals to calibrate their skepticism; and third, introduce tools under supervision.

      The manner in which the tool is utilized is more important than mere usage. In the Anthropic trial, high scorers posed conceptual questions or sought explanations in addition to using code, whereas low scorers relied heavily on the assistant or expected it to handle debugging for them.

      Employers are already adapting to this reality. Gartner forecasts that critical-thinking decline will lead half of global organizations to require “AI-free” skills assessments by 2026, indicating a growing mistrust among hiring managers of portfolios. Ford, for example, is currently rehiring engineers to rectify mistakes made by its AI systems, illustrating the consequences of losing personnel capable of catching such errors.

      The trial does have certain limitations, as acknowledged by its authors. The sample size was small, the quiz assessed understanding immediately rather than after a longer period, and it utilized a sidebar assistant instead of an autonomous coder. The researchers anticipate that the effects in a real-world scenario may be more significant than what was observed.

      It is noteworthy who conducted the study; Anthropic sells the assistant and has published research suggesting that careless use of the assistant can worsen job performance. This could be seen as either an unusual level of transparency or the beginning of a pitch for alternative learning methods, and both interpretations could be valid.

      The findings do not specifically advocate for juniors to code by hand; rather, they indicate that shortcuts and skill development do not equate, and the industry has spent the last two years treating them as though they were the same path.