A London dermatologist finds a 2015 AI diagnoses skin lesions better than GPT-6 and Claude

Two hundred skin lesions, all analyzed in a lab after biopsy. Three artificial intelligences to identify them: OpenAI's GPT-6 Astra, Anthropic's Claude Fable 5.1, and Inception v3, a neural network specialized in images whose architecture was published in 2015. The oldest one wins.

The test was published on October 25 on his blog by Magnus Lynch, a dermatologist and surgeon at Guy's Hospital in London and honorary academic at King's College London. It is not a peer-reviewed study. The old network, trained on 68,011 images from a public archive, gives the correct diagnosis on the first try for 125 of 200 lesions (62%). GPT-6 Astra and Claude Fable 5.1 top out at 84 and 85 (42%).

How the test was run

The 200 lesions were drawn at random from MILK10k, a public database that provides, for each one, a regular photo and a photo taken with a dermatoscope, the dermatologist's magnifying tool. Half of them needed to be seen by a doctor: melanomas, carcinomas, actinic keratoses (precancerous lesions). The other half were benign: moles and seborrheic keratoses. The two large models received the images and a prompt, once per case.

The specialized network is above all much better at recognizing benign lesions: 37 moles out of 50, versus 33 for Claude and 16 for GPT-6, and 26 seborrheic keratoses out of 50, versus 13 and 6. The two large models do slightly better on some precancerous lesions. Of the 30 melanomas in the sample, Inception v3 names 13 on the first try, Claude 11 and GPT-6 8.

More accurate, but not necessarily more cautious

The picture reverses when you look at the question that matters to a patient: should I see a doctor? The large models send almost everyone to the doctor. They flag 97 and 96 of the 100 lesions that need monitoring, but reassure only 32 and 22 of 100 patients with a benign lesion. With its default tuning threshold, the specialized network reassures 76 of 100 benign patients. But it lets 18 of 30 melanomas go, versus 3 for GPT-6 and 4 for Claude.

In other words, the old network recognizes lesions better, but as tuned it would send too many cancers home, while the large models avoid that risk by sending everyone to be seen. With an adjusted threshold, however, the author notes it would flag 98 or 99 of 100 lesions needing monitoring and let only 2 or 1 melanoma go.

The limits the author acknowledges

• The sample is small: with 30 melanomas, the uncertainty is wide.
• Biopsied lesions are harder than average. On ordinary images, the specialized network scores better than in this test.
• Because the database is public, the large models may have seen it during training.
• Inception v3 serves as a point of comparison, not a current benchmark: more recent specialized networks exist.

The author also states that he designed the test and decided the clinical cases himself, but had the analysis and first drafts written with Claude Code, Anthropic's coding tool. His conclusion: for recognizing skin lesions, a small, old, specialized network clearly outperforms two frontier models. He also points out that regulated products, built on more recent networks, do significantly better.