CLEAR：一款基于临床概念构建、具备可审计特性的放射学基础大模型 CLEAR an auditable foundation model for radiology grounded in clinical concepts

NatureBiomedicalEngineering

naturebiomedicalengineering

Article

/10.1038/s41551-026-01741-4

CLEAR:anauditablefoundationmodelforradiologygroundedinclinicalconcepts

Received:26September2025

Accepted:8June2026

Check

forupdates

TianyuHan

o

1,2,3,12区,RigaWu4,12,YuTian4,FirasKhader3,LisaC.Adams

®

5,

KenoK.Bressem5,6,ChristosDavatzikos1,4,JakobNikolasKather7,8,9,10,

LiShen

11,DavidA.Mankoff2,EduardoMortaniBarbosaJr2&DanielTruhn

3

‘Blackbox’deeplearningmodelsformedicalimageinterpretationlimit

clinicaltrustandanalysisofperformancedegradation.HereweintroduceConcept-LevelEmbeddingsforAuditableRadiology(CLEAR),anauditablefoundationmodelbasedonclinicalconcepts.Trainedonover0.87millionimage–reportpairsfrom239,391patients,CLEARlearnsavisual

representationandprojectschestX-raysintoasemanticallyrichspace

definedbylargelanguagemodelembeddings,makingeveryprediction

decomposableintoweightedcontributionsfromindividualradiologicalobservations.Externalvalidationonfourlarge,physician-annotated

datasetsfromtheUnitedStates,EuropeandAsiashowsthatCLEARnot

onlyachievesstate-of-the-artclassificationperformancebutalsoenablesapplications:auditablezero-shotpathologydetection,systematic

identificationofradiologicalconfoundersandthecreationofexpert-levelconceptbottleneckmodelsfromdata-drivenconcepts.Byintegrating

clinicalknowledgedirectlyintoitsreasoningprocess,CLEARoffersa

frameworkforrobustmodelauditing,saferdeploymentandenhancedphysician–AIcollaboration,advancingtowardstrustworthymedicalAI.

Medicalimagingerrorscauseanestimated40,000to80,000preventa-bledeathsannuallyintheUnitedStates,withradiologicalmisdiagnosisaffectingupto1in10patients

1

,

2

.Diagnosticimagingaccountsforover100billiondollarinannualspending,yetinterpretationerrorsdriverepeatscans,delayedtreatmentandmalpracticecostsexceeding4bil-liondollareachyear

3

.Substantialprogressincomputer-assisteddiag-nosis,particularlymethodsleveragingend-to-endmachinelearning,hasbeenmadeacrossradiologicaltasks,includingdiseaseprogression

prediction

4

–

10

,tissuesegmentation

11

–

15

,findingsclassification

16

–

21

,sur-vivalprediction

22

,

23

,medicalreportgeneration

24

–

27

andmultimodalsituationawareness

28

–

35

.Inaddition,foundationmodelscanuselargecohortsofunlabelledtrainingimagestoaddressawiderangeofclini-caltasks,suchasdiagnosingrare(long-tailed)thoracicdiseases

21

andpredictingcancerimagingbiomarkers

36

.

However,thedecision-makingprocessesandpredictivefeaturesofthesemodelsremainlargelyuninterpretabletoevenexperienced

1CenterforAIandDataScienceforIntegratedDiagnostics(AI2D),PerelmanSchoolofMedicine,UniversityofPennsylvania,Philadelphia,PA,USA.

2DepartmentofDiagnosticRadiology,PerelmanSchoolofMedicine,UniversityofPennsylvania,Philadelphia,PA,USA.3DepartmentofDiagnosticand

InterventionalRadiology,UniversityHospitalRWTHAachen,Aachen,Germany.4ArtificialIntelligenceinBiomedicalImagingLaboratory(AIBIL),PerelmanSchoolofMedicine,UniversityofPennsylvania,Philadelphia,PA,USA.5DepartmentofDiagnosticandInterventionalRadiology,SchoolofMedicineandHealth,KlinikumrechtsderIsar,TechnicalUniversityofMunich,Munich,Germany.6DepartmentofCardiovascularRadiologyandNuclearMedicine,

SchoolofMedicineandHealth,GermanHeartCenter,TechnicalUniversityofMunich,Munich,Germany.7ElseKroenerFreseniusCenterforDigital

Health,TechnicalUniversityDresden,Dresden,Germany.8NationalCenterforTumorDiseases,HeidelbergUniversityHospital,Heidelberg,Germany.

9DepartmentofMedicalOncology,HeidelbergUniversityHospital,Heidelberg,Germany.10DepartmentofMedicineI,UniversityHospitalDresden,

Dresden,Germany.11DepartmentofBiostatistics,EpidemiologyandInformatics,PerelmanSchoolofMedicine,UniversityofPennsylvania,Philadelphia,PA,USA.12Theseauthorscontributedequally:TianyuHan,RigaWu.区e-mail:

tianyu.han@

Article

/10.1038/s41551-026-01741-4

NatureBiomedicalEngineering

368,294radiologicalobservationsextractedfrom227,835studies

•Mildcardiomegaly

•Tortuousandcalcifiedthoracicaorta

•Elongationofthedescendingaorta

•Lungsarehyperinflated

•Lowlungvolumes

•Patchyopacitiesatthelungbaseslikelyatelectasis

•Ill-definedopacitywithintheleftlowerlung

•Ill-definedpatchyopacitiesintheleftlungbase

•Smallbilateralpleurale仟usions

•Bluntingofthecostophrenicanglesbilaterally

•LeftPIClineendsintheupperSVC

•Mildpulmonaryvascularcongestion

•Noconsolidationorlargee仟usions

•Nopneumothorax

•Bronchovascularcrowding

•Atelectasisintheleftlowerlobesimulatingconsolidation

•Heartsizeisexaggeratedbylowlungvolume

•

…

aConceptspacecuration

Youareahelpfulassistant.PleaserespondinvalidJSONonly.Question:Whatarethedescriptiveobservationsinthereport?

ReportText:WETREAD:8:55PM

Batterypackoverliestheleftcostophrenicangleonthis

singlefrontalportableimage.Exactlocationdi仟icultto

assess.Wirefromgeneratorascendstoleveloftheleft

neck.Mildpulmonaryvascularcongestion.Noconsolidationorlargee仟usions.Nopneumothorax.

IMPRESSION:APchestreviewedintheabsenceofpriorchestradiographs:

Lowlungvolumesexplainbronchovascularcrowding,butmakeitdi仟iculttoexcludemildinterstitialedema.Italsois

likelyresponsibleforatelectasisintheleftlowerlobe

simulatingconsolidation.Thereisnopneumothoraxor

appreciablepleurale仟usion.Heartsizeisexaggeratedbylowlungvolume.Electrodepassesfromtheleftlow

pectoralpowerpacktotheleftparaspinalneck.

{

"observations":[

"Mildpulmonaryvascularcongestion",

"Noconsolidationorlargee仟usions",

"Nopneumothorax",

"Lowlungvolumes",

"Bronchovascularcrowding",

"Atelectasisintheleftlowerlobesimulatingconsolidation",”Heartsizeisexaggeratedbylowlungvolume",

]

}

cOverviewofCLEAR-conceptannotation

bPretrainingalgorithm

CXRRadiologicalreports

Focalconsolidationattheleftlungbase,

possiblyrepresentingaspirationor

pneumonia.

Centralvascularengorgement.

Contrastivelearning

*DINOv2pretrained

Imageencoder*

Textencoder

dOverviewofCLEAR-imageembeddinggeneration

PairwisecosinesimilaritiesObservationLLMembeddingsConcept-basedimageembeddings

Hyperdensenodularstructuresin

therightlowerlungcompatible

withcalcifiedgranulomas

Multiplemediastinalclipsnoted

anteriorly

Calcifiedgranulomas

Mediastinalclips

Imageencoder

Textencoder

CXRN

CXR1

np.dot

(I_e,T_e.T)

CXRRadiologicalobservations(forexample,text-embedding-3-small)(forexample,text-embedding-3-small)

…

…

…

Observation1

ObservationN

…

Embed.dim1

Embed.dimNEmbed.dim1

…

Embed.dimN

…

Observation1CXR1

CXRN

ObservationN

×=

Fig.1|TheCLEARframework.a,Conceptspacecuration:anLLMextracted

368,294radiologicalobservationsfromthereporttextof227,835studies,includingfindingssuchas‘mildcardiomegaly’,‘tortuousandcalcified

thoracicaorta’,‘elongationofthedescendingaorta’andvariousotherclinical

observationsrangingfromlungpathologiestocardiacfindings.b,Pretraining

algorithm:CXRandradiologicalreportsareprocessedthroughanimageencoder(DINOv2pretrained)andtextencoderconnectedviacontrastivelearningtoalignvisualandtextualrepresentations.c,OverviewofCLEAR-conceptannotation:

radiologicalobservationsaremappedtospecificimagefeatures.d,Overview

ofCLEAR-imageembeddinggeneration:CXRsandobservationsareprojected

intoasharedsemanticspacethroughthreestages:pairwisecosinesimilarities,LLM-generatedobservationembeddings(forexample,SFR-Embedding-Mistral)andfinalconcept-basedimageembeddings.Thisunifiedrepresentationenablesdirectcomparisonbetweenradiologicalimagesandclinicalconceptsthroughmatrixmultiplicationofsimilarityscoresandobservationembeddings.PIC,

peripherallyinsertedcatheter;SVC,superiorvenacava.

radiologists.Moreimportantly,theyarenotnecessarilygroundedinestablishedclinicalconceptsorbiologicalprocesses

37

–

41

.Thislackoftransparencyisnotmerelyanacademicconcern.Asaresult,theyarevulnerabletolearningspuriouscorrelationsfromconfoundersinthedata,makingtrainingviacomputationalapproachessuchasgradi-entback-propagationinherentlyunreliable

39

,

42

–

46

.Consequently,thegeneralizationofevenstate-of-the-artfoundationmodelsmustoftenbetakenonfaith,withlittletonoinsightintotheunderlyingcausesofperformancedegradation.Thisstandsinstarkcontrasttothepracticeofradiologists,whorelyonwrittenlanguagetointerpretandcommu-nicateradiologicalfindingsinreportstopatientsandclinicalteams.

Tofacilitateauditingwithinthetargetapplication,inherentlyinterpretablemodelssuchasconceptbottleneckmodels(CBMs)

47

,LaBo

48

,CEM

49

andposthocCBM

50

usedomain-specific,expert-annotatedconceptstoguidepredictions.Inbiomedicine,CBMshavebeenappliedtotaskssuchasskinlesionclassification

39

,

51

,choroidneoplasiasdiagnosis

52

,neurologicaldisorderdiagnosis

53

andmedi-calimageretrieval

54

.However,CBMshaveimportantlimitations:thechoiceofconceptsoftenreflectsthesubjectivityandconstraintsofindividualreportersandthusmaybeneithercomprehensivenoroptimalforthepredictivetask.Moreover,thesetsarenoteasilyscalableandrequiredenseconceptannotationsinthetrainingset.

CBMsgenerallydemonstratelowerperformancethanend-to-endmodelsandhavenotbeensuccessfullyappliedtozero-shotimageclas-sificationinmedicine

39

,

47

.Addressingtheselimitationsisfundamentaltobuildingtrustworthymodelsformedicalimageinterpretationthatarebothgeneralizableandapplicabletoroutineclinicalpracticeonabroadscale.

Inthisstudy,weintroduceConcept-LevelEmbeddingsforAudit-ableRadiology(CLEAR),whichleveragesthecollectiveknowledgeoftheradiologicalcommunity,asencapsulatedinpubliclyavailableradio-logicalreports(Fig.

1a

),toconstructacomprehensiveandsemanticallyrichconceptspaceforchestX-ray(CXR)interpretationusing368,294largelanguagemodel(LLM)-embeddedradiologicalobservations.CLEARwasdevelopedusingover0.87millionimage–reportpairsfromthepubliclyavailableMIMIC-CXR

55

,CheXpert-Plus

56

andReXGradi-ent

57

datasetsthroughtask-agnosticcontrastivepretraining(Fig.

1b

).WefirsttrainedaDINOv2imageencoder

58

andacorrespondingtextencoder

59

tomapvisualandtextualinformationintoasharedseman-ticspace.Atinference,theseencodersworkintandem:themodelassignsascoretoeachinputCXRforagivenradiologicalobservationbymeasuringthesimilaritybetweentheoutputsoftheimageencoderandthetextencoder,therebyindicatingthedegreetowhichtheimagerepresentsthatobservation(Fig.

1c

).Theresultingvectorofsimilarity

Article

/10.1038/s41551-026-01741-4

NatureBiomedicalEngineering

scoresisthentransferredintothesemanticallyrichembeddingspaceofastate-of-the-artLLM,suchasOpenAI’stext-embedding-3-small(2025),Qwen3-embedding-8B(2025)

60

,

61

,SFR-Embedding-Mistral(2024)

62

orBiomedBERT(2021)

63

(Fig.

1d

andExtendedDataFig.1c),throughmatrixmultiplication,wherethevectorofsimilarityscoresismultipliedbythepre-computedLLMembeddingsforallradiologi-calobservations(Fig.

1d

).Thisfinal,concept-basedembeddingtakesadvantageoftheLLM’sabilitytocapturenuancedsemanticrelation-shipswithincomplexmedicaltext,allowingpreciseinterpretationandcomparison.WedemonstrateCLEAR’ssuperiorzero-shotperfor-manceanditscapabilityforauditingzero-shotclassificationacrossawidearrayoftasks,includingclassificationofcardiac,pulmonary,boneandvascularfindings,usingfourexternalCXRbenchmarksfromtheUnitedStates,SpainandVietnam.CLEARoutperformsothervision-languagefoundationmodels,includingCheXzero

20

,Biomed-CLIP

64

andOpenAICLIP

65

,ontask-specificsupervisedprobing.WealsodemonstratethatboththestandardCBMandourCLEARmodelachieveexpert-levelpathologydetectionontheCheXpertcompetitionusinghigh-qualityconceptsdiscoveredbyourdata-drivenapproach,withoutsacrificinginterpretability.

Results

Auditablezero-shotclassificationofdiversefindings

CLEARperformszero-shotclassificationbyfirstencodingeachCXRasavectorofsimilarityscoresbetweentheimageand368,294clinicalobservations(concepts)extractedfromradiologyreports,computedviapairwisedotproducts.Thisimage-conceptscorevectorquantifiestheextenttowhichtheCXRexpresseseachclinicalconceptandissubsequentlyprojectedintoanLLMembeddingspace(suchasSales-force’sSFR-Embedding-Mistral)bymultiplyingitbythecorrespondingconceptembeddings,resultinginaconcept-basedimageembedding.Forzero-shotclassification,CLEARconstructsapairofpromptsforeachdiseaseorclinicalfinding:apositiveprompt(forexample,‘atelec-tasis’)andanegativeprompt(forexample,‘noatelectasis’).Encod-ingbothpromptsusingthesameLLMembeddingmodel

66

ensuresthatboththeimageembeddingandpromptembeddingsresideinaunifiedsemanticspace.Themodelthencomputeslogitsbymeasur-ingsimilaritiesbetweentheimageembeddingandeachpromptpair.Finally,asoftmaxfunctionappliedtothesepairedlogitsproducesaprobabilityscorethatdirectlyindicatesthelikelihoodofthespecificdiseaseorfindingbeingpresent,enablingmulti-labelclassificationwheremultiplepathologiesmaycoexist(Fig.

2a

).

WeexternallyevaluatedCLEARon3physician-annotatedCXRdatasets:VinDr-CXR(3,000images;21findings),PadChest(7,943images;55findings)andtheIndianaUniversityCXRCollection

(Indiana;7,466images;30findings).Forallevaluations,theareaunderthereceiveroperatingcharacteristiccurve(AUROC)wasusedastheprimaryperformancemetrictocompareCLEARwithstate-of-the-artvision-languagefoundationmodels,includingCheXzero,Biomed-CLIPandOpenAICLIP.DetailedperformanceresultsareprovidedinSupplementaryTables8–11,anddatasetdescriptionscanbefoundinMethods.

OntheVinDr-CXRbenchmark(Fig.

2c

),CLEARachievedanaver-agezero-shotAUROCof78.2%,outperformingthenext-bestmodel,CheXzero(meanAUROC75.0%),withstatisticallysignificantimprove-ments(P<0.01,two-sidedpairedpermutationtest)onfindingssuchas‘nodule/mass’,‘lungtumour’,‘pulmonaryfibrosis’and‘nofinding’.OntheIndianabenchmark(Fig.

2d

),CLEARoutperformedCheXzeroon10findingsandunderperformedon2(P<0.01),outperformedBiomedCLIPonall22findingsandoutperformedOpenAICLIPonall20findings.OnthePadChestbenchmark(Fig.

2b,e

),CLEARachievedameanAUROCof70.0%,outperformingCheXzero(66.8%),Biomed-CLIP(59.2%)andOpenAICLIP(52.0%).TheseresultsdemonstratethatCLEAR,withitsinherentlyinterpretabledesign,consistentlyout-performsstandardvision-languagefoundationmodelsinzero-shotCXRinterpretation.

WhenclassifyingaCXRusingzero-shottransferwithCLEAR,aconceptauditingplot(ExtendedDataFig.2)canbegeneratedtovisualizethecosinesimilaritybetweentheCXRandeachconsid-eredradiologicalobservation.WeobservedthatCLEARsuccessfullyretrievesrelevantclinicalimagesforavarietyofradiologicalterms(SupplementaryFig.1).Conceptswithhighsimilarityscoresareinter-pretedbythemodelascloselymatchingtheground-truthdiagno-sis,forexample,atelectasis(ExtendedDataFig.2a),interstitialandalveolarpattern(ExtendedDataFig.2b),orcongestiveheartfailure(ExtendedDataFig.2c).Notably,CLEARcanalsodetectsemanticallymeaningfulconceptsthatcontributetopredictionerrors,evenunderzero-shotsettings,byanalysingvisuallysimilarclustersthatgiverisetosignificantdifferencesinperformance.Forinstance,inpneumoniadetection,testperformancedifferedsignificantlybetweenVinDr-CXR(Vietnam)andPadChest(Spain),withAUROCsof0.93and0.83,respec-tively(Fig.

2f

).UsingCLEAR,weidentifiedthattheimageclusterswiththehighesterrorratewithinthePadChesttestsetwerethoselabelled‘interstitiallungpattern’and‘lingularatelectasis’(Fig.

2f,g

).Theabilityofinterstitialchangesandatelectasistomaskpneumonia-relatedcon-solidationonchestimagingisawell-knowndiagnosticchallenge.OurfindingsalignwithestablishedradiologicalprinciplesdocumentedintheFleischnerSocietyguidelines

67

,

68

,whichrecognizethatinterstitialpatternscreateoverlappingdensitiesandarchitecturaldistortionthatcanobscureunderlyingpathology.

Fig.2|Zero-shotclassificationandauditingcapabilitiesofCLEAR.

a,SchematicoftheCLEARzero-shotclassificationpipeline.CXRsareprocessedthroughanimageencodertocomputepairwisedotproductswithtext-encodedradiologicalobservationsfromabankof368,294concepts.Theresulting

similarityscoresaremultipliedbyLLM-encodedobservationembeddings

(forexample,text-embedding-3modelorSFR-Embedding-Mistralmodel)

tocreateconcept-basedimageembeddings.Classificationisperformedby

computingnormalizedsimilaritiesbetweentheimageembeddingandpositive/negativepromptpairsforeachpathology.b,Circularheatmapshowingzero-

shotAUROCperformanceonthePadChestdatasetacross55phenotypefindings.ColourintensityrepresentsAUROCvaluesfrom0(blue)to1.0(red),withCLEARoutperformingcomparisonmodels.c–e,Zero-shotperformancecomparison

acrossexternaldatasets.BoxplotsshowAUROCdistributionsforCLEAR,

CheXzero,BiomedCLIPandOpenAICLIPonVinDr-CXR(c)(3,000images;21

findings),Indiana(d)(7,466images;30findings)andPadChest(e)(7,943images;55findings).EachdatapointrepresentstheAUROCforoneofthefindingslistedabove;thecentrelinedenotesthemedian,boxboundsdenotethe25thand

75thpercentiles,andwhiskersextendto1.5×theinterquartilerange.StatisticalcomparisonsbetweenCLEARandeachbaselinewereperformedperfindingbya

two-sidedpairedpermutationtest(10,000permutations),withoutadjustmentformultiplecomparisons;exactPvaluesareshowninSupplementary

Tables8–11.f,Modelauditingrevealsperformancedisparitiesinpneumonia

detectionbetweentheVinDr-CXRandPadChesttestsets(VinDr-CXR,246

pneumonia/3,000totalimages,AUROC=0.938;PadChest,336pneumonia/7,943totalimages,AUROC=0.835).Barsshowthebootstrap-meanAUROC;error

barsdenotethe95%percentileCI(2.5thand97.5thpercentiles)over1,000non-parametricbootstrapresamplesdrawnwithreplacementfromeachtestset;

overlaidjittereddotsshowtheindividualbootstrapAUROCresamples.Statisticalsignificancewasassessedbyatwo-sidedpermutationtestonthedifferenceof

bootstrap-meanAUROCs(10,000permutations,seed=42),withoutadjustmentformultiplecomparisons;****P<10−4(nopermuteddifferencereachedthe

observedgap).CLEARidentifiesthatthePadChestclusterwiththehighesterrorratecontainsimageswithinterstitiallungdiseaseandlingularatelectasis,knownconfoundersthatcanmaskpneumonia-relatedconsolidation.g,RepresentativeCXRs(12)fromthePadChestclusterwiththehighestpneumoniadetectionerrorrate.Thetrueinterstitialpatternandtruepneumonialabelsforeachimagearerepresentedbythecolourandsymbolsintheupperleftandlowerrighttrianglesinthesmallbox,respectively.

Article

/10.1038/s41551-026-01741-4

a

b

PadChest(n=7,943)

CLEARzero-shotclassification

Cardiomegaly

Infiltrates

Hypoexpansionbasal

Hiatalhernia

Fibroticband

Calcifieddensities

Miscellaneous

Chronicchanges

Azygoslobe

Nodule

Pneumonia

Airtrapping

Groundglasspattern

Laminaratelectasis

Parenchymal

Atelectasis

Interstitialpattern

COPDsigns

Granuloma

Pulmonarymass

Consolidation

Lobaratelectasis

Calcifiedgranuloma

Cardiac

Pseudonodule

Heartinsu仟iciency

Hyperinflatedlung

Volumeloss

Hilarenlargement

Hypoexpansion

Goitre

Aorticbuttonenlargement

Pleuralthickening

Mediastiniclipomatosis

Pleurale仟usion

Mediastinal

Vascularhilarenlargement

Pleural

Apicalpleuralthickening

Calcifiedadenopathy

Costophrenicangleblunting

Superiormediastinalenlargement

Hilarcongestion

Hemidiaphragmelevation

Flatteneddiaphragm

Aorticatheromatosis

Chestwall/softtissue

Diaphragmaticeventration

Aorticelongation

Nippleshadow

Vascular

Descendentaorticelongation

Gynecomastia

Supra-aorticelongation

d

CXR

Anembeddingwithashapeof

[

368

,

294

,

1

,

536

]

Radiologicalobservationbank

(#C=368,294)

ortuousandcalcifiedhoracicaorta

ortuousandcalcifiedhoracicaorta

Radiologicalobservationbank

Tt

Tt

BullasBronchiectasis

Tubercl

CLEAR(ours)CheXzero

BiomedCLIPOpenAICLIP

AirwayOsseous(bone)

1.0AUROC0.8

0.6

0.4

0.2

0

Scoliosis

Ribfracture

ScleroticbonelesionKyphosis

Vertebralanteriorcompression

Osteopenia

Text-embedding-3model

Image

encoderPairwisedotproductI_e

0.570.020.180.240.06

T_e

Textencoder

Positiveprompt{pathology}

Negativeprompt

No{pathology}

Normalizedsimilarities

Text-embedding-3model

0.3

0.7

AUROC

AUROC

c

1.0

CLEAR

0.6

0.4

0

CheXzero

0.8

0.6

0.4

0.2

0

VinDr-CXR(n=3,000)

e

1.0

BiomedCLIP

Pneumoniadetection(n=3,000/7,943)

Indiana(n=7,466)

0.8

0.2

AUROC

1.0

0

0.8

0.6

0.4

0.2

PadChest(n=7,943)

Clusterofimageswiththehighesterrorrate

Themodelhaddi仟icultydistinguishingbetweenpneumoniaandchronicinterstitiallungdisease

characterizedbycoarsened

markingsattheleftlungbase,complicatedbypossibleleftupperlobeorlingular

atelectasis.

口OpenAICLIP

f

Modelauditing

AUROC

VinDr-CXR

PadChest

PadChest(test)

0.75

VinDr-CXR(test)

CLEARzero-shot

1.00

0.95

0.90

0.85

0.80

****

CLEAR

modelauditing

–

gClusterofPadChestimageswiththehighesterrorrate

–

–

–

–

–

–

Pneumonialabel

Interstitialpatternlabel

Article

/10.1038/s41551-026-01741-4

CLEARimprovesconcept-basedimagerepresentationsfortrainingmodels

Togainadeeperunderstandingofthecapabilitiesofourconcept-basedimageembeddings,wetrainedlinearprobesinasupervisedmannerusinglabelsfromtheMIMIC-CXRtrainingsetandevaluatedthemexter-nallyonthelabelsfromtheStanfordCheXperttestset.Concept-basedimageembeddingswerefirstcomputedbyCLEARandthenusedasfixedinputfeaturesforalogisticregressionmodelwithasigmoidactivationfunctiontopredictthepresenceorabsenceofeachlabel(Fig.

3a

).OnCheXpertbenchmarks(Fig.

3b

andSupplementaryTable13),CLEARachievedAUROCscoresof87.6%,90.5%,72.0%,88.6%,93.3%,94.5%,83.9%,89.4%and92.8%foratelectasis,consolidation,fracture,lunglesion,lungopacity,pleuraleffusion,pneumonia,pneumothoraxandsupportdevices,respectively,significantlyoutperformingthestate-of-the-artCheXzerobaseline(P<0.01).Overall,CLEARachievedanaverageAUROCof87.0%acrossalltasks,comparedwith71.8%forBiomedCLIPand67.6%forOpenAICLIP.CLEARunderperformedonlyonthe‘pleuralother’classcomparedwithBiomedCLIP(AUROC,90.8%versus91.5%,P<0.01).Thus,CLEARprovidesastrongimageencoderthatperformsbetterthanallvisualencoderstested,includingstrongcross-modalself-supervisedbaselines.Beyondclassificationperfor-mance,CLEARalsodemonstratesexcellentprobabilitycalibrationforlinearprobing,withanexpectedcalibrationerror(ECE)of0.072thatmatchedoroutperformedallbaselines(SupplementaryTable14).

Inadditiontomakingindividual-levelpredictions,linearprobesofCLEARcanbeusedtogeneratecohort-levelbarplotsthatvisualizethecontributionofeachconcepttothepredictedclasslabel.ThisisachievedbymultiplyingtheLLM-generatedconceptembeddingsbythetrainedweightsofthelogisticregressionmodel(SupplementaryFig.2).Conceptswithhighimportancescoresareinterpretedbythemodelascloselyalignedwiththetargetdiagnosis(SupplementaryFigs.3and4).Forexample,SupplementaryFig.3fshowsthattheobserva-tion‘calcifiednodulesintherightupperlobe’stronglysupportsadiagnosisoflunglesion.

However,whenexaminingtheconceptscontributingtotheclas-sificationofenlargedcardiomediastinum(EC),weobservedthatthemostinfluentialconceptswereinfactrelatedtoatelectasisratherthanmediastinalenlargement(Fig.

3e

,conceptsmarkedwithanasterisk).Thissuggeststhepresenceofaconfoundingfactor:intheMIMIC-CXRdataset,caseslabelledwithECareoftenco-labelledwithatelectasis.Theresultingshiftinconceptattributionledtoanotice-ableperformancedropwhentransitioningfromzero-shotclassifica-tion(AUROC=0.890)tolinearprobing(AUROC=0.727)forCLEARandCheXzero,asillustratedintheROCanalysis(Fig.

3c

).Tofurtherinvestigatethisissue,weconductedadataauditinganalysiscomparingpositiveandnegativecasesofECfromMIMIC-CXR

39

(Fig.

3d

).Wefoundthatthemostdistinctivephrasesdifferentiatingpositivecasesalsocentredarounddescriptionsofatelectasisandeffusion,ratherthan

mediastinalcontours.Thishighlightstheriskoflatentconfoundinginradiologydata