调整预测方法以支持情报界分析技术（英）

May2026

DAVIDSTEBBINS,CHRISTOPHERDICTUS,TEDGISTARO

AligningForecastingMethodsto

SupportIntelligenceCommunity

AnalyticTradecraft

T

hispaperoutlinesthewaysinwhichforecastingmethodologiescansup-

porttheapplicationofexistingIntelligenceCommunity(IC)analyticstan- dardsandmethodologiesasdefinedinIntelligenceCommunityDirective (ICD)203.1Crowdsourcedforecastingtoolsandtechniques(e.g.,probabi-listicreasoning,uncertaintyquantification,aggregationofdiversejudgments,andcontinuousevaluation)offeranalyticenhancementsthatoperationalizeandextendICD203,sfiveanalyticstandards(andnineanalytictradecraftstandards)within

currentICworkflows.2Theanalyticstandardsandanalytictradecraftstandards

areincludedinAppendixA.ThispaperconsidersforecastingasananalyticsupportfunctionwithintheIC,designedtostrengthentherigorandtransparencyofIC

assessmentsratherthantoprovidedirectdecision-supporttoolsforpolicymakers.

TheRANDForecastingInitiative(RFI)providesthestructuredforecastingframeworkreferencedthroughoutthispaper.RFIappliesrigorousanalyticpro-cesseswithinasecure,internallymanagedforecastingplatformthatgenerates

probabilisticassessmentsonpolicy-relevantquestions.Forecastsareproducedby

2

trainedparticipantswhosupplybothnumericalprob-

abilityestimatesandwrittenrationalesexplainingthe

reasoningandevidencesupportingtheirjudgments.Theseforecastsareaggregatedtoformcalibratedcrowdpredic-tionsandcontinuouslyevaluatedforaccuracythrough

standardscoringmethods,suchasBrierScore.Unlike

openpredictionmarketsorspeculativetradingplatforms,RFI’sapproachisclosed,structured,andresearch-driven,designedtodemonstratehowprobabilisticmethodsandanalyticvalidationcanstrengtheninstitutionallearningandsupportdecisionmakingacrossthenationalsecuritycommunity.

Althoughourforecastingworkdrawsprimarilyonunclassifiedinformationandparticipantsfromoutside

theIC,ourstructureddesignprovidesdemonstrableana-lyticvalue.Eachforecastquestionisframedtobeobjec-tivelyresolvable,supportedbymeasurableindicators,andaccompaniedbywrittenrationalesthatciteopen-sourcedataandanalyticreasoning.Thediversityofforecasters—

Abbreviations

ACH

analysisofcompetinghypotheses

IARPA

IntelligenceAdvancedResearchProjectsActivity

IC

U.S.IntelligenceCommunity

ICD

IntelligenceCommunityDirective

ICPM

IntelligenceCommunityPredictionMarket

NIPF

NationalIntelligencePrioritiesFramework

ODNI

OfficeoftheDirectorofNationalIntelligence

PIR

PriorityIntelligenceRequirement

RFI

RANDForecastingInitiative

SAT

structuredanalytictechnique

rangingfromacademicsandpolicypractitionersto

scientistsandtechnologists—enablesbroaderhypoth-

esisgenerationandhelpssurfacealternativeviewpoints

thatmaynotemergewithinsmall,specializedanalytic

teams.Resultsarecontinuouslyscoredforaccuracyand

calibration,andrepeatedempiricalstudieshaveshownthatdisciplined,crowdsourcedforecastscanoutperformtra-

ditionalexpertjudgmentonmanygeopoliticalquestions.Althoughforecastingdoesnotreplaceclassifiedreportingorexpertanalysis,itcomplementsthoseinputsbyofferingindependentlyderivedprobabilityestimatesandalternativereasoningthatimproveevaluabilityandmitigatepotentialanalytic(orcognitive)bias.

ProblemStatement

Whyforecasting?Insupportofpolicymakersdealingwithuncertainty,intelligenceanalystsperformthreeprincipalfunctions:Theyformjudgments,prepareforecasts,and

discoverinsights.Forecastsaredefinedas“judgmentsaboutthefuture.”3Akeyaimofthispaperistoshowhowbetteralignmentbetweenforecastingmethodsandstructured

analytictechniques(SATs)willenhanceanalysts’judgmentsaboutthefuture—therebyimprovingthedecision-supportadvantagethatpolicymakersexpectfromintelligence.

Making“predictions”or“knowingthefuture”havebeenaperennialcoremissionofintelligence.KnowledgeandforeknowledgehaveheldcentralrolesinintelligenceatleastsincethetimeofSunTzuinthe5thcenturyBC.4Butforecastingaccuracyfromintelligencehasbeenelu-sive.Momentoussurpriseeventsthathaveevadedintel-ligencewarninginthepasteightdecadesincludePearlHarborin1941,theCubanMissileCrisisin1962,andthe

3

ABriefNoteontheForecastingBrierScore

ABrierScoreisastandardmeasureofforecasting

accuracythatcalculatestheaveragesquareddifferencebetweenforecastedprobabilitiesandactualoutcomes.Scoresrangefrom0(perfectaccuracy)to1(totalerror),meaningthatlowerscoresindicatemore-precisefore-castsandbettercalibrationofprobabilisticjudgment.

Therefore,thecloseraforecaster’sestimatedlikelihoodistowhatactuallyhappens,thebetterthatforecaster’sscoreandthemorereliablethepredictions.Formore,seeCultivateLabs,“WhatIsaBrierScoreandHowIsitCalculated?”

September11,2001,terroristattacks(9/11).TheICwas

remindedoftheseconsequentialwarningfailuresshortlyfollowingthedisastrousNationalIntelligenceEstimateitissuedonlytwoyearsafter9/11.ThatregrettableestimatewronglyclaimedthatIraqwascovertlyracingtorebuilditsweaponsofmassdestructionprograms.Thesesystemicanalytic(andcollection)breakdownsforcedpolicymakerstodemandbetterintelligencetoimprovetheirabilitytoanticipatesignificantfuturethreatsfromabroad.

TheU.S.IC,principallytheCIA,hasrespondedwithanaggressivefocusonnewornovelmethodologiesforintel-ligenceanalysis—analytictradecraft—whosemostpromis-ingexamplesaregroupedasSATs.5Althoughthesenew

analytictechniquestookcenterstageinthemuch-neededreformoflow-performingmethodsofanalysis,lessereffortsweregiventointelligenceforecastingasapromisingcor-

rectivetowarningdeficiencies.Todate,noefforthasbeenmadetoaligntheseseparateanalyticfixes—forecastingand

SATs—tocapitalizeontheiruniquebutcomplementarystrengths.Progresshasbeengrudgingasurgencygrows.

ForICanalysts,challengesremaininhowuncertaintyisexpressedandempiricallymeasured,reflectingboth

theinherentlimitsofforesightandtheanalyticcaution

encouragedbylongexperience.Forecastingmethodologiescancomplementtheseexistingpracticesbyprovidingaddi-tionaltoolsforquantifyingconfidence,validatingjudg-

ments,andintegratingdiverseperspectivesintoassess-

ments.6Currentprocessesgenerallysucceedatstructuringinformationcollectionandanalyticevaluationprocesses,thoughoftenrelyonqualitativeestimationsorstatic

judgmentsthatcanobscureprobabilityandconfidence.7ICD203directsanalyticproductstomapspecificlanguagetocertainprobabilityranges.Forexample,ICD203directsanalyticproductstomapspecificlanguagetoprobabilityrangesbutprovideslittleguidanceonhowanalystsshoulddetermineorjustifythoseprobabilitiesinpractice.The

resultsaremoreoftensubjectivethansystematic.

Forecastingmethodologies,particularlycrowdsourcedandprobabilisticapproaches,offeranadditive,evidence-basedcomplementtothisexistingapproach.Rather

thanreplacingtraditionalanalyticmethods,forecasting

strengthensthembyintroducingcontinuouscalibration,explicitmeasurementofuncertainty,andquantitative

validationofanalyticjudgments.Integratingthesefore-

castingmethodswithotherpromisingoreffectiveanalyticapproaches,consistentwithICD203tradecraftstandards,canhelpclosegapsinevaluability,transparency,and

speed,ultimatelycreatingmore-adaptiveandempiricallygroundedintelligenceproducts.The“PotentialForecastingContributionstoAddressAnalyticTradecraftGaps”box

onthenextpagehighlightspotentialforecastingcontribu-

4

PotentialForecastingContributionstoAddressAnalyticTradecraftGaps

Quantifyinguncertainty:Traditionalanalysisoften

expressesconfidencequalitatively,reflectingthecau-

tiousjudgmentappropriatetocomplexforecasting

problems.Forecastingcomplementsthisapproachby

introducingexplicitprobabilisticmeasuresandcontinu-ouscalibrationoflikelihoods,providinganadditionaltoolforquantifyingandcommunicatinguncertainty.

Validatingjudgments:Traditionalvalidationofintel-

ligenceassessmentsoftenoccursthroughcomplex,

subjective,andirregularposthocreviewsorauditsthatdependoninstitutionalprocessesandanalystjudgment.Forecastingdiffersbyprovidingasystematicandtrans-parentmechanismforobjectiveempiricalevaluation,

usingrecognizedquantitativescoringmethods,suchasaBrierScorethatcomparespredictionsagainstactualoutcomesandenablescontinuousimprovementinana-lyticperformance.

Harnessingcollectiveinsight:Standardworkflows

mayreflectonlyexpertperspectives;crowdsourced

forecastingsystematicallyintegratesdiverseviewpointstoimprovereasonedestimates.

tionstoaddressexistingmethodologicalgaps.8Illustra-tiveapplicationsofthesecontributions,suchastheuseofprobabilisticscoringandforecastingintegrationwiththeNationalIntelligencePrioritiesFramework(NIPF),arediscussedinsubsequentsections.

Ratherthanservingasaneworseparateanalyticframework,forecastingcomplementsestablishedIC

methodsinsuchSATsasscenariodevelopmentandRed

teamanalysis.Itsstructuredquantitativeandqualitativeapproachtoassessingfutureeventsprovidesadisciplinedwaytoexpressprobabilisticjudgments,testalternative

hypotheses,andcontinuouslymeasureanalyticaccuracy.Indoingso,forecastingcanreinforcetheIC’scommit-mentstoobjectivity,transparency,andevaluability,re-inforcingthefoundationalprinciplesthatunderpinallhigh-qualityanalysisunderICD203.

StructuredAnalyticTechniquesandTheirAlignmentwith

Forecasting

BecauseSATsrepresentthefoundationofdisciplinedintel-ligenceanalysis,afullertreatmentherehelpsillustrate

howforecastingcanbestaugmentthem.WithintheIC,

SATsincludeadiversesuiteofmethods,suchasalternativefuturesanalysis,indicatorsandsignpostsofchange,Redteamanalysis,keyassumptionschecks,analysisofcompet-inghypotheses(ACH),brainstorming,devil’sadvocacy,

eventmapping,forcefieldanalysis,andwhat-ifanalysis

thatguideanalystsinexploringuncertainty,structured

reasoning,andcompetinghypotheses.Thesetechniquescanbedirectlystrengthenedbyadoptingprobabilistic

forecastingconcepts,includingexplicitquantificationofuncertainty,calibrationandfeedbackloops,andsystematicintegrationofdiversejudgments.

Tohelpvisualizehowthesemethodsalign,Table1

introducesaconceptualmatrixhighlightingrepresentativeSATsalongsidekeyforecastingtechniques.Suchamatrixfacilitatesanalyticpairingexercises,illustratingwhich

SATsmightnaturallycomplementparticularforecastingtoolsandwheremismatchesmayoccur.Thesecombina-

5

TABLE1

PotentialForecastingandSATAlignment

ForecastingTechnique

CoreSAT

PotentialAlignmentandApplication

Crowdsourcedforecasting

Structured

brainstorming

Collectiveforecasting

exercisescancomplement

brainstormingbyacceleratingideagenerationand

systematicallyaggregating

diverseperspectivesto

identifyalternativehypothesesanddriversofchange.

Probabilisticreasoning

Keyassumptionscheck

Assigningexplicitprobabilitiestoassumptionsenables

validationandcalibrationof

judgments;forecasterscan

helpexposeweakoroutdatedassumptionsandmeasure

howchangesaffectanalyticconfidence.

Uncertainty

quantification

ACH

Forecastingdatasupport

ACHbyprovidingmeasurablelikelihoodsforevidencethatconfirmsordisconfirms

eachhypothesis,enablingtransparentcomparisonacrosscompeting

explanations.

Continuous

assessment/calibration

Indicatorsanalysis

Forecasttrackingparallelsindicatormonitoring:

Continuouslyupdated

probabilitiesrevealwhethereventsaretrendingtowardorawayfromanticipated

outcomes,improvingearlywarningjudgments.

NOTE:Thistablebrieflydemonstratesindicativepairingsonly;combinationscanberefinedthroughempiricaltestingoranalytictrainingexercises.

tionscaninformfuturepilotprojectsortrainingmoduleswithintheICaimedatlinkingforecastingwithstructuredanalyticapproaches.

PhersonandHeuerhavehighlightedthatanalyticrigordependsoncultivatingdisciplinedhabitsofthought.9Astheyobserved,

AgoodanalyticthinkerwhohasmasteredthecoreSATswillinstinctively:(1)knowwhentochallengekeyassumptions;(2)consideralternativeexpla-

nationsorhypothesesforallevents;(3)lookfor

inconsistentdatathatjustifiesdiscardingweaker

hypotheses;(4)focusonkeydriversthatbestexplainwhathasoccurredorwhatisabouttohappen;and

(5)anticipatecustomers’needsandunderstandtheoverarchingcontextwithinwhichheorsheisworking.10

Thesesameprinciplesunderlieeffectiveforecasting.Skilledforecasters,muchlikeexpertanalysts,repeatedlytesttheirassumptions,explorecompetinghypotheses(e.g.,includingnullordeceptionalternatives),trackandweighdisconfirmingevidence,identifycausaldriversthatinflu-enceoutcomes,andtailortheirreasoningtothedecision

contextofendusers.Inthissense,bothSATpractitionersandprobabilisticforecastersoperatewithinanevidence-basedcycleofhypothesisgeneration,testing,andrefine-mentthatreinforcesanalyticrigorandtransparency.His-toricalexamplesunderscorewhatcanhappenwhensuchassumptionsareleftunexamined,asshowninFigure1.

6

FIGURE1

HistoricalExamplesofStrategicAssumptionsLeftUnexamined

StrategicAssumptionsThatWereNotChallenged

GiventhatU.S.superioritywouldonlyincrease,JapanmightviewafirststrikeastheonlywaytoknockAmericaoutofthewar.

1941WORLDWARII

Japanwouldavoidall-outwarbecauseitrecognizedU.S.militarysuperiority.

Chinacouldmakegoodonitsthreatstocounterso-calledU.S.aggressionagainsttheNorth.

1950sKOREANWAR

ChinawouldnotcrosstheYaluRiverinsupportoftheNorthKoreangovernment.

TheKremlincouldmiscalculateandbelieveitcouldcreateafaitaccomplithatayoungU.S.presidentwouldnotbepreparedtoreverse.

1962CUBANMISSILECRISIS

TheSovietUnionwouldnotintroduceoffensivenuclearweaponsintoCuba.

1973YOMKIPPURWAR

AcoalitionofArabstatesknewtheycouldnotwinbecausetheyhadfailedtocooperateinthepastandstilllacked

sufficientairdefensestocounterIsraeliairpower.

Asurpriseattack,evenifrepelled,couldwoundIsrael

psychologicallyandpromptinternationalcallsforceasefiresanddiplomaticnegotiations.

SOURCE:AdaptedfromCenterfortheStudyofIntelligence,ATradecraftPrimer,p.3.

TheSovietUnion—underGorbachev—mightnotbepreparedtointervenemilitarilyinEasternEuropeasithadinthepast.

1989GERMANUNIFICATION

EastGermanycouldnotunifywithWestGermanyagainstthewishesoftheSovietUnion.

AsuccessfulandsurprisenucleartestcouldboostIndiannationalistprideandsolidifypublicsupportforashaky

coalitiongovernment.

IfIraqiauthoritieshaddestroyedtheirWMDstocksand

abandonedtheirprograms,theymightrefusetofullyacknowledgethistotheUNtomaintainIraq’sregionalstatus,deterrence,andinternalregimestability.

2003IRAQ’SWEAPONSOFMASSDESTRUCTIONPROGRAMS

SaddamHusseinfailedtocooperatewithUNinspectorsbecausehewascontinuingtodevelopweaponsofmassdestruction.

1998INDIANNUCLEARTEST

ConductinganucleartestriskedinternationalcondemnationandU.S.sanctionsandwouldthreatenanewlyelected

coalitiongovernment.

7

OverviewofProbabilistic

CrowdsourcedForecastingand

AlignmentwithAnalyticTradecraft

Crowdsourcedforecastingaggregatesindependentproba-bilityestimatesfromdiverseforecastersintoasinglecollec-tivejudgment,awisdomofcrowdsapproach.11Thesemeth-odsdrawonastructured,empiricallyvalidatedforecastingdesign.Forecastersprovidenumericallikelihoodsand

qualitativerationalesthatexplainthelogicandevidencesupportingtheirjudgments.Thiscombinationofquanti-tativeandqualitativereasoningfacilitatesbothempiricalmeasurementandtransparency.Thestrengthofthiscol-lectiveprocessderivesfromthediversityofitsparticipants,whohavedifferentbackgrounds,areasofexpertise,and

analyticperspectives.12

DemographicdataonparticipantsfromtheRFIsum-

marizedinAppendixCillustratethisdiversityinpractice.TheyshowthatRFIforecastersvarywidelybyage,occupa-tionalsector,educationalbackground,andgeographic

location,factorsthattogetherhelpmitigategroupthinkandbroadentherangeofperspectivesincludedincollectivejudg-ments.Suchdiversitybroadenshypothesesandimproves

overallaccuracywithoutsacrificingthecarefulmanagementthatensurescoherenceandsharedunderstanding.13

WithintheRFI,participantsaredrawnfromacuratedpoolofvolunteersandsubject-matterexpertswhocompletestructuredonboardingandtrainingfocusedonprobabilis-ticreasoning,calibration,andethicalanalyticconduct.Allforecastersubmissions—includingprobabilityestimatesandqualitativerationales—arereviewedbyprojectanalystsusingautomatedandmanualquality-assurancecheckstodetectsuperficial,duplicated,ormachine-generatedtext.Contribu-torswhoserationalesdonotmeetcompletenessorrelevancestandardsreceivefeedbackandretrainingtomaintaindataintegrity.Thisprocess,coupledwithperiodicperformancescoringanddiversitymonitoring,ensuresthatforecasts

arebasedontransparentreasoningandverifiableevidenceratherthanunexaminedorsyntheticinputs.Together,thesesafeguardsmaketheRFI’sforecastingoutputssubstantivelydifferentfrompublicpredictionmarkets,whichmayrelyonanonymousandunverifiedsources.14

Forecastingoperatesasastructuredprocessthattrans-formscomplexquestionsaboutuncertainfutureevents

intomeasurablehypotheses.Thefollowingfourstepspres-entageneralizedviewoftheoverallforecastingprocess

(seeFigure2;thefirsttwostepsoutlinetheanalyticmodel,whilethelattertwodescribeitsdeliverables—specifically,

FIGURE2

GeneralForecastingProcessOverview

Scenario,driver,signaldevelopment

Crowdforecasts

Analysis

Strategicissueintake

SOURCE:FeaturesinformationfromRFI,homepage.

8

Generatinginsights

fromforecasts,analysts

synthesizeforecasting

data,refineorsharpenthehypothesis,andproducerationalestogenerate

actionableinsightsforpolicymakers.

thegenerationofcollectiveforecastsandthedevelopmentofactionableinsights):

1.Strategicissueintake:Policymakers,researchers,oranalystsidentifykeystrategicquestionsthatwouldbenefitfromprobabilisticassessment,whichistheexplicitestimationofhowlikelyspecificfuture

eventsoroutcomesaretooccurbasedonavailable

evidence.Thesequestionsserveasthefoundationforstructuredforecastingandensurealignmentwithdecisionmakerpriorities.

2.Scenario,driver,andsignaldevelopment:Incol-laborationwithsubject-matterexperts,broadstra-tegicissuesaretranslatedintospecificforecastingquestionssupportedbyclearlydefinedscenarios,keydrivers,andmeasurablesignalsthatcanbe

trackedovertime.ThisstagealignscloselywiththeSATsdiscussedinthesectiononSATsandtheir

alignmentwithforecasting,reinforcinghowfore-castingandestablishedanalyticmethodscanbeintegratedtoidentifythemostrelevantdriversandsignalsforprobabilisticassessment.

3.Crowdforecastsgeneration:Aftertheforecastingquestionsarelaunched,forecastersprovideprob-abilityestimatesandsupportingrationales.This

collectiveinputconstitutesthecrowdforecastandreflectsaggregatedjudgmentsdrawnfromvariedexpertiseandperspectives,whichmayincludebothhumanandmachine-assistedinputs.

4.Analysis:Generatinginsightsfromforecasts,ana-lystssynthesizeforecastingdata,refineorsharpenthehypothesis,andproducerationalestogenerateactionableinsightsforpolicymakers.Theseinsightsmaytaketheformofspecificjudgmentsaboutlikelyfutureevents,developments,ortrends(e.g.,assess-ingtheprobabilityofaconflictescalation,leader-shipchange,ortechnologicalbreakthroughwithinasettimeframeandexplainingthedriversand

signalsthatsupportthosejudgments).Theresultscanhelpclarifyemergingtrends,assessnewornon-traditionalsourcesofinformation,revealuncon-

sciousbiases,identifyearlywarningindicators,andinformdecisionmakingprocessesacrosspolicyandstrategydomains.

Forecastingcontributesthefollowinguniqueproce-duraladvantagesalignedwithICD203:

•Calibrationandfeedback:Quantitativescoring

createsobjectiveperformancemeasures,encourag-

9

ingmethodologicalrefinement.However,accuracyscoringshouldbetreatedaspartofthebroader

analytictradecraftprocessratherthanimmediateanalyticproduction.Onceforecastoutcomesareknown,scoringbecomesanafter-actionactivitytoevaluatehowwellforecastsalignedwithrealityandtostrengthenfutureanalyticperformance.

•Earlywarningcapability:Continuouslyupdatedprobabilitiesserveasquantifiableindicatorsforemergingthreatsoropportunities.

•Transparencyofreasoning:Ina“showusyour

homework”process,documentedrationalesclarifyassumptionsandlogic,strengtheninganalytic

accountability.15

•Communitylearning:Diverseforecastingcontrib-utorsfostercross-domainreasoning,improvingcol-lectiveaccuracy,thusenablinganalyticinnovation.

Institutionalizingforecastingalongsideconventionalanalyticalmethodscouldhelpapplyandoperationalize

ICD203,sanalyticframeworkwithinprobabilisticreason-ing,linkingevidence-basedanalysiswithdynamicmodel-ing.Whenaggregatedandanalyzedacrossmultiplerelatedquestions,forecastdatacanalsobereconstitutedtoaddressPriorityIntelligenceRequirements(PIRs)andotherstra-tegicintelligencetasks.Thisreaggregationprocessallowsanalyststotranslateindividualeventprobabilitiesinto

broaderinsightsaboutregionalstability,technological

trends,oradversaryintent.Linkingforecastoutputsto

PIR-levelanalysisconnectstacticalprobabilitiestostrategicunderstanding,ensuringthatprobabilisticinsightsdirectlyinformdecisionmakers,highest-priorityquestions.Scoringandcalibrationresultscanthenfeedbackintoacontinu-

ousimprovementcycle,helpinganalystsrefinemethods,

questiondesign,andtradecraftovertime.Thefollowingsubsectionsbrieflyalignforecastingmethodologieswithkeyanalyticprocesses.

ForecastingAlignmentwithIntelligenceQuestionsandAnalysis

Forecastingcouldofferadisciplined,evidence-based

approachforaddressingintelligenceprioritiesbytrans-

formingcomplexstrategicquestionsintostructured,

measurable,andtestablehypotheses.Whenincorporatedintoexistinganalyticworkflows,forecastingcomplementsestablishedICmethods,suchasscenariodevelopmentandrelatedSATs.Itintroducescontinuousfeedbackloopsthathelpanalystsbetterevaluateuncertaintyandcommunicateprobabilisticfindingsmoreclearlyinsupportofpolicy-

makerneeds.16Theprocessdescribedbelowoutlineshowforecastingcansupportintelligenceanalysisfromquestiondesignthroughafter-actionevaluationandintegration(see

Diverseforecasting

contributorsfostercross-domainreasoning,

improvingcollective

accuracy,thusenablinganalyticinnovation.

10

ExampleForecastingAlignmentwithNationalIntelligenceICPriorities

ForecastingmethodscouldcomplementtheNIPF,thesystemestablishedintheearly2000sunderthen–DirectorofCentral

IntelligenceGeorgeTenetandmanagedtodaybytheOfficeoftheDirectorofNationalIntelligence(ODNI)toaligncollectionandanalysiswithpresidentialandNationalSecurityCouncilpriorities.AlthoughNIPFcontentisclassified,itsstructureprovidesa

transparentmechanismbywhichforecastingcouldcontributetosettingandevaluatingnationalintelligencepriorities.AccurateprobabilisticforecastsmappedtoNIPFissuescouldhelpdecisionmakersidentifyemergingtrends,allocateanalyticresources,andmeasureperformanceagainstprioritytargets.a

TodemonstratehowforecastingcanaugmentestablishedICanalytictradecraft(i.e.,includingSATsandothermethodsusedacrossthecommunity)considerahypotheticalintelligencequestion:WillNorthKoreaconductanucleartestwithinthenextsixmonths?

Inthiscase,analystswouldbeginbydefiningmeasurablesignals,suchasincreasedactivityatknowntestsites,specificdiplo-maticortechnicalindicators,andrelevantregionaldevelopments.Forecastersfromdiversebackgroundswouldprovideproba-bilisticestimatesandrationales,balancinghistoricalpatternsoftesttimingwithcontemporarygeopoliticalconsiderations.

Aggregatedcrowdforecastswouldyieldacollectiveprobabilitythatinformsanalystsabouttheconfidencelevelofthispotentialdevelopment,complementingqualitativeassessmentsdrawnfromclassifiedreportingandtechnicalanalysis.Oncetheevent

resolves,theaccuracyoftheseforecastswouldbescoredusingstandardizedmetrics,suchastheBrierScore,feedinginto

anafter-actionevaluationthatenhancesfutureforecastingdesignandcalibration.Thisexampleillustrateshowincorporating

forecastingintoanalyticworkflowshelpsquantifyuncertainty,improvetheexpressionofconfidence,andstrengthenevidence-basedjudgmentssupportingstrategicwarningassessments.

•Questiondevelopment:Analystsdefinearesolvablequestion(e.g.,WillNorthKoreaconductanucleartestbeforeJuly2026?)andidentifymeasurablesignalsandindicatorstiedtotheevent.

•Forecasting:Forecastersassignprobabilitiesandrationalesthatbalancehistoricalpatternswithsuchcontextualfactorsasdiplomaticactivityordomesticpressures.

•Aggregationofprobabilisticforecasts:Individualestimatesarecombinedintoacrowdforecastthatupdatesasnewinformationemerges,revealingshiftsinconsensusanduncertaintyovertime.

•Evaluationandintegration:Aftertheforecastperiodends,resultsarescoredforaccuracyandusedtorefinequestiondesign,calibratefuturefo