英国大学课件ppt新闻分析在金融中的应用 applications of news analytics in

Applications of Applications of news analytics in finance: news analytics in finance: a reviewa review Gautam Mitra Co-author Leela Mitra Summary and scopeSummary and scope In this talk we set out a structured (reading) guide to the published research outputs: Journal papers, white papers, case studies which are emerging in the domain of “news analytics” applied to finance. We aim to provide insight into the subtle interplay of information technology (including AI), the quantitative models and behavioural biases in the context of trading and investment decisions. Applications such as low frequency and high frequency trading are presented; some desirable/potential applications are discussed. OutlineOutline lIntroduction lNews data lData sources lPre analysis of data lDetermining sentiment scores lGeneral overview lDas and Chen lLo lModels and applications in summary form l(abnormal ) Returns lVolatility and risk control lDesirable industry applications lSummary and discussions News. Market Environment. Sentiment. Investment Decisions. Risk Control. IntroductionIntroduction Traders High Frequency Fund Managers Low Frequency Desktop Market Data NewsWire Data WareHouse DataMart IntroductionIntroduction R Information Systems AI, in particular, Natural Language Processing Financial Engineering/quantitative Modelling ( including behavioural finance ) IntroductionIntroduction IntroductionIntroduction Data analysis Datamart quant models Mainstream News Pre-News Web 2.0 Social Media Pre-Analysis Classifiers Sentiment Scores (Numeric) financial market data AnalysisConsolidated Datamart Updated beliefs, Ex-ante view of market environment Quant Models 1. Return Predictions 2. Fund Management / Trading Decisions 3. Volatility estimates and risk control OutlineOutline lIntroduction lNews data lData sources lPre analysis of data lDetermining sentiment scores lGeneral overview lDas and Chen lLo lModels and applications in summary form l(abnormal) Returns lVolatility and risk control lDesirable industry applications lSummary and discussions News data: Data sourcesNews data: Data sources Sources of news/informational flows (Leinweber) News: Mainstream media, reputable sources. Newswires to traders desks. Newspapers, radio and TV. Pre-News: Source data SEC reports and filings. Government agency reports. Scheduled announcements, macro economic news, industry stats, company earnings reports Social media: Blogs, websites and message boards Quality can vary significantly Barriers to entry low Human behaviour and agendas News data: Data sourcesNews data: Data sources Web based news Individual investors pay more attention than institutional investors (Das and Rieger) “Collective Intelligence” large group of people (no ulterior motives) their collective opinion may be useful. SEC does monitor message boards Far from perfect vetting of information. Financial news can be split between Scheduled news (Synchronous) Unscheduled news (Asynchronous, event driven) News data: Data sourcesNews data: Data sources Scheduled news (Synchronous) Arrives at pre scheduled times Much of pre news Structured format Often basic numerical format Typically macro economic announcements and earnings announcements News data: Data sourcesNews data: Data sources Macro economic announcements Widely used in automated trading Impact large and most liquid markets (foreign exchange, Govt. debt, futures markets) Naturally affects trading strategies. Speed and accuracy are key. technology requirements substantial Providers in this space Trade the News, Need to Know News, Market News International, Thomson Reuters, Dow Jones, Bloomberg Earnings announcements Directly influences stock prices Widely anticipated and used in trading strategies News data: Data sourcesNews data: Data sources Unscheduled news (Asynchronous, event driven) Arrives unexpectedly over time Mainstream news and social media Unstructured, qualitative, textual form Non-numeric Difficult to process quickly and quantitatively May contain information about effect and cause of an event To be applied in quant models needs to be converted to an input time series OutlineOutline lIntroduction lNews data lData sources lPre analysis of data lDetermining sentiment scores lGeneral overview lDas and Chen lLo lModels and applications in summary form l(abnormal) Returns lVolatility and risk control lDesirable industry applications lSummary and discussions News data: Pre analysis of dataNews data: Pre analysis of data Collecting, cleaning and analysing news data challenging Major newswire providers collect news from a wide range of sources e.g. Factiva database from Dow Jones, news from 400 sources Tagging Machine readable meta data Major newswire providers tag incoming news stories Reporters tag stories as they enter them to system Machine learning techniques also used to identify relevant tags (RavenPack) Unstructured stories into basic machine readable form Tags held in XML Reveals storys topic areas and other useful meta data News data: Pre analysis of dataNews data: Pre analysis of data Need to identify news which is relevant and current “Information events” distinguish stories reporting on old news from genuinely “new” news Tetlock et al. event study shows “information leakage” News data: Pre analysis of dataNews data: Pre analysis of data Need to identify news which is relevant and current Reuters give for each article Relevance scores measures by how much the article is about a particular company Novelty/uniqueness determines the repetition among articles RavenPack Distinguish stories which are events Carry first mention of a particular theme Stories which are not events are excluded To minimise number of duplicate stories News data: Pre analysis of dataNews data: Pre analysis of data Classification of news Tagged stories provide hundreds of event types Need to distinguish what types of news are relevant to our application Market may react differently to different types of news e.g. Moniz et. al. find market reacts more strongly to earnings news than strategic news Different news is available for different assets Larger companies with more liquid stock, tend to have higher news coverage News data: Pre analysis of dataNews data: Pre analysis of data Classification of news Accounting related news Earnings Announcements of earnings Restatements of Operating Results etc Trading updates Announcements of Sales/Trading Statement etc Strategic news M 1 high disagreement Das and ChenDas and Chen Relationship between sentiment indices and market variables ? Nature of sentiment index? Positive sentiment bias Fig shows histogram of normalised sentiment for a stockpositively skewed RavenPack find positive bias in classifiers more marked in bull markets Das and ChenDas and Chen Relationship between sentiment indices and market variables Sentiment and stock levels are related determining precise nature of price relationship is difficult Sentiment inversely related to disagreement Disgreement rises, sentiment falls Sentiment correlated to posting volume Discussion increases, indicates optimism about stock is rising Strong relationship between message volume and volatility (Antweiler and Frank (2004) also) Strong relationship between trading volume and volatility OutlineOutline lIntroduction lNews data lData sources lPre analysis of data lDetermining sentiment scores lGeneral overview lDas and Chen lLo lModels and applications in summary form l(abnormal) Returns lVolatility and risk control lDesirable industry applications lSummary and discussions LoLo Reuters NewsScope Event Indices (NEI) are constructed to have predictive power for returns and realised volatility integrated framework, returns and volatility used in calibrating indices News data Reuters newsalerts -quick news flashes issued when newsworthy events occur timely and relevant Tags machine readable Headlines concise, small vocabularygood for machine learning analysis LoLo The following parameters are used lList of keywords and phrases with real valued weights lA rolling “sentiment window” of size r (say 5/10 minutes) lA rolling calibration window of size R (say 90 days) l is the vector of keyword frequencies over Raw score is defined as this will tend to be high when news volume is high normalised score LoLo Normalised score At all times t in R days of calibration window record raw score news volume; Normalised score determined by comparing current raw score against raw scores where news volume equals current news volume St =0.92: 92 % of time news volume is at current level, the raw score is less than it currently is. LoLo Model calibration Determine keywords Create list of keywords by hand Tool to extract news from periods when scores are high determine whether keywords are legitimate or need adjusting Optimal weights for intraday return sentiment index regress word frequencies against intraday returns Optimal weights for intraday volatility sentiment index regress word frequencies against (deseasonalised) intraday realised volatility LoLo Model calibration Determining optimal weights more general classification problem Other techniquesmachine learningperceptron algorithm, support vector machines LoLo Index validation to establish empirical significance of indices event study analysis Event is defined when (return/volatility sentiment) index exceeds a threshold value (0.995) Remove events that follow in less than one hour of another event consider only “new” events Tests null hypothesis: Distribution of returns / deseasonalised realised volatility is the same before / after an event. Visual inspection t test for equality of means Levenes test for change in standard deviation Chi squared goodness of fit LoLo Index validation to establish empirical significance of indices event study analysis LoLo Index validation to establish empirical significance of indices event study analysis RavenPack Sentiment ScoresRavenPack Sentiment Scores Reuters NewsScope Sentiment Reuters NewsScope Sentiment EngineEngine OutlineOutline lIntroduction lNews data lData sources lPre analysis of data lDetermining sentiment scores lGeneral overview lDas and Chen lLo lModels and applications in summary form l(abnormal) Returns lVolatility and risk control lDesirable industry applications lSummary and discussions Average Stock Price Reaction to Negative News EventsAverage Stock Price Reaction to Negative News Events Source: Macquarie Quant Research May 2009 Model Lintner 1965), APT (Ross 1976) additional sources of information to market “Profits may be viewed as the economic rents which accrue to the competitive advantage of superior information, superior technology, financial innovation” (Lo ) lTetlock, Saar-Tsechansky and Mackassy (2008) Investors perception determined from their “information sets” Model diBartolomeo and Warrick 2005) Incorporating measures or observations of the market environment in risk estimation is important Applications: Risk managementApplications: Risk management The risk structure of assets may change over time Patton and Verardo find news impacts beta of stocks and in particular most of beta increase comes from rising covariance, suggesting there is contagion in information content of news releases. Applications: Risk managementApplications: Risk management Relationship between information release and volatility widely reported Ederington and Lee (1993) macro economic announcements and foreign exchange and interest rate futures Stock message board activity is a good predictor of volatility Antweiler and Frank (2004); Wysocki (1999) GARCH model with news inputs Kalev et al. (2004); Robertson, Geva and Wolff (2007) OutlineOutline lIntroduction lNews data lData sources lPre analysis of data lDetermining sentiment scores lGeneral overview lDas and Chen lLo lModels and applications in summary form l(abnormal) Returns lVolatility and risk control lDesirable industry applications lSummary and discussions Desirable Industry ApplicationsDesirable Industry Applications 1. Enhanced Strategies ( Asset Management) Low Frequency Portfolio (rebalancing) early trigger based on “draw down” rules/risk. High Frequency Trading “wish to” trade signals. Trading “have to/need to trade sell and buy” signals. News analytics market views taken into consideration for the “optimal trade execution” algorithms. VWAP, Almgre