关于数学公式语音项目的调研.ppt

关于数学公式语音 项目的调研,报告人：彭云辉 2010-4-8,纲要,国外关于数学语音的一些相关的研究 这些项目表示数学公式所采用的语言/格式 相关项目的大体思路及意图 如何消除歧义 项目的优点和局限性及对项目的改进意见,主要相关项目及对应机构或人物,另外，还有一些机构、团体对数学公式语音作了深入的研究，并取得了一定的成果。如University of California at Berkeley(How can we speak math/Math Speak & Write, a Computer Program to Read and Hear Mathematical Input) 和 School of Computing, Dublin City University (Mathematics: How and What to Speak)等。,RETURN,数学表达式源语言,RETURN,an excellent prototype for speaking mathematics LaTeX to audio documents. Can speak both literary texts and highly technical documents that contain complex mathematics. the adequacy of the audio rendering depends on how well the electronic document captures the essential internal structure of the information. produced a structured representation and an audio formatting language (AFL) to provide an interactive environment for listening to and browsing technical documents. uses the Emacs(文本编辑器) front-end (Linux).,有关AsTeR( Audio System for Technical Readings ) (Raman 1994),AsTeR（续）,creates an internal representation easier,used to help the audio rendering,Mathtalk的大体思路及意图 1. a set of rules to insert prosodic cues into spoken algebraic expressions. 2. analyzed the way mathematics teachers speak mathematical expressions and integrated these natural voice inflections（音调变化）. 3. only insertion of prosodic cues (pitch, amplitude, and pauses) into computer-spoken mathematical expressions ,none insertion of lexical cues.,creates the necessary HTML/XML tags for visually-impaired and blind users to use their current screen-reading tools (e.g. JAWS and Window-Eyes for Windows) to read HTML and MathML/XML pages that contain math expressions, to read them in a spoken language.,This markup is not displayed in the browser. Only the MathML visual markup, or a PNG image, or a LiveMath Plug-In interactive image - whatever the author intended, is shown. The “MathSpeak This“ function makes it possible to hear the expression read during the creation/editing process,大体思路： 1. deriving Braille and Spoken Output from LaTeX Documents 2. render spoken mathematics from MathML using prosodic features such as pauses and speaking rate . 3. the use of prosody in synthesized speech to indicate nesting structure. 主要意图： To take a subset of LaTeX and produce both Braille and Spoken out from it. To accurately model a document and to present this to the blind user using a simple and intuitive interface. To harness the capabilities of synthetic speech devices to give more meaningful spoken output to the user.,TechRead的大体思路及意图,AudioMath的大体思路及意图 made use of its own database of prosodic rules in the generation of the spoken expressions. Available in 4 different ways: - ActiveX DLL - .NET component - CGI interface - Executable EXE,Auto-Discovery (the “brain” of the operation that recognizes or identifies elements in the document and calls the respective conversion modules ) Numerals (conversion of several types of numeric forms) Abbreviations Acronyms Network References Mathematical (MathML expressions ),6 modules for the conversion part,AudioMath设计流程图,MathPlayer,a plug-in to Microsoft Internet Explorer (IE) and Adobe Acrobat/Reader that renders MathML visually. is able to dynamically display a mathematical expression according to its font and the color set, users can choose the most suitable font or color scheme for their reading needs. For example, visually impaired readers are likely to set a large font and high color contrast.,上述公式在MathPlayer中的读法为： cap U bar sup h equals one minus exponent open minus fraction 8 cap T sup h over end fraction close,读法： equals ln open fraction n over s end close plus open fraction k sup h over k prime sup h end fraction close ln open s close minus zero point seven five plus open two l z minus z squared close fraction k sup h over q sup w end fraction.,MathSpeak Project的大体思路及意图 The project is one of the proposed methods, consisting of a group of rules to dictate mathematical contents. However it is not a standard and it is intended to serve blind people that want to transcribe their documents into Nemeth Code 18, and later on into Braille.,RETURN,如何消除歧义,两大策略：,Use of lexical indicators (a) x plus begin fraction one over x end fraction minus one (b) begin fraction xplusone over x end fraction minus one Use of prosodic indicators(pauses, modifications of pitch and tempo, rhythm and tone) (a) x plus one over x minus one (b) “ xplus one over x minus one,在消除歧义这一部分，MathPlayer没什么优势，而AudioMath在这一方面做得很不错，其余的一些相关软件也没什么出众的地方，下面着重谈谈AudioMath,AudioMath(葡萄牙语),Lexical Square root of power base a exponent two, end of power, plus power base b exponent two, end of power, end of radicand Prosodic Square root of (LP) a squared (SP) plus b squared (LP) end of radicand,AudioMath tone rules:,1- Rising tone: used when a lower hierarchical level is starting. (root of) 2- Falling and Rising tone: used to mark the smaller separating pause. (a squared) 3- Falling tone: used when level is ended. (b squared) 4- Emphatic Falling tone: used at the end of the expression that simultaneously is the higher hierarchical level (end of radicand).,LP,SP,LP,RETURN,AudioMath优点：,supports usermode options. An example ： 1.25 one point twenty five OR one point two five,Future Work of AudioMath - Complete the support for MathML Content Markup - Study in more detail mathematical prosody - Implement a proper blind tool - Add more languages - Enhancements on XHTML support - Implement SAPI, SSML support for TTS technologies,MathPlayer局限性,the use of tables and the representation of matrices and the possibility of some ambiguous readings no math formulae navigation support. gets complicated with complex math expressions no provision for any kind of user adapted preferences scheme（usermode） has ambiguous rendering in some mathematical expressions. Does not use prosody to render mathematical expressions by speech output. It generates text strings made up of the names of mathematical symbols and commas and periods to set pauses.,MathPlayer优势,allows web browsers users to copy a MathML expression and paste it in a MathML-aware program. This is particularly useful for computation, but might also be useful when used in conjunction with other software aimed at making math accessible (e.g. the LAMBDA system) or with mainstream applications used to process scientific documents (e.g. MathType or Scientific Notebook).,Changes in MathPlayer 2.2,MathPlayer 2.2 (released February 2010) is an upgrade and includes the following: Significantly improved font handling and rendering: Improved support for STIX, Cambria, and other Unicode fonts. Improvements for anti-aliased rendering. Better protection against fonts that contain errors in their tables. More characters are displayed. Improved performance when Internet Explorers zoom is not 100%. Improved compatibility with ASCIIMathML. Fixed bugs with content MathML (handling of , “Copy MathML“),Future work of MathPlayer,MathPlayers speech rules are based upon a pattern matcher/rule system. The rules are able to specify synchronization points and prosody in addition to text to speak. The rules provide a great deal of flexibility and allow users to match structures such as limits and integrals so that they are spoken in the customary manner rather than treating them as general expressions with limits and/or scripts.,Future work of MathPlayer （续）,The downside to this power is that