《基于马氏距离的自适应高斯混合模型的设计与实现路径》10000字

基于马氏距离的自适应高斯混合模型的设计与实现路径目录1引言．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．11.1选题背景及意义．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．11.2设计基本架构．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．2马氏距离．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．2.1马氏距离概述．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．2.2马氏距离的几何意义．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．2.3公式推导．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．2.4马氏距离的优缺点．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．3EM算法．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．3.1极大似然估计．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．3.2EM算法．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．3.2.1推导．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．3.2.2EM算法收敛性证明．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．3.2.3EM算法性质．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．3.2.4EM算法应用．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．4高斯混合模型．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．4.1无监督学习．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．4.2模型结合及概述．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．．结束语参考文献附录1引言1.1选题背景及意义高斯密度函数属于参数模型，包括单高斯模型（SingleGaussianModel,SGM）和高斯混合模型（Gaussianmixturemodel，GMM）两种情况，SMG无法适应复杂的背景状态只能进行微小的渐变，当混合具有不同分布的多个样本时，单个高斯模型就没办法准确的显示样本特征，也无法清楚地分类样本。为更精确的表示样本所具有的不同统计规律，从而能更精确的描述样本的统计特性，弥补SMG这方面的缺陷，因此引出高斯混合模型（陈浩宇，杨静萱，2022）。如果有足够多的高斯模型可以融合，并且它们之间的权重设置得当这个合理的高斯混合模型可以拟合任何分布的样本，生成任何形状的非线性函数，在这种理论框架的指引下可推导出并通过多次优化迭代来抵消隐藏的变量错误，从而生成更好的参数。为祛除数据之间的相关性，利用取自高斯分布部分参数所表示的马氏距离来更好的描述具有不同统计概率的重叠率关系。无监督学习已经成为机器学习的趋势，为实现自适应选择，本文在查找大量文献，进行对比分析算法性能，选择贝叶斯相关学习方法。提出采用基于马氏距离的自适应高斯混合模型确定最优数量的高斯混合模型并生成最优自适应高斯混合模型（邱天佑，秦文轩，2023）。1.2设计基本架构本研究针对高斯混合模型拟合训练样本成分数量难以确定问题、提出一种基于马氏距离的增量高斯混合模型自适应确定成分数量区间、拟采用贝叶斯最优化准则通过百次运算法则确定最终自适应成分数量，实现高斯混合模型最优拟合给定的数据集，进行多次迭代优化，直至最优结果。最后通过对仿真数据集和实测数据集对所提算法进行性能评估。算法概述如下：（1）通过对数据的成分进行自适应分类来确定样本间距。（2）根据贝叶斯信息准则（BIC）对数据进行分类，显而易见的是并通过协方差获得不同数据之间的位置关系，并确定类别。（3）分类的数据最适合于高斯混合模型，并经过持续迭代优化。（4）通过比较模拟和实际测量数据集来评估所提出算法的性能。2马氏距离2.1马氏距离概述马氏距离（MD）由印度统计学家P.C.Mahalanobis提出，基于变量之间的相关性，通过该相关性可以识别和分析不同的模式，衡量未知样本集与已知样本集的相似性，是样本点与分布之间的距离。它表示数据的协方差距离，且在总体样本的基础上进行计算（邓嘉伟，李秀敏，2021）。也就是说如果拿同样的两个样本，从这些反应可以推断出放入两个不同的总体中，最后计算得出的两个样本间的马氏距离通常是不相同的，除非这两个总体的协方差矩阵相同。对于一个均值μ=μ1，D同样的当两个数据点时，马氏距离表示为：D其中∑是多维随机变量所组成的协方差矩阵，μ为样本均值，如果协方差矩阵是单位矩阵，也就是各维度独立同分布，马氏距离就变成了欧氏距离。本文在数据分析策略上，既运用了诸如描述性统计、回归分析等传统统计方法，也纳入了现代数据挖掘技术及其算法。像利用聚类分析识别数据内部结构，或是通过决策树进行趋势预测。这些先进技术增强了对复杂现象的理解能力，并有助于挖掘大数据中的潜在关系。同时，本文注重融合定量与定性研究方法，力求提供一个全方位的研究视角。2.2马氏距离的几何意义用主成分分析将变量进行旋转，使维度之间相互独立；进行标准化，使维度同分布。由PCA可知，主成分就是特征向量的方向，每个方向的方差即对应的特征值，故按照特征向量的方向旋转，再缩放特征值倍数就可得所求（尤智渊，吴芳菲，2021）。在多维高斯向量中，A图1.卡方分布（A）和生成的数据与拟合的A图1.卡方分布（A）和生成的数据与拟合的GMM的混合分量之间的马氏距离（B）B2.3公式推导通过上述分析结果看首先将样本点旋转至主成分，使其维度间线性无关，假定此时坐标为：Aμ（其次，变换之后维度的特征值为方差，且其维度间线性无关，则有：A−==将马氏距离进行规范化，旋转缩放之后即为欧式距离，故马氏距离的计算公式为（侯俊杰，宁晓红，2022）：D===2.4马氏距离优缺点在这个大前提下欧氏距离（Euclideandistance），是一个通常采用的距离定义，是在k维空间中两个点之间的直线距离。在二维和三维空间中的欧氏距离的就是两点之间的距离。但是在大部分的统计类问题中，坐标往往会有不同程度的波动。此时马氏距离的提出，在这样的前提之下有效的解决的这类问题（余睿德，穆俊驰，2023）。它展示了两个服从于同一分布的且协方差矩阵为∑的随机变量的差异程度。但是由于协方差矩阵的影响，马氏距离的计算并不稳定。基于当前阶段性的研究成果总结，对后续工作产生了指导意义。尤其是在研究方法上，本文识别出多个方面可以进行优化的空间。前一阶段的经验教训为本文展示了哪些做法有效，哪些则需改进或淘汰。比如，在数据采集时，本文需要更加关注样本的广泛性和代表性，确保选取的样本能够全面代表目标人群的特点。另外，面对不同研究议题时，采用多种数据搜集技术有助于增强数据的覆盖面和可信度。因为它有求逆矩阵的过程，协方差矩阵必须满秩，所以要求数据要有原维度等量的特征值。在计算马氏距离过程中，要求总体样本数大于样本的维数，否则得到的总体样本协方差矩阵逆矩阵不存在。虽然满足条件总体样数大于样本维数，但是协方差矩阵仍不存在也需要采用欧氏距离计算(邱雨昕,唐羽澄,2022)。同时，马氏距离不被量纲所影响，且与原始数据的测量单位无关，原始数据和均值的差计算出的两个样本点的马氏距离是一样的，从这能见其概此外还可以排除变量之间的相关性的干扰(许睿羽,黄泽谦,2022)。因此，本文提出采用基于马氏距离的自适应高斯混合模型。在经典欧氏距离的求解中，我们很直观可以看出其分布均为球型分布，也就是说欧氏距离只适用于理想情况下的，欧几里得空间内的直线距离；若有若干复杂模型，因互相之间的干扰构成椭球型，通过上述事实能知晓欧氏距离明显无法完美的解决。马氏距离就应运而生，在各变量符合正态分布的情况下，排除各成分之间的量纲干扰，无单位影响，扩大一些微小的量，排除离群值(孙羽航,周佳慧,2021)。在图2.中，四个点到观测中心的欧氏距离都相等，然而却并不是都属于该类。此时的蓝色点的马氏距离明显要比黄色点的马氏距离小，我们有理由判断蓝色点更有可能属于这个类（郭浩，韩雨萱，2022）。图2.马氏距离图解图2.马氏距离图解3自动模型的选择这在一定层面上表露大量的无监督学习中，聚类和降维的方法成为最基本的问题。高斯混合模型因为其包含的协方差矩阵，需要有充足的运行数据才能够保证模型参数的准确性；但是当将其协方差矩阵换为对角协方差矩阵时，则需要足够多的高斯模型才能提供较高的识别能力。在高斯混合模型中的降维方法中，在这般的环境中局部因子分析成为主流方法。通过该方法，降低协方差矩阵的自由度，提高准确性(邓泽洋,吴彤彤,2021)。数据分析期间，本文借助多种统计方法来核实数据的可靠性，并探寻潜在的异常数值。通过深入探究数据分布特性，本文精确地移除了异常数据点，同时确保核心样本信息得到保留。除此之外，本文还利用敏感性实验来测定参数波动对研究结论的稳定性及通用性的影响。为选择局部因子分析的成分数量和局部的维度时，通常借助的统计准则之一是极大似然学习。但是其计算较为复杂，在1994年提出BayesianYing-Yang(BYY)，经过不断地发展完善，成为通用的学习框架。BYYharmonylearning由BYY系统和基本的harmonylearning原则组成，与局部因子分析组合，成为一种正则化的方法，执行参数学习和自动模型选择(何炳福,周志时,2022)ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"DOI":"10.1007/11840930_27","ISBN":"3540388710","ISSN":"16113349","abstract":"AfurtherinvestigationismadeonanadaptivelocalfactoranalysisalgorithmfromBayesianYing-Yang(BYY)harmonylearning,whichmakesparameterlearningwithautomaticdeterminationofboththecomponentnumberandthefactornumberineachcomponent.Acomparativestudyhasbeenconductedonsimulateddatasetsandseveralrealproblemdatasets.ThealgorithmhasbeencomparedwithnotonlyarecentapproachcalledIncrementalMixtureofFactorAnalysers(IMoFA)butalsotheconventionaltwo-stageimplementationofmaximumlikelihood(ML)plusmodelselection,namely,usingtheEMalgorithmforparameterlearningonaseriescandidatemodels,andselectingonebestcandidatebyAIC,CAIC,andBIC.ExperimentshaveshownthatIMoFAandML-BICoutperformML-AICorML-CAICwhiletheBYYharmonylearningconsiderablyoutperformsIMoFAandML-BIC.Furthermore,thisBYYlearningalgorithmhasbeenappliedtothepopularMNISTdatabasefordigitsrecognitionwithapromisingperformance.©Springer-VerlagBerlinHeidelberg2006.","author":[{"dropping-particle":"","family":"Shi","given":"Lei","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Xu","given":"Lei","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"LectureNotesinComputerScience(includingsubseriesLectureNotesinArtificialIntelligenceandLectureNotesinBioinformatics)","id":"ITEM-1","issue":"Ml","issued":{"date-parts":[["2006"]]},"page":"260-269","title":"Localfactoranalysiswithautomaticmodelselection:Acomparativestudyanddigitsrecognitionapplication","type":"article-journal","volume":"4132LNCS"},"uris":["/documents/?uuid=d0548008-1e94-46a5-a4c6-1ae0e7632c17"]}],"mendeley":{"formattedCitation":"[2]","plainTextFormattedCitation":"[2]","previouslyFormattedCitation":"[2]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[2]。在文献ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"DOI":"10.1007/11840930_27","ISBN":"3540388710","ISSN":"16113349","abstract":"AfurtherinvestigationismadeonanadaptivelocalfactoranalysisalgorithmfromBayesianYing-Yang(BYY)harmonylearning,whichmakesparameterlearningwithautomaticdeterminationofboththecomponentnumberandthefactornumberineachcomponent.Acomparativestudyhasbeenconductedonsimulateddatasetsandseveralrealproblemdatasets.ThealgorithmhasbeencomparedwithnotonlyarecentapproachcalledIncrementalMixtureofFactorAnalysers(IMoFA)butalsotheconventionaltwo-stageimplementationofmaximumlikelihood(ML)plusmodelselection,namely,usingtheEMalgorithmforparameterlearningonaseriescandidatemodels,andselectingonebestcandidatebyAIC,CAIC,andBIC.ExperimentshaveshownthatIMoFAandML-BICoutperformML-AICorML-CAICwhiletheBYYharmonylearningconsiderablyoutperformsIMoFAandML-BIC.Furthermore,thisBYYlearningalgorithmhasbeenappliedtothepopularMNISTdatabasefordigitsrecognitionwithapromisingperformance.©Springer-VerlagBerlinHeidelberg2006.","author":[{"dropping-particle":"","family":"Shi","given":"Lei","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Xu","given":"Lei","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"LectureNotesinComputerScience(includingsubseriesLectureNotesinArtificialIntelligenceandLectureNotesinBioinformatics)","id":"ITEM-1","issue":"Ml","issued":{"date-parts":[["2006"]]},"page":"260-269","title":"Localfactoranalysiswithautomaticmodelselection:Acomparativestudyanddigitsrecognitionapplication","type":"article-journal","volume":"4132LNCS"},"uris":["/documents/?uuid=d0548008-1e94-46a5-a4c6-1ae0e7632c17"]}],"mendeley":{"formattedCitation":"[2]","plainTextFormattedCitation":"[2]","previouslyFormattedCitation":"[2]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[2]中，比较了局部因子分析的性能，测试的范围为极大似然估计与AIC，CAIC，BIC三者的分别组合，为了避免初始化导致的局部最优，分别进行十次BYYharmony算法并且对应实施EM算法(许珂茜,付明哲,2019)。进行100次模拟得到平均值。结果表明BYY-LFA在性能和计算时间上的表现是最好的。为了验证和优化理论结构，本文积累了丰富的数据材料。这些数据不仅涉及多样的研究对象，还横跨不同的历史时期和社会背景，为理论框架的全方位验证提供了重要依据。借助统计软件对数据进行解析，有助于确认理论中的各项假设是否成立，并发现其不足。后续的研究工作考虑加入更多因素或者采用更大量的样本，以提升理论框架的有效性和前瞻性。4极大似然估计极大似然估计（MLE）就是在知道结果的情况下，通过概率的计算比较，找到最有可能导致这个结果的参数值，就需要考虑到先验概率和后验概率ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"ISBN":"9781467347143","author":[{"dropping-particle":"","family":"Wang","given":"Pingbo","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Wang","given":"Yu","non-dropping-particle":"","parse-names":false,"suffix":""}],"id":"ITEM-1","issued":{"date-parts":[["2013"]]},"page":"1454-1458","publisher":"IEEE","title":"TwoIterativeAlgorithmsforMaximumLikelihoodEsitimationofGaussianMixtureParameter","type":"article-journal"},"uris":["/documents/?uuid=ebf02573-8fec-40a3-a5ce-9a2aae377728"]}],"mendeley":{"formattedCitation":"[3]","plainTextFormattedCitation":"[3]","previouslyFormattedCitation":"[3]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[3]，在此，先介绍一下较为简单的贝叶斯准则。经典的贝叶斯公式如下（林志博，何梦琪，2021）：p其中，pa为先验概率，即模型内不同类别的分布概率；pab为类条件概率;相应的pba在统计学的实际应用中，似然函数因结构复杂导致难以计算最大化时，通常会有参数估计的问题ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"ISBN":"9788578110796","ISSN":"1098-6596","PMID":"25246403","abstract":"Hasilpenelitian,sistemkomisimerupakansalahsatusistempenjualan(sistemoperasi)jasaangkutantaksiyangberlaku/diterapkandiDKIJakarta.Penelitianinimencobamengungkapduahalyangberkaitandengansistemkomisiditinjaudariaspekpengemudi,yaitu(1)bagaimanasikappengemuditaksiterhadapsistemkomisijasaangkutantaksidan(2)bagaimanatingkatkepuasanpengemuditaksiterhadapsistemkomisijasaangkutantaksi.Hasilkajianmenunjukkansikappengemuditaksibluebirdterhadapsistemkomisijasaangkutantaksibluebirdumumnyarelatifpositif,setujuataumaumenerimadengantingkatjawabandiataslimapuluhpersen.Tingkatkepuasanresponden(pengemudi)atassistemkomisiyangditerapkanrelatifrendahdibawahlimapuluhpersen,Banyakfaktoryangmempengaruhikepuasanatauketidakpuasanseorangpengemudidalammelaksanakanpekerjaandalamsistemkomisi,diantaranyadidugadipengaruhiolehtargetsetoranyangrelatiftinggidantingkatpersainganyangsemakinketatsesamaoperatortaksi.Dampaknyamenurunnyahasilpenjualanyangakhirnyadapatmengurangitingkatkesejahteraanpengemudi.Perludilakukanpenelitianlanjutanterhadapsistempenjualan(sistemkomisi)jasataksiditinjaudariaspekperusahaanatauoperatortaksi.Perusahaantaksilebihmemperhatikanaspekkesejahteraanpengemuditaksi,sepertimenurunkanbatastargetsetoransecararealistisagartingkatkepuasanpengemudimeningkat.","author":[{"dropping-particle":"","family":"Basuki","given":"Kustiadi","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"ISSN2502-3632(Online)ISSN2356-0304(Paper)JurnalOnlineInternasional&NasionalVol.7No.1,Januari–Juni2019Universitas17Agustus1945Jakarta","id":"ITEM-1","issue":"9","issued":{"date-parts":[["2019"]]},"number-ofs":"1689-1699","title":"済無NoTitleNoTitle","type":"book","volume":"53"},"uris":["/documents/?uuid=d8e5aab1-d8b5-45d5-9252-6ad320509c84"]}],"mendeley":{"formattedCitation":"[4]","plainTextFormattedCitation":"[4]","previouslyFormattedCitation":"[4]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[4]，EM算法可以有效地解决极大似然估计中更复杂的情形，是最常见的隐变量估计方法之一。这在一定范围内显示了它采用一种迭代优化的思想，可用来求解最优值，被广泛用于查找高斯概率密度函数或简要地以高似然性拟合样本测量向量的高斯分量的混合参数(邱晨曦,蒋涵瑶,2019)。EM算法的每次迭代都涉及两个步骤，我们将其称为期望步骤和最大化步，其过程之所以引人注目，部分原因是相关理论的简单性和普遍性，部分原因是因为它涵盖了广泛的示例。当基础完整数据来自易于计算其最大似然估计的指数族时，在这种理论框架的指引下可推导出同样容易计算EM算法的每个最大化步骤。然而EM算法不局限于找到概率密度函数的参数，还可用于(朱文静,高梦媛,2020)：（1）检测偏离先验已知的样本；（2）找到达到最低预测误差的特征子集；（3）使用加权最小二乘法找到加权参数。算法具体组成如图1.所示EMEM算法已知结果，寻求使该结果出现的可能性最大的条件，以此作为估计值。极大似然估计Jensen不等式图3.算法组成5EM算法期望最大化（EM）是可迭代计算最大似然（MLE）计值的一种广泛适用的方法，可用于各种不完整数据问题。最大似然估计和基于似然的推断在统计理论和数据分析中至关重要。最大似然估计是一种非常重要的通用方法。它是概率论框架中最常用的估计技术。它也与贝叶斯框架相关ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"author":[{"dropping-particle":"","family":"Geoffrey","given":"J","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Ket","given":"See","non-dropping-particle":"","parse-names":false,"suffix":""}],"id":"ITEM-1","issued":{"date-parts":[["2004"]]},"title":"www.econstor.eu","type":"article-journal"},"uris":["/documents/?uuid=950e228d-878f-4cfd-9576-0b7fa4db1d30"]}],"mendeley":{"formattedCitation":"[5]","plainTextFormattedCitation":"[5]","previouslyFormattedCitation":"[5]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[5]，显而易见的是贝叶斯解决方案在似然和最大似然估计的帮助下证明得出的，贝叶斯解决方案与惩罚似然估计ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"author":[{"dropping-particle":"","family":"Sankhyā","given":"Source","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Indian","given":"The","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Series","given":"a","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Feb","given":"No","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Url","given":"Stable","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"OrderAJournalOnTheTheoryOfOrderedSetsAndItsApplications","id":"ITEM-1","issue":"1","issued":{"date-parts":[["2012"]]},"page":"49-66","title":"IndianStatisticalInstituteConsistentEstimationoftheOrderofMixtureModelsAuthor(s):C.KeribinCONSISTENTESTIMATIONOFTHEORDEROF","type":"article-journal","volume":"62"},"uris":["/documents/?uuid=50a925de-9be2-4ef4-a819-e5b860b7be6c"]}],"mendeley":{"formattedCitation":"[6]","plainTextFormattedCitation":"[6]","previouslyFormattedCitation":"[6]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[6]，最大似然估计是一种普遍存在的技术，并且在统计学域中得到广泛使用。在处理数据时，以往研究的经验表明应加大新兴技术工具的应用力度。随着信息技术的日新月异，大数据分析、机器学习等前沿手段已成为科研不可或缺的部分。这类技术不仅提高了处理大规模数据的效率，还能揭示传统方法无法触及的深层次结构和模式。因此，后续研究需深入探讨如何将这些先进技术整合进分析流程中，以增强研究成果的准确性和深度。经典的似然函数是用数值迭代的方法求解，例如牛顿-拉夫森（NR）ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"DOI":"10.1109/ICOMSSC45026.2018.8941745","ISBN":"9781538667514","abstract":"Systemidentificationmethodandhybridmodelingmethod,usingamathematicalmodelestablishedbythesametheoreticalmethod,duetotheuncertaintyofthestructure,parameters,andenvironmentofthecontrolledobject,areaffectedbyfactorssuchasthespecificenvironment.Indifferentenvironments,thespecificstructuralparametersofthemathematicalmodelarenotexactlythesame.Therefore,thesystemidentificationmethodhasmorepracticalapplicationvaluethanthetheoreticalmodelingmethod.Inthispaper,aNewton-Raphsonevaluationsystemidentificationmethodisproposed.Theshortcomingsofthepreviousidentificationsystemcanbesatisfactorilyidentified,andsimulationexperimentsandphysicalexperimentscanbecarriedout.Thefeasibilityandeffectivenessoftheproposedidentificationandmodelingmethodareverified.","author":[{"dropping-particle":"","family":"Zhang","given":"Wanjun","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Zhang","given":"Feng","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Zhang","given":"Jingxuan","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Zhang","given":"Jingyi","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Zhang","given":"Jingyan","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"Proceedingsof2018InternationalComputers,SignalsandSystemsConference,ICOMSSC2018","id":"ITEM-1","issue":"1","issued":{"date-parts":[["2018"]]},"page":"737-742","publisher":"IEEE","title":"StudyonSystemRecognitionMethodforNewton-RaphsonIterations","type":"article-journal"},"uris":["/documents/?uuid=277a3eaf-b9a1-474c-b8f9-76597eb5fb78"]}],"mendeley":{"formattedCitation":"[7]","plainTextFormattedCitation":"[7]","previouslyFormattedCitation":"[7]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[7]及其系列算法，例如Fisher评分方法(郑思远，谢雨欣,2019)：从这些反应可以推断出在对样本做出合理假设和足够准确的初始值的情况下，由NR算法生成相应的离散序列具有局部二次收敛的特性，这也是该算法的主要优势。但是在分析和计算上，就显得非常局限ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"DOI":"10.1111/j.2517-6161.1977.tb01600.x","abstract":"Abroadlyapplicablealgorithmforcomputingmaximumlikelihoodestimatesfromincompletedataispresentedatvariouslevelsofgenerality.Theoryshowingthemonotonebehaviourofthelikelihoodandconvergenceofthealgorithmisderived.Manyexamplesaresketched,includingmissingvaluesituations,applicationstogrouped,censoredortruncateddata,finitemixturemodels,variancecomponentestimation,hyperparameterestimation,iterativelyreweightedleastsquaresandfactoranalysis.","author":[{"dropping-particle":"","family":"Dempster","given":"A.P.","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Laird","given":"N.M.","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Rubin","given":"D.B.","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"JournaloftheRoyalStatisticalSociety:SeriesB(Methodological)","id":"ITEM-1","issue":"1","issued":{"date-parts":[["1977"]]},"page":"1-22","title":"MaximumLikelihoodfromIncompleteDataViatheEMAlgorithm","type":"article-journal","volume":"39"},"uris":["/documents/?uuid=b135b1c9-b1c3-4072-aa15-e323fde49701"]}],"mendeley":{"formattedCitation":"[8]","plainTextFormattedCitation":"[8]","previouslyFormattedCitation":"[8]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[8]，此时EM算法提供了一种更加完善的替代方法，成为主流的用于迭代似然函数的方法，用于解决数据丢失和信息不完整的各种问题(张志杰,陈雨婷,2022)。5.1推导我们令X=xi，（i=1，2，…，N）作为观测数据，即X是一组随机变量且属于任意的样本空间；θ为g（XL其中fx|θ为x的pdf，最终的目的是找到最优参数θ，记作θ∗，让Lμ，θ我们引入隐变量pa(xi)L式中fqxi|θq的先验概率。与K-means有很大的不同，两者之间最明显的区别在于在K-means中每个样本都被以0或1的概率进行分类(冯天宇,张紫怡,2020)；在这个大前提下而在EM算法中是在0到1之间的概率值。在特殊情况下，密度函数fqp式中||Sq||是矩阵的行列式；D是Xp5.2EM算法收敛性在Dempster等人对EM算法进行常规描述之前，有学者在深入研究了该算法一般性的几个收敛问题之后提出，当完整数据来自具有紧凑参数空间的弯曲指数族，并且当Q函数满足某个适度的微分条件时，则任何EM序列都收敛到似然函数的固定点（不一定是最大值）(曾俊杰,韩璇莹,2018)ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"DOI":"10.1109/TSP.2008.917350","ISSN":"1053587X","abstract":"Inthispaper,theexpectation-maximization(EM)algorithmforGaussianmixturemodelingisimprovedviathreestatisticaltests.ThefirsttestisamultivariatenormalitycriterionbasedontheMahalanobisdistanceofasamplemeasurementvectorfromacertainGaussiancomponentcenter.Thefirsttestisusedinordertoderiveadecisionwhethertosplitacomponentintoanothertwoornot.ThesecondtestisacentraltendencycriterionbasedontheobservationthatmultivariatekurtosisbecomeslargeifthecomponenttobesplitisamixtureoftwoormoreunderlyingGaussiansourceswithcommoncenters.Ifthecommoncenterhypothesisistrue,thecomponentissplitintotwonewcomponentsandtheircentersareinitializedbythecenterofthe(old)componentcandidateforsplitting.Otherwise,thesplittingisaccomplishedbyadiscriminantderivedbythethirdtest.Thistestisbasedonmarginalcumulativedistributionfunctions.ExperimentalresultsarepresentedagainstsevenotherEMvariantsbothonartificiallygenerateddata-setsandrealones.TheexperimentalresultsdemonstratethattheproposedEMvarianthasanincreasedcapabilitytofindtheunderlyingmodel,whilemaintainingalowexecutiontime.©2008IEEE.","author":[{"dropping-particle":"","family":"Ververidis","given":"Dimitrios","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Kotropoulos","given":"Constantine","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"IEEETransactionsonSignalProcessing","id":"ITEM-1","issue":"7I","issued":{"date-parts":[["2008"]]},"page":"2797-2811","title":"GaussianmixturemodelingbyexploitingtheMahalanobisdistance","type":"article-journal","volume":"56"},"uris":["/documents/?uuid=a7bb29e6-30ee-493a-8160-67bcfae0b7e5"]}],"mendeley":{"formattedCitation":"[9]","plainTextFormattedCitation":"[9]","previouslyFormattedCitation":"[9]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[9]。研究所得出的初期结果与先前的计算数据和文献回顾结果大体一致，这表明了本研究在方法上的可靠性和有效性。一致性不仅重申了早期研究的结论，也给现有的理论框架带来了新的证据。通过精心设计的研究方案、详尽的数据收集及严格的分析程序，本文能够复制前人的关键发现，并在此基础上进行深化研究。这不仅提高了对研究假设的信心，也体现了所选研究方法的科学依据。此外，这种一致性为跨学科研究间的对比分析提供了依据，有助于建立更加综合和统一的理论体系。为解决极大似然估计中的寻找最佳参数θ，需用到EM算法来解决这个复杂情况。在面临仅知道结果却不知道是哪个概率分布实现的问题，我们需要根据现有的观测数据先行估计一个大概的值，再根据这个估计值来推敲未观测数据x的条件概率的期望，最终由求到的期望根据极大似然估计获得最优值θ∗。但是仅仅由这一次的估计所得到的最优，并不是最终的结果，在这样的前提之下再不断地重复进行该操作，直到最终收敛时，得到的结果才是我们需要的值。若想在不断优化之后得到值，则算法一定趋于某个值，也就是说，其结果一定是收敛的。证明如下(孔鹏飞,谢茹洁,2022)：用极大似然函数递推得到某模型的参数，公式表示如下：L模型之中含有未知的隐藏变量，假设为Z，则：L从这能见其概式中的求和实际上就是求Z的期望，假定它的概率为yi(Z),取值的分布为g（Z）L==由简森不等式（见图4.）可以推出，凸函数f(E(L图4.简森不等式图解很容易得出Lθ有下届，则每次的迭代优化过程中，提供一个下届J(Z,θ)，不断地提高图4.简森不等式图解在收敛性推导过程中，引入了一个新的变量yi当简森不等式相等时，pxi则(王文涛,王雪霏,2020)y==p(Z|5.32EM算法的初始值当初始值选择出现偏差时，EM算法的收敛速度会降低ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"DOI":"10.1109/TSP.2008.917350","ISSN":"1053587X","abstract":"Inthispaper,theexpectation-maximization(EM)algorithmforGaussianmixturemodelingisimprovedviathreestatisticaltests.ThefirsttestisamultivariatenormalitycriterionbasedontheMahalanobisdistanceofasamplemeasurementvectorfromacertainGaussiancomponentcenter.Thefirsttestisusedinordertoderiveadecisionwhethertosplitacomponentintoanothertwoornot.ThesecondtestisacentraltendencycriterionbasedontheobservationthatmultivariatekurtosisbecomeslargeifthecomponenttobesplitisamixtureoftwoormoreunderlyingGaussiansourceswithcommoncenters.Ifthecommoncenterhypothesisistrue,thecomponentissplitintotwonewcomponentsandtheircentersareinitializedbythecenterofthe(old)componentcandidateforsplitting.Otherwise,thesplittingisaccomplishedbyadiscriminantderivedbythethirdtest.Thistestisbasedonmarginalcumulativedistributionfunctions.ExperimentalresultsarepresentedagainstsevenotherEMvariantsbothonartificiallygenerateddata-setsandrealones.TheexperimentalresultsdemonstratethattheproposedEMvarianthasanincreasedcapabilitytofindtheunderlyingmodel,whilemaintainingalowexecutiontime.©2008IEEE.","author":[{"dropping-particle":"","family":"Ververidis","given":"Dimitrios","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Kotropoulos","given":"Constantine","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"IEEETransactionsonSignalProcessing","id":"ITEM-1","issue":"7I","issued":{"date-parts":[["2008"]]},"page":"2797-2811","title":"GaussianmixturemodelingbyexploitingtheMahalanobisdistance","type":"article-journal","volume":"56"},"uris":["/documents/?uuid=a7bb29e6-30ee-493a-8160-67bcfae0b7e5"]}],"mendeley":{"formattedCitation":"[9]","plainTextFormattedCitation":"[9]","previouslyFormattedCitation":"[9]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[9]。通过上述事实能知晓但是存在这种可能性，在参数空间的边缘不受限制的情况下，如果选择初始值太接近边界，则由EM算法生成的估计序列可能会发散。另外对于似然方程具有对应于局部最大值的多个根的应用，在对所有局部最大值检索时，这在一定层面上表露应从广泛的初始值选择中应用EM算法。其一种变体使用间隔分析方法ADDINCSL_CITATION{"citationItems":[{"id":"ITEM-1","itemData":{"DOI":"10.1109/ISCAS.2010.5537044","ISBN":"9781424453085","abstract":"ThisworkproposesalowcomplexitycomputationofEMalgorithmforGaussianmixturemodel(GMM)andacceleratestheparameterestimation.Inpreviousworks,theauthorsrevealedthatthecomputationalcomplexityofGMM-basedclassificationcanbereducedbyusinganintervalcalculationtechnique[1],[2].ThisworkappliestheideatoEMalgorithmforGMMparameterestimation.Fromexperiments,itisconfirmedthatthecomputationalspeedoftheproposalachievesmorethantwicethatofthestandardmethodwith'exp()'function.Therelativeerrorsarelessthan0.6%and0.053%whenthenumberofbitsfortableaddressingare4and8,respectively.©2010IEEE.","author":[{"dropping-particle":"","family":"Watanabe","given":"Hidenori","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Muramatsu","given":"Shogo","non-dropping-particle":"","parse-names":false,"suffix":""},{"dropping-particle":"","family":"Kikuchi","given":"Hisakazu","non-dropping-particle":"","parse-names":false,"suffix":""}],"container-title":"ISCAS2010-2010IEEEInternationalSymposiumonCircuitsandSystems:Nano-BioCircuitFabricsandSystems","id":"ITEM-1","issue":"8","issued":{"date-parts":[["2010"]]},"page":"2686-2689","title":"IntervalcalculationofEMalgorithmforGMMparameterestimation","type":"article-journal"},"uris":["/documents/?uuid=efc0cd48-a65f-4b7e-a496-0359ea57137d"]}],"mendeley":{"formattedCitation":"[10]","plainTextFormattedCitation":"[10]","previouslyFormattedCitation":"[10]"},"properties":{"noteIndex":0},"schema":"/citation-style-language/schema/raw/master/csl-citation.json"}[10]在参数空间的任何指定区域内找到对数似然函数的多个固定点。本文在这篇文章中构建了一个新的框架模型，该模型在信息流通及数据分析技术上均反映了对以往学术成就的认可和继承，并在其之上实现了突破和进步。尤其是在信息流通的设计上，采用了传统的信息管理原则，确保信息从获取到分析的每一个阶段都准确无误。借助严格的数据甄选机制和规范化的处理流程，提升了信息品质，同时也强调了信息流的公开透明和可追溯性。在指定混合模型的初始值时，有以下方法(徐星宇,李若彤,2020)：在g个成分的混合模型的独立数据，E-step的作用在更新该成分的后验概率，可通过以下方法找到第一个E-step：指定τj(0)（j=1，…,n）来执行第一步，τZ将整体分为x个组件，比如在g=1的正态分量和对应的协方差矩阵混合的情况下，采用初步划分数据的方法，在这般的环境中提供两个p变量。对于高维