《并行计算概述》PPT课件.ppt

1、并行计算Parallel Computing,基本概念,2020/10/9,2,如何满足不断增长的计算力需求？,用速度更快的硬件，也就是减少每一条指令所需时间 优化算法（或者优化编译） 用多个处理机(器)同时解决一个问题 并行计算,2020/10/9,3,串行计算与并行计算,2020/10/9,4,并行的层次,程序级并行 子程序级并行 语句级并行 操作级并行 微操作级并行,并行粒度,粗,细,2020/10/9,5,FLOPS,Floating point number Operations Per Second -每个时钟周期执行浮点运算的次数 理论峰值CPU主频每时钟周期执行浮点运算数CPU

2、数目 部分处理器每时钟周期执行浮点运算数：,2020/10/9,6,,2020/10/9,7,Top5002007年11月,高居榜首的依然是来自IBM的“蓝色基因/L”。自从2004年11月以来，该系统已经连续三年遥遥领先，而且计算能力不断提升，Linpack基准测试性能478.2 TFlop/s(每秒478.2万亿次运算)，而半年前还是280.6 TFlop/s 拿下亚军位置的还是IBM，不过换成了一台落成不久的“蓝色基因/P”。位于德国尤里希研究中心的这套新系统运算能力167.3 TFlop/s，不过按照IBM的设计规划，蓝色基因/P的性能将有望突破1 TFlo

3、p/s大关，即每秒一千万亿次运算。,2020/10/9,8,Top5002007年11月,第三名也是个新面孔，同时也是新墨西哥计算应用中心(NMCAC)的第一套超级计算机，由SGI基于Altix ICE 8200打造，计算能力126.9 TFlop/s。 同时印度史上首次杀入了TOP10行列，印度计算研究实验室的HP Cluster Platform 3000 BL460c以117.9 TFlop/s的性能拿到了第四位,2020/10/9,9,供应商系统数量,2020/10/9,10,供应商计算能力,2020/10/9,11,国家分布系统数量,2020/10/9,12,国家分布计算能力,202

4、0/10/9,13,体系结构系统数量,2020/10/9,14,体系结构计算能力,2020/10/9,15,应用领域系统数量,2020/10/9,16,应用领域计算能力,2020/10/9,17,操作系统系统数量,2020/10/9,18,操作系统计算能力,2020/10/9,19,处理器家族系统数量,2020/10/9,20,处理器家族计算能力,2020/10/9,21,系统数量,2020/10/9,22,计算能力,2020/10/9,23,2007年中国高性能计算机性能TOP100,2020/10/9,24,并行化方法,域分解（Domain decomposition） 任务分解（Task

5、 decomposition） 流水线（Pipelining）,2020/10/9,25,域分解,First, decide how data elements should be divided among processors Second, decide which tasks each processor should be doing Example: Vector addition,2020/10/9,26,域分解,Find the largest element of an array,2020/10/9,27,域分解,Find the largest element of an

6、 array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,28,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,29,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,30,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,31,域分解,Find the largest e

7、lement of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,32,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,33,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,34,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,35,域分解,Find t

8、he largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,36,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,37,域分解,Find the largest element of an array,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,38,任务（功能）分解,First, divide tasks among processors Second, decide which data elem

9、ents are going to be accessed (read and/or written) by which processors Example: Event-handler for GUI,2020/10/9,39,任务分解,f(),s(),r(),q(),h(),g(),2020/10/9,40,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU 1,2020/10/9,41,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU 1,2020/10/9,42,任务分解,f(),s(),r(),q(),h

10、(),g(),CPU 0,CPU 2,CPU 1,2020/10/9,43,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU 1,2020/10/9,44,任务分解,f(),s(),r(),q(),h(),g(),CPU 0,CPU 2,CPU 1,2020/10/9,45,流水线,Special kind of task decomposition “Assembly line” parallelism Example: 3D rendering in computer graphics,Rasterize,Clip,Project,Model,Inp

11、ut,Output,2020/10/9,46,Processing One Data Set (Step 1),Rasterize,Clip,Project,Model,2020/10/9,47,Processing One Data Set (Step 2),Rasterize,Clip,Project,Model,2020/10/9,48,Processing One Data Set (Step 3),Rasterize,Clip,Project,Model,2020/10/9,49,Processing One Data Set (Step 4),Rasterize,Clip,Proj

12、ect,Model,The pipeline processes 1 data set in 4 steps,2020/10/9,50,Processing Two Data Sets (Step 1),Rasterize,Clip,Project,Model,2020/10/9,51,Processing Two Data Sets (Time 2),Rasterize,Clip,Project,Model,2020/10/9,52,Processing Two Data Sets (Step 3),Rasterize,Clip,Project,Model,2020/10/9,53,Proc

13、essing Two Data Sets (Step 4),Rasterize,Clip,Project,Model,2020/10/9,54,Processing Two Data Sets (Step 5),Rasterize,Clip,Project,Model,The pipeline processes 2 data sets in 5 steps,2020/10/9,55,Pipelining Five Data Sets (Step 1),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU

14、2,CPU 3,2020/10/9,56,Pipelining Five Data Sets (Step 2),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,57,Pipelining Five Data Sets (Step 3),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,58,Pipelining Five Data Sets

15、 (Step 4),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,59,Pipelining Five Data Sets (Step 5),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,60,Pipelining Five Data Sets (Step 6),Data set 0,Data set 1,Data set 2,Data set 3,Data set 4,CPU 0,CPU 1,CPU 2,CPU 3,2020/10/9,61,Pi