molehill介绍翻译.docx

MoleHill 3D API介绍Actionscript 3 Experiments and Flash Player love ;)发掘更多关于MoleHill的信息- Thibault Imbert几个月前，在洛杉矶我们介绍过MoleHill API在手机和桌面上运行的情况。更多的信息点击“Molehill” page，访Adobe Labs。我要给你们更多关于molehill的详细信息，一些actionscript开发者角度的技术细节。那么，我们开始吧。什么是MoleHill？“MoleHill”是一组GPU加速的3D API的研发代码，即将表现于Adobe Flash Player 和Adobe AIR 中的actionscript 3.0。这将为Adobe Flash Player提供高档的3D表现力.MoleHill 在windows平台上依赖于DrectX9，MacOS和Linux上依赖于openGL 1.3。在手机平台上，如android中，MoleHill将依赖于OpenGL ES2。技术上，MoleHill API是真正基于可编程性着色器的，而且表现出一些3D开发者在flash中期待已久的特征，例如可编程性顶点和片段着色器，使得支持通过GPU 进行骨骼动画的节点修饰，以及原生的Z缓冲，模板颜色缓冲，立方体纹理等。就表现力来说，现在的Adobe Flash Player 10.1可以以近30HZ的频率下渲染几十万个非Z缓冲三角形，但是通过新的3D API，开发者可以想象以后的Adobe Flash Player能够在60HZ的频率，高清全屏的环境下渲染几十万个Z缓冲三角形。将高端的3D体验传送到每一台接入到网络内的电脑和设备，MoleHill将使之变得可能。点击this观看示例视频，以了解MoleHill平台。MoleHill 原理已经存在于Flash Player 10中的的Flash Player 2.5D API是不被推荐的，Molehill API将提供一个需要完全的GPU加速以促进3D表现力的解决方案。当然，选用何种API取决于你的工程项目。在Adobe Labs里，我们推荐一种“Stage Video”的概念，作为10.2可用的beta版。Stage Video 依赖于同样的设计，解码时通过完全的GPU加速。通过这种新的渲染器模式，Adobe Flash Player不再将视频图像或3D缓冲区呈现在显示序列中，而是将其放入后台的GPU着色模块中。这允许Adobe Flash Player直接将显存中充足可用的资源显示在屏幕上。通过CPU中的显示序列，不需要更多的重读便可检索GPU中的图像并将其显示在屏幕上。结果，由于后台的3D内容不再显示序列中，Context3D和Stage3D都不是显示类。所以，记住你不能像使用其他显示对象一样使用它们，不能对其应用旋转，混合模式，滤镜和很多其它的效果。下面这幅图说明了这个概念：当然，正如你所知道的，2D内容可以完善地融入3D内容，但是相反是不可能的。不管怎样，我们将提供一种API，允许你将3D内容加入BitmapData类。从一个ActionScript API的立场，作为一个开发者，你主要会用到这两个类，Stage3D和Context3D。你向Adobe Flash Player请求一个3D 对象，Context3D类会为你创造。那么现在你也许在想，如果显卡驱动不兼容会怎么样，我会默默地得到一块黑色的屏幕？Flash Player会返还给你一个Context3D对象，但是用的是内部的软件，所以你会体验到相同的Molehill特征和相同的API，但是运行在CPU上。为了达到目的，我们依赖于TransGaming 公司被称为“SwiftShader”的高速CPU光栅器。一个令人振奋的消息是，即使运行在软件上（on software），SwiftShader要比现在Flash Player 10.1中可用的矢量光栅器快10倍，所以你可以期待更非凡的表现里运行在软件里（in software）的时候。MoleHill API的美好在于你不用考虑内部在发生些什么。我是运行在DirectX, OpenGL 或者 SwiftShader?我是否应该用不同的API当在MacOS和 Linux的openGL环境下，或者手机平台的OpenGL ES2环境下？不，对于一个开发者，任何事都是透明的，你只需要用一种API编程，Adobe Flash Player会在内部替你处理这些，并且在屏幕后面完成转换工作。记住这点很重要，Molehill API不使用固定函数管道，而是一种可编程性函数管道，这意味着你可以利用顶点和片段着色器将任何图像显示在屏幕上。那么，你能够你的着色器以纯粹的底层AGAL (“Adobe Graphics Assembly Language”)字节作为字节数组上传到显卡。作为一个开发者，你有两种方式可以选择：在汇编级上编写你的着色器，这需要你对着色器的原理非常了解；或者使用高层的语言例如：Pixel Bender 3D，这将表现出一种更自然的方式去编写你的着色器及编译合适的AGAL字节码。The way it works.In order to represent your triangles, you will need to work withVertexBuffer3DandIndexBuffer3Dobjects by passing vertices coordinates and indices, and once your vertex shaders and fragment shaders are ready, you can upload them to the graphics card through aProgram3Dobject. Basically, a vertex shader deals with the position of the vertices used to draw your triangles whereas a fragment shader handles the appearance of the pixels used to texture your triangles.The following figure illustrates the difference between the types of shaders:As stated before, Molehill does not use a fixed function pipeline, hence developers will be free to create their own custom shaders and totally control the rendering pipeline. So lets focus a little bit on the concept of vertex and fragment shaders with Molehill.Digging into vertex and fragment shadersTo illustrate the idea, here is a simple example of low-level shading assembly you could write to display your triangles and work at the pixel level with Molehill. Lets get ready, cause we are going to go very low-level and code shaders to the metal. Of course if you hate this, do not worry, you will be able to use a higher-level shading language like Pixel Bender 3D.Note : To compile the assembly String to AGAL bytecode, download the AGALMiniAssemblerhere.To create our shader program to upload to the graphics card; we need first a vertex shader (Context3DProgramType.VERTEX) which should at least output a clip-space position coordinate. To perform this, we need to multiplyva0(each vertex position attributes) byvc0(vertex constant 0)our projection matrix stored at this index andoutput the result through theopkeyword (standing for output position of the vertex shader):1./ create a vertex program - from assembly2.varvertexShaderAssembler : AGALMiniAssembler =newAGALMiniAssembler();3.4.vertexShaderAssembler.assemble( Context3DProgramType.VERTEX,5.m44 op, va0, vc0 n/ 4x4 matrix transform from stream 0 (vertex position) to output clipspace6.);Now you may wonder, what is thism44thing? Where does it comes from?It is actually a 4x4 matrix transformation, it projects our vertices according to the projection matrix we defined, we could have written our shader like the following, by manually calculating the dot product on each attribute, but them44instruction (performing a 4x4 matrix transform on all attributes in one line) is way shorter:01./ create a vertex program - from assembly02.varvertexShaderAssembler : AGALMiniAssembler =newAGALMiniAssembler();03.04.vertexShaderAssembler.assemble( Context3DProgramType.VERTEX,05.dp4 op.x, va0, vc0 n+/ 4x4 matrix transform from stream 0 (vertex position) to output clipspace06.dp4 op.y, va0, vc1 n+07.dp4 op.z, va0, vc2 n+08.dp4 op.w, va0, vc3 n+09.);Remember thatvc0(vertex constant 0), it is actually just our projection matrix stored in this index, passed earlier as a constant through thesetProgramsConstantMatrixAPI on theContext3Dobject:1.context3D.setProgramConstantsFromMatrix(Context3DProgramType.VERTEX,0, modelMatrix,true);As with our matrix constant,va0(vertex attributes 0) for the position, needs to be defined, and we did this through thesetVertexBufferAtAPI on theContext3Dobject.1.context3D.setVertexBufferAt (0, vertexbuffer,0, Context3DVertexBufferFormat.FLOAT_3 );In our example, the vertex shader passes the vertices color (va1) to the fragment shader throughv0and themovinstruction to actually paint our triangles pixels. To do this, we could write the following:1./ create a vertex program - from assembly2.varvertexShaderAssembler : AGALMiniAssembler =newAGALMiniAssembler();3.4.vertexShaderAssembler.assemble( Context3DProgramType.VERTEX,5.m44 op, va0, vc0 n+/ 4x4 matrix transform from stream 0 (vertex position) to output clipspace6.mov v0, va1 n/ copy stream 1 (vertex color) to fragment shader7.);And as you can imagine,va1(vertexattributes1) for the color was defined throughsetVertexBufferAt, to expose our pixel colors (float 3) in the shaders:1.context3D.setVertexBufferAt(1, vertexbuffer,3, Context3DVertexBufferFormat.FLOAT_3 );Our vertices position and colors are defined into ourVertexBuffer3Dobject :1./ create a vertex buffer2./ format is (x,y,z,r,g,b) = 3 vertices, 6 dwords per vertex3.vertexbuffer.uploadFromVector ( Vector.(4.-1,-1,0,255/255,0,0,/ red5.0,1,0,193/255,216/255,47/255,/ green6.1,-1,0,0,164/255,228/255/ blue7.),0,3);/ start at offset 0, count 3We have our vertex shader defined, now we need to define and upload our fragment shader (Context3DProgramType.FRAGMENT), the idea is to retrieve each vertex color passed (copied fromva1tov0) and output this color through theocopcode:1.varfragmentShaderAssembler : AGALMiniAssembler=newAGALMiniAssembler();2.fragmentShaderAssembler.assemble( Context3DProgramType.FRAGMENT,3.mov oc, v0/ output color4.);As you can imagine, a fragment shader should always output a color. Then, we need to upload all this to theContext3Dobject:1./ upload the AGAL gram = context3D.createProgram();3.program.upload( vertexShaderAssembler.agalcode, fragmentShaderAssembler.agalcode );If we compile and run those shaders, we would get the following result:Now, lets say we need to invert the colors of each pixel, it would be really easy. As this operation is performed on the pixels color only, we would just modify our fragment shader and use thesubopcode to subtract the color, as following:1.varfragmentShaderAssembler : AGALMiniAssembler=newAGALMiniAssembler();2.fragmentShaderAssembler.assemble( Context3DProgramType.FRAGMENT,3.sub ft0, fc1, v0 n+/ subtract the color ( 1 - color)4.mov oc, ft0/ output color5.);Here, we invert the color of each pixel by subtracting each pixel color from 1 (white). The white pixel we subtract from is stored in a fragment constant (fc1) that we passed by using thesetProgramConstantsFromVectorAPI:1.context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT,1, Vector.( 1,1,1,1 ) );The final pixel color is then stored in a fragment temporary register (ft0) and passed as the final output color.By using this modified fragment shader, we end up with the following result:As another exercice, lets process a sepia-tone filter.To achieve this, we need to first convert to gray scale then to sepia. We would use the following fragment shader for this:1.varfragmentShaderAssembler : AGALMiniAssembler=newAGALMiniAssembler();2.fragmentShaderAssembler.assemble( Context3DProgramType.FRAGMENT,3.dp3 ft0, fc1, v0 n+/ convert to grayscale4.mul ft1, fc2, ft0 n+/ convert to sepia5.mov oc, ft1/ output color6.);As usual, we would have defined our constants using thesetPrograConstantsFromVectorAPI:1./ grayscale2.context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT,1, Vector.( 0.3,0.59,0.11,1 ) );3./ sepia4.context3D.setProgramConstantsFromVector(Context3DProgramType.FRAGMENT,2, Vector.( 1.2,1.0,0.8,1 ) );By using such a fragment shader, we would end up with the following result:As you can imagine, this gives you a lot of power and will allow you to go way further in terms of shading and handle things like lightning through vertex or fragment shading, fog, or even animation through vertex skinning and even more.Ok, so last one, lets now apply a texture to our triangle from aBitmapData, to do this, we would need to pass uv values from our vertex shader to our fragment shader and then use those values to apply our texture that we sampled in our fragment shader.To pass the uv values, we would need to modify our vertex shader this way :1.vertexShaderAssembler =newAGALMiniAssembler();2.vertexShaderAssembler.assemble( Context3DProgramType.VERTEX,3.m44 op, va0, vc0 n+/ 4x4 matrix transform from stream 0 to output clipspace4.mov v0, va1 n/ copy texcoord from stream 1 to fragment program5.);Our uv coordinates are now copied fromva1tov0, ready to be passed to the fragment shader. Notice that we do not pass any vertex color anymore to the fragment shader, just the uv coordinates.As expected, we defined our uv valuesfor each vertex (float2) throughva1withsetVertexBufferAt:1.context3D.setVertexBufferAt(1, _vertexBuffer,2, Context3DVertexBufferFormat.FLOAT_2 );Our vertices position and uv values are defined into ourVertexBuffer3Dobject :01.vertexBuffer.uploadFromVector(02.Vector.(03.04./ x,y,u,v05.-1,-1,0,1,06.0,1,1,0,07.1,-1,1,1,08.09.),10.0,311.);Then we retrieve the uv values in our fragment shader and sample our texture :1.fragmentShaderAssembler.assemble( Context3DProgramType.FRAGMENT,2.mov ft0, v0 n+3.tex ft1, ft0, fs1 n+/ sample texture 14.mov oc, ft1 n5.);To define our texture, we instantiate ourBitmapData, upload it to aTextureobject and upload it to the GPU:1.texture = context3D.createTexture(256,256, Context3DTextureFormat.BGRA,false);2.varbitmap:Bitmap =newMolePeopleBitmap();3.texture.uploadFromBitmapData( bitmap.bitmapData );And then to access it fromfs1, we set it:1.context3D.setTextureAt(1, texture );By using this modified shader program, we end up with this :I will cover in later tutorials new effects like per-fragment fog or heat signature with texture lookup too.Of course, we just covered here how shaders work with Molehill. To control your pixels you need triangles and vertices and indices defining them in your scene. For this, you will need other objects likeVertexBuffer3DandIndexBuffer3D, attached to yourContext3Dobject.The following figure illustra