WBOIT in OpenGL: transparency without sorting

    We will talk about “Weighted blended order-independent transparency” (hereinafter WBOIT) - the technique described in JCGT in 2013 ( link ).

    When several transparent objects appear on the screen, the color of the pixel depends on which one is closer to the observer. Here is a well-known color mixing formula for this case:

    $ \ begin {matrix} C = C_ {near} \ alpha + C_ {far} (1- \ alpha) && (1) \ end {matrix} $


    The order of the fragment arrangement is important for it: the color of the near fragment and its opacity are denoted as C near and α , and the resulting color of all the fragments that are located behind it is denoted as C far . Opacity is a property that takes values ​​from 0 to 1, where 0 means that the object is so transparent that it is not visible, and 1 - that it is so opaque that nothing is visible behind it .

    To use this formula, you must first sort the fragments by depth. Imagine how much headache this involves! In general, sorting should be done in each frame. If you are sorting objects, then some objects of complex shape will have to be cut into pieces and sorted by depth of the cut parts (in particular, for intersecting surfaces, this will definitely need to be done). If you sort the fragments, then the sorting will occur in the shaders. This approach is called “Order-independent transparency” (OIT), and it uses a linked list stored in the video card's memory. To predict how much memory will have to be allocated for this list is almost unrealistic. And if there is not enough memory, artifacts will appear on the screen.

    Lucky for those who can control how many translucent objects are placed on the stage, and where they are relative to each other. But if you do CAD, then you will have as many transparent objects as the user wants, and they will be located randomly.

    Now you understand the desire of some people to simplify their lives and come up with a formula for mixing colors that does not require sorting. Such a formula is in the article to which I referred at the beginning. There are even several formulas there, but the best one according to the authors (and in my opinion too) is this one:

    $ \ begin {matrix} C = {{\ sum_ {i = 1} ^ {n} C_i \ alpha_i} \ over {\ sum_ {i = 1} ^ {n} \ alpha_i}} (1- \ prod_ {i = 1} ^ {n} (1- \ alpha_i)) + C_0 \ prod_ {i = 1} ^ {n} (1- \ alpha_i) && (2) \ end {matrix} $




    In the screenshot are groups of translucent triangles located in four layers in depth. On the left, they are rendered using the WBOIT technique. On the right is a picture obtained using formula (1), classic blending of colors, taking into account the order of the arrangement of the fragments. Next I will call it CODB (Classic order-dependent blending).

    Before we begin rendering transparent objects, we must render all opaque ones. After that, transparent objects are rendered with a depth test, but without writing to the depth buffer (this is done like this:glEnable(GL_DEPTH_TEST); glDepthMask(GL_FALSE);) That is, this is what happens at a point with some screen coordinates (x, y): transparent fragments that are closer than opaque pass the depth test, regardless of how they are located in depth relative to already drawn transparent fragments, and transparent fragments that appear farther opaque, do not pass the depth test, and, accordingly, are discarded.

    C 0 in formula (2) is the color of an opaque fragment, on top of which transparent fragments are drawn, of which we have n pieces, indicated by indices 1 through n. C i is the color of the i-th transparent fragment, α i is its opacity.

    If you look closely, then formula (2) is a bit like formula (1). If you imagine that this is C near, C 0 is C far , and is α , then this will be the 1st formula, one to one. Indeed, this is a weighted average of the colors of transparent fragments (the center of mass is determined in the mechanics by the same formula), it will pass for the color of the nearest fragment C near . C 0 is the color of the opaque fragment located behind all the fragments, for which we calculated this weighted average, and it will pass for C far . That is, we replaced all transparent fragments with one “averaged” fragment and applied the standard formula for mixing colors - formula (1). What is this cunning formula for α that the authors of the original article offer us?

    $ \ alpha = 1- \ prod_ {i = 1} ^ {n} (1- \ alpha_i) $


    This is a scalar function in n-dimensional space, so let’s recall the differential analysis of the functions of several variables. Given that all α i belong to the range from 0 to 1, the partial derivative with respect to any of the variables will always be a non-negative constant. This means that the opacity of the “average” fragment increases with increasing opacity of any of the transparent fragments, and this is exactly what we need. In addition, it increases linearly.

    If the opacity of a fragment is 0, then it is not visible at all, it does not affect the resulting color.

    If the opacity of at least one fragment is 1, then αequal to 1. That is, an opaque fragment becomes invisible, which is generally good. Only here the transparent fragments located behind the fragment with opacity = 1 still shine through it and affect the resulting color:



    Here lies an orange triangle on top, green under it, and gray and cyan under green, and all this against a black background. Blue opacity = 1, all others - 0.5. The picture on the right is what it should be. As you can see, WBOIT looks disgusting. The only place where the normal orange color appears is the edge of the green triangle, surrounded by an opaque white line. As I just said, an opaque fragment is invisible if the opacity of the transparent fragment is 1.

    It can be seen even better here:



    The orange triangle has an opacity of 1, the green one with the transparency turned off is simply drawn with the opaque objects. It looks as if the GREEN triangle shines through ORANGE through the orange triangle.

    To make the picture look decent, the easiest way is not to assign objects high opacity. In my working project, I do not allow setting opacity greater than 0.5. This is 3D CAD, in which objects are drawn schematically, and special realism is not required, so such a restriction is permissible there.

    With low opacity values, the pictures on the left and right look almost the same:



    And with high opacity they noticeably differ:



    Here is the transparent polyhedron:




    The polyhedron has orange lateral and green horizontal faces. Unfortunately, you won’t understand this at first glance, i.e. the picture does not look convincing. Where there is an orange wall in front, you need more than orange, and where green is more than green. It will be much better to draw faces in one color:



    Depth-based WBOIT


    In order to somehow compensate for the lack of sorting by depth, the authors of the article came up with several options for adding depth to formula (2). This makes implementation more difficult, and the result less predictable and dependent on the characteristics of a particular three-dimensional scene. I did not delve into this topic, so who cares - I propose to read the article.

    It is argued that WBOIT is sometimes capable of something that classical sorting transparency cannot. For example, you draw smoke as a system of particles using only two particles - with dark and light smoke. When one particle passes through another, classical color mixing with sorting gives an ugly result - the color of the smoke from the light sharply becomes dark. The article says that depth-sensitive WBOIT allows for a smooth transition and looks more believable. The same can be said about modeling fur and hair in the form of thin tubes.

    Code


    Now about how to implement formula (2) on OpenGL. The example code is on Github ( link ), and most of the pictures in the article are from there. You can collect and play with my triangles. The Qt framework is used.

    For those who are just starting to study the rendering of transparent objects, I recommend these two articles:

    Learn OpenGL. Lesson 4.3 - Mixing colors
    Order-Independent Transparency algorithm using linked lists on Direct3D 11 and OpenGL 4

    The second, however, is not so important for understanding this material, but the first is required to be read.

    To calculate formula (2), we need 2 additional framebuffers, 3 multisample textures and a render buffer, in which we will write the depth. In the first texture - colorTextureNT (NT means non-transparent) - we will render opaque objects. It has type GL_RGB10_A2. The second texture (colorTexture) will be of type GL_RGBA16F; in the first 3 components of this texture we will write down this piece of formula (2):, in the fourth - . Another texture of type GL_R16 (alphaTexture) will contain .

    First you need to create these objects to get their identifiers from OpenGL:

        f->glGenFramebuffers (1, &framebufferNT    );
        f->glGenTextures     (1, &colorTextureNT   );
        f->glGenRenderbuffers(1, &depthRenderbuffer);
        f->glGenFramebuffers(1, &framebuffer );
        f->glGenTextures    (1, &colorTexture);
        f->glGenTextures    (1, &alphaTexture); 

    As I said, the Qt framework is used here, and all OpenGL calls go through an object of type QOpenGLFunctions_4_5_Core, which is always denoted as f for me.

    Now you should allocate memory:

        f->glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, colorTextureNT);
        f->glTexImage2DMultisample( GL_TEXTURE_2D_MULTISAMPLE, numOfSamples,
                                    GL_RGB16F, w, h, GL_TRUE                 );
        f->glBindRenderbuffer(GL_RENDERBUFFER, depthRenderbuffer);
        f->glRenderbufferStorageMultisample( GL_RENDERBUFFER, numOfSamples,
                                             GL_DEPTH_COMPONENT, w, h        );
        f->glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, colorTexture);
        f->glTexImage2DMultisample( GL_TEXTURE_2D_MULTISAMPLE, numOfSamples,
                                    GL_RGBA16F, w, h, GL_TRUE                );
        f->glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, alphaTexture);
        f->glTexImage2DMultisample( GL_TEXTURE_2D_MULTISAMPLE, numOfSamples,
                                    GL_R16F, w, h, GL_TRUE                   );

    And configure framebuffers:

        f->glBindFramebuffer(GL_FRAMEBUFFER, framebufferNT);
        f->glFramebufferTexture2D( GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
                                   GL_TEXTURE_2D_MULTISAMPLE, colorTextureNT, 0
                                 );
        f->glFramebufferRenderbuffer( GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT,
                                      GL_RENDERBUFFER, depthRenderbuffer
                                    );
        f->glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
        f->glFramebufferTexture2D( GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0,
                                   GL_TEXTURE_2D_MULTISAMPLE, colorTexture, 0
                                 );
        f->glFramebufferTexture2D( GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1,
                                   GL_TEXTURE_2D_MULTISAMPLE, alphaTexture, 0
                                 );
        GLenum attachments[2] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1};
        f->glDrawBuffers(2, attachments);
        f->glFramebufferRenderbuffer( GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT,
                                      GL_RENDERBUFFER, depthRenderbuffer
                                    );

    On the second rendering pass, the output from the fragment shader will go to two textures at once, and this must be explicitly specified using glDrawBuffers.

    Most of this code is executed once, at program startup. Code that allocates memory for textures and render buffers is called every time the window is resized. Next comes the rendering code, which is called every time the window is redrawn.

        f->glBindFramebuffer(GL_FRAMEBUFFER, framebufferNT);
        // ... код рендеринга ...

    We just drew all the opaque objects on the colorTextureNT texture, and wrote the depths into the render buffer. Before using the same renderbuffer at the next stage of drawing, you need to make sure that all the depths of opaque objects are already written there. For this, GL_FRAMEBUFFER_BARRIER_BIT is used. After rendering transparent objects, we call the ApplyTextures () function, which will launch the final stage of rendering, in which the fragment shader will read data from the textures colorTextureNT, colorTexture and alphaTexture to apply formula (2). The textures should have been completely written by then, so before calling ApplyTextures () we use GL_TEXTURE_FETCH_BARRIER_BIT.

        static constexpr GLfloat clearColor[4] = { 0.0f, 0.0f, 0.0f, 0.0f };
        static constexpr GLfloat clearAlpha = 1.0f;
        f->glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
        f->glClearBufferfv(GL_COLOR, 0,  clearColor);
        f->glClearBufferfv(GL_COLOR, 1, &clearAlpha);
        f->glMemoryBarrier(GL_FRAMEBUFFER_BARRIER_BIT);
        PrepareToTransparentRendering();
        {
            // ... код рендеринга ...
        }
        CleanupAfterTransparentRendering();
        f->glMemoryBarrier(GL_TEXTURE_FETCH_BARRIER_BIT);
        f->glBindFramebuffer(GL_FRAMEBUFFER, defaultFBO);
        ApplyTextures();

    defaultFBO is the framebuffer through which we display the image. In most cases it is 0, but in Qt it is QOpenGLWidget :: defaultFramebufferObject ().

    Each time the fragment shader is called, we will have information about the color and opacity of the current fragment. But at the output in the colorTexture texture we want to get the sum (and in the alphaTexture texture the product) of some functions of these quantities. Blending is used for this. Moreover, since for the first texture we calculate the sum, and for the second - the product, the blending settings (glBlendFunc and glBlendEquation) for each attachment must be set separately.

    Here is the contents of the PrepareToTransparentRendering () function:

        f->glEnable(GL_DEPTH_TEST); f->glDepthMask(GL_FALSE);
        f->glDepthFunc(GL_LEQUAL);
        f->glDisable(GL_CULL_FACE); 
        f->glEnable(GL_MULTISAMPLE);
        f->glEnable(GL_BLEND);
        f->glBlendFunci(0, GL_ONE, GL_ONE);
        f->glBlendEquationi(0, GL_FUNC_ADD);
        f->glBlendFunci(1, GL_DST_COLOR, GL_ZERO);
        f->glBlendEquationi(1, GL_FUNC_ADD);

    And the contents of the CleanupAfterTransparentRendering () function:

        f->glDepthMask(GL_TRUE);
        f->glDisable(GL_BLEND);

    In my fragment shader, opacity is indicated by the letter w. The product of the color by w and w itself we output to one output parameter, and 1 - w to another. For each output parameter, a layout qualifier is set in the form “location = X”, where X is the index of the element in the attachment array, which we passed to glDrawBuffers in the 3rd listing (specifically, the output parameter with location = 0 is sent to the texture bound to GL_COLOR_ATTACHMENT0 , and the parameter with location = 1 - in the texture attached to GL_COLOR_ATTACHMENT1). The same numbers are used in the glBlendFunci and glBlendEquationi functions to indicate the attachment number for which we set the blending parameters.

    Fragment Shader:

    #version 450 core
    in vec3 color;
    layout (location = 0) out vec4 outData;
    layout (location = 1) out float alpha;
    layout (location = 2) uniform float w;
    void main()
    {
        outData = vec4(w * color, w);
        alpha = 1 - w;
    }

    In the ApplyTextures () function, we simply draw a rectangle over the entire window. The fragment shader requests data from all the textures we have created, using the current screen coordinates as texture coordinates and the current sample number (gl_SampleID) as the sample number in the multisample texture. Using the gl_SampleID variable in the shader automatically turns on the mode when the fragment shader is called once for each sample (under normal conditions, it is called once for the whole pixel, and the result is written to all samples that were inside the primitive).

    There is nothing remarkable in the vertex shader:

    #version 450 core
    const vec2 p[4] = vec2[4](
         vec2(-1, -1), vec2( 1, -1), vec2( 1,  1), vec2(-1,  1)
                             );
    void main() { gl_Position = vec4(p[gl_VertexID], 0, 1); }

    Fragment Shader:

    #version 450 core
    out vec4 outColor;
    layout (location = 0) uniform  sampler2DMS colorTextureNT;
    layout (location = 1) uniform  sampler2DMS colorTexture;
    layout (location = 2) uniform  sampler2DMS alphaTexture;
    void main() {
        ivec2 upos = ivec2(gl_FragCoord.xy);
        vec4 cc = texelFetch(colorTexture, upos, gl_SampleID);
        vec3 sumOfColors = cc.rgb;
        float sumOfWeights = cc.a;
        vec3  colorNT = texelFetch(colorTextureNT, upos, gl_SampleID).rgb;
        if (sumOfWeights == 0)
        { outColor = vec4(colorNT, 1.0); return; }
        float alpha = 1 - texelFetch(alphaTexture, upos, gl_SampleID).r;
        colorNT = sumOfColors / sumOfWeights * alpha +
                  colorNT * (1 - alpha);
        outColor = vec4(colorNT, 1.0);
    }

    And finally, the contents of the ApplyTextures () function:

        f->glActiveTexture(GL_TEXTURE0);
        f->glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, colorTextureNT);
        f->glUniform1i(0, 0);
        f->glActiveTexture(GL_TEXTURE1);
        f->glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, colorTexture);
        f->glUniform1i(1, 1);
        f->glActiveTexture(GL_TEXTURE2);
        f->glBindTexture(GL_TEXTURE_2D_MULTISAMPLE, alphaTexture);
        f->glUniform1i(2, 2);
        f->glEnable(GL_MULTISAMPLE); f->glDisable(GL_DEPTH_TEST);
        f->glDrawArrays(GL_TRIANGLE_FAN, 0, 4);

    Well, it would be nice to free OpenGL resources after it's over. I have this code called in the destructor of my OpenGL widget:

        f->glDeleteFramebuffers (1, &framebufferNT);
        f->glDeleteTextures     (1, &colorTextureNT);
        f->glDeleteRenderbuffers(1, &depthRenderbuffer);
        f->glDeleteFramebuffers (1, &framebuffer);
        f->glDeleteTextures     (1, &colorTexture);
        f->glDeleteTextures     (1, &alphaTexture);

    Also popular now: