Pixel Shader UI Components: Writing Your First Shader

    Who can be called a “pixel shader boss and a pixel commander”? Denis Radin , who works at Evolution Gaming on photorealistic web games using React and WebGL: he is known to many just under the name Pixels Commander.

    In December, at our HolyJS conference, he made a presentation on how using GLSL can improve the work with UI components compared to “regular javascript”. And now for Habr we have prepared a text version of this report - welcome to kat! At the same time we attach a video of the speech:



    First, a question for the audience: how many languages ​​are well supported on the web? (Voice from the audience: “Not a single one!”)
    Well, the languages ​​in the browser, let’s say so. Three? Let's assume there are four of them: HTML, CSS, JS, and SVG. SVG can also be considered a declarative language, another kind, it's still not HTML.

    But actually there are even more of them. There is VRML, he died, you can not count it. And there’s GLSL (“OpenGL Shading Language”). And GLSL is a very special language for the web.

    Because the rest (JS, CSS, HTML) originated on the web, and from web pages began a victorious march on other platforms (for example, mobile). And GLSL was born in the world of computer graphics, in C ++, and came to the web from there. And what's great about it: it works wherever OpenGL works, so if you learned it, you can use it anywhere (in Unity, Swift, Java, and so on).

    GLSL stands for crazy special effects in computer games. And I like that with its help you can develop interesting and unusual UI components, and we'll talk about them later. It is also a technology for parallel computing, which means you can mine cryptocurrencies using GLSL. What interested?

    History


    Let's start with the history of GLSL. When and why did he appear? This diagram displays the OpenGL rendering pipeline:



    Initially, in the first version of OpenGL, the pipeline rendering looked like this: vertices are sent to the input, primitives are gathered from the vertices, the primitives are rasterized, the frame is trimmed and then the framebuffer is output.

    There is a problem here: it is not customizable. Since we have a clearly defined pipeline, you can upload textures there, but you can’t do anything special with any exact request.

    Let's look at the simplest example: draw a fog. There is a scene. It all consists of vertices, they are superimposed texture. In the first version of OpenGL, it looked like this:



    How can fog be made? In the fogvalue formula, this is the distance to the camera times the density of the fog, and the pixel color is equal to the current pixel color times the color of the fog and the amount of fog. If we perform this operation for each pixel on the screen, we will get the following result:



    GLSL shaders appeared in 2004 in OpenGL v2, and this was the biggest breakthrough in the history of OpenGL. It appeared in 1991, and now, 13 years later, the next version was released.

    From then on, the pipeline rendering began to look like this:



    The vertices are fed to the input, the vertex shader is executed first, which allows you to change the geometry of the object, then the primitives are built, they are rasterized, then the fragment shader is executed (“fragment” means “pixel”, in English terminology often uses “fragment shader”), then it cropped and displayed.

    Okay, let's talk about some of the features of GLSL, because it has so many, many things that are unusual and sound strange, for JS developers so for sure.

    Well, first, what's important to us, for web developers: GLSL is part of the WebGL specification. The gateway to GLSL will be.

    GLSL is compiled using the GPU driver. Thanks to this, it is cross-platform, because it compiles for each specific platform, and it is amazingly fast. It is very fast, thousands of times faster than JavaScript, because it compiles specifically for the platform and runs on a special piece of hardware for which it was intended.

    At the same time, it starts in many processes, for example, a card, if you follow the news of hardware, the GTX 970 card, it simultaneously runs 1664 shader processes. Imagine how much you can mine?

    In general, this is how mining is performed, and everything else, all parallel computing - these are CUDA platforms, work through shaders. They come in different forms, not always GLSL, but on the web we have GLSL shaders, part of the OpenGL specification.

    There are certain features associated with the fact that data is sent only once. Since the execution is parallel, for the entire passage, for the entire screen, the data is loaded into the shader once, and this should be taken into account.

    GLSL is a strongly typed language. There are types float, integer, boolean, 2-3-4 component vectors (which, in fact, are 2-3-4-element arrays), there are also 2-3-4-dimensional matrices.

    This is a language sharpened for mathematics, and it has all the wonderful trigonometric and mathematical functions that you can imagine: radians into degrees, degrees into radians, sine, arc cosine, tangent, matrix multiplication, vector multiplication, various derivatives, etc. .

    Practice


    Good. From theory, let's move on to practice. Consider the simplest pixel shader.



    First we set the radius of the circle, this is a variable of type float. Then the center is a two-component vector. Note that the origin in GLSL is not in the upper left corner, but in the lower left. You can play around a bit with the coordinates, move this circle somewhere.

    Then comes the main function, this is the entry point to any shader. It first calculates the distance to the center using the built-in distance function, the coordinates of the current pixel and the coordinates of the center.

    Next, the inCircle float variable is calculated: if our pixel is inside the circle, it is equal to one, and if outside it is zero.

    And the last operation is the output parameter gl_FragColor, which determines the color that will be on the screen. We assign it the aforementioned inCircle. That is, if we are inside the circle, there will be one.

    This is a very interesting shorthand: a four-component vector is created from one float variable, all the components of the vector are immediately assigned the value of this variable. It uses RGBA notation, that is, the four components are RGB and the alpha channel.

    And you can change it like this:



    What is going on here? We assign the resulting value not to all channels at once, but only to green.

    Okay, let's move on from the simplest example, which is almost useless, to solving a practical problem. Once upon a sad Amsterdam autumn morning, I got a task at JIRA, which made my life a little more fun.

    The task was about a spinner. We wrote the operating system in JS, and we had such a cool spinner in the OS. It worked, but there was one small problem: when there was some kind of background process, the spinner sometimes twitched. I was asked to sort it out.



    I started digging and saw that the spinner was implemented using the sprite sheet: the background-position of the element changed, and all these frames scroll.



    Basically, it worked, but you probably know that if the background-position changes, then what happens? Repaint. There is constant ripping, and this loaded the processor, it did not work very fast.

    How can this be fixed? It is possible through CSS. Naturally, I did not immediately go into the jungle of GLSL, at first we did it all under the simplest way, through CSS, through hardware-accelerated properties. You many know that there are hardware-accelerated properties that without re-can allow you to perform some kind of animation. Here, all this can be changed to opacity, that is, from background-position we move to opacity.

    How can this be done with opacity? To decompose all frames into layers and with the help of opacity gradually hide them and show, in general, the same effect is obtained, but without any replay. Hooray, QA-department confirmed the increase in performance, everyone is happy.

    The next day I received another task in JIRA. Denis, we know that you are already an expert in spinners, we have exactly the same spinner, only blue and a slightly different width.

    I knew there were a lot of spinners, and I realized that there is a small problem. Firstly, this spinner for 150 frames in video memory is deployed for more than 8 megabytes, I specifically calculated by the resolution and bitness of these textures (because a texture is created for each frame. And it takes 10 megabytes in RAM. And for this you need . download 100 KB In general, each spinner is worth about 20-30 megabytes, considering that it is necessary to loosen the spinner 30 megabytes -. it is, frankly, a lot of them if 3-4 -. it is 100 MB RAM on the spinners.

    In us in the browser there was a limit of 256 megabytes: as soon as they were reached, the system and the whole was emptied I think, even to mobile 100 megabytes of spinners -. the same luxury.

    Okay, I get it, we have a problem. It can be solved using GLSL. How pragmatic this is, we will discuss later.

    Writing a shader


    And now we can write a pixel shader together. Spinner can be represented in the form of an animated arch that collapses and pops open: it changes the angle of the beginning and end, and this arch rotates with a certain periodicity, attenuation, acceleration. Therefore, we need to learn how to draw an arch using mathematics in GLSL.

    First, install the extension for Chrome Refined GitHub , it is necessary to copy diff from commits. If you don’t put it, when you try to copy the diff text, line numbers will be copied, and you will have to delete them manually. Therefore, Refined Github helps a lot: it puts out line numbers on a separate list, and it's cool.

    Then open the online shader editor and GitHub repositoryPixelsCommander / pixel-shaders-workshop , in which you need to go through the steps.

    Where do we start



    ? Copy-paste the first step in the GLSL editor, thanks to which we will have a circle: What is happening here? At the top of the new block, it was not in the previous example, here comes a uniform variable. “Uniform” means a variable sent from JavaScript. We see here u_time, u_resolution and u_mouse from JavaScript. The most interesting of them is u_resolution. What she says is the canvas dimension. JavaScript removed the canvas dimension and sent us a two-component vector in GLSL, now we know the canvas size in GLSL.

    In PI, we determined the number pi, so as not to write it constantly with our hands. Then we multiplied u_resolution by 0.5: this is a two-component vector (there is width and height there), and when you multiply the vector by 0.5, all its components are multiplied by 0.5 at once. So we found half our dimension. After that we took the radius as the minimum of width and height.

    Now we have the Circle function: earlier we just defined in main whether we were inside the circle or not, and now we moved it into a separate function, where we launch the coordinate of the current pixel, center and radius.

    And in main, we get isFilled as the result of executing the Circle function, and subtract isFilled from the unit, because we want the background not to be white. That is, they inverted all this wealth.

    Now step two: we will cut off the sector on the circle.



    We add a function that draws a sector, and a function that says whether the angle lies between two given angles. And besides, we now make isFilled the product of the results of circle and sector. If in both cases one, then we are within our figure. If circle were not taken into account, then the sector would be infinite, and not limited to a circle. The result looks like this:



    Now the third step . We draw an arch.



    A new arch feature is added here. Now we need to know its thickness, for this we will calculate the internal radius and what do we see?

    Now we have isFilled - this is the result of executing the arc function, to which we pass the start angle, end angle, inner and outer radii. Here it is all built internally on the sector that we already have, and on two functions of the circle that invert each other. That is, two circles are cut off, one hides the other.

    Everything is great, everything is fine, we have an arch, we are almost ready, but if you look closely, and I will try to help you, then you will notice that the arch is pixelized, there are “cloves” without smoothing:



    This is because there is no smoothing, this is because when we draw a circle, we use the step function here, when we determine whether the point lies in the circle or not, and the step function hard cuts off, discretely, 0 or 1, if the value is lower given, then this is 1, if higher it is 0. Accordingly, our pixel can be either black or white.

    Let's get rid of this, this will be step 4. Add antialiasing.



    We replace step function with smoothstep. And smoothstep doesn’t just say “either 0 or 1”, but interpolates between the two values. Here we have "distanceToCenter minus two pixels" and there is just distanceToCenter, that is, we have anti-aliasing by 2 pixels smearing. Here you can argue about the terms, but in reality we just added antialiasing to our shader.



    And the arch became smooth and silky.

    Now let's move on to the most difficult - to the animation. To draw an arch is, in general, trigonometry of the 5th grade, and there is nothing complicated. With animation, things are a little more complicated, because you first need to recognize and decompose it.

    By decomposing the spinner animation, we find that there are actually two animations. One is the collapse-unblock animation, and the second is the rotation animation. In addition, the animation accelerates at the beginning of the cycle, and slows down at the end. This is very similar to the behavior of the sine function: in the interval from - pi / 2 to pi / 2, acceleration first starts, soars up sharply, and then slows down.



    Step Five We will apply this function to our corners of the beginning and end of the arch. We get an animation of collapsing, unfolding, albeit for the time being a little respectable (this will be fixed). What's going on here? Time closes in the period from - pi / 2 to pi / 2, then the sine function is applied to this, and all the time we get a value from zero to one - how much we are collapsed, unhappy. That is, in fact, the easing function is used here, this is what is used everywhere in twins, in CSS, it is implemented on GLSL here. Then we multiply 360 by the result of this easing function and get the start angle, the end angle, which we pass to the arch function that we wrote earlier.

    The next step is the rotation of the entire spinner.



    With rotation, everything is simple, we have already prepared the theoretical base, we know that the sine drives and we add to the startAngle and endAngle a value that is obtained, again, from the sine, but with a twice as large period, because we have two collapsing, collapsing, it turns out just one revolution.

    Thus, we got a spinner, which already practically corresponds to our technical task. It remains to add a little parameterization:



    To do this, you need the RGB function. It’s not necessary to use it, but it’s good, because we usually take colors from Photoshop, and they have channel byte values ​​from 0 to 255, you saw that in GLSL from 0 to 1, and this function allows you to send it to it familiar to us 255/255/255 and get 1/1/1 at the output.

    We use this function in main, and there is also added customization of the background, just in case of fire.



    It turned out a wonderful animated vector spinner, in which you can change the width and color. The component is ready, it works, is rendered on the GPU, and all this stuff takes 70 lines of code. If it comes to rest, you can probably squeeze up to 5 lines, which, of course, can not be compared with the amount of information that we transmitted in the sprite sheet - just heaven and earth. If we just had 30 megabytes of pictures there, plus you need to initialize the same contexts for textures and so on, then there is obvious progress.

    What can we do about it


    How to use the GLSL component in your web application? As already mentioned, this is done through the WebGL context.



    There is an easy way. There is a web component called the GLSL component, and you put it in the right place on your page, this tag, put the GLSL code that we got in the editor inside. And you will receive in the size in which you have this block, you will receive your GLSL component that works online.

    Earlier, we implemented what CSS could do with a stretch through a sprite sheet or other tricks, albeit not always quickly. But in fact, shaders are much cooler: they give control over every pixel.

    Here is the gif that shows the spinner responding to the cursor:



    And on the videoyou can see an even more impressive example of how GLSL provides disproportionately more features and allows you to control each pixel. There, the spinner has already turned into something else.

    That is, using fairly simple mathematics, you can get some kind of component, and after working a little more, we can add new unusual properties to it. The possibilities of pixel shaders are essentially limitless and limited only by your knowledge of mathematics and your skills in writing shaders.

    And what else is good for GLSL: in addition to these limitless possibilities, it gives JavaScript developers, front-end developers, a breath of fresh air. You write JavaScript for some years, you understand that you are good at it, and you want something new, but you don’t want it in the backend. In this case, GLSL is a good option to change and diversify your life.

    Thanks a lot!

    If you liked the report, pay attention: HolyJS 2018 Piter will take place already next week , and Denis will also speak there , now with the theme “Mining crypto in browser: GPU, WebAssembly, JavaScript and all the good things to try”. And in the discussion area after the report, it will be possible to properly question him both on the topic of the new report and about the shaders. In addition to Denis, there will be dozens of other speakers - see all the details on the HolyJS website .

    Also popular now: