April 22, 2013 at 09:5906 - DreamWorks and HP: Creating the Kruds Family
Four cities. Eighty million render hours. Numerous workstations, the most powerful blade servers, the best specialists in the field of animation and computer graphics, the network architecture of HP FlexNetwork , 2166 copies in Russia and the best start of the cartoon rental movie in our country. What is it about? About the “Kruds Family” animated film by DreamWorks Animation, in which HP was directly involved.
The production of modern computer graphics takes a truly huge number of computing modules, consumes kilowatts of electricity with terrifying speed, drives tens of gigabits of traffic through networks. Each frame contains many millions of polygon triangles that form character models, terrain, objects, and all-all-all that you see. The most complex shader programs that recreate ultra-realistic water, fire and dust are responsible for creating special effects. Creating five minutes of video material takes many weeks of painstaking work, and despite the progress in both algorithms and application software, everything depends on the computing power of computers, network bandwidth and ruthlessly ticking time.
How did 90 minutes of the cartoon, which fit in eighty million render hours, fit into several years of development? The first sketches, working drawings, storyboards, ideas and characters appeared in 2005. In early 2007, DreamWorks began to work on ideas, reworked the script and characters, breathed new life into the project, and work on the graphics itself began in 2009. The development of models, textures, scenes, special effects and other materials took several years, the final rendering, film assembly and editing were carried out in 2011 and 2012.
DreamWorks has been working with HP for many years in software and hardware solutions to achieve the highest performance results, and this cartoon was no exception.
The main "workhorse" of artists, animators, specialists in special effects and editing became Z820 workstations .
Two server Xeons, up to 512 GB of RAM, a multi-layered sandwich of professional video adapters, high-speed disk subsystems, a reliable cooling system - all for working with super-complex scenes in an interactive environment.
Work with sound was carried out on the HP Z800 - a time-tested workstation equipped with an effective and silent cooling system.
Some sounds are born very simply, some as a result of complex and painstaking work, and the creation of a three-dimensional sound stage that will take into account what is happening not only on the screen, but also behind it, from all sides, with all the obstacles, the model of the environment and geometry space, its materials and their properties in the field of reflection and absorption of sound waves - the task is no less difficult than creating realistic looking wool, hair and skin. Professionals in the field of sound and special effects worked hard, and now you hear the roar of a prehistoric beast, the rustle of leaves, the sound of water and fire as if they really exist and are near you, but our workstations helped them with this.
It is difficult to imagine an office that would accommodate all DreamWork Animation employees and provide them with the opportunity to work on one project within the same room. Artists and animators are located not just in four different buildings, but in four different cities: Glendale, Redwood City, Las Vegas and Bangalore. It’s not easy to work on one project at such great distances, and working simultaneously, in real time, on one scene, one character and one frame seems to be a task beyond real. But the task is real, and most importantly, its solution is just as real as the very need for such technology.
HP Remote Graphics Software has combined three cities in the United States and one in India into one efficient work environment, providing people with everything they need to work together and be creative.
Awesome network technology that allows you to “be present” at another computer, hundreds and thousands of kilometers away, is a reality, and HP’s powerful network technologies have achieved minimal delays in video transmission and feedback.
Stability and uninterrupted operation of the network under the highest load was provided by the HP FlexNetwork architecture (there will be separate material about it a bit later). Work with remote employees, scalability, real-time transmission of HD video content, the ability to work with HP Remote Graphics Software technology on such a complex and highly loaded project is the merit of this interesting technology and HP hardware systems, which create the very “low level” that enables transmission data and traffic control.
HP Intelligent Resilient Framework (HP IRF)- a technology that works wonders out of the blue. It combines the flexible virtualization of server technologies, the management of traffic routes, the elimination of unnecessary entities and idle capacity, the distribution of loads and computing power, the scalability of network solutions and, most importantly, the use of well-known and open technologies as the basis. The result is a converged, simple, transparent network with a minimum of losses and a maximum of efficiency, easy to manage and the ability to easily expand existing solutions.
At the hardware level, network infrastructure and HP IRF were supported by switches HP 12518 , 5820 , 5800 and 6125G / XG .
The most delicious iron! DreamWorks used our eighth generation Blade systems, the HP ProLiant BL460c Gen8, as render servers .
Two eight-core processors, 16 slots for RAM, 10 Gigabit network controller that supports HP FlexFabric (one of the components of FlexNetwork ) - impressive power, and in one “rack” of such kids there can be more than a dozen.
It was they that made it possible to accommodate an impressive 80 million render hours per year of work, parallelizing the execution of tasks for rendering each frame.
In the process of working on such projects, you inevitably have to use the help of printers, scanners and copiers. Hand-drawn storyboards, sketches of scenes, characters, objects, elements, landscapes and just scenes - all this is easier to keep on the wall in front of you, and not in your head or on the computer. I looked away from the monitor - looked at the illustration that interests you - continued to work. No alttabs or searches in the network folders.
As displays used products from the professional line of HP DreamColor . One of the most popular solutions was the HP LP2480zx monitor .
24 inches, 1920x1080, H-IPS-matrix and factory color calibration of each instance for the most complete and uniform disclosure of color space. It would seem nothing impressive, but the LP2480zx is not an ordinary monitor. It contains RGB LED backlighting that provides the most accurate control of the “white point” of the display, five basic preset profiles (sRGB, Adobe RGB, Rec. 601 (standard calibration for analog TV) and Rec. 709 (standard calibration for HDTV) and special color space for standard colors developed by DreamWorks). In addition, there is full support for specialized software and work with a colorimeter, support for ten-bit color space, work with a 200 Hz signal source.HP DreamColor LP2480zx is deservedly considered one of the best professional displays, and our software allows you to keep dozens or even hundreds of such monitors in perfect accordance with each other in terms of displayed colors, brightness and contrast.
The source code for such a cartoon can occupy many hundreds and thousands of terabytes, and such information must be stored as reliably as possible. The loss of several models, finished scenes or fragments can cost a lot of time and money. To ensure reliable storage of the results of painstaking work, our best solutions were used: HP 3PAR StoreServ . We already wrote about this storage system last year, but, besides it, there are a number of products and technologies that I would like to mention.
This is the HP MSA P2000 hard drive array and our versatile storage and backup systems HP StoreAll Storage and HP StoreOnce Backup. Efficient, fast and reliable, they provided storage of finished materials, convenient installation work and flexibility in working with backups.
As you can see, HP equipment was used at all stages and in all possible aspects of creating a cartoon. From working on the characters themselves to animation and special effects. From rendering finished frames to final editing. From recording sound to creating complex sound scenes, taking into account the environment and the graphic scene. From displaying images on screens to storing all parts of the work throughout the entire life cycle of the product. Many sophisticated technologies provided professionals with simple and effective work in their difficult task: to accommodate eighty million hours in several years, and to receive ninety minutes of the finished film at the output.
UPD from Apr 22, 2012: We launched a special project on Habré, which we talked about in the zero post of this season. It is located at:habrahabr.ru/special/hp/commercial and there is a lot of interesting information, the promised feedback form, as well as a test for knowledge in the field of virtualization. We are waiting for you and your questions!
Technique and technology
The production of modern computer graphics takes a truly huge number of computing modules, consumes kilowatts of electricity with terrifying speed, drives tens of gigabits of traffic through networks. Each frame contains many millions of polygon triangles that form character models, terrain, objects, and all-all-all that you see. The most complex shader programs that recreate ultra-realistic water, fire and dust are responsible for creating special effects. Creating five minutes of video material takes many weeks of painstaking work, and despite the progress in both algorithms and application software, everything depends on the computing power of computers, network bandwidth and ruthlessly ticking time.
How did 90 minutes of the cartoon, which fit in eighty million render hours, fit into several years of development? The first sketches, working drawings, storyboards, ideas and characters appeared in 2005. In early 2007, DreamWorks began to work on ideas, reworked the script and characters, breathed new life into the project, and work on the graphics itself began in 2009. The development of models, textures, scenes, special effects and other materials took several years, the final rendering, film assembly and editing were carried out in 2011 and 2012.
DreamWorks has been working with HP for many years in software and hardware solutions to achieve the highest performance results, and this cartoon was no exception.
Models, animations, and special effects
The main "workhorse" of artists, animators, specialists in special effects and editing became Z820 workstations .
Two server Xeons, up to 512 GB of RAM, a multi-layered sandwich of professional video adapters, high-speed disk subsystems, a reliable cooling system - all for working with super-complex scenes in an interactive environment.
Sound and audio effects
Work with sound was carried out on the HP Z800 - a time-tested workstation equipped with an effective and silent cooling system.
Some sounds are born very simply, some as a result of complex and painstaking work, and the creation of a three-dimensional sound stage that will take into account what is happening not only on the screen, but also behind it, from all sides, with all the obstacles, the model of the environment and geometry space, its materials and their properties in the field of reflection and absorption of sound waves - the task is no less difficult than creating realistic looking wool, hair and skin. Professionals in the field of sound and special effects worked hard, and now you hear the roar of a prehistoric beast, the rustle of leaves, the sound of water and fire as if they really exist and are near you, but our workstations helped them with this.
4 cities - one company
It is difficult to imagine an office that would accommodate all DreamWork Animation employees and provide them with the opportunity to work on one project within the same room. Artists and animators are located not just in four different buildings, but in four different cities: Glendale, Redwood City, Las Vegas and Bangalore. It’s not easy to work on one project at such great distances, and working simultaneously, in real time, on one scene, one character and one frame seems to be a task beyond real. But the task is real, and most importantly, its solution is just as real as the very need for such technology.
HP Remote Graphics Software has combined three cities in the United States and one in India into one efficient work environment, providing people with everything they need to work together and be creative.
Awesome network technology that allows you to “be present” at another computer, hundreds and thousands of kilometers away, is a reality, and HP’s powerful network technologies have achieved minimal delays in video transmission and feedback.
Networking and HP FlexNetwork
Stability and uninterrupted operation of the network under the highest load was provided by the HP FlexNetwork architecture (there will be separate material about it a bit later). Work with remote employees, scalability, real-time transmission of HD video content, the ability to work with HP Remote Graphics Software technology on such a complex and highly loaded project is the merit of this interesting technology and HP hardware systems, which create the very “low level” that enables transmission data and traffic control.
HP Intelligent Resilient Framework (HP IRF)- a technology that works wonders out of the blue. It combines the flexible virtualization of server technologies, the management of traffic routes, the elimination of unnecessary entities and idle capacity, the distribution of loads and computing power, the scalability of network solutions and, most importantly, the use of well-known and open technologies as the basis. The result is a converged, simple, transparent network with a minimum of losses and a maximum of efficiency, easy to manage and the ability to easily expand existing solutions.
At the hardware level, network infrastructure and HP IRF were supported by switches HP 12518 , 5820 , 5800 and 6125G / XG .
Servers and render farms
The most delicious iron! DreamWorks used our eighth generation Blade systems, the HP ProLiant BL460c Gen8, as render servers .
Two eight-core processors, 16 slots for RAM, 10 Gigabit network controller that supports HP FlexFabric (one of the components of FlexNetwork ) - impressive power, and in one “rack” of such kids there can be more than a dozen.
It was they that made it possible to accommodate an impressive 80 million render hours per year of work, parallelizing the execution of tasks for rendering each frame.
Color, displays and printing
In the process of working on such projects, you inevitably have to use the help of printers, scanners and copiers. Hand-drawn storyboards, sketches of scenes, characters, objects, elements, landscapes and just scenes - all this is easier to keep on the wall in front of you, and not in your head or on the computer. I looked away from the monitor - looked at the illustration that interests you - continued to work. No alttabs or searches in the network folders.
As displays used products from the professional line of HP DreamColor . One of the most popular solutions was the HP LP2480zx monitor .
24 inches, 1920x1080, H-IPS-matrix and factory color calibration of each instance for the most complete and uniform disclosure of color space. It would seem nothing impressive, but the LP2480zx is not an ordinary monitor. It contains RGB LED backlighting that provides the most accurate control of the “white point” of the display, five basic preset profiles (sRGB, Adobe RGB, Rec. 601 (standard calibration for analog TV) and Rec. 709 (standard calibration for HDTV) and special color space for standard colors developed by DreamWorks). In addition, there is full support for specialized software and work with a colorimeter, support for ten-bit color space, work with a 200 Hz signal source.HP DreamColor LP2480zx is deservedly considered one of the best professional displays, and our software allows you to keep dozens or even hundreds of such monitors in perfect accordance with each other in terms of displayed colors, brightness and contrast.
Data storage
The source code for such a cartoon can occupy many hundreds and thousands of terabytes, and such information must be stored as reliably as possible. The loss of several models, finished scenes or fragments can cost a lot of time and money. To ensure reliable storage of the results of painstaking work, our best solutions were used: HP 3PAR StoreServ . We already wrote about this storage system last year, but, besides it, there are a number of products and technologies that I would like to mention.
This is the HP MSA P2000 hard drive array and our versatile storage and backup systems HP StoreAll Storage and HP StoreOnce Backup. Efficient, fast and reliable, they provided storage of finished materials, convenient installation work and flexibility in working with backups.
Summary
As you can see, HP equipment was used at all stages and in all possible aspects of creating a cartoon. From working on the characters themselves to animation and special effects. From rendering finished frames to final editing. From recording sound to creating complex sound scenes, taking into account the environment and the graphic scene. From displaying images on screens to storing all parts of the work throughout the entire life cycle of the product. Many sophisticated technologies provided professionals with simple and effective work in their difficult task: to accommodate eighty million hours in several years, and to receive ninety minutes of the finished film at the output.
UPD from Apr 22, 2012: We launched a special project on Habré, which we talked about in the zero post of this season. It is located at:habrahabr.ru/special/hp/commercial and there is a lot of interesting information, the promised feedback form, as well as a test for knowledge in the field of virtualization. We are waiting for you and your questions!
