Unity 3D Test Automation Team
- Transfer
Hello everyone, my name is Elvis Alistar, I have been working with Unity for over 2 years. I am the head of the Testing Engineers (IT) team, which is part of the QA Unity department. As the name implies, we are a team of software developers who love to test. In this post, I would like to tell you why it is so important for Unity to have a team of experienced developers focusing on test automation.
Unity is a fast-growing company with more than 450 employees from 50 countries operating in 27 locations worldwide. The large distribution of our company means that we need to work with developers from different time zones, different cultures. We also have more than 2.5 million registered developers, and we often interact with them through our forums, the feedback website, alpha and beta groups, and by processing some error reports we receive from them. Close collaboration with our development teams means that good communication skills are becoming key.
Unity supports the 12 largest platforms and several smaller ones. Users can also use three different programming languages to write game code. Add to this the presence of runtime tests, graphic tests, integration tests, unit tests, UI tests, performance tests, etc. - and the phenomenal number of different combinations that our test automation infrastructure must support and cope with.
Take our runtime tests, for example. We now have over 1,300 unique Runtime API tests. Over 13,000 tests are performed in a single, automated test suite assembly. At the same time, we work in more than 100 development areas, which means that there are about 500,000 control points that are carried out on our build farm every day. IT needs to write, maintain, optimize and improve them daily.
Just two years ago, we had only a few test platforms that were very poorly documented, we had very few tests, and no one supported or improved the platforms or test code. An IT team was formed to take on the support of these platforms, tidy up, document and optimize all our tests. We also made sure that all developers know how to use these platforms and that they can easily create new tests using them.
The code base in Unity is growing very fast, and we need to be sure that the product works very stable with every new release that we release. That is why we pay great attention to Unity testing automation. Just to see the prospect, we now have about 2.2 million lines of code, of which approximately 400,000 are test code, broken into more than 7,000 files. This ensures that we can maintain high quality, avoid the appearance of new regressions and make sure that known errors that have been fixed will never appear in our product again.
Now what I call a task!
We are a team of 8 developers working on tools, platforms and automation functions, with a great emphasis on the latter. Two of us are in Copenhagen (Denmark), one in Silicon Valley (USA) and five in Odessa (Ukraine). Our job is to find ways to improve the overall workflow for all Unity developers by improving automation on the build farm, creating regression sets to prevent pop-up errors, and promoting the use of the platform by testers and developers.
Unity is a very complex product, and to work more efficiently, we divided it into various areas: Scripting, Lightmapping, 2D, Animation, etc. Each IT is responsible for writing, maintaining and reviewing tests in one or more areas. New ITs usually start with one area and work on it until they become proficient in using our processes, platforms and strategies. Once they gain enough knowledge and earn trust they will gain more areas. All ITs ultimately work to improve our existing platforms or add new ones. We have 7 different kinds of automation platforms at Unity, and IT should be able to work with all of them and help those who need to work with them.
Our work requires close cooperation with other teams in the QA structure and teams in our R&D department. We need to keep in touch with developers and software testers (TVEs) to make sure that we know who is working on this feature, which parts have been developed and tested, and what should be automated. We all participate in the main QA tests in each release, even if they do not include test automation: exploratory testing, full testing, and pre-release testing (see the Testing Reference in the article “Testing Unity 4.0” on our blog).
Automation work is rarely tied to a specific Unity release, and it gives us more freedom and flexibility to focus on what we want to improve. It also allows us to visit our various offices to help developers or to work with local testers. We attend large testing conferences and speak at them whenever possible. We participate in our now famous HackWeeks, where we can experiment with any ideas and projects that we have, no matter how strange they may seem.
As ITs, we are required to have extensive knowledge in programming, object-oriented design and test automation (unit testing, TDD, high-level tests, etc.). We need to disseminate information about testing platforms and methods, maintain feedback in code analysis, and be able to work well with developers. We value clean code and the development of best practices.
In Unity, we also need to be able to work effectively with many of the tools that we use internally, such as FogBugz, QMetry, TeamCity, RhodeCode, Sikuli, NUnit, Moq, Unittest ++, Atomiq, etc.
That's it! Being IT at Unity is both great and beneficial, and the variety of tasks that we solve is simply breathtaking.
There is a task! Are you ready for her?
Then get a job with us!
Translator note: I will be glad to all comments and advice regarding improving the quality of translations.
Task
Unity is a fast-growing company with more than 450 employees from 50 countries operating in 27 locations worldwide. The large distribution of our company means that we need to work with developers from different time zones, different cultures. We also have more than 2.5 million registered developers, and we often interact with them through our forums, the feedback website, alpha and beta groups, and by processing some error reports we receive from them. Close collaboration with our development teams means that good communication skills are becoming key.
Unity supports the 12 largest platforms and several smaller ones. Users can also use three different programming languages to write game code. Add to this the presence of runtime tests, graphic tests, integration tests, unit tests, UI tests, performance tests, etc. - and the phenomenal number of different combinations that our test automation infrastructure must support and cope with.
Take our runtime tests, for example. We now have over 1,300 unique Runtime API tests. Over 13,000 tests are performed in a single, automated test suite assembly. At the same time, we work in more than 100 development areas, which means that there are about 500,000 control points that are carried out on our build farm every day. IT needs to write, maintain, optimize and improve them daily.
Just two years ago, we had only a few test platforms that were very poorly documented, we had very few tests, and no one supported or improved the platforms or test code. An IT team was formed to take on the support of these platforms, tidy up, document and optimize all our tests. We also made sure that all developers know how to use these platforms and that they can easily create new tests using them.
The code base in Unity is growing very fast, and we need to be sure that the product works very stable with every new release that we release. That is why we pay great attention to Unity testing automation. Just to see the prospect, we now have about 2.2 million lines of code, of which approximately 400,000 are test code, broken into more than 7,000 files. This ensures that we can maintain high quality, avoid the appearance of new regressions and make sure that known errors that have been fixed will never appear in our product again.
Now what I call a task!
Team organization
We are a team of 8 developers working on tools, platforms and automation functions, with a great emphasis on the latter. Two of us are in Copenhagen (Denmark), one in Silicon Valley (USA) and five in Odessa (Ukraine). Our job is to find ways to improve the overall workflow for all Unity developers by improving automation on the build farm, creating regression sets to prevent pop-up errors, and promoting the use of the platform by testers and developers.
Unity is a very complex product, and to work more efficiently, we divided it into various areas: Scripting, Lightmapping, 2D, Animation, etc. Each IT is responsible for writing, maintaining and reviewing tests in one or more areas. New ITs usually start with one area and work on it until they become proficient in using our processes, platforms and strategies. Once they gain enough knowledge and earn trust they will gain more areas. All ITs ultimately work to improve our existing platforms or add new ones. We have 7 different kinds of automation platforms at Unity, and IT should be able to work with all of them and help those who need to work with them.
Our work requires close cooperation with other teams in the QA structure and teams in our R&D department. We need to keep in touch with developers and software testers (TVEs) to make sure that we know who is working on this feature, which parts have been developed and tested, and what should be automated. We all participate in the main QA tests in each release, even if they do not include test automation: exploratory testing, full testing, and pre-release testing (see the Testing Reference in the article “Testing Unity 4.0” on our blog).
Automation work is rarely tied to a specific Unity release, and it gives us more freedom and flexibility to focus on what we want to improve. It also allows us to visit our various offices to help developers or to work with local testers. We attend large testing conferences and speak at them whenever possible. We participate in our now famous HackWeeks, where we can experiment with any ideas and projects that we have, no matter how strange they may seem.
Skills and Installation
As ITs, we are required to have extensive knowledge in programming, object-oriented design and test automation (unit testing, TDD, high-level tests, etc.). We need to disseminate information about testing platforms and methods, maintain feedback in code analysis, and be able to work well with developers. We value clean code and the development of best practices.
In Unity, we also need to be able to work effectively with many of the tools that we use internally, such as FogBugz, QMetry, TeamCity, RhodeCode, Sikuli, NUnit, Moq, Unittest ++, Atomiq, etc.
That's it! Being IT at Unity is both great and beneficial, and the variety of tasks that we solve is simply breathtaking.
There is a task! Are you ready for her?
Then get a job with us!
Translator note: I will be glad to all comments and advice regarding improving the quality of translations.