How to create an operating system
According to the report of Ivan Ruzanov “Windows - how to create an operating system” from the conference “ Platform 2009 ”.
As antonms already wrote , reports of reports from the Platform became available. I looked at a few of them and want to share the one I liked most. Considering the busyness of an ordinary habr-user and the duration of the initial report recording (1 hour 20 minutes), below I give a detailed presentation of the material.bought / stole Windows NT. Of course, this is not so. To work on the new operating system, Microsoft invited Dave Cutler, who worked at DEC and was one of the developers of the DEC VAX VMS operating system . In 1988, the new operating system development team was only 12 people, among whom, along with Dave Cutler, were also former DEC employees who participated in the development of the DEC VAX VMS. And all the code for the new Windows NT was written by Microsoft.
The abbreviation "NT" stands for marketing as "New Technologies", but in the design documentation, it meant a completely different thing. The fact is that Windows NT was developed for the new Intel i860 processor , which was not yet released in 1988 . Its code name was"N10" ( NT en) .
The first version - Windows NT 3.1, was released 5 years later, in 1993. At this point, the team already had 250 developers.
Specifications → Design → Implementation → Testing → Delivery.
But such a methodology only works for small projects. For a product like Windows today, other methodologies are needed:
For Windows, as a product in general, the Product Cycle Model is used:
The whole Windows development process is built around a daily build:
Daily development cycle:
What is Windows written on?
Now, after the release (RTM) on Windows, there are 2 versions of the source code:
Thanks to everyone who read to the end =)
As antonms already wrote , reports of reports from the Platform became available. I looked at a few of them and want to share the one I liked most. Considering the busyness of an ordinary habr-user and the duration of the initial report recording (1 hour 20 minutes), below I give a detailed presentation of the material.
- Windows History
- Windows today
- Windows development
- Windows 7 kernel changes
- Windows Maintenance Updates / Fixes
Windows History
Few people know when the development of Windows NT began - more than 20 years ago, in November 1988. Rumor has it that MicrosoftThe abbreviation "NT" stands for marketing as "New Technologies", but in the design documentation, it meant a completely different thing. The fact is that Windows NT was developed for the new Intel i860 processor , which was not yet released in 1988 . Its code name was
The first version - Windows NT 3.1, was released 5 years later, in 1993. At this point, the team already had 250 developers.
Windows today
- 1 billion users
- 140 million lines of code (including test code and tools)
Windows code is very different. Some parts were written 20 years ago, some appeared only in the current version. For example, the Web Services on Devices (WSD) code in Windows Vista exists in its first version, the GDI code is at the final stage of its development and is almost unchanged, DirectX code is already well developed, but it is actively changing now. - 8000 developers
- 36 languages of localization
- 20 years of development
Windows development
20-30 years ago, only one Waterfall programming methodology was used. It is a sequence:Specifications → Design → Implementation → Testing → Delivery.
But such a methodology only works for small projects. For a product like Windows today, other methodologies are needed:
- Product cycle model
- Team Software Process
- " Extreme Programming "
For Windows, as a product in general, the Product Cycle Model is used:
- 3-4 months periods
- Inside the period there is a “waterfall”
- Requirements will change
- Opportunities will change
- Work schedule will change
- Project will change
- Users will change
- Release of intermediate versions (milestones, beta, CTP) for the masses of testers
- Release of internal assemblies with short cycles (1 day)
- Simplicity and reliability of design
- Personal and team code readings
- Unit tests
- Verification Tests
- Any intermediate assembly should be of high quality (what is written should work)
The whole Windows development process is built around a daily build:
- This is the pulse of the product.
- Development never stops
- Daily auto testing
- Early integration
- Developer Responsibility
- Apparent Product Condition
Daily development cycle:
- 15:00 - Changes to the source code control system allowed for integration
- Build 6 versions (Free / Checked - x86, x64, IA64)
- 18:00 - New versions are available for testing
- The new version is installed on several thousand workstations and servers for testing.
- Automated Stress Test
- 05:00 - Test reports are analyzed, failures are diagnosed
- 09:00 - Summary reports are automatically sent to teams
- 09:30 - Team Leadership Summit
What is Windows written on?
- C, C ++, C #, Assembler (x86, x64, IA64)
Assemblers are used to a rather limited extent in those situations where you cannot do without it. - Visual Studio, Source Insight, build, nmake
- Source Depot - source control system
- WinDbg, KD, NTSD - debuggers
Windows 7 kernel changes
The Windows 7 kernel has undergone the following changes:- Refactoring.
Why can't I remove the graphics subsystem in Windows?
The answer to this question from a technical point of view is that the graphics subsystem in Windows is not independent, it is part of the Win32 subsystem.
Windows 7 refactored many low-level components in order to break down dependencies. This will not be noticeable to users, only new Dlls will appear, for example,kernel32.dll is divided intokernel32.dll andkernelbase.dll .
This partition made it possible to allocate a minimal core called MinWin (20 megabytes per disk). - EFI support for x86 and x64 (as in Vista SP1)
Many manufacturers are trying to get rid of the BIOS in favor of EFI. - Booting from VHD (Virtual Hard Disk)
- Parallel device initialization and start of services
When loading Windows, it takes quite a long time to build a device tree. The PNP manager should poll bus drivers (PCI, USB, FireWire, etc.) for what devices they have. And most of the time the processor waits for the devices to respond (or not). After all, in order to identify devices on the bus you need to interview them. If they are, then they will answer, but if not, then you have to wait, and the processor is idle. Performing these tasks in parallel reduces load times. - Removing Dispatcher lock from the scheduler and PFN lock from the memory manager Over the
past few years, the processor clock speeds have not increased, and development is going towards increasing the number of parallel instructions both at the core level and at the system level (multicore). In this regard, a lot of work has been done to improve scaling.
The two hottest locks that were in the kernel, Dispatcher lock and PFN lock, were removed.
Dispatcher lock was used by the scheduler when changing the state of threads. This lock was deleted, and the status of the "wait" thread was divided into several:- Waiting: In progress
- Waiting: Completed
- Pending: Canceled
- Support for 256 logical processors
Previously, in Windows, a machine word was used as an affinity mask. This was due to the fact that it was so easy to find free processors - each bit represents a processor. Accordingly, 32 logical processors were supported in a 32-bit system, and 64 in a 64-bit system.
In Windows 7, as a result of the transition to the affinity mask segment model, it became possible to support 256 logical processors. Processors began to be grouped into groups / segments. Each group can contain up to 64 processors. The result is backward compatibility, old programs only see processors in one group, and new programs that use new interfaces work with all processors in the system. - Improved energy saving: disabling processor sockets Today, there is a serious problem of energy saving not only for laptop owners, but also for data center owners. In the United States, 2% of electricity is consumed by computer data centers. Many of them shut down part of their servers during low user activity (weekends).
It was found that it is much more profitable to disconnect the entire processor socket than one core per several, because in this case, you can disable the entire socket support infrastructure (memory controller).
Windows Maintenance Updates
Previously, updates were often cumulative (cumulative). This meant that if the erroneous code was contained in an early update of the component, then later versions would also contain this code. But not all users need all the updates, they have a different configuration.Now, after the release (RTM) on Windows, there are 2 versions of the source code:
- RTM GDR (General Distribution Release)
Includes those few changes that are for everyone. Mostly security fixes. - RTM LDR (Limited Distribution Release)
During the installation of the update, the Windows Update client selects the branch it needs and installs the code from it.
Create a security update
Work on creating a security update begins with a vulnerability discovery. There are tons of different detection methods — internal security teams, security partners, developers. When a vulnerability is discovered, 2 parallel processes begin:- Patch development for all platforms
- Search for “options”
Large-scale search for similar vulnerability options on all platforms. Search not for an identical code, but for a similar one.
- Manual and automatic component testing
- Automatic testing of file format distortions, network components, etc. (more than a million options)
- Testing the entire system, including backward compatibility testing
Thanks to everyone who read to the end =)