Oculus Rift helmets did not work for almost a day due to an expired code signing certificate
On March 7, 2018, the owners of the Oculus Rift virtual reality helmets were in an extremely unpleasant situation. One day, their gadgets suddenly stopped working , giving the error "Can't Reach Oculus Runtime Service".
As it turned out, the problem arose due to the expiration of the code signing certificate for the OculusAppFramework.dll dynamic library , which is part of the Oculus Runtime Service. She just didn't boot. The file indicates the expiration date of the certificate:
Valid to: Wednesday, March 7, 2018 01:00:00 PM
In this case, Oculus employees showed complete incompetence. It's not even that they forgot to renew the certificate, but that the file was signed incorrectly. They neglected the control signature from the timestamping server. If they put this signature, then the files would be signed forever, because the signature from the time stamp server confirms the validity of the certificate at the time of signing the file, so you can not check the date on the certificate in the future. But since Oculus neglected this signature, the library stopped loading at the moment the certificate expired, that is, 03/07/2018 at 01:00:00 PM.
Signing drivers has become mandatory since Windows 10 build 1607 . Unsigned drivers simply do not load, with some exceptions (for example, if the Secure Boot option is disabled or if the file is signed with a cross certificate issued before July 29, 2015).
As the investigation showed, the signature from the time stamp server disappeared after the upgrade of Oculus from version 1.22 to 1.23, which took place a little more than a month ago. The cause of the incident is not yet clear. Versions are being expressed that the signature might not have been set during automatic assembly if the time stamp server was down at this point in time.
Within a day, the company released a patch replacing the OculusAppFramework.dll file in the system. To run the patch in the Windows operating system, you need to disable the antivirus (in Windows Defender, just click on the More info link and click the Run Anyway button ). After installing the patch, Oculus Runtime Service is updated from the server - and the helmet works again.
The patch appeared on the morning of March 8th. That is, because of this, without downplaying the slovenliness of the company's employees, all Oculus Rift helmets in the world went out of order for almost a day. These are the consequences of one wrong code signing certificate.
The co-founder of the company, Nate Mitchell, made a public apology and promised to give out loans to the victims in the Oculus Store in the amount of $ 15.
At first, loans were handed out only to those who specifically apply for them. The fact is that VR helmets still loaded in Oculus Home mode , where you can perform some actions. So not all users formally became victims. But later a message appeared that within seven days loans should be credited to everyone who installed the update.
What conclusions can be drawn from this story?
Thousands of users were affected. Due to its own oversight, the company itself suffered damage if it really accrued a $ 15 loan to a significant part of Oculus Rift users. The company was lucky that it did not receive lawsuits from large customers - after all, Oculus VR systems are also used in the corporate sector: for example, for presentations, promotions, etc. One of the customers said that on March 6 their company held a large presentation for major brand. If the problem with the certificate happened a day earlier, then the event would have to be canceled.
Another victim saysthat their startup has been developing software for training surgeons in the VR environment for several months. Last week, they were preparing to make a presentation at a large medical conference, but the Oculus Rift helmets went down on the morning of the conference day . Fortunately, one company programmer quickly figured out the problem - and found out that you can run the program if you roll back the Windows system clock a couple of days ago.
A single expired certificate could result in more serious financial losses for Oculus than a $ 15 loan for all affected users. One can imagine that VR helmets and other IoT devices are ubiquitous. For example, they will be used in real surgical operations - and then all of a sudden all at once fail at the same time due to a similar software error.
Inattention to the validity of the code signing certificate is entirely Oculus's fault, and no one else. If you forget to attach the signature from the time stamp server, then the signed files actually turn into a "time bomb" that will explode as soon as the certificate expires. For example, GlobalSign issues code signing certificates for 1, 2, and 3 years.
Someone may ask why you should install this “time bomb” in your software at all, that is, why sign the code. But there is no other way: this is Microsoft's requirement for certain categories of files. If the executable code for Windows is signed, then at almost any time you can "prohibit" its execution on all computers. This is a compromise between freedom and security. In this case, the choice has been made in favor of security. So the blame for such incidents in a sense lies with Microsoft, which created a “single point of failure” to protect against code injection.
We can assume that in the future such incidents will occur more often, since verification of the code signature becomes more thorough due to security risks, and also because operating system developers want to get a percentage of profit from the sale of programs through proprietary program catalogs - and encourage all developers to sign the code. For Microsoft, this is a potential source of billions of dollars of additional revenue, which it was previously deprived of.
Oculus is not distributed through the Windows Store and does not pay Microsoft a 30% deduction (at least not yet). But for system drivers, it is required to implement a code signature to protect against injection and guarantee the safety of the original files. Although in the end, Microsoft probably expects to attract both Oculus and all other developers in its Windows Store.
One way or another, but the developers have almost no choice. I have to use certificates. But at the same time, you need to be careful to avoid incidents like Oculus.