May 10, 2013 at 10:16Evaluation of the effectiveness and security of authentication mechanisms
In a previous article , I gave my classification of authentication mechanisms. Now I will share the methods for evaluating and comparing them.
In total, 4 main indicators of the authentication system can be distinguished:
1. Costs for installation and maintenance.
2. Efficiency.
3. Reliability.
4. Security.
But first, remember what is meant by the authentication process?
It can be represented as a function that calculates the degree of similarity of the images of authenticators (a) available in the system (S) and the user (P), and checks them for occurrence in the confidence interval. In case of entry, authentication is considered successful.
Why exactly the images (a '), and not the authenticators themselves?
An authenticator can be: password, e-token, iris, etc. and in most cases it is stored either by the user or in the system, and on the opposite side only his image is stored, for example, a password hash, a fingerprint or a picture of the iris. And for comparison, the first side also does not transmit the authenticator itself, but only its image. These images can coincide completely, in the case of using passwords, or can be very similar when using biometrics. The similarity check method differs depending on the authentication method, as well as the boundaries of the confidence interval - k min and k max, which are in the range from zero to one. With password or hardware authentication, both boundaries of the confidence interval are usually equal to one, and when using biometrics or security questions, the interval often has a small range in the vicinity of the unit. Moreover, k max - is not always equal to unity. In the case of the use of authenticators having a dynamic nature, for example, voices or handwriting, the complete coincidence of authenticator images speaks more about its fake.
Now let's move on to the main indicators.
- a cost indicator, which includes 
- time, effort or funds of the system administrator for its installation and maintenance, and the user to create or change their account, divided by a given coefficient 
, which by default is the average cost for this class of systems.
This is perhaps the most difficult of the indicators, because it’s not entirely clear how to correctly consider the costs of installing and maintaining the system as an administrator? Take them in whole or per user?
It is possible to calculate= ((system installation costs + administrator’s maintenance costs per year) / number of users) + user costs per year;
In addition, you can add factors for different types of costs.
- an efficiency indicator that describes the time, effort, or other costs required by the account holder each time he or she authenticates in the system, where 
are the costs of entering and analyzing the authenticator, and 
by default is the average cost of entering and analyzing the authenticator for this class of systems.
This is a much more tangible parameter. Authenticator input time, for example, password input time, connecting a USB flash drive, voice recording and their further analysis can be easily counted, and do not have large fluctuations.
- a reliability indicator equal to the probability that the account owner will be able to successfully authenticate. The higher the level of reliability, the lower the indicator of overall security.
k min is taken from the description of the authentication process. If for authentication you need to answer a few simple security questions, for example 7 out of 10, then the reliability will be 30%. Accordingly, the higher the reliability of the access function, the easier it is for the user to authenticate. Therefore, with a fall in the priority of using the authentication mechanism (from the primary through the backup to the mechanism of last resort), the reliability of the mechanism should increase.
- a security indicator describing the total costs of the attacker required for successful authentication (false authentication) using a certain type of attack, where 
is the cost of creating one authenticator and checking it, 
is the skill level of the attacker, and 
is the complexity of the authenticator. By default, an attack is a complete enumeration of all possible authenticator options. For other types of attacks, the formula may differ from this.
Here, too, everything is generally clear. If the mechanism does not have obvious vulnerabilities, then the only way to hack is an exhaustive attack. And here it all depends on how quickly an attacker can, for example, generate passwords and verify them.
Odds are added so that the indicators can be made dimensionless. In addition, when comparing several authentication mechanisms by setting the necessary values for the coefficients, one can obtain orders of numbers more convenient for perception.
The difficulty of calculating these indicators lies in the fact that costs can include immediately time, money, and specific resources.
Two derivatives can be created from these key indicators:
5. General security.
6. Overall efficiency.
- general security of the authentication mechanism, against a certain type of attack, in this case, against a brute-force attack.
- The overall effectiveness of the authentication mechanism.
1. The more authentication mechanisms used in your system, the better for the user and worse for you. The golden mean is the presence of three mechanisms: the main, reserve and last resort.
2. The primary criterion for selecting and configuring authentication mechanisms used in the system is general security. It should be approximately the same for all mechanisms, while with the decrease in the priority of using the authentication mechanism, its reliability and security should increase.
3. When choosing between mechanisms belonging to the same class and having the same security, choose the one with higher overall efficiency.
4. It makes sense to compare only mechanisms that belong to the same class, or that have one common characteristic.
The comments welcome constructive criticism, clarifications, and case studies.
In total, 4 main indicators of the authentication system can be distinguished:
1. Costs for installation and maintenance.
2. Efficiency.
3. Reliability.
4. Security.
But first, remember what is meant by the authentication process?
It can be represented as a function that calculates the degree of similarity of the images of authenticators (a) available in the system (S) and the user (P), and checks them for occurrence in the confidence interval. In case of entry, authentication is considered successful.
Why exactly the images (a '), and not the authenticators themselves?
An authenticator can be: password, e-token, iris, etc. and in most cases it is stored either by the user or in the system, and on the opposite side only his image is stored, for example, a password hash, a fingerprint or a picture of the iris. And for comparison, the first side also does not transmit the authenticator itself, but only its image. These images can coincide completely, in the case of using passwords, or can be very similar when using biometrics. The similarity check method differs depending on the authentication method, as well as the boundaries of the confidence interval - k min and k max, which are in the range from zero to one. With password or hardware authentication, both boundaries of the confidence interval are usually equal to one, and when using biometrics or security questions, the interval often has a small range in the vicinity of the unit. Moreover, k max - is not always equal to unity. In the case of the use of authenticators having a dynamic nature, for example, voices or handwriting, the complete coincidence of authenticator images speaks more about its fake.
Now let's move on to the main indicators.
Installation and Maintenance Costs
- a cost indicator, which includes - time, effort or funds of the system administrator for its installation and maintenance, and the user to create or change their account, divided by a given coefficient , which by default is the average cost for this class of systems. This is perhaps the most difficult of the indicators, because it’s not entirely clear how to correctly consider the costs of installing and maintaining the system as an administrator? Take them in whole or per user?
It is possible to calculate
= ((system installation costs + administrator’s maintenance costs per year) / number of users) + user costs per year; In addition, you can add factors for different types of costs.
Efficiency
- an efficiency indicator that describes the time, effort, or other costs required by the account holder each time he or she authenticates in the system, where are the costs of entering and analyzing the authenticator, and by default is the average cost of entering and analyzing the authenticator for this class of systems. This is a much more tangible parameter. Authenticator input time, for example, password input time, connecting a USB flash drive, voice recording and their further analysis can be easily counted, and do not have large fluctuations.
Reliability
- a reliability indicator equal to the probability that the account owner will be able to successfully authenticate. The higher the level of reliability, the lower the indicator of overall security. k min is taken from the description of the authentication process. If for authentication you need to answer a few simple security questions, for example 7 out of 10, then the reliability will be 30%. Accordingly, the higher the reliability of the access function, the easier it is for the user to authenticate. Therefore, with a fall in the priority of using the authentication mechanism (from the primary through the backup to the mechanism of last resort), the reliability of the mechanism should increase.
Security
- a security indicator describing the total costs of the attacker required for successful authentication (false authentication) using a certain type of attack, where is the cost of creating one authenticator and checking it, is the skill level of the attacker, and is the complexity of the authenticator. By default, an attack is a complete enumeration of all possible authenticator options. For other types of attacks, the formula may differ from this. Here, too, everything is generally clear. If the mechanism does not have obvious vulnerabilities, then the only way to hack is an exhaustive attack. And here it all depends on how quickly an attacker can, for example, generate passwords and verify them.
Odds are added so that the indicators can be made dimensionless. In addition, when comparing several authentication mechanisms by setting the necessary values for the coefficients, one can obtain orders of numbers more convenient for perception.
The difficulty of calculating these indicators lies in the fact that costs can include immediately time, money, and specific resources.
Two derivatives can be created from these key indicators:
5. General security.
6. Overall efficiency.
General security
- general security of the authentication mechanism, against a certain type of attack, in this case, against a brute-force attack.Overall performance
- The overall effectiveness of the authentication mechanism.Recommendations for choosing and configuring authentication systems
1. The more authentication mechanisms used in your system, the better for the user and worse for you. The golden mean is the presence of three mechanisms: the main, reserve and last resort.
2. The primary criterion for selecting and configuring authentication mechanisms used in the system is general security. It should be approximately the same for all mechanisms, while with the decrease in the priority of using the authentication mechanism, its reliability and security should increase.
3. When choosing between mechanisms belonging to the same class and having the same security, choose the one with higher overall efficiency.
4. It makes sense to compare only mechanisms that belong to the same class, or that have one common characteristic.
The comments welcome constructive criticism, clarifications, and case studies.