Password Evolution: A Guide to Authentication in the Modern Age

Original author: Troy Hunt
  • Transfer
It all started simply: you have two character sets (username and password) and someone who knows both can log in. Nothing complicated.

However, the ecosystems in which they functioned were also simple — say, a time-sharing system from MIT , which is considered the first computer system to use passwords.



But this state of affairs developed in the sixties of the last century, and since then much water has flowed. Until the last couple of decades, we had a very small number of accounts with a limited number of connections, which made the spectrum of threats narrow enough. Only those who were in close proximity could harm you - that is, people who were able to directly, physically gain access to the system. Over time, they were also joined by remote users who could connect via telephone, and the range of threats expanded. What happened after that, you yourself know: more relationships, more accounts, more intruders and more data leaks, especially in recent years. The initial scheme with a simple reconciliation of characters no longer seems such a brilliant idea.

I wrote a couple of months agoabout reusable passwords, throwing in of registration data and that another billion entries were added to the Have I been pwned service database . Things are as follows: billions of access details lie and wait for some villain to start using them to hack any site he likes. This poses a very interesting question for us: how to protect ourselves in such circumstances? I mean, you’re trying to keep the online system afloat, and some kind of outside have working data from the accounts of some users - how can you forbid him to enter the system? A simple juxtaposition of characters from the sixties is not enough.

But the evolution of authentication does not boil down to an infinite expansion of the bases for throwing in credentials - the login process has changed in many other respects. In some cases, this led to the fact that everything turned upside down, and those truths about creating and managing accounts that were once considered capital, are now irrelevant. Nevertheless, we can see how some organizations apply outdated schemes to modern threats today. This post aims to analyze this discrepancy and explain how to actually design this critical part of the system in our time. I hope someday it will become the resource where well-wishers will send companies that say: "We do this stupidity for the sake of security."

Listen to what the government (and smart IT companies) say


I’ll start with links to other sources, because recently a lot of good materials on this topic have appeared on the Web, and I will refer to them. I want to lay it all at once, to make one point clear: most of the recommendations, which will provide further - this is not just my personal view of the problem, and a direct quotation of organizations such as the National Institute of Standards and Technology (National Institute of Standards and Technologies, or NIST). In fact, their work Digital Identity Guidelines , which came out literally last month, can perhaps be considered the impetus that prompted me to take up this post - there were so many interesting things.

National Cybersecurity CenterThe UK National Cyber ​​Security Center is another great resource that I’ll use. They regularly publish very informative articles on the topic of security, and present one of the rare examples of a government agency that really "thinks" in the IT field.

I will also refer to Microsoft's Password Guidance study by the Identity Protection team. On its first page it says that Microsoft is in a “unique position to understand the role of passwords in account hijacking,” because it survives 10 million attacks every day, so these guys definitely have enough experience! This is a very practical guide, compiled by people who have clearly thought through how to secure their online accounts.

I am sure that there are other interesting materials, and I will be glad if you share your findings in the comments. Here I have listed only a few sources, further down the text you will find references to many others. Well, let's get started!

The authentication process should not be reduced to switching between two states


First, let's look at the authentication process from a more philosophical point of view: until you log into the system, you do not have access to any of its components, then you go in and get access to everything that is written in your rights. No gray zone - either this way or that.

One of the remarkable trends that we have been observing recently in the information security sector is the differentiation of threat and confidence levels, that is, the realization that in some scenarios related to potential risk (such as logging in), we feel more trust in the user, and in some - less. For example: instead of allowing a certain number of login attempts, let's say five (noticed that for some reason the number is always odd?), And then just block the account, it might be worth adding a step with captcha after three. Thus, we show that the level of trust in the user has decreased and we are forced to take additional measures to make sure that the system is not subjected to an automated attack. This is a serious obstacle for bots, and a slight inconvenience for users, in addition,

Continuing the thought, is it worth giving users access to all the features right away when they log in successfully? Suppose they didn’t enter the password correctly the first time and came from a browser that they had not used before, and from another country — should they give them unlimited access to everything in this case? Or is it better to limit ourselves to the most basic functions and require proof that the specified address really belongs to the user before giving him the opportunity to carry out some serious operations?

With the help of these simple examples, I try to make readers think that they can act smarter than traditional binary systems, which still prevail in most services, allow. This topic will continue to develop in the text.

The longer, the better (usually)


Now let's delve into the technical component and start with a simple one: what you see below is not the best policy for creating passwords.



The first sentence in the pop-up window (“Password must be 8-10 characters long”) is probably the most common of the erroneous anti-patterns for creating passwords: a small specified password length. This kills password managers (we will dwell on them later), it kills passphrases and, as a result, it kills usability. Speaking of usability: I took the screenshot from my last year’s article on how we make mistakes in the most basic things , and, in addition to the ineffective Etihad policy (adopted, incidentally, “for the sake of security” ), I describe in it a case where Paypal actually blocked my access to my account thanks to a similar policy.

In addition to the problems mentioned above, a low limit on length often inspires people to suspect that password storage is poorly organized. After cryptographic hashing, all stored passwords have the same fixed length, so such arbitrary restrictions may mean that passwords are stored in plain text and the column can only accommodate ten characters. Of course, the reason may be different, but, you understand, such assumptions are not unfounded.

What then is the length limit to be set? “Nothing” is the wrong answer, because beyond a certain border you will have other problems. For example, if the size exceeds four megabytes, you will not meet the default request size in ASP.NET . Here's what NIST says about this:

Verifiers must allow the input of any secret code up to 64 characters long at the choice of the subscriber.

Not a single normal person, registering on a site with a password length limit of 64 characters, will say: “What kind of crap they have with security, I was not even allowed to make a password longer than 64 characters.” But just in case, you can set a limit of 100 characters. Or pick up the idea of ​​NIST, and set a maximum of 256 characters - what's the difference, anyway, after hashing, everything will be aligned.

NIST points to another important, although not so lying, point on the surface:

Truncation of the secret code is not allowed.

These are, in fact, the most basic things: do not specify a short length and do not truncate the end characters of the password proposed by the user. At the worst, companies that advocate the legitimacy of such a practice should add: “We know that this is not possible, we have problems with heredity, we will put this on the list for urgent revision.”

All characters are special (but optional)


I want to consider two aspects of using special characters. I'll start with this:



Usually, some characters in passwords are not allowed. This rule is introduced to protect against possible attacks. Say, angle brackets can be used in XSS attacks, and apostrophes can be used when injecting SQL code. But such arguments show that there are serious flaws in the site’s security profile. First of all, passwords should never be displayed on the interface due to the threat of cross-site scripting; secondly, passwords should not be sent to the database without hashing, after which only alphanumeric characters remain. And of course, even if you break these rules by displaying passwords in the interface and saving them in the database as plain text, you still have the encoding during output and parameterization.

NIST expresses this very clearly - do not do this:

Secret codes should allow the use of any printed ASCII characters [RFC 20], including a space. Unicode characters [ISO / ISC 10646] must also be accepted .

It is worth noting their reservation regarding Unicode characters - there were precedents when developers imposed a ban on characters quite acceptable from the point of view of the language, simply because they allegedly went beyond the framework of "normal" ones. On the other hand, of course, there are also passwords like this:



Well, if someone really wants to put the first couple of Letters and Forgets from Frozen written down with emoticons - please!

Another aspect that I wanted to touch upon is also mentioned by NIST:

Verifiers should not introduce additional rules for composing a secret code (for example, require the use of different types of characters or prohibit the introduction of identical characters in a row) .

Wait, how so ?! That is, people do not have to include capital, lowercase letters, numbers and symbols in the password ?! This is so contrary to the generally accepted approach that you have to sit down and think logically in order to understand that they are right. As an illustration: I recently wrote about how evaluating a password with a system as “good” or “bad” pushes people to make wrong decisions . The basic message was this: a purely mathematical approach does not help us come up with good passwords. Such requirements allow passwords such as “Password!” And cut off passphrases just because they do not contain capital letters.

In the document I mentioned above, Microsoft echoes the recommendations from NIST:

Refuse password requirements.

If there are requirements for password complexity, the situation usually unfolds like this: people first try to enter something primitive, and then correct the characters until they include the very minimum of required components. Microsoft reveals this problem as follows:

For the most part, people resort to the same patterns (the first letter is a capital letter, a special character, or two numbers at the end). Cyber-crooks know this, therefore, searching through the dictionary, they include all replacements made according to standard schemes ("$" instead of "s", "@" instead of "a", "1" instead of "l" and so on) .

As a result, the password will still turn out bad, and most likely the user has already used it somewhere before. All that can be achieved by these measures is to pat the user’s nerves without any result!

Tips In no case!


According to my personal observations, now tips are far from being as popular as they were once. This idea started in “listen, people constantly forget passwords, let's help them remember!” And resulted in the fact that when creating an account the user was able to enter not only a password, but also a hint to it. The problem is that this hint will be shown to unauthorized users - only at this stage it is needed. Another problem is that this information is entered by the user and, accordingly, there will most likely be complete horror.

Adobe kept password hints in its database, which was made publicly available in 2013. To demonstrate how bad everything is, I looked at this database and decided to share individual samples:

  • my name
  • adobe
  • as usual
  • password
  • e-mail

They were all stored in plain text, so you can imagine what happened when the system was hacked. It's clear why NIST thinks this is a bad idea:

Secret code verifiers should not allow subscribers to leave a “hint” that will be available to an unauthorized person when they try to log in.

As I said, now you’ll rarely see hints in online systems, but just in case you thought about something like this - don’t think about it!

Love Password Managers


I’ve been saying for a long time about the value that password managers have in themselves - already since the beginning of 2011, when I wrote in one of the articles that the only reliable password is one that you don’t know for memory . The idea is simple and can be stated literally in a few points:

  1. We know that passwords must be “strong”, that is, they must be difficult to guess or calculate by exhaustive search. In other words, the more characters and the more randomly chosen, the better.
  2. We know that it is impossible to use one password several times: if one service is hacked, then user accounts on other resources will be at risk. The problem with the registration data stuffing, which I mentioned above, is precisely this.
  3. People cannot invent and memorize strong, unique passwords for each service, relying only on their memory.


Now someone will object that using a password manager means collecting everything in one place, and he will be right: if someone gets into this place, you will have big troubles. But this is a very rare phenomenon compared with cases of hacking of individual services and subsequent leakage of account data. Moreover, as I recently said, password managers do not have to be perfect, it’s enough to be better with them than without them .

But this post was not written as a guide for users to create good passwords, it is intended for those who create services. Accordingly, no matter how personally you relate to password managers, this is not worth doing:



Good afternoon! We do not recommend using a password manager, your device can be hacked.

Such shortsightedness is common, I can immediately find a dozen of these tweets. They generally do not take into account the three points that I touched on above, and in fact say something like this: “Hey, create passwords that are easy to remember, they will, of course, be weak and most likely you will repeat them from account to account, but it’s necessary for the sake of security. ”

NCSC plainly talks about this problem: in the infographics for Password Guidance: Simplifying Your Approach , they have this fragment:



Create the ability for users to store passwords safely

In other words, do not break password managers or make such statements as in the tweet above. But the NCSC goes even further and offers the following recommendation, clearly with the expectation that organizations enable their employees to create passwords and manage them without risking security:

You must also provide appropriate password storage equipment with a level of protection consistent with the confidentiality of the information you seek to protect. These storages can be either material (for example, safes), or technical (for example, special programs for managing passwords) or combine aspects of both. It is imperative that your organization provides an authorized mechanism to help users control passwords: this eliminates the need to resort to unreliable “hidden” methods to deal with the abundance of passwords.

Think about the environment in which you are currently working. Are there any criteria for password strength? Are there annual trainings, or maybe posters hanging on the walls urging you to create unique, complex passwords? Of course, some kind of control and instruction related to passwords is definitely present, but do they provide you with an “authorized mechanism” that would help achieve your goals? Most likely not, and I say this with such confidence because I often ask this question when I conduct trainings in various organizations. It's amazing how often you see such a mismatch.

Let them insert passwords


Using the password manager is closely related to the ability to insert passwords into the input field. There are many password managers who fill in the fields automatically, but there are also cases when you still have to use the insert or the service blocks attempts to enter the password not manually (this can be easily verified using JavaScript). The result is the following:



In 2014, I wrote about the “cobra effect” that occurs when paste is not allowed in the password field. I explain the meaning of this term in detail in the article by reference, but if to summarize briefly: the cobra effect manifests itself when an attempt to solve the problem only exacerbates it. When the site blocks the ability to insert a password to ensure security, this forces some users to simplify passwords until they become so primitive that it is easy to type them.

NCSC support me here :



They even introduced a special term for this anti-pattern - SPP (Stop Pasting Passwords, “Stop inserting passwords”) and debunk popular myths about the risks associated with pasting passwords. They mention my article, which spoke about the effect of cobra - it's nice to get such a protection from the British government!

NIST joins the NCSC position, as evidenced by their statement:

Verifiers should allow the person undergoing authorization to use the insert function when entering the secret code. This facilitates the use of password managers, which are widespread and in many cases encourage users to create more complex secret codes.

So do not even doubt it: people on both sides of the Atlantic Ocean unanimously support the use of password managers and oppose active actions to block them.

Do not require a regular password change


At my previous work in a private company, I came up with a simple way to calculate how much I already use my current password: I simply took the number that it contained and which I increased once a quarter, when the company required to replace the password, and divided it by four. Thus, I extended a single password for almost six years, each time changing no more than a couple of characters in it. If at your workplace it is also customary to regularly change passwords, you probably do the same - it is an easy and natural way out of the situation when you come across a technical requirement that is inconvenient.

Let's analyze the rationale for this approach: the argument in favor of periodically updating passwords is that if the password falls into the wrong hands, it will become irrelevant after a forced replacement, which means scammers will not be able to use it. The problem here is that in my example, the hackers had up to three months of time, and in other cases, perhaps more:



If you think so, hackers usually do not wait idly by, they immediately get to work with accounts, trying to benefit from them. The Lifeboat team did not take this into account in their attempts to smooth out the consequences of last year's data leak, and the chain of reasoning turned out to be incorrect:

After what happened in January, we decided that the best course of action is to quietly replace all passwords so that hackers do not know that they have little time left to implement their plans.


As mentioned in the tweet above, the NCSC shares the view that today, forced password replacement is an anti-pattern in information security. Here's what they say about this practice in their password management guide :

[she] does not bring any real benefit, since hacked passwords are usually used immediately.

They bring this idea into infographics:



Ask users to change their password only if there are signs or suspicions that the old password has been compromised.

Microsoft is saying the same thing:

The password expiration policy does more harm than good, as it encourages users to create highly predictable passwords from consecutive numbers and words that are closely related to each other (as a result, each subsequent password is easy to guess from the previous one). With regard to reducing risks, changing the password does not play a big role here either, since in most cases cyber scammers use their details as soon as they get access to them.

Their words, of course, should not be understood as "do not force users to change passwords, and nothing more is required of you." Rather, it is an indication that a broader, more advanced approach to password management is needed. For example, NCSC also gives such recommendations:

  1. Track all authorization cases to detect unusual behavior.
  2. Tell users the details of login attempts, whether they were successful or failed. In turn, users should notify you if any of these attempts were not from them.


And then we smoothly move on to the next section.

Report abnormal behavior to users


This is an important aspect of the “advanced” approach to the authentication process and it is possible that you have already seen it in action. I recently logged into my Yammer account with my new Lenovo Yoga 910 - until then I hadn’t used their service on this machine. A little later I received such an alert:



Later, I went to Dropbox through a browser on the same machine and immediately received a message:



Of course, these letters did not prevent me from logging into the system and, of course, in my place there could have been an intruder who got the victim's access details. But they immediately notified me of the incident, and it is of great value - guided by this information, I could, if necessary, go and expel the one who invaded my account. Dropbox even showed me all the devices that I have authorized over the past five years, and provided the opportunity to cancel authorization.



Being able to track account behavior in this way is very useful; for example, you can see which devices are currently logged into my Facebook account.



As in the case of Dropbox and Yammer, it also offers the option to log out on any device from the list. This means that the rightful owner can at least to some extent regain control over the account in a situation where someone got unauthorized access to it. Many of the modern services offer this opportunity; A good example of this is Github, which provides even more detailed information, including the IP address and specific events significant from a security point of view, for example, a request for two-factor identification or the creation of a public key.

Here is another possible scenario:



I would like to be informed of login attempts when the password was entered correctly, but two-factor identification did not pass. This clearly indicates that the password fell into the wrong hands.

It also seems very reasonable, and when you think about various scenarios in which it would be worth taking the initiative and warning the user, you begin to understand how much can be done with minimal labor.

Lock compromised passwords


Let's go back for a moment to the stuffing of registration data and everything related to them. If passwords are highlighted somewhere, they should be considered “defiled” - in the sense that they can no longer be used. Never. Now that they have been in the public domain, a huge number of people have information about these details, which significantly increases the risks for those who use them. Just imagine: access to a billion pairs of e-mail address-password, taken from leaks of real data, which I spread about in a post about throwing in registration data :



NIST says the following about this issue:

При обработке запросов на создание или изменение секретного кода верификаторы должны сверять предложенные коды со списком тех, которые считаются широко распространенными, предсказуемыми или скомпрометированными. Такой список может включать, среди прочего, пароли из корпусов, составленных на материале утечек баз данных.

In simple terms: when someone creates an account or changes a password, you must ensure that this password is not among those that appear in data leaks. It does not matter that this may not be the same user who set the password for the hacked account - the mere fact that the password was in the public domain increases the likelihood of its use in hacker attacks. The citation also mentions that the password should not be a word in its dictionary form or “a word generated by context”. Describing when the CloudPets Database Has Been Shared, I drew readers' attention to the fact that bcrypt hashes are extremely easy to crack using a small password dictionary, which also includes words such as “cloudpets”. Do not give users the opportunity to set the password of the name of the resource on which they are registered - otherwise they will do just that!

Next level


There are many other subtleties that I deliberately did not include in this article. Say, the various multi-step verification mechanisms that are now available are, by the way, also mentioned by NIST in their documentation. Here I wanted to focus on the most basic principles, although, of course, additional layers of protection (for example, an authentication application) significantly improve the security situation - it is a pity that almost no one uses them.



Last January, only 1% of Dropbox users used two-factor authentication (even before data leakage).

Another aspect that should be addressed is the ability to detect and repel automated hacker attacks. Think about it: how prepared is your system to identify the unexpected influx of attempts to enter the system as an attack, for example, when throwing in credentials? Will it all become clear only when users start massively reporting account hacks, or will you be in the know from the very beginning? And if, suppose you manage to detect an attack, can you immediately take countermeasures to minimize the risks? For example, like on Cloudpet, where you can put down the desired level of protection?



Also, I did not analyze in detail the methods for storing passwords, having decided to focus on those components of the authentication policy that can be seen with the naked eye. In order not to leave this topic completely untouched, I suggest that you familiarize yourself with the OWASP Password Storage Cheat Sheet and learn all the necessary parting words there. In addition, I advise you to carefully read the article on how Dropbox stores your passwords - this is a very interesting text, where the modern approach to hashing is combined with encryption.

The problems presented are not limited to what I wrote here. The bottom line is that the principles outlined by me above should be a mandatory minimum these days, but this does not mean that you should stop at this stage.

Summarizing what was said


Here's a general picture of what all of these guides from government agencies and IT companies recognize: security is increasingly associated with a combination of controls that together provide more reliable protection. In this post you will find a set of recommendations that, complementing each other, help create a powerful solution. Accordingly, individual recommendations, such as rejecting complexity requirements, may seem strange at first glance, but when you think about how people usually act in reality (just choose bad passwords with a combination of different types of characters), and take into account other instructions, for example, advice to block previously cracked passwords, everything starts to fall into place.

And here's another thing: theoretically, all of these guidelines are a good thing, but putting them into practice can be a little more difficult. I am going to develop this topic in the next post, which will be released this week. There I’m going to tell you something interesting and new that will help site owners ensure the safety of their subscribers. I worked on it for several weeks, I spent a lot of effort and am going to share the result with you for free, so check back soon on my site. I also intend to update this post by adding a link to the resource as soon as I post it in the public domain. Stay tuned!

Also popular now: