Tom Moertel's Weblog: Category security

The Insecurity of Security Questions: Why I met my wife in CWmKryWzuxCSAnMDuIg.

Tom Moertel — Fri, 19 Sep 2008 11:06:00 -0400

Via Dare Obasanjo’s blog, I learned that the much-publicized cracking of Sarah Palin’s Yahoo! email accounts was accomplished by exploiting the weakness of “security questions”. In short, all the attacker needed to do to convince Yahoo’s computers that he was Palin was answer three questions as if he were Palin:

What’s your birthday?
What’s your Zip code?
Where did you meet your spouse?

The attacker says he obtained the answers to these questions in less than an hour. Everything he needed was already public knowledge, and Google and Wikipedia made that knowledge easy to find.

And that’s why when I sign up for web sites that ask me to provide baseline answers for those annoying security questions, I claim that I met my spouse in CWmKryWzuxCSAnMDuIg. What? You’ve never been there? Well, that’s not surprising. It’s not a real place: it’s a password, randomly generated, and remembered for me by password-management software on my computer.

That’s right. Every time I’m asked to establish my “secret” answer to a security question, I generate a random string and use that. Here’s a script I use:

#!/usr/bin/perl

use MIME::Base64;

open my $random, "/dev/urandom" 
    or die "can't open /dev/urandom";
my $bytes;
read $random, $bytes, 16;
close $random;

my $pw = encode_base64($bytes);
$pw =~ tr/A-Za-z0-9//cd;
print "$pw$/";

Then I store the string in my password-management software, just in case the web site asks me for it later. Which should only happen if I forget my primary password for that site. Which should only happen if I can’t get into my password-management software. Which should only happen if I’m totally screwed, anyway, so what are the security questions buying me again?

In sum, if you care about your security, you’re probably picking good passwords already. In that case, security questions can’t help you, but they can harm you by making it easier for an attacker bypass your passwords. That’s how the Palin-email cracker did it. So treat your answers to security questions as if they were passwords – in effect, that’s what they are.

A bright future: security and modern type systems

Tom Moertel — Wed, 15 Aug 2007 16:07:00 -0400

The recent defacement of the United Nations web site is a prime example of why we programmers shouldn’t trust ourselves to write secure code – at least not without our computers’ help. The U.N. web site, according to Slashdot’s coverage of the incident, was defaced by way of a common, well-known attack: SQL injection. What’s interesting is that programmers can render this attack harmless by employing simple, readily available programming tools such as placeholders and prepared statements. Why, then, are so many web sites, including the UN site apparently, still vulnerable?

Some say it’s because the programmers of these sites are incompetent, but that argument ignores that programmers are human, while the security tools we give them offer meaningful protection only if wielded with inhuman perfection. Having the tools to plug security holes, even if the tools are simple to use and readily available, is not enough to ensure that every single security hole will be identified, let alone plugged. Even the most experienced programmer can be expected to overlook a hole now and then. Unfortunately, one hole is all it takes.

That’s because security is not like other software-quality challenges: its costs are fundamentally asymmetric. For the attacker, the bad guy, the challenge is to find just a single exploitable hole. For us, the good guys, the challenge is to achieve perfection: to plug all of the holes in our code, every single one. That’s because attackers, unlike regular users, can be expected to probe our code until they find a hole to exploit.

How then do we ensure that we have plugged every single hole in our code? Testing isn’t sufficient: we can easily overlook holes when writing tests – a perfectly human error. We could supplement testing with code reviews, painstakingly searching for remaining holes while enforcing the use of hole-preventing best practices, but reviews are expensive and, again, subject to human error. A better approach, both less costly and more reliable, is to delegate this burden to our computers, which can do the job correctly, every single time.

This kind of delegation is possible today with modern static type systems. For example, in A Type-Based Solution to the Strings Problem, I offered a tiny “safe strings” library for the Haskell programming language. The library takes advantage of Haskell’s powerful type system to detect unsafe string interactions at compile time. If we faithfully build our code on top of the library, and our code compiles without error, we can be assured that our code is free – completely free – of SQL-injection (and XSS) holes.

While this result is indeed quite beautiful, it certainly isn’t novel. Researchers have been proving interesting properties via type systems for a long time. As Oleg Kiselyov and Chung-chieh Shan pointed out in a comment on my earlier article, the foundational idea is over three decades old.

More recently, Kiselyov and Shan have extended the idea to guarantee more-interesting properties using a trusted kernel and types that represent lightweight static capabilities. The kernel, which is small enough to be reasoned about and formally verified, carefully hands out capabilities to untrusted application code. The untrusted code, in turn, presents the capabilities back to the kernel to invoke operations, which, thanks to the kernel’s trustworthiness, are guaranteed to be safe. (My safe-string library can be seen as a trivial implementation of this programming style.)

When static type systems are used in this way, they don’t merely catch typos and bugs that good testing would have caught as a matter of course, but offer programmers guarantees that would have been impractical to obtain any other way.¹ If you consider security important, you might bear this fact in mind when choosing languages for your next project.

Going further, the security benefits of rich static type systems are only now starting to trickle into mainstream industry. As libraries like “safe strings” and idioms like static capabilities become more familiar and get woven into future generations of development frameworks, we can expect marked improvements in the security and robustness of our applications.

In the not-too-distant future, perhaps, we might look back in amazement at the days when important security properties were neither free nor guaranteed but expensive and uncertain, underwritten only by the heroic efforts of individual programmers, struggling against impossible odds to achieve inhuman perfection.

Then again, it sure took garbage collection a long time to catch on.

1. How, for example, could you eliminate the possibility of SQL-injection and XSS holes via testing?

I suppose you could do it if you worked at it hard enough. You could augment your string data structures with run-time information about what they represent: this string represents SQL, this string represents plain-old text, and so on. Then you could redefine your string operations and template interpolation systems to assert that their string inputs were compatible. Of course, if these assertions ever failed, they would do so only at run time, when it would be too late to do anything but die rather ungracefully. So you would be forced to augment your code-coverage tools to ensure that every string-path was covered during testing. That way you could catch all potential run-time string failures – indicating holes – during testing and eliminate the holes (and the subsequent need to fail at run time) before you deployed your application for real.

So, yes, you could do it. But to do so would require you, in effect, to write a crude, single-purpose type system that checks types at test time. That says something, doesn’t it?

Don't let password recovery keep you from protecting your users

Tom Moertel — Fri, 09 Feb 2007 15:36:00 -0500

In 2006’s most-read article on my blog, Never store passwords in a database!, I urged web programmers, unsurprisingly, not to store passwords in their user databases. I tried to persuade them to salt and hash the passwords instead: store the salts and hashes in the database and throw the passwords away. The article, posted shortly after the Reddit blog announced the theft of its unprotected user database, generated buckets of comments. Reading over them today, I noticed something that I had missed earlier.

It seems that a decent slice of programmers think that switching to a salted-and-hashed password scheme implies giving up the ability to assist users who have forgotten their passwords. If the passwords are irretrievably hashed away, the programmers reason, there’s no way to recover forgotten passwords and email them to stranded users. Hence those users are screwed.

And that wrinkle, it might seem, is a good reason not to switch to a salted-and-hashed password scheme.

But that wrinkle turns out to be imaginary. Not being able to recover an account’s password does not mean that you can’t recover the account itself. The password, after all, is not the thing of value; the account is. And, as we shall see, we can recover an account without knowing its password.

Recall that the primary benefit of using a hash is that it is a one-way operation. Once you salt and hash a password, there is no practical way to retrieve it. That’s what protects it from would-be attackers. But that also means you can’t get at it, either. Thus sending password reminders to people who have forgotten their passwords is no longer an option.

How, then, can you help your stranded users? One method is to send them account-recovery tokens, which you can think of as one-time, special-purpose passwords. (This method is suitable only if you require no stronger authentication than knowing that your site’s users own the email addresses they claim to own. This is the case for most “low security” sites such as Slashdot, Reddit, and Digg, as well as most blogging systems.)

Here’s how it works. Say Joe has lost his password and can’t log in to your site. He clicks that button that says “I’ve lost my password. Help me!” Now what?

Here’s what you do:

Generate a big, random, unique token and stuff it into Joe’s account record in the database. Stuff the current date and time in there, too.
Send an email to Joe, but instead of enclosing his password (which you can’t recover), tell Joe to click on the enclosed account-recovery link, which includes the random token: http://example.com/recover-account?token=pCIqq1unxntVqc8XtCXg.
Joe receives the email and clicks on the link, which sends his token to your site.
Look up the token in the user database. Is it there?
1. No? Render a screen that says, “Sorry, bub, that token is no longer valid.” Stop.
2. Yes? Excellent. Grab the user record associated with the token. (It will, of course, be Joe’s record.)
Is the date and time stamp on that record more than a few hours old?
1. Yes? Render that screen that says, “Sorry, bub, that token is no longer valid.” Stop.
2. No? Congratulations. Joe has effectively authenticated himself via his email address.
Render a confirmation screen that explains the following to Joe:
1. His account password is going to be reset to the following random string: ocZodbew. (Generate a new random string each time.)
2. If he likes the password, great. If not, he can use the change-password feature immediately after the password is reset.
3. If he understands the above and wants to continue, he should confirm by clicking the big “Reset My Account Password” button.
Joe clicks the button.
You, in response, do the following:
1. Delete the recovery token from Joe’s user record in the database. (This prevents somebody from using the old token to steal his account, should, for example, Joe’s email get stolen.)
2. Replace Joe’s old password with the new, randomly generated password from above. (You will, of course, use the salted-and-hashed method and not store the new password itself.)
3. Log Joe in.
4. Render a screen saying, “Joe, please don’t forget that your new password is ocZodbew. If you would like to change it, just visit Change My Password in your account preferences [provide a link]. Otherwise, you’re logged in and ready to go. Enjoy the site!”
And you’re done.

The code required to make it happen is shorter than the explanation above. It’s one of those easier-done-than-said things.

So, if concerns about account recovery have been holding you back from protecting your users’ passwords, you need hold back no longer. It’s time to “do” your due diligence.

Update 2007-09-10: I made clear that the account-recovery method I describe above is suitable only for low-security sites where a valid email address is sufficient to authenticate users.

Never store passwords in a database!

Tom Moertel — Fri, 15 Dec 2006 13:25:00 -0500

Recently, the folks behind Reddit.com confessed that a backup copy of their database had been stolen. Later, spez, one of the Reddit developers, confirmed that the database contained password information for Reddit’s users, and that the information was stored as plain, unprotected text. In other words, once the thief had the database, he had everyone’s passwords as well.

Had the folks at Reddit salted and hashed the passwords, the thief would now be in a very different situation. Instead of holding all the keys to the kingdom, he would face the prospect of a potentially expensive search for each and every user’s password he wanted to extract from the database. The expense of the search would likely have dissuaded him from making the attempt in earnest, given how little exploitable value a Reddit account represents. In short, the passwords would have been secure, even though the database had fallen into the thief’s hands.

Why, then, didn’t Reddit’s programmers salt and hash the passwords before storing them in their database? Because, according to the earlier post by spez, they wanted to be able to send forgotten passwords to users via email. It was a design decision: they weighed the risks of having plain-as-day passwords in the database against the convenience of being able to email users their forgotten passwords and decided that, in the balance, convenience carried more weight. It’s a decision they now regret. (It’s a doubly unfortunate decision because you don’t need to store passwords in your user database in order to offer convenient account recovery.)

The reason I’m writing about this event isn’t to kick the good folks at Reddit while they’re down. Rather, I’m trying to make a point:

If you are storing passwords in a database, you are almost certainly making a mistake.

The guys at Reddit are known for being smart. They thought they had a good reason for storing passwords in their database. They were wrong. If smart programmers can make this mistake, lots of programmers can. Do you think you have a good reason for storing passwords in your database? If so, you’re probably wrong, too.

How can I be so sure? Because, when it comes to web-app authentication, cutting corners doesn’t buy you anything. It doesn’t save you coding time. It doesn’t give your users a better experience. All it does is weaken the security of your web application, needlessly putting your users, your employer, and yourself at risk.

So please let me take this opportunity to ask if you know of (or perhaps work on) any software systems that store passwords as plain, unprotected text in a database. If so, fix your software now:

Salt and hash each and every password (use an expensive hashing function such as bcrypt that was designed for password applications)
Store the salt and hash – not the password – in your database.
Throw the password itself away.

You’ll be glad you did.

Update: Minor edits for clarity.

Update 2007-02-13: Salting and hashing does not get in the way of account recovery. You do not need to email users their forgotten passwords: there are other account-recovery options that are just as convenient but much more secure. See Don’t let password recovery keep you from protecting your users for more.

Update 2007-10-03: Revised text slightly to emphasize that there is no benefit to be had by implementing a weak password system, and therefore there is no reason not to implement a secure system. Pointed more directly to bcrypt, too.