Tom Moertel's Weblog: Tag safe

Google Web Accelerator vs. unsafe linking: Round Two!

Tom Moertel — Tue, 25 Oct 2005 16:12:00 -0400

The good folks at 37signals are once again up in arms about Google Web Accelerator (GWA). David Heinemeier Hansson (DHH), in particular, writes in a recent post to Signal vs. Noise that “[GWA] was evil enough the first time around, but this time it’s downright scary.”

The problem, it seems, is that GWA automatically, silently, and unblockably follows hypertext links to web pages that are linked to by the pages you visit. It does this in order to cache those pages so that if you visit them later, it will have cached copies ready in an instant, thus “accelerating” your web surfing. But some web developers use hypertext links to trigger potentially unsafe actions, such as deleting records in a database, and when GWA automatically follows such links, it triggers the actions.

Oops.

Let’s do the time warp again…

Now, if this story sounds familiar, that’s because half a year ago, the exact same thing happened. GWA was unveiled to the public. People started using it. And some of those people started losing data from their accounts with popular web applications, such as 37signal’s own Backpack. 37signals publicized the problem in their blog and DHH even called for a recall on GWA.

And then the community responses came in. For the most part, the responses could be divided into two camps, based on who was blamed for the problem. The first camp blamed the web designers who used links to trigger unsafe actions (in violation of applicable standards), and the second camp blamed Google for unleashing GWA upon a web where standards aren’t always followed.

Both viewpoints had some merit, but I was in the first camp and thus argued for following the standards and against unsafe linking practices:

What surprised me was that so many people in the second camp argued in defense of unsafe linking practices, which I had thought indefensible. I didn’t have any problem with arguments against Google’s unleashing GWA on an imperfect web, but arguing for the web’s imperfections seemed like an odd way of making the case. The supportive arguments boiled down to the following:

Lots of web sites use action-triggering links, so the practice is de facto acceptable.
The existing palette of user-interface options is too limited for today’s web applications; thus, designers are justified in breaking the rules.
The standards don’t actually prohibit the practice (they say “SHOULD NOT,” not “MUST NOT”); thus, the practice is allowable.

None of the arguments seem to withstand scrutiny. The first argument breaks down like so: That lots of web sites do it only means that those sites get away with it, not that the practice is acceptable. Further, as GWA demonstrates, those sites may not get away with the practice much longer.

The second argument breaks down when one examines the uses of unsafe linking practices. Most of them could be replaced by safe practices through modest UI refactoring. Given that safe alternatives exist, the unsafe practices are not justified by virtue of being the only realistic option.

The third argument breaks down when one actually reads the relevant standards. Then it becomes clear that one should not use links to trigger potentially unsafe actions. The wiggle room created by the use of “SHOULD NOT” instead of “MUST NOT” does not admit the large problems caused by unsafe linking.

Finally, even if there were some justification for unsafe linking, the practice would still be a bad idea: its costs and risks outweigh its benefits. Why hold back the potential of efficient caching architectures for the web? Why risk data loss for your users? It’s not worth it.

Back to the Future

So where are we now? Given how little justification there is for unsafe linking practices, one would hope that we would have abandoned them by now. But, as the recent cries about the second coming of GWA suggest, the web-development community is not yet ready to give up those sexy, action-triggering links.

It’s not that the means aren’t available. Rails, for example, has plenty of support for sane and safe practices for triggering actions. Rather, the problem is cultural. Too many influential people, especially in the Rails community, are unrepentant users of – and, dare I say it, apologists for – action-triggering links. Until this changes, I expect many new web developers to pick up dangerous habits from the very people they respect most.

Fortunately, many other respect-worthy people are pointing toward a better way:

Sam Ruby: “I’m on the other side of this debate. While this appears to be a purely philosophical concern, in reality this stuff matters.”
Bill de hÓra: “The GWA is back and following GET links again… The technology itself is interesting insofar as we are going to see more and more highly automated robots enter the web over the next few years…. Even more interesting is the kind of outrage holding forth in places like Signal v Noise….”
Joe Gregorio : “And now we begin the next chapter in which Pooh discovers that five months after the first time Google turned on GWA that standards still matter.”

I hope that this time around the web-development community answers the wake-up call. It’s time to abandon action-triggering links.

How to change symlinks atomically

Tom Moertel — Mon, 22 Aug 2005 12:00:00 -0400

Many people don’t realize that changing the target of a symbolic link (symlink) is not an atomic operation. “Changing” a symlink really means deleting it and creating a new link with the same file name. For example, if I have a symlink current that points to a directory old, and I want to change it to point to a directory new, I might use the following command:

$ ln -snf new current

Strace shows what really happens when I run the command:

$ strace ln -snf new current 2>&1 | grep link
unlink("current")         = 0
symlink("new", "current") = 0

First, the existing symlink is deleted via the unlink system call. Then a new, identically named symlink is created via the symlink system call. It’s a two-step process, and in between the steps, there is no symlink.

This can be a problem if you expect the symlink to be there always, such as when using the link to point to the active version of a live web site. If you change the symlink while deploying a new version of your site, for example, the web server might try to dereference the link during the small window of time when it doesn’t exist. Oops.

The solution to this problem is to effect the change by creating a new symlink and then renaming it over the old symlink. On Unix-like systems, renaming is an atomic operation, and thus the symlink “change” will be atomic too. By hand, the process looks like this:

$ ln -s new current_tmp && mv -Tf current_tmp current

In Ruby, I make atomic symlinking available everywhere by extending the Pathname class with a new method atomic_symlink:

require 'pathname'

class Pathname
  def atomic_symlink(old)
    suffix = [Array.new(6){rand(256).chr}.join].pack("m").strip.tr('/','_');
    tmplink = Pathname.new(self.to_s + "_" + suffix)
    tmplink.make_symlink(old)
    begin
      tmplink.rename(self)
    rescue
      # if rename fails, we must remove the temporary link manually
      File.unlink(tmplink.to_s)
      raise
    end
  end
end

This code is nothing more than a robustified version of the by-hand method. It picks better names for temporary links, and it cleans up after itself, should something go wrong, but otherwise it does the same thing.

Given how easy it is to change symlinks atomically, why do it any other way? Life is hard enough without having to worry about another race condition.

The button_to helper is now part of Rails!

Tom Moertel — Thu, 16 Jun 2005 12:00:00 -0400

I am delighted to report that the button_to helper has been added to the Ruby on Rails web-development framework. David applied the patch earlier today, and so button_to will be in the much-anticipated Rails 1.0 release.

David’s change-log entry summarizes the patch well:

Added button_to as a form-based solution to deal with harmful actions that should be hidden behind POSTs. This makes it just as easy as link_to to create a safe trigger for actions like destroy, although it’s limited by being a block element, the fixed look, and a no-no inside other forms.

David does a good job of highlighting the helper’s limitations. I’ll take this opportunity to elaborate on each.

It is a block element

The button_to helper creates a small form, which in HTML is considered block content, just like the p, div, and blockquote elements are. Basically, block content cannot be mixed into runs of text. But links can: links are inline content. Thus button_to cannot be used as a drop-in replacement for every occurrence of link_to that might be unsafe; it works only for those occurrences within block-accepting contexts.

Luckily for us, when designers use links to trigger unsafe actions, they rarely slip such links into the middle of ordinary looking text. Naughty uses of link_to almost always occur within contexts that accept block content. In Rails-generated scaffolding code, for instance, the unsafe uses of link_to occur within table cells, and table cells have a flow content model, which accepts both inline and block content. So button_to works great for the default cases in Rails.

It has a fixed look

As its name implies, button_to creates buttons. Buttons don’t look like links and aren’t styled the same way that links are. For some design scenarios, this might be a problem.

(My view is that links should not be used to trigger unsafe actions. In the same way that action-triggering GET requests violate the spirit of the HTTP standards, action-triggering hypertext links violate the spirit of the HTML standards. For this reason, I view this limitation as a feature.)

It is a no-no inside other forms

Forms cannot be nested, and so button_to cannot be used inside of forms.

Fortunately, this limitation usually doesn’t matter because when we are inside of a form, we can use its buttons instead of button_to-created buttons to trigger actions. Still, there are some circumstances where it does matter, such as the “Amazon.com wish list” scenario. In this scenario, we should consider other options.

The bottom line: Pick the low-hanging fruit

While button_to has its limitations, it does provide a simple solution to the unsafe-GET problem for most real-world cases. I am glad that it is now a part of Rails, and I offer a big thank-you to David for accepting the patch.

Taking the unsafe GETs out of Rails

Tom Moertel — Sun, 08 May 2005 12:00:00 -0400

Update 2005-06-17: The button_to helper, introduced below, has been incorporated into the Rails framework and will be a part of the Rails 1.0 release. See Good news: The button_to helper is now part of Rails! for more.

Update 2005-05-28: I now have a more-recent version of the button_to code, which adds support for the disabled HTML attribute. Thanks to Sean T Allen for the great idea and initial implementation.

As I wrote earlier, it’s time for web developers to do away with the fundamentally broken practice of using hypertext links to trigger dangerous events such as deleting things. One of the first places we ought to clean house is in the burgeoning Rails web-application framework, where this practice is pervasive.

The primary culprit in Rails is the all-too-easy link_to method, which is (presently) the orthodox means of creating links to any action, even unsafe ones. For example:

link_to "Destroy", :controller => 'accounts',
        :action => 'destroy', :id => 6

The above code generates the following HTML hypertext link, which when followed will merrily delete account number 6:

<a href="/accounts/destroy/6">Destroy</a>

Because this practice is dangerous and contrary to the decade-old convention that links be safe, the link_to method thoughtfully lets us request that a Javascript confirmation dialog be tacked onto the link for added protection:

link_to "Destroy", ...,  :confirm => "Are you sure?"

The resulting “safe” HTML:

<a href="/accounts/destroy/6" 
   onclick="return confirm('Are you sure?');">Destroy</a>

Unfortunately, the Javascript protection doesn’t work. First, not all web browsers care about it. Lots of people surf with Javascript turned off. Second, a whole slew of things besides web browsers live on the Internet, and almost all of them are oblivious to Javascript. Web crawlers fall into this category. They will be more than happy to follow any link you feed to them. “Hey, Googlebot just deleted every account in our database!” Oops.

Thus another layer of protection is commonly used: authorization. The theory is that dangerous links can be safely corralled in the private parts of a web application, where the public and web crawlers cannot go. Only authorized users can get into those parts, and those users will be smart enough not to click on the truly dangerous links unless they really mean it.

The problem is, any number of intermediary agents can be operating on behalf of an authorized user, and these agents are free to do anything the user is allowed to do, such as follow dangerous links. Google’s Web Accelerator is one such agent. It tries to make your surfing faster by (among other things) pre-fetching the resources that are linked to on the pages you visit. And what happens if you, an authorized user, visit a page containing dangerous links? That’s right, Web Accelerator will fetch the “resources” those links point to – and delete a bunch of your stuff.

I hope by this point that I have argued convincingly that using links for unsafe actions is a bad idea. Even if you feel justified in ignoring the applicable parts of the HTTP RFCs, it’s a bad idea. Even if you tack on Javascript confirmations and hide your links in authorization-protected zones of your site, it’s a bad idea. It is, all around, a bad idea. Don’t do it.

So what alternatives are there? Read on for one possibility, button_to.

A link_to alternative: button_to

If you shouldn’t use links for unsafe actions, what should you use instead? Form buttons. Forms can be submitted via HTTP POST requests, and POST requests are understood to do potentially unsafe things. Web crawlers will not try to click your buttons. Intermediary user agents will not try to pre-submit your forms.

So, how do we make doing the right thing as easy as creating a link? My answer is button_to, a method that takes the same parameters as the ever-popular link_to but creates a tiny form that contains a single button instead of a link:

button_to "Destroy", { :action => 'destroy', :id => 6 },
          :confirm => "Are you sure?"

The resulting HTML (reformatted for your viewing pleasure):

<form method="post" action="/accounts/destroy/6" class="button-to">
  <div><input onclick="return confirm('Are you sure?');" 
              value="Destroy" type="submit">
  </div>
</form>

The forms I create are given the class button-to, which makes it easy to apply styles to them. With a little work, the buttons can look pretty darn good:

So that’s my plea: Use a button. It’s a simple solution to a potentially ugly problem. There’s no need for Ajax or other non-portable Javascript trickery. Just use a button.

And it’s easy, too. In a few minutes, I was able to “clean house” on the Rails application I’m developing.

The code

If you’re interested, here’s the code for button_to. It’s only ten lines, but the docs make it look much longer.

# Generates a form containing a sole button that submits to the URL
# given by _options_.  Use this method instead of +link_to+ for
# dangerous actions that do not have the safe HTTP GET semantics
# implied by using a hypertext link.
#
# The parameters are the same as for +url_to+.  Any _html_options_
# that you pass will be applied to the inner +input+ element.  The
# generated form element is given the class 'button-to', to which
# you can attach CSS styles for display purposes.
#
# Example 1:
#
#   # inside of controller 'feeds'
#   button_to "Edit", :action => 'edit', :id => 3
#
# Generates the following HTML (sans formatting):
#
#   <form method="post" action="/feeds/edit/3" class="button-to">
#     <div><input value="Edit" type="submit"></div>
#   </form>
#
# Example 2:
#
#   button_to "Destroy", { :action => 'destroy', :id => 3 },
#             :confirm => "Are you sure?" 
#
# Generates the following HTML (sans formatting):
#
#   <form method="post" action="/feeds/destroy/3" class="button-to">
#     <div><input onclick="return confirm('Are you sure?');" 
#                 value="Destroy" type="submit">
#     </div>
#   </form>
# 
# *NOTE*: This method generates HTML code that represents a form.
# Forms are "block" content, which means that you should not try to
# insert them into your HTML where only inline content is expected.
# For example, you can legally insert a form inside of a +div+ or +td+
# element or in between +p+ elements, but not in the middle of a run
# of text.  (Bottom line:  Always validate your HTML before going
# public, especially if this paragraph seems confusing.)

def button_to(name, options = {}, html_options = nil)
  html_options = (html_options || {}).stringify_keys
  convert_confirm_option_to_javascript!(html_options)
  url, name = options.is_a?(String) ? 
    [ options,  name || options ] :
    [ url_for(options), name || url_for(options) ]
  html_options.merge!("type" => "submit", "value" => name)
  "<form method='post' action='#{h url}' class='button-to'><div>" +
    tag("input", html_options) + "</div></form>" 
end

Thanks for reading and happy unsafe-link hunting!

Google Web Accelerator offers web developers an important opportunity

Tom Moertel — Fri, 06 May 2005 21:55:00 -0400

Google Web Accelerator offers web developers an important opportunity There has been a lot of heat lately about Google’s Web Accelerator (GWA) exposing serious problems in some popular web applications. The problem, in short, is that these applications use GET-based links that when followed perform dangerous actions such as deleting records in a database. According to web standards going back a decade, that is a no-no: Links should be safe to follow. Thus GWA, expecting links to be safe, tries to help you out by pre-fetching various resources that are linked to by the pages you visit. Unfortunately, if the page you happen to be visiting contains lots of dangerous links, GWA will innocently try to pre-fetch the “resources” that the links point to, and in doing so will accidentally delete a bunch of stuff. Oops.

That’s the backdrop for our real story, which is the response from the community of web developers. What I find fascinating, and somewhat disheartening, is the number of people who say the problem is Google’s to fix. Yes, there are a lot of broken web apps out there, and Google could have been smarter about working around the minefields those apps represent. But that’s a side problem. The real problem is that there are a lot of broken web apps out there, and they do represent minefields. Worse, a lot of web developers think it is acceptable to brush aside fundamental conventions of the web going back a decade when they find it sexier to use GET instead of POST.

What these developers overlook is that the web is not a bunch of colorful pages with buttons, clickable links, and pretty pictures. Rather the web is a distributed collection of hypertext documents, each of which has a meaning that is given by standards that most people have agreed to follow. While the collection may look like a bunch of colorful pages in one particular visual presentation, it really, truly is not.

Nevertheless, many web applications are designed with the prevailing mindset that the meaning of the web is nothing more than how it looks and behaves in a web browser. Even if those web applications are not intended for use outside of a few approved browsers – the escape hatch that is often used to justify departures from the standards – this mindset is wrong.

The reason it is wrong is because, like it or not, web applications are implemented in terms of protocols and languages that mean something. The bits and pieces of a web application each mean something, even if what they mean is not in harmony with the designer’s overall vision. What a web designer may see as a bold, red link that says “delete” and is protected by a Javascript confirmation dialog, is actually a hypertext reference – a pointer to another resource that the standards say should not be dangerous to follow. Thus there is a fundamental mismatch between what the designer is trying to say – “follow this link only if you really, no-kidding want to delete something” – and what his markup actually means: “this link is safe to follow.”

The bottom line is that hypertext links are supposed to be safe. Dangerous links can’t be made “safe” by layering tricks on them. Wrapping them with Javascript confirmation dialogs doesn’t make them safe. This trick fails fundamentally because the meaning of a link doesn’t change when there is Javascript associated with it. It fails practically because not all consumers of hypertext evaluate Javascript, nor do any standards suggest that they should. (This is also why client-side validation cannot be trusted on the server side.)

Hiding dangerous links in authorization-controlled portions of a web application does not make them safe, either. This trick might shield the links from external spiders, but the standards allow for any number of intermediary agents (such as Google’s Web Accelerator) to work on behalf of an authorized user. Anything the user is authorized to do, so are the user’s agents. If the user can click the “delete” link, so can the agents.

Let’s answer the wake-up call.

GWA is only the first of a new breed of smarter user agents that promise to make the web a better place for all of us. If you’re a web developer, take the slew of problems that GWA uncovered as a wake-up call. Even if Google works around your problems, other user agents may not. As developers, it’s time to admit our mistakes and fix the stupid things that our web applications do.

First, let’s drive a stake in the notion that dangerous links are OK if we’re “careful.” They’re not. Dangerous links are lazy. “Confirming” them with Javascript or “hiding” them behind authorization doesn’t work.

Second, let’s clean house. Let’s find those places where we have laced our web applications with dangerous links and remove them. Break out the forms! Long live the POST!

Third, let’s take a look at how we got into this mess and try to learn from our mistakes. Do we value sexiness more than substance? We like to think that form follows function and that good design and good implementation go hand in hand, but the reality of this debacle suggests otherwise. I think the truth is that on the web, sexiness and hype are what gets attention, and we seek attention more than we like to admit.

Maybe the best thing for us is a good dose of humility. And, come to think of it, that’s just what Google’s Web Accelerator offered us.