Posted by Tom Moertel
Thu, 15 Nov 2007 07:30:00 GMT
I recently started using Puppet
to automate my server-build processes. The basic idea behind Puppet
is that you create “manifests” that declare
a directed graph of “resources” that represents the desired state of
your machines. Puppet-managed machines on your network then query a
master server to obtain the latest copy of the graph, which they then
reconcile with their current states to make whatever changes are
necessary to bring themselves up to date.
For the most part, everything works well. I have encountered a couple
of snags when writing manifests, however, so I’m going to explain them
here as reminder until I get the time to fix them in the Puppet code and send
patches upstream.
First, don’t use hyphens in class names. While hyphens are legal
in class names, they are not allowed in qualified variables, thus
variables defined within hyphen-named classes are inaccessible
from the outside world.
Second, and this one is both tricky and important, Puppet handles
prerequisites for definitions by silently passing those prerequisites on
to all of the resources within the definitions. Definitions, in
effect, don’t really have their own prerequisites, they just pass them on to
their children. But – and here’s the problem – if those child
resources declare their own prerequisites, those prerequisites will
overwrite the passed-on prerequisites, effectively causing them to
be ignored.
This problem bit me hard when trying to create a definition for
installing Ruby Gems from a local cache of gems:
define local_gem($gem) {
$path = "/var/local/local-gems/$gem"
file { $path:
ensure => present,
source => "puppet://puppet/files/gems/$gem",
require => File["local-gems-dir"],
owner => root,
group => root,
mode => 0664,
}
package { $title:
ensure => installed,
provider => "gem",
require => [ Package["rubygems"], File[$path] ],
source => $path,
}
}
The intent was to be able to declare a local gem like so:
local_gem { "sqlite3-ruby":
gem => "sqlite3-ruby-1.2.1.gem",
require => Package["sqlite-devel"]
}
Thus the “sqlite3-ruby” local gem has the single prerequisite of the
“sqlite-devel” package – or at least that’s what I expected. What
happened on deployment was that the prerequisite was ignored because
when it was passed on to the inner file and package resources, those
resources had their own require parameters, and those parameters
overwrote the passed-on prerequisite.
The work-around is somewhat hacky. I augmented the definition with a do-nothing resource
that has no require parameter of its own. This
resource does nothing but capture the passed-on prerequisites. Then I made
all of the other resources in the definition include the do-nothing
resource as one of their prerequisites. Thus they are made to inherit the
passed-on prerequisites.
My final definition looks like this:
define local_gem($gem) {
# dummy exec to propagate requires from local_gem
exec { $name: command => "/bin/true" }
$path = "/var/local/local-gems/$gem"
file { $path:
ensure => present,
source => "puppet://puppet/files/gems/$gem",
require => [ Exec[$name], File["local-gems-dir"] ],
owner => root,
group => root,
mode => 0664,
}
package { $title:
ensure => installed,
provider => "gem",
require => [ Exec[$name], Package["rubygems"], File[$path] ],
source => $path,
}
}
Notice how the file and package resource both require the dummy exec resource.
That’s the trick that allows them to require the prerequisites passed on from
the local_gem definition.
It’s not pretty, but it works. See this email on the puppet-users mailing list for more on the problem.
Posted in sysadmin
Tags gems, manifests, puppet, rails
2 comments
no trackbacks

Posted by Tom Moertel
Fri, 30 Mar 2007 04:34:00 GMT
How much content have I written for my blog? Let’s find out.
My blog runs on Typo, which is built upon
Ruby on Rails. Let’s fire up the Rails
console and gather a quick word count:
$ cd ~/blog
$ ruby script/console
Loading development environment.
>> require 'article'
=> true
>> Article.find(:all).inject(0) { |sum,a| sum +=
(a.body + a.extended.to_s).split(/\s+/).length }
=> 66665
So I have written about 66 kilo-words, which is entering novel
territory. Paperback-wise,
it’s about 190 pages.
All I need now is a villain and some cool cover art.
;-)
Posted in site news, rails, writing
Tags blog, rails, word_count, words, writing
2 comments
no trackbacks

Posted by Tom Moertel
Wed, 07 Feb 2007 17:08:00 GMT
Via
eigenclass.org
I learned that Ruby 1.9 will sport a new Object method
called
tap,
which is something I’ve been hoping
for.
What’s tap? It’s a helper for call chaining. It
passes its object into the given block and, after the block finishes,
returns the object:
The benefit is that tap always returns the object it’s called on, even if the block returns some other result. Thus you can insert a tap block into the middle of an existing method pipleline without breaking the flow. MenTaLguY has some nifty examples of other things you can do with tap.
Fans of Ruby on Rails may recognize tap as similar to RoR’s own
returning helper.
Looks like Ruby 1.9 is going to be extra cool for a number of reasons.
Posted in ruby, rails
Tags helpers, rails, ruby, tap
2 comments
no trackbacks

Posted by Tom Moertel
Thu, 12 Oct 2006 20:06:00 GMT
Recently I wrote about unit testing being a tool, not a goal in
itself.
I argued that unit testing was not a reliable way to fight
certain kinds of common coding errors and, therefore, that unit testing
ought to be supplemented with other tools.
To support my argument, I gave an example of a common, important
coding error that unit testing does a bad job of helping programmers
control. That error is failing to manage and escape strings
properly: the “strings problem.” It is the mother of XSS and
SQL-injection security vulnerabilities, not to mention the cause of
legions of broken links and bad HTML on the web.
If you think I’m overstating the problem, or if you think that unit
testing is a good way of solving it, let me show you how easy it is
for even smart developers to get it wrong.
Consider Ruby on
Rails, a great framework
for developing web applications.
Rails has an extensive suite of unit tests, and the Rails development guidelines require that changes to Rails be accompanied by unit tests that “prove [the] change works.”
Now consider that one of Rails’s most-used and most-scrutinized methods – the venerable link_to helper – contains a fundamental string-escaping error:
require 'rubygems'
require_gem 'rails'
include ActionView::Helpers::UrlHelper
url = "http://example.com?ohms_law?volt=1&=3"
puts link_to("TEST", url)
The code, when executed, prints the following HTML snippet:
<a href="http://example.com?ohms_law?volt=1&=3">TEST</a>
The HTML snippet represents a hypertext link. The link should point
to the URL given in the code, but because the URL was not properly
escaped when it was converted into HTML by the link_to helper, the
link is broken:
CORRECT: http://example.com?ohms_law?volt=1&=3
LINK_TO: http://example.com?ohms_law?volt=1&=3
^ oops
Here’s what’s going on. Because the URL was not escaped, web browsers
misinterpret its “amp” parameter as a character-entity reference,
which gets gobbled up when the link’s href attribute is parsed.
(To see this for yourself, save the output of the Ruby code into an
HTML file, open the file with your favorite web browser, and see where
the link points.)
Now, how come the unit tests didn’t catch this problem?
It turns out, the tests got it wrong, too, by expecting
broken output:
# in url_helper_test.rb
def test_link_tag_with_query
assert_dom_equal \
"<a href=\"http://www.example.com?q1=v1&q2=v2\">Hello</a>",
link_to("Hello", "http://www.example.com?q1=v1&q2=v2")
end
The point isn’t that the Rails developers are dumb. The point is that
the Rails developers are smart. If they can’t get the strings
problem right, even with all their brains and all their unit testing,
what reason does any programmer have to think that unit testing is going
to solve this problem reliably?
If, then, you want to solve the strings problem – and you really,
seriously ought to want to solve the strings problem – you should
consider options beyond unit testing.
Update 2007-09-04: I just noticed that the documentation for
link_to has been revised to state that if you pass a
string as its options parameter, the string will be interpreted not
as a URL but as an HTML href attribute value, that is, an
HTML-encoded URL. The old documentation:
def link_to(name, options = {}, html_options = nil, *parms)
Creates a link tag of
the given name using an URL created by the set of options.... It’s
also possible to pass a string instead of an options hash to get a
link tag that just points without consideration.
The relevant part of the revised documentation:
It’s also possible to pass a string instead of an options hash to
get a link tag that uses the value of the string as the href for the
link.
So, according to the updated documentation, the test I described in my
article is actually correct. Does this mean that string-handling code
is Rails is worry free? The existence of helper methods like fix_double_escape
suggests the answer is no.
Posted in programming, testing, rails
Tags escaping, problem, rails, strings, testing
5 comments
no trackbacks

Posted by Tom Moertel
Thu, 24 Aug 2006 19:41:00 GMT
In an earlier post I wrote about stability
problems that have plagued my blog since upgrading from Typo 4.0.0 to 4.0.3. I have finally traced the problem to its source, and here’s the deal:
If you’re serving Typo up via Mongrel, do not configure ActiveRecord to allow concurrency.
One of the changes between Typo 4.0.0 and 4.0.3 is this
addition to the environment.rb file:
config.active_record.allow_concurrency = true
Comment out this line, restart Typo, and the problem is solved.
Apply Changeset 1255, and the problem is solved. (See
Update 2, below.)
Discussion
When ActiveRecord::Base.allow_concurrency is set to
true, AR will give each thread its own database
connections and cache them in thread-localized storage. The idea is
that, in a multi-threaded environment, this simple policy prevents
unsafe interactions between threads and the database. (Imagine what
would happen if one thread “borrowed” a connection over which
another thread had opened a transaction. Oops, there goes
transactional isolation.)
This policy, however, does place a burden on the owner of the threads to
make sure that each thread’s local connection cache is cleared when
the thread is joined, a burden that is not, it would seem, being
carried by Typo under Mongrel. As a result, Typo rapidly chews
through the allotment of file descriptors that the operating system
kindly had reserved for Mongrel:

(On my Linux server, the Mongrel process gets an allotment of 1024
file descriptors.)
Lucky for us, this each-thread-gets-its-own-connections policy is unnecessary under
Mongrel because Mongrel, while being multi-threaded itself, serializes
all access to the Rails-based applications it serves up:
Q: Is [Mongrel] multi-threaded or can it handle concurrent requests?
Mongrel is uses a pool of thread workers to do it’s processing. This means that it is able to handle concurrent access and should be thread safe. This also means that you have to be more careful about how you use Mongrel. You can’t just write your application assuming that there are no threads involved. ...
Ruby on Rails is not thread safe so there is a synchronized block around the calls to Dispatcher.dispatch. This means that everything is threaded right before and right after Rails runs. While Rails is running there is only one controller in operation at a time.
(Source: Mongrel FAQ list)
Thus we can safely turn off (i.e., comment out in Typo’s
environment.rb file) ActiveRecord’s allow-currency option
without having to worry about nasty concurrency or performance issues:
# the following line is commented out
# config.active_record.allow_concurrency = true
For more on this subject, see Rails ticket
#2162 and Rails ticket
#2742.
Now, here’s my question: Are there any environments in which
Typo can run with the allow-concurrency option enabled and not
leak database connections? Inquiring minds want to know.
Update: Upon further investigation, turning off
concurrency might not be altogether without risk. Some of the Typo
code that handles potentially long tasks, such as making trackbacks
and pings, spawns new threads in which to carry out its work. I’m
looking further into this risk. Updates to come.
Update 2: Piers Cawley added Changeset
1255, which turns AR’s
allow-concurrency flag back off and revises the ping code so that
it does not attempt concurrent database access. Apply the patch version of
1255
and restart Typo to get the fix. A tip of the hat to Piers for making
the quick fix when he was supposed to be on holiday.
Posted in ruby, typo, rails
Tags activerecord, concurrency, rails, sqlite3, typo
8 comments
no trackbacks

Posted by Tom Moertel
Thu, 24 Aug 2006 04:41:00 GMT
Since I upgraded my blog from Typo 4.0.0 to
4.0.3, it has been somewhat unstable. About once a day it starts
responding with “500 Internal Server Error” and stays that way until I
restart it.
The root of the problem seems to be the database
connection, as evidenced by this exception showing up in the
production log:
SQLite3::CantOpenException (could not open database)
Unfortunately, the exception doesn’t provide anything specific
to go on.
A quick look at the
sqlite3-ruby code
suggested that I was not going to get the specifics, either. The Ruby-based wrapper
never calls sqlite3_errmsg after a call to sqlite3_open fails on behalf of SQLite3::Database.new.
A quick patch, however, fixed the problem:
--- sqlite3-ruby-1.1.0.orig/lib/sqlite3/database.rb
+++ sqlite3-ruby-1.1.0/lib/sqlite3/database.rb
@@ -109,7 +109,7 @@
@statement_factory = options[:statement_factory] || Statement
result, @handle = @driver.open( file_name, utf16 )
- Error.check( result, nil, "could not open database" )
+ Error.check( result, self, "could not open database" )
@closed = false
@results_as_hash = options.fetch(:results_as_hash,false)
(Submitted as Ticket 5504 on RubyForge.)
Before applying the patch, opening a database at a nonexistent path results in
a generic error message:
$ ruby -r rubygems -e 'require_gem "sqlite3-ruby";
SQLite3::Database.new("/no/such/path/db")'
... could not open database (SQLite3::CantOpenException) ...
After applying the patch, we get additional error information:
... could not open database: unable to open database file
(SQLite3::CantOpenException) ...
With the patch in place, all I have to do is wait for Typo to start
acting up again. Then I’ll have some interesting information in the
log.
Until then, I’m relying on cron
and a short monitoring script to restart Typo when it tips into
foolishness:
#!/bin/bash
url=http://blog.moertel.com/admin
addrs=tom@moertel.com
response=$(GET -sd $url 2>&1)
if [ "$response" != "200 OK" ]; then
{ echo "Response was: $response"; echo; service typo restart; } |
mail -s "Blog site not responding! (Restarting)" $addrs
fi
We’ll see how it goes.
Update: That was fast. The error popped up
again and this time the log told me something useful: “unable to open
database file.” Now, why couldn’t Typo open the database file,
especially since the file is perfectly fine and had been opened
successfully (many times) by the very same Typo process earlier? Here’s
a hint:
$ ls /proc/28788/fd | wc -l
1023
Seems like there’s a resource leak in Typo 4.0.3 (or Rails 1.1.6).
Under some conditions, instead of reusing existing database
connections, Typo keeps trying to open new ones. Eventually, it uses
up its allotment of file descriptors and the operating system is forced
to say, “That’s enough, pal,” (EMFILE).
I’ll look in to it more in the morning.
Update 2: Problem solved.
Posted in ruby, typo, rails, sysadmin
Tags rails, sqlite3, typo
1 comment
no trackbacks

Posted by Tom Moertel
Tue, 25 Oct 2005 20:12:00 GMT
The good folks at 37signals are once again up
in arms about Google Web
Accelerator (GWA). David
Heinemeier Hansson (DHH), in particular, writes in a recent post to
Signal vs. Noise that “[GWA] was evil
enough the first time around, but this time it’s downright scary.”
The problem, it seems, is that GWA automatically, silently, and
unblockably follows hypertext links to web pages that are linked to by
the pages you visit. It does this in order to cache those pages so
that if you visit them later, it will have cached copies ready in an
instant, thus “accelerating” your web surfing. But some web
developers use hypertext links to trigger potentially unsafe actions,
such as deleting records in a database, and when GWA automatically
follows such links, it triggers the actions.
Oops.
Let’s do the time warp again…
Now, if this story sounds familiar, that’s because half a year
ago, the exact same thing happened. GWA was unveiled to the public.
People started using it. And some of those people started losing data
from their accounts with popular web applications, such as
37signal’s own Backpack. 37signals
publicized the problem in their blog and DHH even
called for a recall on GWA.
And then the community responses came in. For the most part, the
responses could be divided into two camps, based on who was
blamed for the problem. The first camp blamed the web designers who used
links to trigger unsafe actions (in violation of applicable standards),
and the second camp blamed Google for unleashing GWA upon a web where
standards aren’t always followed.
Both viewpoints had some merit, but I was in the first camp and thus
argued for following the standards and against unsafe linking
practices:
What surprised me was that so many people in the second camp argued in
defense of unsafe linking practices, which I had thought indefensible.
I didn’t have any problem with arguments against Google’s unleashing
GWA on an imperfect web, but arguing for the web’s imperfections
seemed like an odd way of making the case. The supportive arguments
boiled down to the following:
- Lots of web sites use action-triggering links, so the practice is de facto acceptable.
- The existing palette of user-interface options is too limited for today’s web applications; thus, designers are justified in breaking the rules.
- The standards don’t actually prohibit the practice (they say “SHOULD NOT,” not “MUST NOT”); thus, the practice is allowable.
None of the arguments seem to withstand scrutiny. The first argument
breaks down like so: That lots of web sites do it only means that
those sites get away with it, not that the practice is acceptable.
Further, as GWA demonstrates, those sites may not get away with the
practice much longer.
The second argument breaks down when one examines the uses of unsafe
linking practices. Most of them could be replaced by safe practices
through modest UI refactoring. Given that safe alternatives exist,
the unsafe practices are not justified by virtue of being the only realistic option.
The third argument breaks down when one actually reads the relevant
standards. Then it becomes clear that one should not use links to
trigger potentially unsafe actions. The wiggle room created
by the use of “SHOULD NOT” instead of “MUST NOT” does not admit
the large problems caused by unsafe linking.
Finally, even if there were some justification for unsafe linking, the
practice would still be a bad idea: its costs and risks outweigh its
benefits. Why hold back the potential of efficient caching
architectures for the web? Why risk data loss for your users? It’s
not worth it.
Back to the Future
So where are we now? Given how little justification there is for
unsafe linking practices, one would hope that we would have abandoned
them by now. But, as the recent cries about the second coming of GWA
suggest, the web-development community is not yet ready to give up those
sexy, action-triggering links.
It’s not that the means aren’t available. Rails, for example, has
plenty of support for sane and safe practices for triggering actions.
Rather, the problem is cultural. Too many influential people,
especially in the Rails community, are unrepentant users of – and, dare
I say it, apologists for – action-triggering links. Until this changes, I
expect many new web developers to pick up dangerous habits from the
very people they respect most.
Fortunately, many other respect-worthy people are pointing toward
a better way:
- Sam Ruby: “I’m on the other side of this debate. While this appears to be a purely philosophical concern, in reality this stuff matters.”
- Bill de hÓra: “The GWA is back and following GET links again… The technology itself is interesting insofar as we are going to see more and more highly automated robots enter the web over the next few years…. Even more interesting is the kind of outrage holding forth in places like Signal v Noise….”
- Joe Gregorio : “And now we begin the next chapter in which Pooh discovers that five months after the first time Google turned on GWA that standards still matter.”
I hope that this time around the web-development community answers
the wake-up call. It’s time to abandon action-triggering links.
Posted in web development
Tags get, gwa, rails, rest, safe, unsafe
no comments
no trackbacks

Posted by Tom Moertel
Thu, 16 Jun 2005 16:00:00 GMT
I am delighted to report that the button_to
helper
has been added to the Ruby on Rails
web-development framework. David
applied the patch earlier
today, and so button_to will be in the much-anticipated Rails 1.0
release.
David’s change-log entry summarizes the patch well:
Added button_to as a form-based solution to deal with harmful
actions that should be hidden behind POSTs. This makes it just as
easy as link_to to create a safe trigger for actions like destroy,
although it’s limited by being a block element, the fixed look,
and a no-no inside other forms.
David does a good job of highlighting the helper’s limitations. I’ll
take this opportunity to elaborate on each.
It is a block element
The button_to helper creates a small form, which in HTML is considered
block content, just
like the p, div, and blockquote elements are. Basically, block
content cannot be mixed into runs of text. But links can: links are
inline content. Thus
button_to cannot be used as a drop-in replacement for every
occurrence of link_to that might be unsafe; it works only for those
occurrences within block-accepting contexts.
Luckily for us, when designers use links to trigger unsafe actions,
they rarely slip such links into the middle of ordinary looking
text. Naughty uses of link_to almost always occur within contexts
that accept block content. In Rails-generated scaffolding code, for
instance, the unsafe uses of link_to occur within table cells, and
table cells have a flow content
model, which accepts
both inline and block content. So button_to works great for the
default cases in Rails.
It has a fixed look
As its name implies, button_to creates buttons. Buttons don’t look
like links and aren’t styled the same way that links are. For some
design scenarios, this might be a problem.
(My view is that links should not be used to trigger unsafe
actions. In the same way that action-triggering GET requests violate
the spirit of the HTTP standards, action-triggering hypertext links
violate the spirit of the HTML standards. For this reason, I view this
limitation as a feature.)
It is a no-no inside other forms
Forms cannot be nested, and so button_to cannot be used inside of
forms.
Fortunately, this limitation usually doesn’t matter because when we
are inside of a form, we can use its buttons instead of
button_to-created buttons to trigger actions. Still, there are some
circumstances where it does matter, such as the “Amazon.com wish list”
scenario. In this scenario, we should consider other
options.
The bottom line: Pick the low-hanging fruit
While button_to has its limitations, it does provide a simple solution
to the unsafe-GET problem for most real-world cases. I am glad that it
is now a part of Rails, and I offer a big thank-you to David for
accepting the patch.
Posted in web development, rails
Tags get, gwa, link_to, post, rails, safe, unsafe
no comments
no trackbacks

Posted by Tom Moertel
Sun, 08 May 2005 16:00:00 GMT
As I wrote earlier, it’s time for web developers to do away with the fundamentally broken practice of using hypertext links to trigger dangerous events such as deleting things. One of the first places we ought to clean house is in the burgeoning Rails web-application framework, where this practice is pervasive.
The primary culprit in Rails is the all-too-easy link_to method, which is (presently) the orthodox means of creating links to any action, even unsafe ones. For example:
link_to "Destroy", :controller => 'accounts',
:action => 'destroy', :id => 6
The above code generates the following HTML hypertext link, which when followed will merrily delete account number 6:
<a href="/accounts/destroy/6">Destroy</a>
Because this practice is dangerous and contrary to the decade-old convention that links be safe, the link_to method thoughtfully lets us request that a Javascript confirmation dialog be tacked onto the link for added protection:
link_to "Destroy", ..., :confirm => "Are you sure?"
The resulting “safe” HTML:
<a href="/accounts/destroy/6"
onclick="return confirm('Are you sure?');">Destroy</a>
Unfortunately, the Javascript protection doesn’t work. First, not all web browsers care about it. Lots of people surf with Javascript turned off. Second, a whole slew of things besides web browsers live on the Internet, and almost all of them are oblivious to Javascript. Web crawlers fall into this category. They will be more than happy to follow any link you feed to them. “Hey, Googlebot just deleted every account in our database!” Oops.
Thus another layer of protection is commonly used: authorization. The theory is that dangerous links can be safely corralled in the private parts of a web application, where the public and web crawlers cannot go. Only authorized users can get into those parts, and those users will be smart enough not to click on the truly dangerous links unless they really mean it.
The problem is, any number of intermediary agents can be operating on behalf of an authorized user, and these agents are free to do anything the user is allowed to do, such as follow dangerous links. Google’s Web Accelerator is one such agent. It tries to make your surfing faster by (among other things) pre-fetching the resources that are linked to on the pages you visit. And what happens if you, an authorized user, visit a page containing dangerous links? That’s right, Web Accelerator will fetch the “resources” those links point to – and delete a bunch of your stuff.
I hope by this point that I have argued convincingly that using links for unsafe actions is a bad idea. Even if you feel justified in ignoring the applicable parts of the HTTP RFCs, it’s a bad idea. Even if you tack on Javascript confirmations and hide your links in authorization-protected zones of your site, it’s a bad idea. It is, all around, a bad idea. Don’t do it.
So what alternatives are there? Read on for one possibility, button_to.
Read more...
Posted in ruby, web development, rails
Tags get, gwa, link_to, post, rails, safe, unsafe
9 comments
no trackbacks

Posted by Tom Moertel
Sat, 07 May 2005 01:55:00 GMT
Google Web Accelerator offers web developers an important opportunity
There has been a lot of heat lately about Google’s Web Accelerator (GWA)
exposing serious problems in some popular web
applications. The problem, in short, is that these applications use
GET-based links that when followed perform dangerous actions such as
deleting records in a database. According to web standards going back
a decade, that is a no-no: Links should be safe to follow. Thus GWA,
expecting links to be safe, tries to help you out by pre-fetching
various resources that are linked to by the pages you
visit. Unfortunately, if the page you happen to be visiting contains
lots of dangerous links, GWA will innocently try to pre-fetch the
“resources” that the links point to, and in doing so will accidentally
delete a bunch of stuff. Oops.
That’s the backdrop for our real story, which is the response from the
community of web developers. What I find fascinating, and somewhat
disheartening, is the number of people who say the problem is Google’s
to fix. Yes, there are a lot of broken web apps out there, and Google
could have been smarter about working around the minefields those apps
represent. But that’s a side problem. The real problem is that there
are a lot of broken web apps out there, and they do represent
minefields. Worse, a lot of web developers think it is acceptable to
brush aside fundamental conventions of the web going back a decade
when they find it sexier to use GET instead of POST.
What these developers overlook is that the web is not a bunch of
colorful pages with buttons, clickable links, and pretty
pictures. Rather the web is a distributed collection of hypertext
documents, each of which has a meaning that is given by standards that
most people have agreed to follow. While the collection may look like
a bunch of colorful pages in one particular visual presentation, it
really, truly is not.
Nevertheless, many web applications are designed with the prevailing
mindset that the meaning of the web is nothing more than how it looks
and behaves in a web browser. Even if those web applications are not
intended for use outside of a few approved browsers – the escape
hatch that is often used to justify departures from the
standards – this mindset is wrong.
The reason it is wrong is because, like it or not, web applications
are implemented in terms of protocols and languages that mean
something. The bits and pieces of a web application each mean
something, even if what they mean is not in harmony with the
designer’s overall vision. What a web designer may see as a bold, red
link that says “delete” and is protected by a Javascript confirmation
dialog, is actually a hypertext reference – a pointer
to another resource that the standards say should not be dangerous to
follow. Thus there is a fundamental mismatch between what the designer
is trying to say – “follow this link only if you
really, no-kidding want to delete something” – and
what his markup actually means: “this link is safe to follow.”
The bottom line is that hypertext links are supposed to be
safe. Dangerous links can’t be made “safe” by layering tricks on
them. Wrapping them with Javascript confirmation dialogs doesn’t make
them safe. This trick fails fundamentally because the meaning of a
link doesn’t change when there is Javascript associated with it. It
fails practically because not all consumers of hypertext evaluate
Javascript, nor do any standards suggest that they should. (This is
also why client-side validation cannot be trusted on the server side.)
Hiding dangerous links in authorization-controlled portions of a web
application does not make them safe, either. This trick might shield
the links from external spiders, but the standards allow for any
number of intermediary agents (such as Google’s Web Accelerator) to
work on behalf of an authorized user. Anything the user is authorized
to do, so are the user’s agents. If the user can click the “delete”
link, so can the agents.
Let’s answer the wake-up call.
GWA is only the first of a new breed of
smarter user agents that promise to make the web a better place for
all of us. If you’re a web developer, take the slew of problems that
GWA uncovered as a wake-up call. Even if Google works around your
problems, other user agents may not. As developers, it’s time to admit
our mistakes and fix the stupid things that our web applications do.
First, let’s drive a stake in the notion that dangerous links are OK
if we’re “careful.” They’re not. Dangerous links are
lazy. “Confirming” them with Javascript or “hiding” them behind
authorization doesn’t work.
Second, let’s clean house. Let’s find those places where we have laced
our web applications with dangerous links and remove them. Break out
the forms! Long live the POST!
Third, let’s take a look at how we got into this mess and try to learn
from our mistakes. Do we value sexiness more than substance? We like
to think that form follows function and that good design and good
implementation go hand in hand, but the reality of this debacle
suggests otherwise. I think the truth is that on the web, sexiness and
hype are what gets attention, and we seek attention more than we like
to admit.
Maybe the best thing for us is a good dose of humility. And, come to
think of it, that’s just what Google’s Web Accelerator offered us.
Posted in web development
Tags get, gwa, post, rails, safe, unsafe
no comments
no trackbacks
