Closures and the professional programmer

Tom Moertel — Tue, 30 Aug 2005 14:56:00 -0400

I came across Tim Bray’s thoughts on Ruby via the ever-delightful Lambda the Ultimate and found the following bit fascinating:

I’ve had access to languages with closures and continuations and suchlike constructs for years and years, and I’ve never ever written one. While I’m impressed by how natural this stuff is in Ruby, I’m still unconvinced that these are a necessary part of the professional programmer’s arsenal. [Emphasis mine.]

While Tim Bray may be unconvinced, I am a true believer.

I use closures so much that I feel cheated into doing busy work by languages that do not support them. I use continuations less often but frequently enough to appreciate how much time they save me. Neither is strictly required for professional work, but they are potent tools, and a professional who knows how to use them has an advantage over those who do not.

Closures, in particular, are something every professional ought to master. Besides their more celebrated uses, closures make refactoring practical on a small scale. For example, consider the following Ruby method, which we will assume is one of several similar methods belonging to a class that implements some kind of Internet server:

def process
  sn = next_serial()
  log.info "process/#{sn}: stage 1"
  # ... do some work
  log.info "process/#{sn}: stage 2"
  # ... do some more work
  log.info "process/#{sn}: finished"
end

The method first gets a unique serial number, which is used during the processing of requests and also to relate log entries generated by the same processing call. Then the method does its work, logging each stage in passing.

The method makes three logging calls that each hardcode the logger, the logging level, and the format of the log entries. Since these things are repetitive and could very well change, we probably ought to factor them out into an isolated method. After all, we don’t want to rewrite a bucket of logging calls if the log-entry format changes.

Let’s introduce a helper method mylog to hold the common pieces:

def process
  sn = next_serial()
  mylog("process", sn, "stage 1")
  # ... do some work
  mylog("process", sn, "stage 2")
  # ... do some more work
  mylog("process", sn, "finished")
end

def mylog(activity, sn, msg)
  log.info "#{activity}/#{sn} #{msg}"
end

While we managed to isolate the logger, the logging level, and the format of our logging messages, just calling our helper method mylog still requires much redundancy. Worse, the redundancy is on such a low level that we can’t factor it out with another helper method – calling the new helper would be as expensive and redundant as calling mylog directly.

What we need are refactoring tools that scale down to this sub-method level, and that’s where closures come to the rescue. Using them, we can corral the remaining redundancy with a local logging helper llog that “closes over” the relevant state:

def process
  sn = next_serial()
  llog = lambda { |s| mylog("process", sn, s) }
  llog["stage 1"]
  # ... do some work
  llog["stage 2"]
  # ... do some more work
  llog["finished"]
end

Notice how much simpler and less redundant the logging code is? Each stage can now be logged just by giving its name to llog. We don’t need to pass in the activity name or serial number because llog already knows them both. It knows the activity name because we made it part of llog’s definition, but it knows the serial number because sn is captured in llog’s closure – for free. (Note: If f is a Proc object, f[args] is syntactic sugar for f.call(args).)

Of course, if we have several methods that require a unique serial number and a corresponding logger, we could (thanks to closures) factor things further:

def process
  sn, llog = next_serial_and_logger("process")
  llog["stage 1"]
  # ... do some work
  llog["stage 2"]
  # ... do some more work
  llog["finished"]
end

def next_serial_and_logger(activity)
  sn = next_serial()
  [sn, lambda { |s| mylog(activity, sn, s) }]
end

You get the point: Closures reduce the cost of working with local state because they capture it implicitly. There is no need to pass the state back and forth; it’s simply there.

Because the craft of programming is dominated by writing the stuff inside of methods, where local state lives, the potential benefits of closures are immense. If more programmers knew how to use them, the profession would be richer for it.

Update 2006-01-11: First, because this article is getting renewed interest, I have turned the comments back on. Second, I edited the article to improve clarity.

Update 2007-05-04: Added syntax highlighting to code snippets.

Tom Moertel's Weblog: Tag closures

Closures and the professional programmer