Tom Moertel's Weblog: Tag darcs

How I stopped missing Darcs and started loving Git

Tom Moertel — Mon, 10 Dec 2007 16:52:00 -0500

About three years ago, I switched to Darcs as my primary source-code management system. It was simple, intuitive, and powerful, and it made managing my projects more fun and less frustrating than any centralized VCS ever had. That it was written in Haskell, one of my favorite programming languages, made it even better. I was hooked.

Since then, the distributed SCM landscape has changed. Darcs hasn’t improved much, but its competitors have made long strides, especially Git and Mercurial. Both are crazy fast, vigorously developed, and widely used on large, highly active real-world projects, such as the Linux kernel and Mozilla 2. In comparison, Darcs has stagnated.

When I started working for a new company recently, I had to consider whether to advocate Darcs or something else. In the end, I decided that Darcs would be a hard sell. Nobody else at the company uses Haskell, and having to explain how to avoid the occasional corner case seemed liked a losing proposition.

After researching and playing around with Git and Mercurial, I settled on Git. I like Git’s underlying hashed-blobs model better than Mercurial’s revlogs, and Git seems to have slightly more development momentum. Still, it was a close call. Either choice would have been completely reasonable.

Missing Darcs

When I started using Git on real projects, the one thing I really missed was the ability to easily amend earlier patches, something Darcs made trivial. Let me explain. The typical development workflow goes something like this:

Checkout copy of upstream code base.
Implement feature X.
Commit.
Implement independent feature Y.
Commit.
Implement independent feature Z.
Commit.
Push new features back upstream.

Now, what really happens is that when I’m implementing Y or Z, I’ll realize that I made a mistake in X. The trick is then fixing X so that my fix is part of the changeset/patch for X that ultimately gets pushed upstream in the last step. That way, the upstream folks will see only a single, clean patch for feature X – not a mishmash of patches that together represent X.

In Darcs, amending the original patch is easy because its patch theory lets me tweak the patch for X independently of the other patches. Darcs will simply ask me which patch I want to amend, and I’ll select the orignal patch for X:

$ emacs               # fix X
$ darcs amend-record  # amend original patch for X

Mon Dec 10 14:43:13 EST 2007  Tom Moertel <tom@moertel.com>
  * Implemented Z
Shall I amend this patch? [yNvpq], or ? for help: n

Mon Dec 10 14:42:12 EST 2007  Tom Moertel <tom@moertel.com>
  * Implemented Y
Shall I amend this patch? [yNvpq], or ? for help: n

Mon Dec 10 14:41:46 EST 2007  Tom Moertel <tom@moertel.com>
  * Implemented X
Shall I amend this patch? [yNvpq], or ? for help: y
hunk ./x 1
-X1
+X2
Shall I add this change? (1/?)  [ynWsfqadjkc], or ? for help: y
Finished amending patch:
Mon Dec 10 14:43:25 EST 2007  Tom Moertel <tom@moertel.com>
  * Implemented X

That’s it. The exact same process will work regardless of when I realize I need to fix X: before I start Y, while I’m implementing Y, after I’ve committed Y, while I’m working on Z, or after I’ve committed Z.

Learning to love Git

With Git, however, I can amend a commit only if I haven’t committed anything else before making my fix. In Git’s mind, Y depends on X, and Z depends on Y, even if they really are independent of one another.

So if I commit the original patch for X and then immediately realize I need to make a fix, before I start working on Y or Z, it’s easy:

$ emacs               # implement X
$ git commit -m 'Implemented X'

# discover problem in X

$ emacs               # fix X
$ git commit --amend  # amend original patch

More typically, it’s only while I’m working on Y that I’ll realize I need to fix X. Then it’s more complicated to amend the original commit:

$ emacs               # implement X
$ git commit -m 'Implemented X'
$ emacs               # start working on Y

# discover problem in X

$ git stash           # stash away half-completed work on Y
$ emacs               # fix X
$ git commit --amend  # amend original patch for X
$ git stash apply     # restore work on Y
$ emacs               # continue working on Y

While not as convenient as Darcs’s workflow, it’s perfectly workable.

Now let’s consider another fairly typical case: I commit X and Y and then start working on Z before I notice the problem in X. I used to think that Git couldn’t handle this case, but it can, thanks to git rebase --interactive:

$ emacs               # implement X
$ git commit -m 'Implemented X'
$ emacs               # implement Y
$ git commit -m 'Implemented Y'
$ emacs               # start working on Z

# discover problem in X

$ git stash           # stash away half-completed work on Z
$ emacs               # fix X
$ git commit -m 'Fixed X'
$ git rebase --interactive HEAD~3  # see comments below
$ git stash apply     # restore work on Z
$ emacs               # continue working on Z

The git rebase --interactive command is powerful. What the command does, as called in the snippet above, is invoke my editor of choice on a text file describing the last 3 commits (that’s the HEAD~3 part):

# Rebasing 3ad99a7..b9a8405 onto 3ad99a7
#
# Commands:
#  pick = use commit
#  edit = use commit, but stop for amending
#  squash = use commit, but meld into previous commit
#
# If you remove a line here THAT COMMIT WILL BE LOST.
#
pick 0885540 Implemented X
pick 320b115 Implemented Y
pick b9a8405 Fixed X

I can then edit the file to reorder, merge (squash), and/or remove the commits. In this example, I want to merge the fix for X into the original commit that implemented X. So I edit the file like so:

pick 0885540 Implemented X
squash b9a8405 Fixed X
pick 320b115 Implemented Y

Then I save the file, at which point Git takes over and makes the requested changes, merging the fix for X into the original commit for X. Now the log shows the original implementation and fix as one commit:

$ git log
commit f387d650976246c0854d028b040cca40e542be56
Author: Tom Moertel <tom@moertel.com>
Date:   Mon Dec 10 15:11:26 2007 -0500

    Implemented Y

commit 82a1c849ffd1bd688d5bc9d99be0e63548a89c4c
Author: Tom Moertel <tom@moertel.com>
Date:   Mon Dec 10 15:13:03 2007 -0500

    Implemented X

    Fixed X

commit 3ad99a7ef537b7ae99e435e0d2b4b0d03de92c65
Author: Tom Moertel <tom@moertel.com>
Date:   Mon Dec 10 15:11:14 2007 -0500

    Initial checkin

Once I figured out how to use git rebase --interactive, I stopped missing Darcs and started loving Git.

Practical differences between Darcs and Git/Mercurial

Tom Moertel — Thu, 25 Oct 2007 16:26:00 -0400

On the Darcs Users mailing list, I ran across an interesting thread: practical differences between darcs’ patch model and git/mercurial’s?

Among the interesting points of discussion:

Do the mechanics that give rise to Darcs’s strong cherry-picking abilities also make it susceptible to naughty time-complexity behavior?
When you merge non-conflicting changes in Git or Mercurial, you must record a merge patch, which binds the two in the development timeline, but in Darcs the respective patches are free to commute. Which behavior is better for real-world development?

If you’re interested in distributed source-code management, it’s an interesting thread to follow.

Some recent reviews of distributed source-code-management systems

Tom Moertel — Mon, 14 Aug 2006 13:50:00 -0400

John Goerzen recently compared a bunch of distributed source-code-management systems in Whose Distributed VCS Is The Most Distributed? His comparison includes all of the major contenders except for SVK and monotone. He ends up favoring Darcs, which I also prefer and use to manage my projects’ code. If you’re looking for a quick overview of distributed SCM options, check out John’s comparison.

Also check out Bryce “Zooko” Wilcox-O’Hearn’s Quick Reference Guide to Free Software Decentralized Revision Control Systems, which is updated regularly. (He also likes Darcs.)

Update: fixed small typo.

Source code management with darcs: a first look

Tom Moertel — Sat, 12 Feb 2005 12:00:00 -0500

I have been managing the LectroTest project with the monotone revision control system. For the last few months, monotone has been undergoing some growing pains that have made it less stable than I would like for everyday use. Thus I thought that I would give darcs a try.

I have been following the progress of darcs since it was first announced on the Haskell-Cafe mailing list on 9 April 2003. Darcs is written in Haskell, one of my favorite programming languages, and that was my initial draw. Still, until yesterday I had never used it for any of my projects because I felt it was immature and needed some more real-world testing before I committed work to it.

In the last three months, Darcs has gained mainstream attention (triggered by a favorable write-up in Linux Weekly News) and a growing user base. Under the gaze of these new eyeballs, darcs has matured much. I thought it was time for another look.

Darcs has a small, easy-to-understand command set and yet offers "modern" source-code management features such as distributed development (via HTTP, ssh, and email), change sets, and cherry picking. Want to start an experimental branch of your project? Just check out another copy and use it for the branch. Each working copy is a complete, independent repo. Want to publish a project repository to the world? Just copy it to a public web server. Want to start working on someone else's project? A single "darcs get http://other.project.com/project" gives you a complete, stand-alone copy. Your own personal branch. Start hacking.

To try darcs on something I was familiar with, I decided manage my LectroTest development with it. The first thing I did was change to the LT working directory and use "darcs init" to create a darcs repository there.

[tom@bifur Flippi]$ cd ~/work/research/perl/qc/  # LT root dir
[tom@bifur qc]$ darcs init
[tom@bifur qc]$ l
blib/                       mt.db
_build/                     mtdb.dump
Build*                      perl-Test-LectroTest-0.2007-1.src.rpm
Build.PL                    pod2htmd.tmp
Build.PL~                   pod2htmi.tmp
buildrpm*                   posts/
buildrpm~                   prop2.pl
Changes                     prop2.pl~
Changes~                    README
checkpods*                  t/
checkpods~                  Test-LectroTest-0.2001.tar.gz
ctime.pl                    Test-LectroTest-0.2002.tar.gz
ctime.pl~                   Test-LectroTest-0.2003.tar.gz
CVS/                        Test-LectroTest-0.2004.tar
_darcs/                     Test-LectroTest-0.2004.tar.gz
Example1.pl~                Test-LectroTest-0.2005.tar.gz
lib/                        Test-LectroTest-0.2006.tar.gz
Makefile.PL                 Test-LectroTest-0.2007.tar.gz
MANIFEST                    Test-LectroTest-0.2008.tar.gz
MANIFEST~                   Test-LectroTest-0.2009.tar.gz
MANIFEST.bak                Test-LectroTest-0.201.tar.gz
MANIFEST.SKIP               tex/
MANIFEST.SKIP~              THANKS
META.yml                    THANKS~
monotone.db                 TODO
monotone.db.bak             TODO~
monotone.db.pre-changesets  toms-notes.txt
monotone.db-pre-sql3        toms-notes.txt~
MT/

You can see that there is a lot of accumulated cruft in my working directory, including CVS, monotone, and now darcs revision-control artifacts. To prevent Perl’s Module::Build from thinking the _darcs directory is meaningful, I added it to the manifest-skip file.

[tom@bifur qc]$ echo '\b_darcs\b' >> MANIFEST.SKIP

Next I added my LectroTest sources, docs, and related files to the darcs repo.

[tom@bifur qc]$ darcs add Build.PL buildrpm Changes \
    checkpods MANIFEST MANIFEST.SKIP tex THANKS TODO \
    toms-notes.txt tex/Makefile tex/titlepage.ltx

The “darcs whatsnew” command asks darcs to tell me what is changed in the working directory with respect to the repository state.

[tom@bifur qc]$ darcs whatsnew -s
A ./Build.PL
A ./Changes
A ./MANIFEST
A ./MANIFEST.SKIP
A ./THANKS
A ./TODO
A ./buildrpm
A ./checkpods
A ./tex/
A ./tex/Makefile
A ./tex/titlepage.ltx
A ./toms-notes.txt

The files that I added are new because I had not yet recorded them to the repository. Before I did that, I added the remaining LT assets.

[tom@bifur qc]$ darcs add t  # add the tests dir
[tom@bifur qc]$ darcs add t/*.t
[tom@bifur qc]$ darcs add lib
[tom@bifur qc]$ cd lib
[tom@bifur lib]$ l
Test/
[tom@bifur lib]$ darcs add Test
[tom@bifur lib]$ cd Test
[tom@bifur Test]$ l
LectroTest/    LectroTest.pm~     LectroTest::Tutorial.pod~
LectroTest.pm  LectroTest.pm.bak
[tom@bifur Test]$ darcs add LectroTest LectroTest.pm
[tom@bifur Test]$ cd LectroTest
[tom@bifur LectroTest]$ l
Compat.pm      Generator.pm~     Simple.pm~         Tutorial.pod
Compat.pm~     Generator.pm.bak  Test.pm~           Tutorial.pod~
Compat.pm.bak  Property.pm       TestRunner.pm      Tutorial.pod.bak
CVS/           Property.pm~      TestRunner.pm~
Generator.pm   Property.pm.bak   TestRunner.pm.bak
[tom@bifur LectroTest]$ darcs add *.pm *.pod

At this point, it looked like I had all of the files under darcs’s watchful eye.

[tom@bifur LectroTest]$ darcs w -s  # abbreviated: w -> whatsnew
A ./Build.PL
A ./Changes
A ./MANIFEST
A ./MANIFEST.SKIP
A ./THANKS
A ./TODO
A ./buildrpm
A ./checkpods
A ./lib/
A ./lib/Test/
A ./lib/Test/LectroTest/
A ./lib/Test/LectroTest.pm
A ./lib/Test/LectroTest/Compat.pm
A ./lib/Test/LectroTest/Generator.pm
A ./lib/Test/LectroTest/Property.pm
A ./lib/Test/LectroTest/TestRunner.pm
A ./lib/Test/LectroTest/Tutorial.pod
A ./t/
A ./t/001.t
A ./t/002.t
A ./t/003.t
A ./t/004.t
A ./t/005.t
A ./t/compat.t
A ./tex/
A ./tex/Makefile
A ./tex/titlepage.ltx
A ./toms-notes.txt

[tom@bifur LectroTest]$ cd ../../..  # back up to project home

That looked right. It was time to record my changes. This was straightforward.

[tom@bifur qc]$ darcs record --all    # record all changes

Darcs needs to know what name (conventionally an email address) to use
as the patch author, e.g. 'Fred Bloggs <fred@bloggs.invalid>'.
If you provide one now it will be stored in the file
'_darcs/prefs/author' and used as a default in the future.  To change
your preferred author address, simply delete or edit this file.

What is your email address? Tom Moertel <tom@moertel.com>
What is the patch name? Initial checkin of sources
Do you want to add a long comment? [yn] n
Finished recording patch 'Initial checkin of sources'

Now what did darcs think has changed?

[tom@bifur qc]$ darcs w -s
No changes!

Excellent.

One cool feature of darcs is that every working directory is also a complete, independent repository. To make a branch, then, is as simple as checking out a new repository.

Of course, because there is no central repository in the darcs model, “checking out” is a concept that does not really apply. Rather, what I must do is set up a new repository and then “push” my existing repository’s patches to it. I can push in many ways, including via ssh to a remotely hosted repository, but here I will just set up a new repo in /tmp and push to it on the local filesystem.

[tom@bifur qc]$ mkdir /tmp/lt && pushd /tmp/lt
/tmp/lt ~/work/research/perl/qc
[tom@bifur lt]$ darcs init   # set up new repo at /tmp/lt
[tom@bifur lt]$ popd
~/work/research/perl/qc
[tom@bifur qc]$ darcs push /tmp/lt   # push to repo at /tmp/lt
Sat Feb 12 01:26:15 EST 2005  Tom Moertel <tom@moertel.com>
  * Initial checkin of sources
Shall I push this patch? (1/1) [ynWvxqadjk], or ? for help: y
Finished applying...

Now, I can begin working on my new branch in the /tmp/lt working directory.

[tom@bifur qc]$ cd /tmp/lt
[tom@bifur lt]$ l
Build.PL  Changes    _darcs/  MANIFEST       posts/  tex/    TODO
buildrpm  checkpods  lib/     MANIFEST.SKIP  t/      THANKS  toms-notes.txt
[tom@bifur lt]$ emacs lib/Test/LectroTest.pm   # fix typo
[tom@bifur lt]$ darcs record
hunk ./lib/Test/LectroTest.pm 38
-of your software.  LectroTest then checks your software see whether
+of your software.  LectroTest then checks your software to see whether
Shall I record this patch? (1/1) [ynWsfqadjk], or ? for help: y

What is the patch name? Fixed stupid typo in intro text of T::LectroTest.pm
Do you want to add a long comment? [yn] n
Finished recording patch 'Fixed stupid typo in intro text of T::LectroTest.pm'

Now my branch repository contains two patches:

[tom@bifur lt]$ darcs changes
Sat Feb 12 13:20:07 EST 2005  Tom Moertel <tom@moertel.com>
  * Fixed stupid typo in intro text of T::LectroTest.pm
Sat Feb 12 01:26:15 EST 2005  Tom Moertel <tom@moertel.com>
  * Initial checkin of sources

Because the typo that I fixed is not unique to my new branch, I ought to make sure that the original branch gets the fix, too. To do so, I just push it:

[tom@bifur lt]$ darcs push ~/work/research/perl/qc
Pushing to /home/thor/work/research/perl/qc...
Sat Feb 12 13:20:07 EST 2005  Tom Moertel <tom@moertel.com>
  * Fixed stupid typo in intro text of T::LectroTest.pm
Shall I push this patch? (1/1) [ynWvxqadjk], or ? for help: y
Finished applying...

And now my patch has been pushed back up to the mainstream branch! This is an attractive development model.

So far, I like darcs. Its source code–management model is simple and powerful. Its command set is small enough to actually grok. Using darcs has me wondering why other SCM systems have made the problem seem so complicated. My life is complicated enough as it is.

I think I just switched to darcs.