Archive

Two steps forward, one step back

Published
by
lennon
on February 11, 2009
in Uncategorized and code
. 0 Comments

Once upon a time, there was RCS, and then CVS. They tracked normal edits to a set of text files reasonbly well, and coupled with telnet or ssh, even made it relatively straightforward for a trusted group of collaborators to share their changes with each other. Some people used other proprietary tools (Perforce, Visual Source Safe, etc.) but they tended to be either a) expensive b) really, really lousy or c) both. Among the open source crowd, at least, CVS dominated the version control space for many years.

Then came Subversion. It improved on many of the failings of CVS — notably, Windows support was dramatically better, repositories could be shared over HTTP, and many operations that just didn’t work in CVS (renames, binary diffs, etc.) performed reasonably well out of the box. To this day, Subversion is a reasonable choice for many projects, especially given the advanced level of support for it in IDEs, graphical repository browsers, and the like.

Much of the reason for that diversity of useful tooling built atop Subversion, of course, is that it was written in C, and built with an eye towards allowing high-level languages to use bindings into the same runtime libraries upon which the ’svn’ command itself relied. In fact, Python, Perl, Java, and Ruby are all supported by the core Subversion maintainers, and additional bindings using those same underlying libraries are available for a number of other languages.

Enter the distributed version control systems: Git, Mercurial, Bazaar, Darcs, and their ilk. The basic workflow they offer is in some ways more like RCS than it is Subversion: each developer works locally against their own copy of a repository, and they share their work via patch files and periodic synchronization. (This is of course a gross over-simplifaction, as all of them offer much more sophisicated change-tracking under the hood than RCS did, but the user-visible behavior is still reminiscent.) However, their ability to maintain change history across many developers and systems without forcing everyone to eventually squash their work down into a single source tree makes a number of new modes of project management possible, or at least much easier than before.

All of the above DVCS systems potentially offer a huge gain of productivity for many developers, since you can easily experiment with changes locally, selectively share only those modifications which you wish to, and continue working without being connected to the central repository. (This is especially significant for those whose employers maintain draconian firewall rules and disallow off-site access to their source control.)

Unfortunately, none of the popular DVCS systems have anything resembling the level of cross-language API support that Subversion does. Mercurial and Bazaar are both implemented in Python, making access from other Python code quite fast, and that from any other language painfully slow. Git is implemented in C, but without a supported and documented core library of functions designed to be used to facilitate access from other languages. Darcs is written in Haskell, which means only crazy mathematicians and CS majors have any ability or interest in using it. (I’m kidding here, but the point remains that Haskell isn’t exactly the most useful substrate for scripting language bindings.)

The fallout from all of this is that we’re left using wrapper libraries which fork out to the command-line tools for each DVCS. Such wrappers have a number of problems: the performance sucks, the internal APIs are usually only as robust as the set of regular expressions you write to parse the output of the commands, and almost no work is shared between the various wrapper implementations.

Don’t get me wrong: as a simple version control tool, I’ve found Git in particular (and distributed version control in general) to be a big step up from the old centralized-repository model. However, the very eighties-esque fork-and-regexp-scrape model for IPC — coupled with the lack of an obvious “best of breed” leader in the DVCS space — means that I (along with anyone else trying to support DVCS in a general-purpose way) end up doing a lot of low-level grunt work when we could be building real value for users.

Even something as simple as a standard dump format for a common subset of the information available from the popular DVCS types would be a start. I do know that, for the time being, I’m stuck supporting a bunch of very brittle code which relies on the various idiosyncratic console output formats of each version-control system.

Playing prognosticator, I would even go so far as to suggest that the first DVCS system to provide supported, documented interfaces in a number of popular programming languages could climb to the top of the dogpile that exists currently and emerge as a clear standard.

Inauguration playlist

Published
by
lennon
on January 21, 2009
in Uncategorized
. 1 Comment

We had a little family dance party in the street (no, seriously, we did — pictures forthcoming) after re-watching all the coverage of the inauguration tonight.

Our playlist:

  1. The Payback — James Brown
  2. Song 2 — Blur
  3. Gone Daddy Gone — Gnarls Barkley
  4. The Yeah Yeah Yeah Song — The Flaming Lips
  5. The Golden Path (Ewan Pearson Extended Vocal) — The Chemical Brothers Featuring The Flaming Lips
  6. Paper Planes — M.I.A.
  7. All My Friends — LCD Soundsystem
  8. My People — The Presets
  9. Fear Not Of Man — Mos Def
  10. Work On You — MSTRKRFT
  11. Rawnald Gregory Erickson the Second — Starfucker
  12. My Favorite Things — Outkast

The theme is obvious, but we all enjoyed the hell out of it.

Daily git-svn

Published
by
lennon
on January 20, 2009
in code
. 4 Comments

My team at Sun uses Subversion to host our “authoritative” source repository for Project Kenai. However, since most work is done on the trunk, many of us find it more convenient to work locally with Git, using the git svn subcommand heavily to keep ourselves up-to-date without interfering with others’ work.

When I first started using this combo, I had some early trouble keeping my local Git repository from getting horribly b0rked whenever there were edits made to the same files I had been working on locally. Having used CVS and Subversion for so long, I initially assumed such conflicts (and the manual merge steps they entailed) were simply part of the equation, even when working with a proper DVCS. However, by applying a little more discipline to my use of local branches, I’ve been able to basically eliminate manual merges, except in cases where the exact same line has literally been edited by multiple people.

My first, most critical discovery was to never use the fetch command. Instead, use rebase. Second, never pull the latest changes from Subversion into a working feature branch; instead, switch to your master branch, create a new branch for merging (I usually call mine “svn-merge”), and do your rebase there. After the rebase has finished, merge in your feature branch changes, and then use dcommit to push your changes upstream.

As an example, here are to commands I would use to check out a new local Git clone of the main Subverion repository, work on a single command, and then push it back into SVN:

viper:Work$ git svn clone https://example.com/svn/repo/trunk -r500:HEAD repo
# ... lots of Git output here ...
viper:Work$ cd repo
viper:repo$ git checkout -b issue-123
# ... hack, hack, hack...
viper:repo$ git commit -m "fix for issue #123"
viper:repo$ git checkout master
viper:repo$ git checkout -b svn-merge
viper:repo$ git svn rebase
# ... watch results for conflicts ...
viper:repo$ git merge --squash issue-123
viper:repo$ git commit -m "ISSUE-123: fixed"
viper:repo$ git svn dcommit -e
# ... $EDITOR launches, allows you to write useful commit message for svn ...
viper:repo$ git checkout master
viper:repo$ git merge svn-merge

This may seem like a lot of extra branch switching, localized commits, etc., but the end result has been worth it (for me, at least). If you following this process, you can be relatively certain that your master branch will only ever mirror changes that have been made in Subversion.

Insuring that the master branch is always “clean” (i.e., has no conflicting commits) with regard to the shared svn tree makes it easy to switch temporarily to another feature branch if you have an urgent bugfix or simple change to make, while your bigger changes happily sit on another feature branch waiting to be pushed.

Updated Mar. 4, 2009: Changed merge to use --squash option, so that many local Git commits can be combined into a single upstream revision.

Testing

Published
by
lennon
on January 13, 2009
in Uncategorized
. 0 Comments

Is this thing on?

Looks like mobile blogging is a go.

Quick status update

Published
by
lennon
on December 15, 2008
in Uncategorized
. 0 Comments

It’s been two weeks since I posted, which is a bit embarassing. I’ve been pretty busy with the new job, though, and generally sticking to Twitter to get the word out about how I’m doing.

The quick version is: the Project Kenai team is turning out to be just about as good a group to work with as I could hope for. We’ve got enough to do to keep things from getting boring, but it’s the good kind of work: interesting + challenging, but nothing that feels like a death march. Plus, every time I fire up an editor to look at a new piece of code and see the GPL license in the header comments, I feel a little better about the company as a whole. Being somewhere that open source is the default (rather than a special case for which you have to lobby) is a pretty cool feeling for a OSS nerd like me.

Otherwise, things are pretty normal. As my family and friends all give in to the gravitational pull of SE Portland, I also find that my social life is less and less about going out, and more about staying in for social meals + conversation, which suits me just fine, especially in the winter.

That’s it for now. Expect more on the technical side of the work I’m doing after Christmas, when I have some time to write up my impressions of doing JRuby on Rails, and working in a heterogenous Solaris/Linux/OS X environment.

Home charcuterie

Published
by
lennon
on November 30, 2008
in food and photos
. 3 Comments

Those who have followed this blog for a while (or have met me in person) already know that I’m a fan of home-made charcuterie: bacon, sausage, ham, pastrami, etc. Over the last few months, the quality and consistency of said DIY projects has gone up considerably from our initial wonderful-but-inconsistent results.

Behold:

DSC_0012

bacon

While I didn’t actually prepare the cure for either of these batches, I did take at least my share of time at the smoker to insure their juicy-salty-goodness.

Killfile 2.0

Published
by
lennon
on November 25, 2008
in code
. 1 Comment

There’s been an persistent blog-wank-fest making the rounds over the last few weeks about the state of the Ruby community: whether it’s become more or less “fun”, “creative”, etc. I’m not going to reward any of the participants with a link, but I do offer the following balm if you, like me, are a bit sick of hearing about it:

%w(open-uri rss resolv uri rubygems hpricot).each {|lib| require lib }

blog_url = ARGV.shift

blog_hdoc = Hpricot.parse(open(blog_url))
rss_links = blog_hdoc / :head / 'link[@type="application/rss+xml"]'

feed_rss = RSS::Parser.parse(rss_links.first['href'])

rants = feed_rss.items.select {|i| t = i.title; t =~ /rant/i && t =~ /ruby/i }

if rants.empty?
  puts "Okay, you get a pass."
else
  puts "Bad blogger! No biscuit!"
  hostname = URI.parse(blog_url).host
  ip_addr = Resolv.getaddress(hostname)
  `sudo route add -host #{hostname} gw 127.0.0.1`
end

The Me Meme

Published
by
lennon
on November 14, 2008
in Uncategorized
. 1 Comment

  1. Take a picture of yourself right now.
  2. Don’t change your clothes, don’t fix your hair…just take a picture. (should be super-easy with Photobooth)
  3. Post that picture with NO editing.
  4. Post these instructions with your picture.

(via Fredrick Jean)

Granting just enough power, but no more

Published
by
lennon
on November 12, 2008
in code
. 2 Comments

Sometimes you have a group of users who need to run certain commands on a server, and no others. It’s not necessarily that you don’t trust them. The point is simply that they don’t need a full-blown shell account, and you’re understandably reluctant to give it to them.

There are countless ways to set up a restricted account that can use only certain commands, but most of them are either extremely special-purpose, or rather difficult to set up. (Locking down SSH sessions inside a chroot jail, for example, requires almost as work as just setting up a dedicated virtual machine for your untrusted users.)

Furthermore, none of the existing solutions were written by me, to address exactly the needs I have for such a “sandbox” environment. Most notably, I don’t want users to even have to remember which commands are available to them. In many cases, they may only be using these tools every few months, and remembering cryptic UNIX-y command paths and syntax is hard enough even when you use something every day.

And so, I give you menush, a simple shell replacement which presents users with a list of available commands from which the user may choose. It loops until the user exits via the menu (or uses Ctrl-C/Ctrl-D to end the session).

To set it up on your own server, you’ll need to copy the file into a known location (say, /usr/local/sbin), then add a line to your /etc/shells file pointing to it. For each user you want to lock into a sandbox, edit their password entry using vipw (or your passwd editing method of choice) and change the last field from /bin/bash or similar to the full path to menush.

Then, create the directory /etc/menush, and write your default menu file (/etc/menush/__default__). On startup, menush will look in that directory for a file with the same name as the user being logged in; if that file is absent, the default will be loaded instead. The format for the menu files is documented in the README, but it’s just a YAML file. (Also, bonus points for the first person to correctly identify the gaping security hole in the provided example.)

The code is written in a fairly portable, POSIX-y style, so it should work on Linux, BSD, or OS X. Feel free to send me suggestions, pull requests, or rants about the horrible security holes I left because I banged this whole thing out in like two hours and then spent a bunch of time blogging about it instead of reviewing my own code.

And now, for something completely different…

Published
by
lennon
on November 11, 2008
in Uncategorized
. 2 Comments

Things have been far too serious around here lately. Politics, smartphones, and programming are all well and good, but there’s been a notable lack of food pr0n in my posts since Twitter took over my impulse-blogging.

Well, no more. Look upon the awesome power of the “Heart Attack” sandwich, as proudly served during the Sunday brunch at the family BBQ restaurant:

"heart attack" sandwich

You can click through to the Flickr page to see notes breaking down all the constituent parts, but suffice to say, everything on that sandwich, right down to the bun itself, is fried.

It was so good. I think I’ll probably only be able to eat one about once a year, but I will cherish that day.