Devoh

In the Kitchen: Cooking with Chef

I gave a brief introduction to Chef at this month's at this month's Orlando Ruby Users Group meeting. Gregg Pollack was kind enough to record and produce a video of that presentation.

Relaxed and Relieved

I've been working on a rewrite of Relax for a little while now. My desire is to create a DSL for defining web service consumers that's flexible enough to work with the wide variety of different services out there. For example, some web services may require the method to be specified as part of the path, while others use query parameters. Some use HTTP Basic authentication, while others rely on OAuth. Some produce XML responses, but others may return JSON. The ultimate goal is be able to quickly produce a simple consumer for situations where you may not need a full-blown API.

While at RailsConf this week, I had some time between sessions to wrap up some of these changes and have released an initial version of the rewrite. During this final stretch of development I ended up stripping out a lot of unnecessary complexity. I had originally created factories for performers (used to make the actual network requests), authenticators, and parsers, but I had only implemented a single concrete implementation for each. I figure that it's better to simplify the initial release, and then worry about adding in additional implementations as the need arises. I think this will result in better code since development will be driven by practical requirements rather than speculative abstraction.

Here's what a basic Flickr search consumer might look like with Relax:

class Flickr < Relax::Service
  defaults do
    parameter :api_key, :required => true
  end

  endpoint 'http://api.flickr.com/services/rest' do
    defaults do
      parameter :method, :required => true
    end

    action :search do
      set :method, 'flickr.photos.search'
      parameter :per_page, :default => 5
      parameter :page
      parameter :tags

      parser :rsp do
        attribute :stat, :as => :status

        element :photos do
          attribute :page
          attribute :pages
          attribute :perpage, :as => :per_page
          attribute :total

          elements :photo do
            attribute :id
            attribute :owner
            attribute :secret
            attribute :server
            attribute :farm
            attribute :title
            attribute :ispublic, :as => :is_public
            attribute :isfriend, :as => :is_friend
            attribute :isfamily, :as => :is_family
          end
        end
      end
    end
  end
end

And here's a search for photos of cucumbers and lemons:

flickr = Flickr.new(:api_key => FLICKR_API_KEY)
flickr.search(:tags => 'cucumbers,lemons')

An unintentional byproduct of this process was the extraction of the XML parser into a standalone gem called Relief. The API is somewhat inspired by SAX-Machine, but it returns hashses and arrays instead of objects with instance methods. It's also not SAX-based at the moment, but it does support the use of XPath for element definitions.

The parser in the Relax example above is using Relief, but here's a simpler, standalone example:

parser = Relief::Parser.new(:photos) do
  elements :photo do
    element :name
    element :url
  end
end

The same parser could be implemented using XPaths like so:

parser = Relief::Parser.new(:photos) do
  elements '//photo', :as => :photo do
    element 'name/text()', :as => :name
    element 'url/text()', :as => :url
  end
end

I'm happy with the direction these libraries are headed, but I'm happier still to have wrapped up the work that I've been putting into them and have actually released some gems. So, sudo gem install relax relief and let me know what you think.

Installing CouchDB on Mac OS X 10.5 Leopard

Trial and Error

CouchDB is a RESTful, document-based database that uses map-reduce with JavaScript as its query language. Its innovative use of some lesser known projects make its existence possible, but they also tend to complicate its installation process. The large majority of the Mac OS X installation instructions available for CouchDB utilize MacPorts for its dependencies, leaving those of us who prefer to install from source in the dark.

I felt that is was worth pursuing a from-scratch installation, but alas I ended up spending many more hours than any sane person would to accomplish such a trite goal. It ended up being a trial and error process to figure out each missing piece until the puzzle was complete. It is my hope that there are others like myself who will benefit from this information, but, if not, I write it is as a personal reference.

This guide provides step-by-step instructions on installing the requisite dependencies, and, ultimately, CouchDB itself. It should provide a fairly complete description of each task along the way, but let me know if you run into snags or have suggestions on improving the content.

Install Erlang

CouchDB is written in Erlang, so we'll need to install it before we'll be able to run CouchDB.

curl -O http://www.erlang.org/download/otp_src_R12B-5.tar.gz
tar xzvf otp_src_R12B-5.tar.gz
cd otp_src_R12B-5
./configure --enable-hipe --enable-smp-support --enable-threads
make
sudo make install

International Components for Unicode

CouchDB's internationalization support is provided by the ICU library.

curl -O http://download.icu-project.org/files/icu4c/4.0.1/icu4c-4_0_1-src.tgz
tar xzvf icu4c-4_0_1-src.tgz
cd icu/source
./runConfigureICU MacOSX --prefix=/usr/local
make
sudo make install

SpiderMonkey

CouchDB relies on the same SpiderMonkey JavaScript engine that Firefox uses. Unfortunately, the build process for this project seems surprisingly neglected considering its prevalence within Mozilla's products.

You'll first need to install the Netscape Portable Runtime, as SpiderMonkey relies on this library for its thread support.

curl -O http://ftp.mozilla.org/pub/mozilla.org/nspr/releases/v4.7/src/nspr-4.7.tar.gz
cd nspr-4.7/mozilla/nsprpub
./configure
make
sudo make install

Now SpiderMonkey itself can be installed, but we'll have to apply some patches from the MacPorts port to get it to build properly.

Download and extract the source, first.

curl -O http://ftp.mozilla.org/pub/mozilla.org/js/js-1.7.0.tar.gz
tar xzvf js-1.7.0.tar.gz
cd js/src

Next, fetch and apply the patches.

curl http://trac.macports.org/export/44341/trunk/dports/lang/spidermonkey/files/spidermonkey-1.5-build.patch | patch -p0
curl http://trac.macports.org/export/44341/trunk/dports/lang/spidermonkey/files/spidermonkey-1.5-threadsafe.diff | patch -p0
curl http://trac.macports.org/export/44341/trunk/dports/lang/spidermonkey/files/spidermonkey-1.5-header.diff | patch -p0
curl http://trac.macports.org/export/44341/trunk/dports/lang/spidermonkey/files/patch-config-Darwin.mk | patch -p0

Then, replace the placeholders in the Makefile.

sed -i -e 's|__PREFIX__|/usr/local|' Makefile.ref
sed -i -e 's|__USER__|root|' Makefile.ref
sed -i -e 's|__GROUP__|wheel|' Makefile.ref

And, finally, build and install.

make -f Makefile.ref LIBDIR="/lib" SO_SUFFIX=dylib JS_THREADSAFE=1
sudo make -f Makefile.ref install
sudo ranlib /usr/local/lib/libjs.a

CouchDB

We now have all of the dependencies in place, and we're ready to proceed with the installation of CouchDB itself.

curl -O http://www.smudge-it.co.uk/pub/apache/incubator/couchdb/0.8.1-incubating/apache-couchdb-0.8.1-incubating.tar.gz
tar xzvf apache-couchdb-0.8.1-incubating.tar.gz
cd apache-couchdb-0.8.1-incubating
./configure --prefix=/usr/local
make
sudo make install

User and Group

We need to create a user and group now, but I won't go into too much detail here as it has already been covered here before in the PostgreSQL guide.

For the purposes of this tutorial, let's assume an ID of 115 for both the user and the group.

First, create the _couchdb user.

sudo dscl . create /Users/_couchdb UniqueID 115
sudo dscl . create /Users/_couchdb PrimaryGroupID 115
sudo dscl . create /Users/_couchdb RealName "CouchDB Server"
sudo dscl . create /Users/_couchdb NFSHomeDirectory /usr/local/pgsql/
sudo dscl . create /Users/_couchdb Password "*"
sudo dscl . append /Users/_couchdb RecordName couchdb

Then, create the _couchdb group.

sudo dscl . create /Groups/_couchdb
sudo dscl . create /Groups/_couchdb PrimaryGroupID 115
sudo dscl . create /Groups/_couchdb RealName "CouchDB Users"
sudo dscl . append /Groups/_couchdb RecordName couchdb

Firing It Up

Before we can start up CouchDB, we need to fix the permissions on a couple directories.

sudo chown -R couchdb:couchdb /usr/local/lib/couchdb/ \
  /usr/local/log/couchdb/

Then, we'll try starting it up to make sure everything worked.

sudo -u couchdb couchdb

You should be able to open your browser to http://localhost:5984/ and see a welcome message of sorts.

{"couchdb":"Welcome","version":"0.8.1-incubating"}

Daemon

CouchDB installs a daemon for automatic start-up, but we still have to let launchd know about it.

ln -s /usr/local/Library/LaunchDaemons/org.apache.couchdb.plist \
  /Library/LaunchDaemons/org.apache.couchdb.plist
sudo launchctl load /Library/LaunchDaemons/org.apache.couchdb.plist

Getting Started

If you're new to CouchDB, here are a few resources to get you started.

• the Introduction and Technical Overview from the CouchDB site
• the API Reference and How-To Guides from the CouchDB wiki
• CouchDB: The Definitive Guide, a book in progress
• Programming CouchDB with JavaScript, a tutorial on creating a to-do list application

Thanks

Many thanks to Josh Lucas for his tip about applying the patches from the SpiderMonkey port. I don't think I would have succeeded without it. I owe you a beer, Josh.

I'd also like to thank Dan Benjamin for his technical review of this guide, and for his own inspiring install-from-source tutorials.

Supplanting Lighttpd and Mongrel with Nginx and Thin

I've been using lighttpd and Mongrel to deploy my Rails applications for some time now. While I've kept abreast of the other deployment options, I've never really had a need for a more optimal setup, but lately I've started feeling that it's now time to move on to something that's more actively maintained. With my most recent deployment I decided to go ahead and mix things up a bit and try out Nginx and Thin.

I found this choice of servers to be trivial to configure, and I especially like having the ability to use UNIX domain sockets to avoid the overhead of using TCP sockets. I also found Thin's built-in init script installation a breath of fresh air compared to mongrel_cluster's manual configuration process. Even setting up SSL in Nginx proved to be a fairly simple procedure, except for one gotcha in getting the ssl_requirement plugin working. I had to add a X-Forwarded-Proto header to let Rails know that SSL was turned on. In the Nginx server block for the secure site, the location headers should look like this:

location / {
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto https;
  proxy_set_header Host $http_host;
  proxy_redirect false;
  ...
}

The only other part that required some custom configuration was letting Capistrano know how to start and stop Thin. I ended up with this simple addition to my deploy.rb:

namespace :deploy do
  %w(start stop restart).each do |action|
  desc "#{action.capitalize} the application servers."
    task action.to_sym, :roles => :app do
      run "thin #{action} -C /etc/thin/#{application}.yml"
    end
  end
end

I'm happy with the way this little experiment turned out—happy enough that I'll probably stick with this combination for a while.

Anticipating Change

Since the announcement was made right before Christmas that Merb is going to be merged into Rails, everyone and their brother has seemingly had something to say about the situation. I have to admit that when I first heard the news, I fell into the camp of dissenters. I’ve been using Merb for the past six months, and during that period I’ve become quite attached to the framework. While I think Rails brought a lot of great new concepts to the table, Merb proves that there’s still lots of room for improvement in this space. I feared that the advancements made in Merb would be somehow diminished were they to be merged into Rails.

That being said, the new joint Rails and Merb core team has a big challenge ahead of itself in defining the scope of this project. I would imagine that there would be a desire to keep at least a modicum of backwards compatibility with existing Rails applications, but at the same time there needs to be an evolutionary progression. While not quite a paradigm shift, there are major new concepts being incorporated into Rails from Merb.

I have no doubt that the team will be able to pull it off. I’m over my initial hesitation that the merge is a good idea, and more than anything I’m excited to see how the team will choose to proceed and how Rails 3 will take shape. Even if the only beneficial products of the merge are a well-defined public API and a more modular framework there’s still a lot to which we can look forward.