Some weeks ago I needed to build a small, static web page starting from a large number of images. Boring task… copying and pasting chunks of HTML is not for me, so I started looking for a suitable static site generator. I ended up using nanoc but could not stop wondering about Jekyll.

I know, it’s better suited to blog-like sites, but I had some free time (it was a national holiday :-)) and so put together a rough extension for Jekyll which has already slept too long on my disk. Here’s the code, without warranty of any kind (including usefullness).

A couple of days ago I spent the lunch playing, again, with fog; this time I was interested in S3’s multipart upload feature, so fired up fog’s cli and started typing away to test it.

Fog still does not have any model for this recently added feature, so I had to resort to use the lower level mechanism of requests. Here’s a (bad, procedural, defective of any error checking) script I wrote afterward by collecting and cleaning up the cli session:

So, roughly speaking, I started asking AWS the list of pending multipart uploads (and so discovered a couple of them initiated by another tool and dating back from an earlier upload gone south twice), set up a new one and uploaded the previously split parts, ending that section showing the parts’ identifiers assigned by AWS, just to be sure they were really there, and, again, the list of pending multipart uploads to confirm that somehow I did something right; then I asked AWS to reassemble the file, check that there were no more pending uploads and get the new file metadata.

Some notes:

• I think it’s better to pass File objects to fog instead of reading files into memory and passing their contents to fog (a File object will be read and transfered in chunks by excon);
• I prefer to wrap file operation (even the upload of single files) in a block passed to File::open, so that the file is automatically closed when done.

Inspired by the experience (or disgusted by the amount of low level operations?), I jotted down what I’d like to see as higher level interface… perhaps I should try to find some time to fork fog :-)

Chef is growing stronger every day and after reading that Engine Yard has decided to drop their Capistrano based deployment process, I found myself thinking about mapping between Capistrano e Chef…

Chef’s stock deploy resource is approximately equivalent to Capistrano’s deploy task (or deploy:migrations, both with the remote cache strategy thrown in): some things are done in a slightly different order, but the results are largely the same. Even if the process is particularly suited to Ruby on Rails applications, it’s general enough to be usable with other type of web applications. Engine Yard’s deploy resource is quite different from Chef’s one, but the principles are the same.

A key difference between Capistrano’s standard recipe and Chef’s stock deploy resource is that the latter is not so flexible as the former: you can tweak the resource behaviour via a given number of attributes and you have another given, small number of spots where you can insert custom code via callbacks; with Capistrano, you can insert callbacks pratically anywhere. When switching from Capistrano to Chef you are obliged to rework your custom deployment recipes, eventually merging or rearranging your callbacks.

So, roughly speaking:

• whatever you ran before deploy or deploy:migrations, before or after update_code and before or after finalize_update in Capistrano, now you must put it in Chef’s before_migrate callback;

• whatever you ran after migrate and before symlink in Capistrano, now you must put it in Chef’s before_symlink callback;

• whatever you ran after symlink and before restart in Capistrano, now you must put it in Chef’s before_restart callback;

• whatever you ran after restart or after deploy:migrations in Capistrano, now you must put it in Chef’s after_restart callback.

Don’t bother adding code equivalent to Capistrano’s deploy:cleanup task: Chef’s stock deploy resource already cleans up after itself, keeping a small, fixed number of deployed releases (5, at the moment).

Yesterday afternoon I had to share some rather big files with a client, so I decided to put them in a S3 bucket and let him download them at his pleasure. Nothing too special, but I could not let the whole world to see these files nor I could force any form of explict authentication upon this client, so I was left with the signed URLs.

While waiting for the last couple of uploads to finish, I thought “Why not scripting the remaining boring part?”, so fired up fog’s cli and wrote something like following lines (amended to hide the real names and fit the page):

Operations could have been chained a little bit more, but I prefer to build things piece by piece. I used ActiveSupport because I’m too lazy to do the time math by myself; the last #map is “necessary evil” for European buckets (at least with fog 0.4.1, available yesterday afternoon).

Update 2011-02-03: setting the right region in the ~/.fog file (say eu-west-1) has a couple of nice effects: first you avoid a redirection when connecting to S3 and then the signed URLs change from https://s3.amazonaws.com/yourbucket/… to https://s3-eu-west-1.amazonaws.com/yourbucket/… and this makes the last map {} unnecessary.

I took a few days off after Christmas, but I cannot really stop tinkering with computers at large, so I returned to play with knife. I really like it, but have a few wishes, in no particular order, which I hope to see fulfilled.

• I’d like the ssh subcommand optionally made use of known_hosts.

  In an ever changing environment, like a cloud, it would probably be a hassle, but over the years I’ve appreciated this checking of the hosts’ keys at least a couple of time and in a more stable environment I really cannot think about living without it.

• I’d like more things could be specified inside the configuration file (e.g., SSH username and identity files, EC2 region, etc.).

  I like the subcommands’ great flexibility, but for daily use with homogeneously configured machines, I’d rather if most of those connection/identity/service related options could be written once and for all, with CLI flag taking precedence.

• I’d like knife itself were a separate gem, depending on Chef, the various SSH gems and fog.

  Granted, you don’t need knife on every machine, but keeping track of what it needs just in case you have to use it somewhere else, is a (really small) nuisance.

One more thing… Happy New Year!

After the release of Ruby on Rails 3, using Webrat with Cucumber in a Ruby on Rails 3 application is not something that works out of the box as before.

The following assumes Cucumber 0.9.4, Cucumber-rails 0.3.2 and Webrat 0.7.2.

Rack mode

First of all, if we let Cucumber’s generator do its work, we get an env.rb file which is unsuitable for Ruby on Rails 3: the Webrat’s mode is wrong. We can edit that file, trusting that our VCS will help us to recover the needed modifications next time we upgrade Cucumber, or we can drop a new file in features/support/ and override the configuration:

Rack::Test

Then we have to fix Rack::Test, which is used by Webrat’s Rack mode and Rails’s integration testing classes. What’s wrong with it?

Webrat and Rails use “example.com” as default host; Rack::Test uses “example.org” as default host in the fake session automatically built by its methods which simulate HTTP requests. This difference influences negatively Webrat’s capability to follow the usual, “internal” redirection which usually comes after, for example, record updates: the hosts do not match, so the redirection is not considered an internal one and Webrat does not follow it.

We must redefine a Rack::Test’s constant; Ruby will produce a warning, but it’s harmless noise in this case.

Update 2011-03-07: A reader kindly reminded me about Kernel::silence_warnings, so you can optionally redefine that Rack::Test’s constant as follows and never see that noise again:

Please, beware of the peril of “playing with constants” and don’t blame neither of us if your application blows up misteriously :-) any time later.

HTML5 data attributes

So now we have almost everything working again… until we smash into the following problem.

Let’s say we have a list of records, each paired with a “Delete” link. We have the following scenario, spiced up with the help of Pickle:

We run Cucumber and the scenario does not pass. Webrat uses link’s onclick to decide if GET or something else must be used, but Ruby on Rails 3 does not produce the same, old HTML code: it now embraces the unobtrusiveness way of life, so it got rid of onclick attributes and replaced them with HTML5 data attributes:

Until a fix emerges for this, a workaround could be something like the following step definition:

The happy end

There’s no doubt that life was simpler in the old days of Ruby on Rails 2 :-) but with just a bunch of tweaks everything works almost as before and things will probably get better as the dust settles.

This post is for myself (so that I won’t forget) and the only other reader of this blog.

Background: you have a shining new web application written with Ruby on Rails 3.0.*, which uses an engine; this engine employs various static assets (CSS stylesheets, images, etc.) stored in the engine’s public directory.

You code your way and during development everything works. Then, a day, you fire up Thin or Mongrel in production mode and… where the heck are the stylesheets?

Static assets in production

Ruby on Rails 3.0.* does not serve static assets (i.e., the contents of public) in production mode. The reasons are that your application server is usually fronted with a web server and this piece of software is better suited than anything else to quickly serve files.

But you insist: you want to see you app, in production mode, on your development machine, running just Thin. Ok, then change the value of config.serve_static_assets in config/environments/production.rb to true or comment out that row.

Restart Thin, reload the app and there they are, your stylesheets, your images, etc. There’s almost everything, but the engine’s assets! And the app is strangely slow…

Turns out that there’s another optimization for production environment at work: X-Sendfile.

Engine’s static assets in production

Even if you’ve just reenabled serving static assets, that setting influences only the contents of your app’s public directory. Engines’ assets are affected by config.action_dispatch.x_sendfile_header: if its value is one out of a short list and not nil, every response containing your assets is replaced with an empty body and a particular header that a well configured web server would intercept and manage for you trasparently.

So, comment out also that row in config/environments/production.rb, restart Thin and, lo!, all your assets are there.

Clean up before deployment

First of all, remember to restore your production.rb file.

Then, double check that the web server used in production can handle X-Sendfile/X-Accel-Redirect: if it can’t, either replace it with another software or comment out again config.action_dispatch.x_sendfile_header in production.rb. It will slow down your app, because engines’ assets will have to be handled by the application server, but they’ll at least be there.