This rake task had been broken for a while. This fixes the breakages,
adds a test to help avoid future breakages, and adds a few ergonomic
improvements to the task itself.