A Single Programmer's Blog

Today, Craig White asks about svn and ruby structure:

When I set up the repository, I did an import of the entire base directory of the rails application which was great because it allowed me to do a checkout on to my computer with everything.

Of course, now on my home computer, I’ve got a changed database.yml and environment.rb and the log files (because I’ve been running a copy on my home system), and I would suspect that I really only want app and public directories (though the doc and plugins would be nice to manage with svn).

The most common problems Rails developers have with versioning their project are the config/database.yml file and log/ folder. Personally, I have found that doc/appdoc/ and doc/apidoc/ sometimes gets in the way. More on that later.

The following essay discusses how I manage my Rails projects with Subversion. It is assumed you already know about Rails and how to create new Rails applications. I also assume you have a working knowledge of Subversion. If you need help on using Subversion itself, you should really read one of these two books, ideally both:

• Version Control with Subversion
• Pragmatic Version Control using Subversion

If you are on a Windows machine, most of the commands will work as is. Notable exceptions are those that use the backquote (`), find, grep and the yes commands.

Pinkatio – our latest Rails application

We just had a killer idea to get us thousands of dollars of revenue per month. We will call it pinkatio, and we will put it on Rails.

Creating the Subversion repository

This is the most variable step in the whole procedure. If you know your way around a Unix/Linux system, go ahead and put the repository in /var. Watch your permissions, though. Else, I suggest sticking with a HOME based repository, like I did below. You can use the file repository access method. The only caveat is that you will not be able to share your repository with other people using that method.

To create the repository, we simply call svnadmin’s create subcommand:

1
2

$ mkdir ~/svn
$ svnadmin create --fs-type=fsfs ~/svn/pinkatio

$ REPOS=file://`pwd`/svn/pinkatio

Subversion recommends creating all repositories with three folders at the root: trunk, tags and branches. This is if you use the one project per repository. This is explained in more details in Choosing a Repository Layout.

This is by no means a requirement to use Subversion, but I suggest sticking to this convention anyway. Most Subversion repositories I have seen adhere to the convention, and if you have only one project in your repository, it makes sense to be able to tag and branch at will.

$ svn mkdir --message="Initial project layout" $REPOS/trunk $REPOS/tags $REPOS/branches

Committed revision 1.

With the repository creation out of the way, let us now turn to creating our Rails application.

Creating the Rails application and importing into the repository

Creating the Rails application is straightforward:

1
2
3
4
5

$ rails ~/pinkatio
      create
      create  app/controllers
      ...
$ cd ~/pinkatio

At this point, you could do an svn import and import the whole application into Subversion. I recommend against doing that. If you use the “in-place import” procedure, you can commit only the pieces that you want, not the whole tree (log files being ones we don’t want under version control). See Subversion’s How can I do an in-place ‘import’ FAQ for the full details.

1
2

$ svn checkout $REPOS/trunk .
Checked out revision 1.

1
2
3
4

$ svn add --force .
A         app
...
A         README

1
2
3

$ svn mkdir db/migrate tmp 
A         db/migrate
A         tmp

Removing the log files from version control

Right now, Subversion will helpfully track changes to the log files. This is not really useful for us, as the log files can be pruned at any point.

To ease our burden, the easiest thing is to tell Subversion to ignore the logs.

1
2
3
4
5
6
7
8

$ svn revert log/*
Reverted 'log/development.log'
Reverted 'log/production.log'
Reverted 'log/server.log'
Reverted 'log/test.log'

$ svn propset svn:ignore "*.log" log
property 'svn:ignore' set on 'log'

See svn:ignore in the Subversion book for more details on the property format.

Managing the database configuration

Again the Subversion FAQ comes to the rescue: I have a file in my project that every developer must change, but I don’t want those local mods to ever be committed. How can I make ‘svn commit’ ignore the file?.

The solution is to have a template of the file in the repository, and to force each working copy to copy the template file to the real file. Let us simply revert the add of the config/database.yml file, and add a sample file instead:

1
2
3
4
5
6
7
8
9
10

$ svn revert config/database.yml
Reverted 'config/database.yml'

$ mv config/database.yml config/database.yml.sample
$ svn add config/database.yml.sample
A         config/database.yml.sample

$ svn propset svn:ignore "database.yml" config
property 'svn:ignore' set on 'config'
$ cp config/database.yml.sample config/database.yml

The only problem with this procedure is if important changes are made to the config.yml.sample file, the developers might not notice the changes. Most of the time though, the sample file will not change, and leaving it as-is is ok.

Database structure dumps during testing

When you run the tests, Rails dumps the development database’s structure to a file in db/. Usually, this file should not be under version control. Your migration scripts should be under version control instead, and your migrations should enable you to recreate the development database at will.

Additionally, this step will depend on which configuration setting you use for the config.active_record.schema_format. If you use the :ruby (the default on Edge Rails and Rails 1.1), you should ignore the schema.rb file from db/. If you use :sql, simply ignore development_structure.sql instead. Alternatively, you could ignore both files, making this a moot point.

1
2

$ svn propset svn:ignore "schema.rb" db
property 'svn:ignore' set on 'db'

tmp/, documentation, scripts and public

Edge Rails and Rails 1.1 now possess a tmp/ folder. Since this folder will hold socket and session files, we can safely ignore everything in it.

1
2

$ svn propset svn:ignore "*" tmp
property 'svn:ignore' set on 'tmp'

The doc/ folder can hold two subfolders: appdoc/ and apidoc/. If you don’t plan on building the documentation for your project, you can ignore setting svn:ignore on doc/. Else, you should ignore like this:

1
2

$ svn propset svn:ignore "*doc" doc
property 'svn:ignore' set on 'doc'

Subversion also has a property that tells it which files are executable. We can set the property on files that are intended to be run from the command line:

1
2
3

$ svn propset svn:executable "*" `find script -type f | grep -v '.svn'` public/dispatch.*
property 'svn:executable' set on 'script/performance/benchmarker'
...

Last but not least, my projects usually have a default home page served by a Rails action. This means building a route and removing public/index.html:

1
2
3
4

$ svn revert public/index.html
Reverted 'public/index.html'

$ rm public/index.html

Saving our work

After all of these changes, it is important to commit our work to the repository.

1
2
3
4
5
6

$ svn commit --message="New Rails project"
Adding         README
...
Adding         vendor/plugins
Transmitting file data .......................................
Committed revision 2.

After this step, it is time to start coding your application, unless you need to go on the Edge…

Using Rails Edge and protecting against overzealous gem upgrades

When you are going to put your application into production, you don’t want an upgrade in your host’s environment to affect your application. To prevent such problems, you should keep a local copy of Rails in your application’s vendor folder.

If you want to live on the Edge (with all the latest features), this step is a necessity. If you are not so comfortable with Edge, replace trunk/ with tags/rel_1-0-0 (or tags/rel_1-1-0 when Rails 1.1 is out) in the svn:externals property below.

1
2
3
4
5
6
7
8
9
10
11
12

$ svn propset svn:externals "rails http://dev.rubyonrails.org/svn/rails/trunk/" vendor
property 'svn:externals' set on 'vendor'

$ svn update vendor

Fetching external item into 'vendor/rails'
A    vendor/rails/cleanlogs.sh
...
 U   vendor/rails
Updated external to revision 3830.

Updated to revision 2.

If you went for Rails Edge, you should really rerun the rails . command after you update. This will ensure you get the latest version of the scripts and javascript files. And since we have not made any changes to the contents of any files, now is the best time to do this.

1
2
3
4
5

$ yes | rails .
      exists
      exists  app/controllers
...
   identical  log/test.log

1
2
3
4

$ svn commit --message="Living on the Edge - set svn:externals on vendor/ for Rails"
Sending        vendor

Committed revision 3.

Tracking Edge Rails

Next time you svn update, Subversion will go out to the Rails repository and retrieve all changes since your last checkout. If the JavaScript files changed, you should copy them over using the rails:update Rake command:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

$ svn update
...
Updated external to revision 3831.

Updated to revision 2.

$ rake rails:update
(in /home/fbos/pinkachio)

$ svn status
M      public/javascripts/prototype.js
M      public/javascripts/effects.js
M      public/javascripts/dragdrop.js
M      public/javascripts/controls.js

$ svn commit --message="Updated JavaScripts to latest revision"
M      public/javascripts/prototype.js
...

Gems

Gem features a useful subcommand: unpack. When you run it on a gem, it will unpack that gem’s content into the current folder. We can then move the code to our vendor/ folder, and again protect ourselves against host upgrades.

As an example, let us unpack the Money gem:

1
2
3

$ cd vendor
$ gem unpack money
Unpacked gem: 'money-1.7.1'

Gems all have a lib/ folder into which the gem’s source code is stored. Copying the contents of the lib/ folder into vendor/ is the important trick here. If you move lib/ to vendor, it won’t help, as Rails automatic dependency loading mechanism will not know how to find your code.

At the same time, to comply with the library’s license, we copy the library verbatim to our vendor/ folder.

1
2
3

$ cp -Rf money-1.7.1/lib/* .
$ cp -Rf money-1.7.1/MIT-LICENSE LICENSE-money
$ cp -Rf money-1.7.1/README README-money

1
2
3
4

$ svn add bank money support money.rb LICENSE-money README-money
A         bank
..
A         README-money

1
2

$ svn propset version "1.7.1 (Gem)" money.rb
property 'version' set on 'money.rb'

$ rm -Rf money-1.7.1

1
2
3
4
5
6

$ cd ..
$ svn commit --message="Unpacked Money 1.7.1 into vendor/"
Adding         vendor/LICENSE-money
...
Transmitting file data ........
Committed revision 4.

Gem upgrades

When the next version of the Money gem will come around, we simply follow the same procedure as above. Of course, you will use svn status to know what files changed exactly. You might have to add new files, and remove old ones.

One tool that can help automate this process svn_load_dirs.pl, from the Subversion’s contrib/ area.

Plugins

For plugins, you have to take the same decision as for Rails – Edge or safe. For plugins, I have found that sticking to released version is safer for me. YMMV.

As an example, I will use the FileColumn plugin. Unfortunately, this plugin is not ready to be used by the script/plugin install -x procedure. So, we have to resort to a manual one.

1
2
3
4
5
6
7
8
9
10
11
12

$ svn propset svn:externals "file_column http://opensvn.csie.org/rails_file_column/plugins/file_column/tags/rel_0-3-1/" vendor/plugins
property 'svn:externals' set on 'vendor/plugins'

$ svn update vendor/plugins

Fetching external item into 'vendor/plugins/file_column'
A    vendor/plugins/file_column/test
...
 U   vendor/plugins/file_column
Updated external to revision 58.

Updated to revision 4.

1
2
3
4

$ svn commit --message="Added FileColumn plugin"
Sending        vendor/plugins

Committed revision 5.

Creating migrations, models and controllers

1
2
3
4
5
6
7

$ script/generate migration --svn InitialSchema
      exists  db/migrate
      create  db/migrate/001_initial_schema.rb
A         db/migrate/001_initial_schema.rb

$ svn status
A      db/migrate/001_initial_schema.rb

All generate and destroy generators accept the --svn option. This makes it easy for the developer to keep his changes under version control.

Final words

I follow this script more or less verbatim for all of my Rails projects. After the first two or three times, this becomes automatic. For the adventurous, I have a shell script which does most of the steps above automatically. You can get rails2.sh.

# Distributed in the Public Domain by Francois Beausoleil.
# This script might destroy twenty years worth of work, and I cannot be held
# responsible.  You are your own master.  Read this file in detail before
# you use it.
#
# NO IMPLIED WARRANTY.