Just received an email from Apple today:

From: devbugs@apple.com
Subject: Re: Bug ID 6543251: Git is not available by default on Leopard

Hello François,

This is a follow up to Bug ID# 6543251. After further investigation it has been determined that this is a known issue, which is currently being investigated by engineering. This issue has been filed in our bug database under the original Bug ID# 5404556. The original bug number being used to track this duplicate issue can be found in the State column, in this format: Duplicate/OrigBug#.

Thank you for submitting this bug report. We truly appreciate your assistance in helping us discover and isolate bugs.

Best Regards,

Kit Cheung
Apple Developer Connection
Worldwide Developer Relations

This is a followup to Want Git preinstalled on next Mac OS X?

I just found out Howto use Git and svn together, by Flavio Castelli. In there, he lists the steps he uses to develop using Git, but keeping a Subversion repository in the loop.

In his Solve git-svn rebase problems section, Flavio uses git-stash to workaround git-svn rebase problems when your tree has local modifications. I use a different solution. I develop locally on a branch and never commit directly on master. This way I can git-svn rebase with impunity. When I’m ready to merge, I’ll git-svn rebase, followed by git checkout my-branch, git rebase master (to freshen my changes with the remote repository), and finally, I’ll git merge onto master. The steps:


1 git checkout -b francois
2 # work and commit locally on branch francois
3 git checkout master
4 git svn rebase
5 git checkout francois
6 git rebase master
7 # run the tests!
8 git checkout master
9 git merge francois # I might use —squash if the new feature warrants it
10 git svn dcommit # push to Subversion

Overall, my experience with this workflow has been great. I can the features of each system to it’s fullest advantage. I can be on the road, still Gittin’ it done, and the rest of the team still uses Subversion. This is a great combination!

Thanks to Jean-François Couture for showing me the light.

James Bowes has a super simple git-rebase explanation: git rebase: keeping your branches current.

If you have any difficulty understanding the concept, or want an easy way to explain to someone, go ahead.

I’m just starting to use git-svn and git-rebase, and the combination rocks.

Well, it’s working. Piston 2.0 (aka 1.9.0 in Piston’s GitHub repository) can import from a Subversion repository, into a Subversion working copy. And it’s mostly saving the same information as it was previously, in the same way.

But, I received a suggestion/comment from Faisal:

so i’m wondering if it wouldn’t make sense to have the per-vcs metadata formats replaced with something like config/piston.yml, read by piston. the main advantage to doing so would be easier conversion between format types, especially in cases where one type is checking out from another (e.g. svk or git-svn against an svn back-end).

Faisal N Jawdat in a private E-Mail conversation

Actually, this is not a bad idea. Using this method, a Subversion project, imported in a Git repository, would still be able to use Piston to update the vendor branch. The more the information is accessible, the better it will be.

This means Piston needs to provide an upgrade path. I shall do like Subversion, which silently upgrades the working copy when necessary. And to be on the safe side, I shall also include a format in the piston metadata to enable future upgrades with easier handling.

Recapping, Pistonized directories will now have a new, hidden, file:


1 $ ls -A1 vendor/rails
2 .piston-metadata.yml
3 activerecord/
4

And the .piston-metadata.yml file would contain something like this:


1 $ cat vendor/rails/.piston-metadata.yml
2 # What data can Piston expect from this file ?
3 format: 1
4
5 # Which repository handler must we use ?
6 repository: svn
7
8 # Properties that the handler wanted us to save
9 handler-metadata:
10 # Same as piston:remote-revision
11 remote-revision: 9025
12 # Same as piston:root
13 svn-root: http://dev.rubyonrails.org/svn/rails/trunk
14 # Same as piston:uuid
15 svn-uuid: 5ecf4fe2-1ee6-0310-87b1-e25e094e27de

If you were aware of how Piston was storing properties, you might not that the piston:local-revision property is gone. Instead of hard-coding which revision we need to import, I’ll instead use the last changed date/time. It’s less precise, but makes interoperability with different repository handlers much easier. No need to map between revision 8920 of Subversion to Git commits.

That looks promising, no ?

Next step ? Implementing the Git backend. After that, it’s testing Svn+Git, Git+Svn, Git+Git and Svn+Svn. Hurray!

Well, I just got the answer I needed: how to clone just the tip of a remote Git repository ? Here’s how:

Sometimes you just want to distribute the source code without its history, and that’s where git-archive comes in.

git archive <tree-ish> > my_new_archive.tar

Kate Rhodes in Getting just the tip of a Git repo

I think I might use this in Piston. I’m not too sure yet, as I haven’t implemented piston-git, but it looks like a promising candidate.

Jean-François commented on Piston will get Git support:

You might want to check out braid, formerly known as giston.

Jean-François Couture

I knew about Braid. The way I see it, Braid requires Git. Without it, it is useless. What I want to do with Piston instead is to be completely repository and working copy agnostic. Use the best possible solution given the tools at hand: Subversion properties when appropriate, YAML files if I can’t. Use Git to merge if possible, do it using Subversion if I can’t. And so on.

I even had a request for supporting SVK by Faisal. It will be very important for me to allow any combination of server/working copy. Anyway, more code, less talk. Once I have a workinjg release, you’ll be able to judge the quality of my work.

Now that I’ve had a good change to play with Git, I’m ready to implement Git support in Piston. This will involve a couple of refactorings. I can see 4 cases currently:

Repository Working Copy Strategy
Subversion Subversion Use current strategy of storing data in Subversion properties
Subversion Git Clone the Git repository, copy the files over and store the metadata as Subversion properties. Use Git to handle the merging for Piston (Yay!)
Git Subversion svn export the data and use a hidden YAML file to store the metadata in the pistonized directory
Git Git Use Git submodules perhaps ? Or git clone + copy + YAML

I have no idea how git submodules work, so I can’t really say that I will be handling that last case in the most efficient manner. I’m planning on having completed this work by the end of next week (March 14th). Stay tuned for details !

My friend, Carl Mercier, unveiled Defensio in November 2007. At the time, I had repeatedly told Carl I would write a plugin for Mephisto to integrate with Defensio. Then, in December 2007, TheWebFellas released a plugin that integrated Mephisto and Defensio.

I installed the plugin, but it didn’t work out for me. And then, I had other stuff to do (like nobody does, duh). And now, a couple of months later, I am just learning about Git, and how Git empowers people to make wholesale changes to an application, and still be able to exchange that code with everyone.

I was on the #github channel on Friday night, having some problems creating a new repository. Comes alongs halorgium telling me “why didn’t you name your repository Mephiso ?” Good question. And I asked him if Rick Olson (techno-weenie) had a public Git repository for Mephisto. Halorgium replied with Rick’s repository URL, and told me that it was down at the moment. But, he had a recent clone, which he pushed to GitHub. I then simply forked his repository, and started coding like mad.

It’s now 4 days later, and I am releasing a refactored Mephisto. Instead of being intimately tied to Akismet, or Defensio for that matter, this version of Mephisto uses the Strategy and Adapter design patterns to enable any spam detection engine to connect to Mephisto.

I don’t know Rick personally, nor do I know Mark Dagget, but if they wish, they can now pull from my repository, and the whole Mephisto community will have a much better Mephisto available to them all.

So:

  • If halorgium hadn’t been on #github on Friday, I wouldn’t have known about Rick’s Mephisto repository;
  • If halorgium hadn’t had a clone, I wouldn’t have started then;
  • If I hadn’t been using Git, I wouldn’t have attempted this (too many changes in too many places for Subversion);

Git empowered me to make big changes to a foreign code base. I’m not afraid of losing any of my changes, and anyone can pull from my repository. This is a completely different working model than Subversion.

If you wish to play with this Mephisto version, you can pull from my public clone URL: git://github.com/francois/mephisto.git

Update 2008-03-03: After discussion with Halorgium on #github, I have pushed a multiengine branch. Use that instead of my master, which has been reset.


1 git clone git://github.com/francois/mephisto.git mephisto_defensio
2 cd mephisto_defensio
3 git checkout multiengine
4 rake db:bootstrap db:migrate
5 thin start

If you already have a clone of Mephisto’s repository from someone else, add mine as a remote:


1 cd mephisto
2 git remote add francois git://github.com/francois/mephisto.git
3 git fetch francois
4 git branch —track multiengine francois/multiengine
5 git checkout multiengine
6 rake db:migrate
7 thin start

In case you want to look at the code first, you can browse the GitHub repository using http://github.com/francois/mephisto/tree/multiengine

I suggest starting with Mephisto::SpamDetectionEngine::Base, and exploring from there.

It hurts, but this article is now obsolete. Rails 2.3 has templates, there are a variety of starter applications, and the text of this article was reformatted and put into Deploying Rails Applications. Good bye, article.

Pinkatio – our latest Rails application

We just had a killer idea to get us thousands of dollars of revenue per month. We will call it pinkatio, and we will put it on Rails.

Creating the Subversion repository

This is the most variable step in the whole procedure. If you know your way around a Unix/Linux system, go ahead and put the repository in /var. Watch your permissions, though. Else, I suggest sticking with a HOME based repository, like I did below. You can use the file repository access method. The only caveat is that you will not be able to share your repository with other people using that method.

To create the repository, we simply call svnadmin’s create subcommand:


1 $ mkdir ~/svn
2 $ svnadmin create —fs-type=fsfs ~/svn/pinkatio

To ease our poor fingers, let us create an environment variable to refer tot he repository’s root URL:


1 $ REPOS=file://`pwd`/svn/pinkatio

Subversion recommends creating all repositories with three folders at the root: trunk, tags and branches. This is if you use the one project per repository. This is explained in more details in Choosing a Repository Layout.

This is by no means a requirement to use Subversion, but I suggest sticking to this convention anyway. Most Subversion repositories I have seen adhere to the convention, and if you have only one project in your repository, it makes sense to be able to tag and branch at will.


1 $ svn mkdir —message="Initial project layout" $REPOS/trunk $REPOS/tags $REPOS/branches
2
3 Committed revision 1.

With the repository creation out of the way, let us now turn to creating our Rails application.

Creating the Rails application and importing into the repository

Creating the Rails application is straightforward:


1 $ rails ~/pinkatio
2 create
3 create app/controllers
4
5 $ cd ~/pinkatio

At this point, you could do an svn import and import the whole application into Subversion. I recommend against doing that. If you use the “in-place import” procedure, you can commit only the pieces that you want, not the whole tree (log files being ones we don’t want under version control). See Subversion’s How can I do an in-place ‘import’ FAQ for the full details.


1 $ svn checkout $REPOS/trunk .
2 Checked out revision 1.

Next, let us add the whole tree to the working copy. This is no different than if we had done an svn import initially, except all changes are local, and we can selectively revert files and folders.


1 $ svn add —force .
2 A app
3
4 A README

The Rails command helpfully creates most of the tree. Since I use migrations in all of my Rails projects, I immediately create the db/migrate/ folder. Edge Rails and Rails 1.1 also include a tmp/ folder. For completeness’ sake, I create it at the same time.


1 $ svn mkdir db/migrate tmp
2 A db/migrate
3 A tmp

Removing the log files from version control

Right now, Subversion will helpfully track changes to the log files. This is not really useful for us, as the log files can be pruned at any point.

To ease our burden, the easiest thing is to tell Subversion to ignore the logs.


1 $ svn revert log/*
2 Reverted ‘log/development.log’
3 Reverted ‘log/production.log’
4 Reverted ‘log/server.log’
5 Reverted ‘log/test.log’
6
7 $ svn propset svn:ignore "*.log" log
8 property ‘svn:ignore’ set on ‘log’

See svn:ignore in the Subversion book for more details on the property format.

Managing the database configuration

Again the Subversion FAQ comes to the rescue: I have a file in my project that every developer must change, but I don’t want those local mods to ever be committed. How can I make ‘svn commit’ ignore the file?.

The solution is to have a template of the file in the repository, and to force each working copy to copy the template file to the real file. Let us simply revert the add of the config/database.yml file, and add a sample file instead:


1 $ svn revert config/database.yml
2 Reverted ‘config/database.yml’
3
4 $ mv config/database.yml config/database.yml.sample
5 $ svn add config/database.yml.sample
6 A config/database.yml.sample
7
8 $ svn propset svn:ignore "database.yml" config
9 property ‘svn:ignore’ set on ‘config’
10 $ cp config/database.yml.sample config/database.yml

The only problem with this procedure is if important changes are made to the config.yml.sample file, the developers might not notice the changes. Most of the time though, the sample file will not change, and leaving it as-is is ok.

Database structure dumps during testing

When you run the tests, Rails dumps the development database’s structure to a file in db/. Usually, this file should not be under version control. Your migration scripts should be under version control instead, and your migrations should enable you to recreate the development database at will.

Additionally, this step will depend on which configuration setting you use for the config.active_record.schema_format. If you use the :ruby (the default on Edge Rails and Rails 1.1), you should ignore the schema.rb file from db/. If you use :sql, simply ignore development_structure.sql instead. Alternatively, you could ignore both files, making this a moot point.


1 $ svn propset svn:ignore "schema.rb" db
2 property ‘svn:ignore’ set on ‘db’

tmp/, documentation, scripts and public

Edge Rails and Rails 1.1 now possess a tmp/ folder. Since this folder will hold socket and session files, we can safely ignore everything in it.


1 $ svn propset svn:ignore "*" tmp
2 property ‘svn:ignore’ set on ‘tmp’

The doc/ folder can hold two subfolders: appdoc/ and apidoc/. If you don’t plan on building the documentation for your project, you can ignore setting svn:ignore on doc/. Else, you should ignore like this:


1 $ svn propset svn:ignore "*doc" doc
2 property ‘svn:ignore’ set on ‘doc’

Subversion also has a property that tells it which files are executable. We can set the property on files that are intended to be run from the command line:


1 $ svn propset svn:executable "" `find script -type f | grep -v ‘.svn’` public/dispatch.
2 property ‘svn:executable’ set on ‘script/performance/benchmarker’
3

Last but not least, my projects usually have a default home page served by a Rails action. This means building a route and removing public/index.html:


1 $ svn revert public/index.html
2 Reverted ‘public/index.html’
3
4 $ rm public/index.html

Saving our work

After all of these changes, it is important to commit our work to the repository.


1 $ svn commit —message="New Rails project"
2 Adding README
3
4 Adding vendor/plugins
5 Transmitting file data …………………………………
6 Committed revision 2.

After this step, it is time to start coding your application, unless you need to go on the Edge…

Using Rails Edge and protecting against overzealous gem upgrades

When you are going to put your application into production, you don’t want an upgrade in your host’s environment to affect your application. To prevent such problems, you should keep a local copy of Rails in your application’s vendor folder.

If you want to live on the Edge (with all the latest features), this step is a necessity. If you are not so comfortable with Edge, replace trunk/ with tags/rel_1-0-0 (or tags/rel_1-1-0 when Rails 1.1 is out) in the svn:externals property below.


1 $ svn propset svn:externals "rails http://dev.rubyonrails.org/svn/rails/trunk/" vendor
2 property ‘svn:externals’ set on ‘vendor’
3
4 $ svn update vendor
5
6 Fetching external item into ‘vendor/rails’
7 A vendor/rails/cleanlogs.sh
8
9 U vendor/rails
10 Updated external to revision 3830.
11
12 Updated to revision 2.

If you went for Rails Edge, you should really rerun the rails . command after you update. This will ensure you get the latest version of the scripts and javascript files. And since we have not made any changes to the contents of any files, now is the best time to do this.


1 $ yes | rails .
2 exists
3 exists app/controllers
4
5 identical log/test.log

Don’t forget to commit your changes:


1 $ svn commit —message="Living on the Edge – set svn:externals on vendor/ for Rails"
2 Sending vendor
3
4 Committed revision 3.

Tracking Edge Rails

Next time you svn update, Subversion will go out to the Rails repository and retrieve all changes since your last checkout. If the JavaScript files changed, you should copy them over using the rails:update Rake command:


1 $ svn update
2
3 Updated external to revision 3831.
4
5 Updated to revision 2.
6
7 $ rake rails:update
8 (in /home/fbos/pinkachio)
9
10 $ svn status
11 M public/javascripts/prototype.js
12 M public/javascripts/effects.js
13 M public/javascripts/dragdrop.js
14 M public/javascripts/controls.js
15
16 $ svn commit —message="Updated JavaScripts to latest revision"
17 M public/javascripts/prototype.js
18

Gems

Gem features a useful subcommand: unpack. When you run it on a gem, it will unpack that gem’s content into the current folder. We can then move the code to our vendor/ folder, and again protect ourselves against host upgrades.

As an example, let us unpack the Money gem:


1 $ cd vendor
2 $ gem unpack money
3 Unpacked gem: ‘money-1.7.1’

Gems all have a lib/ folder into which the gem’s source code is stored. Copying the contents of the lib/ folder into vendor/ is the important trick here. If you move lib/ to vendor, it won’t help, as Rails automatic dependency loading mechanism will not know how to find your code.

At the same time, to comply with the library’s license, we copy the library verbatim to our vendor/ folder.


1 $ cp -Rf money-1.7.1/lib/* .
2 $ cp -Rf money-1.7.1/MIT-LICENSE LICENSE-money
3 $ cp -Rf money-1.7.1/README README-money

Let us tell Subversion what files we now want to version control:


1 $ svn add bank money support money.rb LICENSE-money README-money
2 A bank
3 ..
4 A README-money

To help me remember which version I unpacked, I set a custom property on the main file of the library I just unpacked:


1 $ svn propset version "1.7.1 (Gem)" money.rb
2 property ‘version’ set on ‘money.rb’

Next, we cleanup after ourselves:


1 $ rm -Rf money-1.7.1

Finally, let us commit our changes back to the repository.


1 $ cd ..
2 $ svn commit —message="Unpacked Money 1.7.1 into vendor/"
3 Adding vendor/LICENSE-money
4
5 Transmitting file data ……..
6 Committed revision 4.

Gem upgrades

When the next version of the Money gem will come around, we simply follow the same procedure as above. Of course, you will use svn status to know what files changed exactly. You might have to add new files, and remove old ones.

One tool that can help automate this process svn_load_dirs.pl, from the Subversion’s contrib/ area.

Plugins

For plugins, you have to take the same decision as for Rails – Edge or safe. For plugins, I have found that sticking to released version is safer for me. YMMV.

As an example, I will use the FileColumn plugin. Unfortunately, this plugin is not ready to be used by the script/plugin install -x procedure. So, we have to resort to a manual one.


1 $ svn propset svn:externals "file_column http://opensvn.csie.org/rails_file_column/plugins/file_column/tags/rel_0-3-1/" vendor/plugins
2 property ‘svn:externals’ set on ‘vendor/plugins’
3
4 $ svn update vendor/plugins
5
6 Fetching external item into ‘vendor/plugins/file_column’
7 A vendor/plugins/file_column/test
8
9 U vendor/plugins/file_column
10 Updated external to revision 58.
11
12 Updated to revision 4.

Again, we must not forget to commit our changes to the repository:


1 $ svn commit —message="Added FileColumn plugin"
2 Sending vendor/plugins
3
4 Committed revision 5.

Creating migrations, models and controllers

The Rails generate has a helpful option: --svn (-c):


1 $ script/generate migration —svn InitialSchema
2 exists db/migrate
3 create db/migrate/001_initial_schema.rb
4 A db/migrate/001_initial_schema.rb
5
6 $ svn status
7 A db/migrate/001_initial_schema.rb

All generate and destroy generators accept the --svn option. This makes it easy for the developer to keep his changes under version control.

Final words

I follow this script more or less verbatim for all of my Rails projects. After the first two or three times, this becomes automatic. For the adventurous, I have a shell script which does most of the steps above automatically. You can get rails2.sh.

rails2 license:

# Distributed in the Public Domain by Francois Beausoleil.
# This script might destroy twenty years worth of work, and I cannot be held
# responsible.  You are your own master.  Read this file in detail before
# you use it.
#
# NO IMPLIED WARRANTY.

Search

Your Host

A picture of me

I am François Beausoleil, a Ruby on Rails and Scala developer. During the day, I work on Seevibes, a platform to measure social interactions related to TV shows. At night, I am interested many things. Read my biography.

Top Tags

Books I read and recommend

Links

Projects I work on

Projects I worked on